Cert-generator job - argocd sync loop

[geothechr]

Describe the bug
When deploying nginx-gateway-fabric helm chart (v2.4.2, for example), everything is deployed successfully initially (cert generator job enabled, by default). Should be noted that I do not directly deploy helm chart, but use it as a dependent chart in my local helm chart.

Then, if something is changed in helm values or argocd application and argo detects it, it shows that argo wants to delete cert-generator ServiceAccount, Role & RoleBinding.

I suspect the reason it the helm pre-install hook that all of the above resources contain in their annotations:

• https://github.com/nginx/nginx-gateway-fabric/blob/v2.4.2/charts/nginx-gateway-fabric/templates/certs-job.yaml#L10C1-L11C1

Tried couple of things (ignoreDifferences section in argocd application manifest) but nothing works and always argocd is stuck in a sync loop:

1. cert-generator job & resources deleted
2. cert-generator job re-created
3. job completes
4. ...
5. repeat

I also tried to set some argocd PreSync annotation through helm values, but the field certGenerator.annotations does not seem to be used inside helm template:

• https://github.com/nginx/nginx-gateway-fabric/blob/v2.4.2/charts/nginx-gateway-fabric/values.yaml#L699

Has anyone come across with this problem? And did you manage to resolve it somehow?

To Reproduce
Steps to reproduce the behavior:

1. Deploy nginx-gateway-fabric using helm chart (v2.4.2) - with argocd
2. Sync ArgoCD application & wait some time
3. Change argocd manifests
4. Refresh argocd application
5. Argo wants to delete cert-generator manifests - sync loop if applied

Expected behavior
Chart is installed & sync loop does not keep happening forever in changes afterwards.

Your environment

• Version of the NGINX Gateway Fabric - release version or a specific commit: helm chart v2.4.2 (but happened in v2.3.0 helm chart too)
• Version of Kubernetes: 1.33.4
• Kubernetes platform (e.g. Mini-kube or GCP): Azure (AKS)
• Details on how you expose the NGINX Gateway Fabric Pod: Service of type LoadBalancer
• Logs of NGINX container: kubectl -n <nginx-deployment-namespace> logs deployments/<nginx-deployment>
• NGINX Configuration: kubectl -n <nginx-deployment-namespace> exec -it deployments/<nginx-deployment> -- nginx -T

Additional context
Add any other context about the problem here. Any log files you want to share.

[nginx-bot]

Hi @geothechr! Welcome to the project! 🎉

Thanks for opening this issue!
Be sure to check out our Contributing Guidelines and the Issue Lifecycle while you wait for someone on the team to take a look at this.

[geothechr]

btw, even if application is synced in argocd and you try to sync again - cert-generator job keeps re-creating and completed forever :(

[trantandat18pm]

I’m experiencing a similar issue.

Whenever I make changes related to Nginx Gateway Fabric, the application gets stuck in the Syncing state when I try to sync it again. You can see the details in the screenshot I attached:

As a temporary workaround, I terminate the operation and then run Sync again. I often have to repeat this process several times before everything works properly.

System information and configuration:

• Argo CD + Nginx Gateway Fabric (v2.4.1)
• Auto Sync: Disabled
• Sync Option: Server-Side Apply

[ciarams87]

Hi @geothechr and @trantandat18pm, thanks for bringing this issue to our attention!

I have a draft PR open with the changes I think will rectify this issue, I haven't fully recreated the initial bug so I still need to confirm my fix is appropriate