feat(chart): optional sidecar on webhook

[aceeric]

What does it do ?

It's possible to run the webhook outside the cluster. See #5585.

Motivation

In my particular environment, I run the webhook outside the cluster and so the webhook sidecar is un-needed.

More

• Yes, this PR title follows Conventional Commits
• Yes, I added unit tests
• Yes, I updated end user documentation accordingly

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign stevehipwell for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• charts/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: aceeric / name: Eric Ace (333f098, 385ec41, 552d115)

[k8s-ci-robot]

Welcome @aceeric!

It looks like this is your first PR to kubernetes-sigs/external-dns 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/external-dns has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @aceeric. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[szuecs]

/ok-to-test

[mloiseleur]

At first glance, I do not see the interest to add an option for only one person in the world. But maybe I missed something.
I'm putting this PR on hold, waiting for the discussion on linked issue
/hold

[stevehipwell]

FYI this was discussed when the chart was first updated to support the sidecar and it was decided to to support this. I'll second the request to open an issue if you want to discuss this, for architectural changes we much prefer an issue over a PR for this. @mloiseleur correct me if I'm wrong but I think ExternalDNS does support this if you provide your own manifests?

[Raffo]

I have no strong preferences here, will completely defer the decision on if we want to support his in helm.

[stevehipwell]

@mloiseleur what do you think?

[mloiseleur]

@stevehipwell This is a valid and interesting use case. It seems there are now two known users with this needs.

/hold cancel
/retitle feat(chart): optional sidecar on webhook

[stevehipwell]

I'm not sure that provider.webhook.sidecar as a boolean is the best API design for this. Personally I think we should redesign the webhook API to look more like the following with the sidecar values moved under sidecar (and the old values deprecated).

provider:
  name:
  webhook:
    readTimeout:
    writeTimeout:
    url:
    sidecar:
      image:
        repository:
        tag:
        pullPolicy:

When in webhook mode, if the provider.webhook.url value is set then the --webhook-provider-url flag can be overridden otherwise the current sidecar logic can be applied.

[aceeric]

@stevehipwell just to confirm - basically all of the keys under the current provider.webhook will indent under provider.webhook.sidecar (comments & defaults removed for clarity):

provider:
  name:
  webhook:
    url:
    readTimeout:
    writeTimeout:
    sidecar:
      image:
        repository:
        tag:
        pullPolicy:
      env:
      args:
      extraVolumeMounts:
      resources:
      securityContext:
      livenessProbe:
        etc...
      readinessProbe:
        etc...
      service:
        port:
      serviceMonitor:
        interval:
        scheme:
        tlsConfig:
        bearerTokenFile:
        scrapeTimeout:
        metricRelabelings:
        relabelings:

Just confirming before making the mod thanks.

[aceeric]

Folks - I've made the requested mods. (Re-based on master.) Also: I fixed the Helm schema generation as discussed in this issue.

[stevehipwell]

@aceeric we need to keep the old values unless we want to make this a breaking change, I think we probably want to keep this as just a patch even if we release a breaking change for the next binary release.

[aceeric]

@stevehipwell Thank you. I think you're saying to revert the PR to the initial commit wherein the change was (essentially) non-breaking. E.g. values from:

provider:
  name: aws
  webhook:
    image:
      (etc.)

To:

provider:
  name: aws
  webhook:
    sidecar: true/false <------added
    image:
      (etc.)

Please confirm, thank you.

[stevehipwell]

@aceeric I'm saying the existing values need to be kept but deprecated, with the new pattern overriding the old.

[aceeric]

@stevehipwell Thanks. Supporting both current and new values together will require the chart to handle four use cases. Even though one use case is still not supported the chart still needs to be able to differentiate among them.

#	want sidecar?	using which values?
1	Yes	deprecated
2	Yes	new
3	No	deprecated (not supported - backward compatible)
4	No	new

One way to do this is to introduce a new variable under the new sidecar variable:

provider:
  webhook:
    sidecar:
      enabled: true/false

The implementation then would be:

#	want sidecar?	using which values?	Helm values
1	Yes	deprecated	provider.name=webhook && provider.webhook.sidecar=null
2	Yes	new	provider.name=webhook && provider.webhook.sidecar.enabled=true
3	No	deprecated	Not supported (backward compatible)
4	No	new	provider.name=webhook && provider.webhook.sidecar.enabled=false

In summary - out of the box the chart will prefer the new sidecar vars but the deprecated functionality can be enabled by setting the sidecar variable to null.

Please let me know your thoughts or recommend an alternate approach. Thanks again.

[stevehipwell]

@aceeric you should be able to use functions like coalesce & default to combine the new value with the deprecated value in a non-breaking way. This looks like a good case for writing unit tests.

[aceeric]

@stevehipwell Thank you. Then the logic to include or exclude the sidecar container will be:

If webhook.sidecar.image.repository (new value) is null and webhook.image.repository (deprecated value) is null
  no sidecar
else
  sidecar with either new or deprecated values, with new having precedence

[stevehipwell]

@aceeric I'd suggest adding a new provider.webhook.sidecar.enabled value defaulting to true to control if the webhook is in sidecar or external mode when the provider is set to webhook.

[aceeric]

@stevehipwell giving it another go...