chore(source/istio): replace kube API calls with caching and ingress informers

[ivankatliarchuk]

What does it do ?

• instead of making API calls for every istio gateway to fetch ingress objects, use informers with caching

follow-up:

• add ingress transfomers to reduce memory footprint
• when sources configured for --istio-gateway and --istio-virtualservice, we fetching same endpoints twice here

  external-dns/source/istio_gateway.go

  Line 229 in b31afb4

  func (sc *gatewaySource) targetsFromIngress(ctx context.Context, ingressStr string, gateway *networkingv1beta1.Gateway) (endpoint.Targets, error) {

  and here

  external-dns/source/istio_virtualservice.go

  Line 410 in b31afb4

  func (sc *virtualServiceSource) targetsFromIngress(ctx context.Context, ingressStr string, gateway *v1beta1.Gateway) (endpoint.Targets, error) {

  plus then we need a deduplicate logic to kickoff as well. In follow-up PR (most likely in next release, as istio has quite few changes already), I'll remove requirement for an ingress fetching when flag --istio-virtualservice present, as the logic is duplicated.

Motivation

• just trying to optimise logic where possible, before we dive into multiple workers, threading and concurrency
• adding missing documentations and use cases
• previous PR refactor(source/istio): add transformers #5728
• issue Provide an option for configure multiple workers to apply DNS records changes #5458

More

• Yes, this PR title follows Conventional Commits
• Yes, I added unit tests
• Yes, I updated end user documentation accordingly

[mloiseleur]

Suggested change

	This is a common setup in modern cloud environments, like on GCP's Service Mesh, where a Kubernetes Ingress directs all public traffic to the Istio Gateway.
	Is used to address a specific architectural pattern:
	It can be used to address a specific architectural pattern, when a Kubernetes Ingress directs all public traffic to the Istio Gateway:

[stevehipwell]

@ivankatliarchuk the Helm test changes look good, if you want to add them to a seperate PR I'm happy to approve them and reduce the size of this PR?

RE the duplicate calls were you thinking of adding something like a singleflight wrapper?

[ivankatliarchuk]

Moved helm tests to a separate PR

[mloiseleur]

It looks like a new annotation to me, a new feature, not directly related a "replace kube API calls with caching and ingress informers".

Wdyt of moving the code and documentation of this new feature into a separate PR ?

[ivankatliarchuk]

Will do. This is not a new annotations, it was added here #3842

[ivankatliarchuk]

moved out documentation

[mloiseleur]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mloiseleur

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mloiseleur]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment