Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -203,3 +203,12 @@ unit-test:

e2e-test:
./scripts/ci_e2e_test.sh

# generate gateway api CRD spec doc
.PHONY: gw-api-ref-docs
gw-api-ref-docs:
crd-ref-docs \
--source-path=${PWD}/apis/gateway/ \
--config=crd-ref-docs.yaml \
--renderer=markdown \
--output-path=${PWD}/docs/guide/gateway/spec.md
14 changes: 14 additions & 0 deletions crd-ref-docs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# This file contains configuration for our reference docs generation. For more
# information about the possible configuration, refer to
# https://github.com/elastic/crd-ref-docs.

processor:
ignoreTypes:
- "()List$"
# RE2 regular expressions describing type fields that should be excluded from the generated documentation.
ignoreFields:
- "TypeMeta$"

render:
# Version of Kubernetes to use when generating links to Kubernetes API documentation.
kubernetesVersion: 1.32
146 changes: 146 additions & 0 deletions docs/guide/gateway/customization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
## Customizing your ELB resources

The AWS Load Balancer Controller (LBC) provides sensible defaults for provisioning and managing Elastic Load Balancing (ELB) resources in response to Kubernetes Gateway API objects. However, to accommodate diverse use cases and specific operational requirements, the LBC offers extensive, fine-grained customization capabilities through two Custom Resource Definitions (CRDs): [LoadBalancerConfiguration](../spec/#loadbalancerconfiguration) and [TargetGroupConfiguration](../spec/#targetgroupconfiguration).

![screen showing all LBC Gateway API components](assets/gateway-full.png)

### Customizing the Gateway (Load Balancer) using `LoadBalancerConfiguration` CRD

The `LoadBalancerConfiguration` CRD allows for the detailed customization of the AWS Load Balancer (ALB or NLB) provisioned by the LBC for a given Gateway.

For a comprehensive list of configurable parameters, please refer to the [LoadBalancerConfiguration CRD documentation](./loadbalancerconfig.md).

**Example:** To configure your Gateway to provision an `internet-facing` Load Balancer, define the following `LoadBalancerConfiguration` resource:

```yaml
apiVersion: gateway.k8s.aws/v1beta1
kind: LoadBalancerConfiguration
metadata:
name: internet-facing-config
namespace: example-ns
spec:
scheme: internet-facing
```
This configuration can then be applied by attaching the `LoadBalancerConfiguration` resource to either a `Gateway` or a `GatewayClass`.

**Attaching to a Gateway:**
When attached directly to a `Gateway` resource, the specified configuration applies specifically to the Load Balancer provisioned for that individual Gateway.

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: test-gw
namespace: example-ns
spec:
gatewayClassName: nlb-gateway
infrastructure:
parametersRef:
group: gateway.k8s.aws
kind: LoadBalancerConfiguration
name: internet-facing-config # Must be in the same namespace as the Gateway
listeners:
...
```

**Attaching to a GatewayClass:**
When attached to a `GatewayClass` resource, the configuration becomes a default for all `Gateway` resources that reference this `GatewayClass`.

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: nlb-gateway
spec:
controllerName: gateway.k8s.aws/alb
parametersRef:
group: gateway.k8s.aws
kind: LoadBalancerConfiguration
name: internet-facing-config
namespace: example-ns
```

#### Conflict Resolution for `LoadBalancerConfiguration`

It is possible for a `LoadBalancerConfiguration` to be attached to both a `Gateway` and its associated `GatewayClass`. In such scenarios, when identical fields are specified in both configurations, the LBC employs a merging algorithm to resolve conflicts. The precedence of values is determined by the `mergingMode` field, which is exclusively read from the `GatewayClass`'s `LoadBalancerConfiguration`. If `mergingMode` is not explicitly set, the `GatewayClass` configuration implicitly takes higher precedence.

The following fields exhibit specific merge behaviors:

* **`tags`**: The tag maps from both configurations are combined. In the event of duplicate tag keys, the value from the higher-priority configuration (as determined by `mergingMode`) will be utilized.
* **`loadBalancerAttributes`**: The attribute lists are combined. For duplicate attribute keys, the value from the higher-priority configuration will prevail.
* **`mergeListenerConfig`**: Listener lists are combined. For duplicate `ProtocolPort` keys, the listener configuration from the higher-priority source will be selected.

-----

### Customizing Services (Target Groups) using `TargetGroupConfiguration` CRD

The `TargetGroupConfiguration` CRD enables granular customization of the AWS Target Groups created for Kubernetes Services.

For a comprehensive overview of configurable parameters, please refer the [TargetGroupConfiguration CRD documentation](./targetgroupconfig.md).

**Example: Default Target Group Configuration for a Service**

To configure the target groups for a specific service (e.g., `my-service`) to use `IP` mode and custom health checks across all routes referencing it, employ the following configuration:

```yaml
apiVersion: gateway.k8s.aws/v1beta1
kind: TargetGroupConfiguration
metadata:
name: custom-tg-config
namespace: example-namespace
spec:
targetReference:
name: my-service
defaultConfiguration:
targetType: ip
healthCheckConfig:
healthCheckPath: /health
healthCheckInterval: 30
healthyThresholdCount: 3
```

Here, `my-service` is referenced within the `targetReference` of `custom-tg-config`. Any target group subsequently created for `my-service` via any route will inherit these default settings. Note that only one `TargetGroupConfiguration` CRD can be declared per service, and it must reside within the same namespace as the service it configures.

**Example: Route-Specific Target Group Configuration**

Alternatively, specific target group settings can be applied based on the individual routes referencing a service. This allows for tailored configurations for different traffic flows.

```yaml
apiVersion: gateway.k8s.aws/v1beta1
kind: TargetGroupConfiguration
metadata:
name: route-specific-tg-config
namespace: example-ns
spec:
targetReference:
name: my-service
defaultConfiguration:
targetType: ip
routeConfigurations:
- routeIdentifier:
kind: TCPRoute
namespace: example-ns
name: api-route
targetGroupProps:
healthCheckConfig:
healthCheckPath: /api/health
healthCheckProtocol: HTTP
- routeIdentifier:
kind: TCPRoute
namespace: example-ns-2
name: admin-route
targetGroupProps:
healthCheckConfig:
healthCheckPath: /admin/health
healthCheckInterval: 10
```

#### How Default and Route-Specific Configurations Merge

When both `defaultConfiguration` and `routeConfigurations` within a `TargetGroupConfiguration` specify the same field, route-specific configurations take precedence. The controller identifies the most relevant route specification from the list of `routeConfigurations` and merges its `targetGroupProps` with the `defaultConfiguration`'s settings. For detailed information on the route matching logic employed, refer to the [Route Matching section](../targetgroupconfig/#route-matching-logic).

The following fields exhibit specific merge behaviors:

* **`tags`**: The two tag maps are combined. Any duplicate tag keys will result in the value from the higher-priority (route-specific) configuration being used.
* **`targetGroupAttributes`**: The two attribute lists are combined. Any duplicate attribute keys will result in the attribute value from the higher-priority (route-specific) configuration being applied.
117 changes: 21 additions & 96 deletions docs/guide/gateway/gateway.md
Original file line number Diff line number Diff line change
@@ -1,120 +1,45 @@
# Gateway API

!!! warning
- Only very basic (and not conforming) support of the Gateway API spec currently exists. The team is actively trying to close conformance and support gaps.
- Using the LBC and Gateway API together is not suggested for production workloads (yet!)
!!! warning
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there anyway to make those warning more obvious, like in a red box?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image This will look this. Its a standard format of displaying the warnings.

- Only very basic (and not conforming) support of the Gateway API spec currently exists. The team is actively trying to close conformance and support gaps.
- Using the LBC and Gateway API together is not suggested for production workloads (yet!)


The AWS Load Balancer Controller (LBC) supports reconciliation for Kubernetes Gateway API objects. It satisfies
L4 routes (TCPRoute, UDPRoute, TLSRoute) with an AWS NLB. It satisfies L7 routes (HTTPRoute, GRPCRoute) using an AWS ALB.
Mixing protocol layers, e.g. TCPRoute and HTTPRoute on the same Gateway, is not supported.


## Current Support

!!! warning
- GRPCRoute and HTTPS Listeners for L7 gateways do not currently work. And only basic support is added for HTTPRoute.

The LBC Gateway API implementation supports the following Gateway API routes:

* L4 (NLBGatewayApi): UDPRoute, TCPRoute, TLSRoute >=v2.13.3
* L7 (ALBGatewayApi): HTTPRoute

## Prerequisites
* LBC >= v2.13.0
* For `ip` target type:
* Pods have native AWS VPC networking configured. For more information, see the [Amazon VPC CNI plugin](https://github.com/aws/amazon-vpc-cni-k8s#readme) documentation.
* Installation of Gateway API CRDs
* Standard Gateway API CRDs: `kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.0/standard-install.yaml` [REQUIRED]
* Experimental Gateway API CRDs: `kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.0/experimental-install.yaml` [OPTIONAL: Used for L4 Routes]
* Installation of LBC Gateway API CRDs:
* `kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/refs/heads/main/config/crd/gateway/gateway-crds.yaml`
* Standard Gateway API CRDs: `kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.0/standard-install.yaml` [REQUIRED]
* Experimental Gateway API CRDs: `kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.0/experimental-install.yaml` [OPTIONAL: Used for L4 Routes]
* Installation of LBC Gateway API specific CRDs: `kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/refs/heads/main/config/crd/gateway/gateway-crds.yaml`

## Configuration

By default, the LBC will _not_ listen to Gateway API CRDs. To enable support, specify the following feature flag(s) in the LBC deployment:

```
- --feature-gates=NLBGatewayAPI=true,ALBGatewayAPI=true
```

### Customization

![screen showing all LBC Gateway API components](assets/gateway-full.png)

The LBC tries to use sensible defaults. These defaults are not appropriate for each use-case.

#### Customizing the Gateway

For full details on what you can configure:
[LoadBalancerConfiguration CRD](./loadbalancerconfig.md)

For example, to configure your Gateway to provision an `internet-facing` LoadBalancer you can use this configuration:

```
apiVersion: gateway.k8s.aws/v1beta1
kind: LoadBalancerConfiguration
metadata:
name: internet-facing-config
namespace: echoserver
spec:
scheme: internet-facing
```

You can then attach your configuration to either the Gateway or GatewayClass
* `NLBGatewayAPI`: For enabling L4 Routing
* `ALBGatewayAPI`: For enabling L7 Routing

Gateway
```
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: test-gw
namespace: echoserver
spec:
gatewayClassName: nlb-gateway
infrastructure:
parametersRef:
group: gateway.k8s.aws
kind: LoadBalancerConfiguration
name: internet-facing-config # Must be in the same namespace as the Gateway
listeners:
...
```

GatewayClass
```
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: nlb-gateway
spec:
controllerName: gateway.k8s.aws/alb
parametersRef:
group: gateway.k8s.aws
kind: LoadBalancerConfiguration
name: internet-facing-config
namespace: echoserver
- --feature-gates=NLBGatewayAPI=true,ALBGatewayAPI=true
```


#### Configuration Conflict Resolution

It is possible to attach a LoadBalancerConfiguration to both the Gateway and GatewayClass resources. When both the
GatewayClass and Gateway LoadBalancerConfiguration specify the same field, the LBC uses a merging algorithm to take one value.
Priority of what value to take is determined by the `mergingMode` field, this field is used to give priority to either the
GatewayClass or Gateway value. When the value is not set, the GatewayClass value takes higher precedence. The `mergingMode` field
is only read from the GatewayClass LoadBalancerConfiguration.

The following fields have differing merge behavior:

- `tags`: The two tags maps will be combined. Any duplicate tag keys will use the tag value from the higher priority config.
- `loadBalancerAttributes`: The two attribute lists will be combined. Any duplicate attribute keys will use the attribute value from the higher priority config.
- `mergeListenerConfig`: The two listener lists will be combined. Any duplicate ProtocolPort keys will use the listener config from the higher priority config.

#### Customizing the Targets

[Not currently supported]


## Protocols

!!! warning
- TLSRoute, GRPCRoute and HTTPS Listeners do not currently work.

The LBC Gateway API implementation supports all Gateway API routes:

L4 (NLB): UDPRoute, TCPRoute, TLSRoute
L7 (ALB): HTTPRoute, GRPCRoute


## Subnet tagging requirements
See [Subnet Discovery](../../deploy/subnet_discovery.md) for details on configuring Elastic Load Balancing for public or private placement.

Expand Down
Loading
Loading