AWS Lambda runtime TypeError: Header 'cf-ipcity' fails validation due to non-ASCII characters (ñ, ó) from Cloudflare

[aberigle]

What version of Bun is running?

1.21

What platform is your computer?

No response

What steps can reproduce the bug?

1. Set up an AWS Lambda function using the Bun runtime
2. Configure a domain to route traffic through Cloudflare Proxy (orange cloud) to the API Gateway endpoint that triggers the Lambda function.
3. Have a request from a geographical location whose city name includes a special character (e.g., Torrejón de Ardoz, Logroño, Córdoba).
4. Cloudflare adds the HTTP header cf-ipcity containing the city name (e.g., cf-ipcity: Torrejón de Ardoz).
5. The Bun runtime initialization code fails when validating or decoding this header value.

The issue is a TypeError raised during the initial handling of the HTTP request before the user's function handler is executed. The error suggests that the Bun runtime's internal mechanism for parsing or validating HTTP headers, specifically the cf-ipcity header injected by Cloudflare, cannot handle the multibyte or non-ASCII characters present in names like 'Torrejón de Ardoz'.

The failure occurs in the code responsible for formatting the event, specifically:

315 | function formatHttpEventV1(event: HttpEventV1): Request {
316 |   const request = event.requestContext;
317 |   const headers = new Headers();
318 |   for (const [name, values] of Object.entries(event.multiValueHeaders ?? {})) {
319 |     for (const value of values) {
320 |       headers.append(name, value);
                    ^
TypeError: Header 'cf-ipcity' has invalid value: 'Torrejón de Ardoz'
      at formatHttpEventV1 (/opt/runtime.ts:320:15)
      at formatRequest (/opt/runtime.ts:459:15)
      at #acceptRequest (/opt/runtime.ts:521:21)
      at #acceptRequest (/opt/runtime.ts:520:24)
      at accept (/opt/runtime.ts:507:14)
      at accept (/opt/runtime.ts:500:16)
      at /opt/runtime.ts:812:35

This prevents the Lambda function from executing any user code. The expected behavior is that the runtime should correctly decode or ignore non-ASCII characters in standard HTTP headers passed from a reverse proxy.

What is the expected behavior?

The Bun runtime should successfully decode or tolerate HTTP header values containing non-ASCII characters (like ñ, ó, á, etc.) that are commonly passed by CDNs and proxies (such as Cloudflare's cf-ipcity), allowing the Lambda function to execute normally.

What do you see instead?

A TypeError that crashes the Lambda instance before the user's code is invoked:

315 | function formatHttpEventV1(event: HttpEventV1): Request {
316 |   const request = event.requestContext;
317 |   const headers = new Headers();
318 |   for (const [name, values] of Object.entries(event.multiValueHeaders ?? {})) {
319 |     for (const value of values) {
320 |       headers.append(name, value);
                    ^
TypeError: Header 'cf-ipcity' has invalid value: 'Torrejón de Ardoz'
      at formatHttpEventV1 (/opt/runtime.ts:320:15)
      at formatRequest (/opt/runtime.ts:459:15)
      at #acceptRequest (/opt/runtime.ts:521:21)
      at #acceptRequest (/opt/runtime.ts:520:24)
      at accept (/opt/runtime.ts:507:14)
      at accept (/opt/runtime.ts:500:16)
      at /opt/runtime.ts:812:35

Additional information

No response

[linear]

ENG-20643 AWS Lambda runtime TypeError: Header 'cf-ipcity' fails validation due to non-ASCII characters (ñ, ó) from Cloudflare

[aberigle]

I've seen that the default behavior in bun is to remove the non-ASCII characters from the header, but maybe we could encode them to avoid losing information?

I was thinking something like this:

function encodeNonAscii(value : string="") {
  return value.replace(/[^\u0000-\u007f]/g, encodeURIComponent)
}

But I'm unsure if adopting this approach would violate any HTTP standards or introduce any unexpected pitfalls related to header parsing on the receiving end.