feat: per-listener dedicated Service type for external listeners

[david-yu]

Summary

Adds opt-in support for per-listener Service type, enabling external listeners to get their own dedicated Service per broker instead of sharing the default one. This aligns with Strimzi's listener model where each listener is an independent access path, while maintaining full backwards compatibility.

Problem

Currently, all external listeners share a single LoadBalancer Service per broker (lb-redpanda-0, etc.) with global annotations from external.annotations. This makes it impossible to have:

• A private listener on an internal LB (e.g., service.beta.kubernetes.io/aws-load-balancer-internal: "true")
• A public listener on an internet-facing LB (e.g., service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing)

...because both listeners share the same Service and therefore the same annotations.

Solution

Three new optional fields on each external listener definition:

Field	Type	Description
type	LoadBalancer or NodePort	When set, the listener gets a dedicated per-broker Service of this type
annotations	map[string]string	Annotations on the dedicated Service (only takes effect when type is set)
loadBalancerSourceRanges	[]string	Source CIDR restrictions (only takes effect when type is LoadBalancer)

Behavior

• No type set (default): Listener ports are included in the shared lb-<podName> Service — existing behavior, fully backwards compatible
• type set: Listener ports move to a dedicated lb-<listenerName>-<podName> Service of the specified type. Those ports are excluded from the shared LB
• If ALL listeners have type set, the shared LB is not created (no empty Services)
• type is the explicit opt-in — annotations alone does not trigger a dedicated Service

Design alignment with Strimzi

Strimzi treats each listener as a fully independent access path with its own type (LoadBalancer, NodePort, Ingress, etc.), and each listener always gets its own Services. Our design follows the same principle — the type field on a listener declares it as independent — but defaults to sharing when type is not set, preserving backwards compatibility.

---

Examples

All examples below use the Redpanda CRD (cluster.redpanda.com/v1alpha2). The same configuration works with Helm-only installs by placing the clusterSpec contents directly in values.yaml.

Previous behavior: Single LB per broker (unchanged, still the default)

This is how external listeners work today and continues to work unchanged:

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  clusterSpec:
    external:
      enabled: true
      type: LoadBalancer
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: nlb
        service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    listeners:
      kafka:
        port: 9092
        external:
          default:
            enabled: true
            port: 9094
            advertisedPorts: [9094]
      admin:
        port: 9644
        external:
          default:
            enabled: true
            port: 9645
            advertisedPorts: [9645]
    statefulset:
      replicas: 3

Result: One LB Service per broker, all listeners combined:

Service	Annotations	Ports
lb-redpanda-0	aws-load-balancer-internal: "true"	kafka-default:9094, admin-default:9645
lb-redpanda-1	aws-load-balancer-internal: "true"	kafka-default:9094, admin-default:9645
lb-redpanda-2	aws-load-balancer-internal: "true"	kafka-default:9094, admin-default:9645

New: Dedicated LB per listener (opt-in via per-listener type)

Private listener on an internal LB, public listener on an internet-facing LB:

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  clusterSpec:
    external:
      enabled: true
      type: LoadBalancer
      annotations:
        # Global annotations apply to the shared LB (used by listeners without their own type)
        service.beta.kubernetes.io/aws-load-balancer-type: nlb
        service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    listeners:
      kafka:
        port: 9092
        external:
          private:
            enabled: true
            port: 9094
            advertisedPorts: [9094]
            # No type → uses shared LB with global annotations (private/internal)
          public:
            enabled: true
            port: 9095
            advertisedPorts: [9095]
            # Per-listener type → gets its own dedicated Service
            type: LoadBalancer
            annotations:
              service.beta.kubernetes.io/aws-load-balancer-type: nlb
              service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
            loadBalancerSourceRanges:
              - 203.0.113.0/24   # restrict public access to known CIDRs
      admin:
        port: 9644
        external:
          private:
            enabled: true
            port: 9645
            advertisedPorts: [9645]
            # No type → shares the private LB
    statefulset:
      replicas: 3

Result: Two LB Services per broker — private (shared) and public (dedicated):

Service	Type	Annotations	Ports	Source Ranges
lb-redpanda-0	LoadBalancer	aws-load-balancer-internal: "true"	kafka-private:9094, admin-private:9645	(global)
lb-public-redpanda-0	LoadBalancer	aws-load-balancer-scheme: internet-facing	kafka-public:9095	203.0.113.0/24
lb-redpanda-1	LoadBalancer	aws-load-balancer-internal: "true"	kafka-private:9094, admin-private:9645	(global)
lb-public-redpanda-1	LoadBalancer	aws-load-balancer-scheme: internet-facing	kafka-public:9095	203.0.113.0/24
lb-redpanda-2	LoadBalancer	aws-load-balancer-internal: "true"	kafka-private:9094, admin-private:9645	(global)
lb-public-redpanda-2	LoadBalancer	aws-load-balancer-scheme: internet-facing	kafka-public:9095	203.0.113.0/24

New: All listeners dedicated (no shared LB)

When every listener has its own type, no shared LB is created:

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  clusterSpec:
    external:
      enabled: true
      type: LoadBalancer
    listeners:
      kafka:
        port: 9092
        external:
          private:
            enabled: true
            port: 9094
            advertisedPorts: [9094]
            type: LoadBalancer
            annotations:
              service.beta.kubernetes.io/aws-load-balancer-internal: "true"
              service.beta.kubernetes.io/aws-load-balancer-type: nlb
          public:
            enabled: true
            port: 9095
            advertisedPorts: [9095]
            type: LoadBalancer
            annotations:
              service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
              service.beta.kubernetes.io/aws-load-balancer-type: nlb
    statefulset:
      replicas: 3

Result: Two dedicated LBs per broker, no shared LB:

Service	Annotations	Ports
lb-private-redpanda-0	aws-load-balancer-internal: "true"	kafka-private:9094
lb-public-redpanda-0	aws-load-balancer-scheme: internet-facing	kafka-public:9095
lb-private-redpanda-1	aws-load-balancer-internal: "true"	kafka-private:9094
lb-public-redpanda-1	aws-load-balancer-scheme: internet-facing	kafka-public:9095
lb-private-redpanda-2	aws-load-balancer-internal: "true"	kafka-private:9094
lb-public-redpanda-2	aws-load-balancer-scheme: internet-facing	kafka-public:9095

GCP example: Internal and external LoadBalancers

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  clusterSpec:
    external:
      enabled: true
      type: LoadBalancer
      annotations:
        networking.gke.io/load-balancer-type: Internal
        cloud.google.com/l4-rbs: enabled
    listeners:
      kafka:
        port: 9092
        external:
          private:
            enabled: true
            port: 9094
            advertisedPorts: [9094]
            prefixTemplate: "broker-private-{{ .Index }}"
          public:
            enabled: true
            port: 9095
            advertisedPorts: [9095]
            prefixTemplate: "broker-public-{{ .Index }}"
            type: LoadBalancer
            annotations:
              # No Internal annotation — GKE creates an external LB by default
              cloud.google.com/l4-rbs: enabled
    statefulset:
      replicas: 3

Azure example: Internal and public LoadBalancers

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  clusterSpec:
    external:
      enabled: true
      type: LoadBalancer
      annotations:
        service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    listeners:
      kafka:
        port: 9092
        external:
          private:
            enabled: true
            port: 9094
            advertisedPorts: [9094]
            prefixTemplate: "broker-private-{{ .Index }}"
          public:
            enabled: true
            port: 9095
            advertisedPorts: [9095]
            prefixTemplate: "broker-public-{{ .Index }}"
            type: LoadBalancer
            annotations:
              # No internal annotation — Azure creates a public LB by default
              service.beta.kubernetes.io/azure-load-balancer-resource-group: my-rg
    statefulset:
      replicas: 3

Result: Per broker, one internal LB (shared, from global annotations) and one public LB (dedicated):

Service	Annotations	Ports
lb-redpanda-0	azure-load-balancer-internal: "true"	kafka-private:9094
lb-public-redpanda-0	azure-load-balancer-resource-group: my-rg	kafka-public:9095
lb-redpanda-1	azure-load-balancer-internal: "true"	kafka-private:9094
lb-public-redpanda-1	azure-load-balancer-resource-group: my-rg	kafka-public:9095
lb-redpanda-2	azure-load-balancer-internal: "true"	kafka-private:9094
lb-public-redpanda-2	azure-load-balancer-resource-group: my-rg	kafka-public:9095

Mixed Service types: LB + NodePort

The type field also enables mixed Service types — one listener as LoadBalancer, another as NodePort:

listeners:
  kafka:
    external:
      lb-listener:
        port: 9094
        # No type → shares default LoadBalancer
      nodeport-listener:
        port: 9095
        type: NodePort
        nodePort: 30095

---

Helm-only usage

The same configuration works without the Redpanda CRD by placing it directly in values.yaml:

# values.yaml
external:
  enabled: true
  type: LoadBalancer
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"

listeners:
  kafka:
    port: 9092
    external:
      private:
        enabled: true
        port: 9094
        advertisedPorts: [9094]
      public:
        enabled: true
        port: 9095
        advertisedPorts: [9095]
        type: LoadBalancer
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing

helm install redpanda redpanda/redpanda -f values.yaml

---

Changes

File	Description
charts/redpanda/values.go	Added Type, Annotations, LoadBalancerSourceRanges to ExternalListener; added HasDedicatedService(), ServicePortsForListener, ServicePortsExcludingListeners methods
charts/redpanda/service.loadbalancer.go	Split LB creation based on Type: shared LB excludes dedicated listeners, new dedicated Service per listener name per broker with per-listener type
operator/api/redpanda/v1alpha2/redpanda_clusterspec_types.go	Added Type, Annotations, LoadBalancerSourceRanges to CRD ExternalListener
charts/redpanda/chart/templates/_service.loadbalancer.go.tpl	Regenerated gotohelm output
charts/redpanda/chart/templates/_values.go.tpl	Regenerated
charts/redpanda/chart/values.schema.json	Updated schema with new fields
charts/redpanda/values_partial.gen.go	Regenerated

Test plan

• Existing chart template tests pass (no behavior change when type not set)
• Add test case with per-listener type verifying dedicated Service creation
• Verify shared LB excludes dedicated listener ports
• Verify dedicated Service has correct type, annotations, and source ranges
• Verify no shared LB created when all listeners have type set
• Verify TestHelmValuesCompat passes (CRD↔Helm round-trip)
• Helm lint passes

🤖 Generated with Claude Code

[github-actions]

This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This PR is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days.