feat: implement per-route timeout configuration

Add three user-facing timeouts configurable via InterceptorRoute CRD:
request (disabled by default), responseHeader (300s default), and
readiness (disabled by default, 30s when fallback is configured).
Timeout errors now return 504 Gateway Timeout instead of 502 Bad
Gateway.

Changes:
- Add timeouts spec to InterceptorRoute CRD and update CRD manifest
- Refactor timeout config to derive internal timeouts from three user-facing ones
- Propagate per-route timeouts through routing table and middleware
- Simplify dial context to single-arg DialContextWithRetry
- Cap dial retries at 1 minute when request timeout is disabled
- Fix fallback not being reached when readiness timeout >= request timeout
- Default readiness timeout to 30s when a fallback service is configured
- Remove legacy per-transport env vars (breaking), deprecate old naming
- Add unit tests for timeout config, routing, and upstream handler
- Expand e2e timeout tests for the new per-route behavior
- Delete unused pkg/k8s/annotations.go

Signed-off-by: Vincent Link <[email protected]>

Author: linkvt

CHANGELOG.md

@@ -27,11 +27,14 @@ This changelog keeps track of work items that have been completed and are ready
 
 ### Breaking Changes
 
+- **Interceptor**: Change default timeout behavior: request timeout (`KEDA_HTTP_REQUEST_TIMEOUT`) defaults to `0` (disabled), response header timeout (`KEDA_RESPONSE_HEADER_TIMEOUT` → `KEDA_HTTP_RESPONSE_HEADER_TIMEOUT`) defaults to `300s` (was `500ms`), and readiness timeout (`KEDA_CONDITION_WAIT_TIMEOUT` → `KEDA_HTTP_READINESS_TIMEOUT`) defaults to `0` (disabled, was `20s`). Timeout errors return 504 instead of 502. ([#1474](https://github.com/kedacore/http-add-on/issues/1474))
 - **Interceptor**: Redesign interceptor metrics: `interceptor_request_count` → `interceptor_requests_total` (labels: `method`, `code`, `route_name`, `route_namespace`), `interceptor_pending_request_count` → `interceptor_pending_requests` (labels: `route_name`, `route_namespace`), added `interceptor_request_duration_seconds` histogram; `path` and `host` labels removed in favor of route identity via InterceptorRoute name/namespace to fix unbounded cardinality OOM issues; non-standard HTTP methods normalized to `_OTHER`; dashboards and alerting rules must be updated ([#1559](https://github.com/kedacore/http-add-on/issues/1559))
+- **Interceptor**: Remove `KEDA_HTTP_TLS_HANDSHAKE_TIMEOUT`, `KEDA_HTTP_EXPECT_CONTINUE_TIMEOUT`, `KEDA_HTTP_KEEP_ALIVE`, `KEDA_HTTP_IDLE_CONN_TIMEOUT`, and `KEDA_HTTP_DIAL_RETRY_TIMEOUT` environment variables; these now use Go's `DefaultTransport` defaults. ([#1474](https://github.com/kedacore/http-add-on/issues/1474))
 
 ### New
 
 - **General**: Add `InterceptorRoute` CRD to separate routing/interceptor config from scaling config; `HTTPScaledObject` remains supported but will be deprecated in a future release ([#1501](https://github.com/kedacore/http-add-on/issues/1501))
+- **Interceptor**: Add per-route timeout configuration via InterceptorRoute `timeouts` spec with `request`, `responseHeader`, and `readiness` fields. When unset, global env var defaults are used. When a fallback service is configured and no readiness timeout is set, it defaults to 30s. ([#1474](https://github.com/kedacore/http-add-on/issues/1474))
 
 ### Improvements
 
@@ -43,7 +46,7 @@ This changelog keeps track of work items that have been completed and are ready
 
 ### Deprecations
 
-- **General**: TODO ([#TODO](https://github.com/kedacore/http-add-on/issues/TODO))
+- **Interceptor**: Deprecate `KEDA_CONDITION_WAIT_TIMEOUT` and `KEDA_RESPONSE_HEADER_TIMEOUT` environment variables in favor of `KEDA_HTTP_READINESS_TIMEOUT` and `KEDA_HTTP_RESPONSE_HEADER_TIMEOUT`. Old vars take precedence when set and log deprecation warnings. ([#1474](https://github.com/kedacore/http-add-on/issues/1474))
 
 ### Other
 

config/crd/bases/http.keda.sh_interceptorroutes.yaml

@@ -208,6 +208,31 @@ spec:
                 x-kubernetes-validations:
                 - message: exactly one of 'port' or 'portName' must be set
                   rule: has(self.port) != has(self.portName)
+              timeouts:
+                description: Timeout configuration for request handling.
+                properties:
+                  readiness:
+                    description: |-
+                      Time to wait for the backend to become ready (e.g. scale-from-zero).
+                      Unset: uses the global KEDA_HTTP_READINESS_TIMEOUT (default: disabled).
+                      Set to "0s" to disable the dedicated readiness deadline so the full
+                      request budget is available for cold starts. When a fallback service
+                      is configured and this is "0s", a 30s default is applied.
+                    type: string
+                  request:
+                    description: |-
+                      Total time allowed for the entire request lifecycle.
+                      Unset: uses the global KEDA_HTTP_REQUEST_TIMEOUT (default: disabled).
+                      Set to "0s" to disable the request deadline.
+                    type: string
+                  responseHeader:
+                    description: |-
+                      Max time to wait for the response headers from the backend after the
+                      request has been fully sent. Does not include cold-start wait time.
+                      Unset: uses the global KEDA_HTTP_RESPONSE_HEADER_TIMEOUT (default: 300s).
+                      Set to "0s" to disable the response header deadline.
+                    type: string
+                type: object
             required:
             - scalingMetric
             - target

docs/integrations.md

@@ -40,20 +40,13 @@ spec:
 
 1. **Error: `context marked done while waiting for workload reach > 0 replicas`**
 
-   - This error indicates that the `KEDA_CONDITION_WAIT_TIMEOUT` value (default: 20 seconds) might be too low. The workload scaling process may not be complete within this timeframe.
-   - To increase the timeout:
-     - If using Helm, adjust the `interceptor.replicas.waitTimeout` parameter (see reference below).
-     - Reference: [https://github.com/kedacore/charts/blob/main/http-add-on/values.yaml#L139](https://github.com/kedacore/charts/blob/main/http-add-on/values.yaml#L139)
-
-2. **502 Errors with POST Requests:**
-
-   - You might encounter 502 errors during POST requests when the request is routed through the interceptor service. This could be due to insufficient timeout settings.
-   - To adjust timeout parameters:
-     - If using Helm, modify the following parameters (see reference below):
-       - `KEDA_HTTP_CONNECT_TIMEOUT`
-       - `KEDA_RESPONSE_HEADER_TIMEOUT`
-       - `KEDA_HTTP_EXPECT_CONTINUE_TIMEOUT`
-     - Reference: [https://github.com/kedacore/charts/blob/main/http-add-on/values.yaml#L152](https://github.com/kedacore/charts/blob/main/http-add-on/values.yaml#L152)
+   - This error indicates that the readiness timeout might be too low. By default the readiness timeout is disabled (bounded only by the `request` timeout); when a fallback service is configured and no readiness timeout is set, it defaults to 30s.
+   - Set the `readiness` timeout in the InterceptorRoute `timeouts` spec or `KEDA_HTTP_READINESS_TIMEOUT` to a value that gives the workload enough time to scale up.
+
+2. **502/504 Errors with POST Requests:**
+
+   - You might encounter timeout errors during POST requests when the request is routed through the interceptor service. This could be due to insufficient timeout settings.
+   - Increase `KEDA_HTTP_RESPONSE_HEADER_TIMEOUT` or set the `responseHeader` timeout in the InterceptorRoute `timeouts` spec.
 
 3. **Immediate Scaling Down to Zero:**
    - If `minReplica` is set to 0 in the HTTPScaledObject, the application will immediately scale down to 0.

interceptor/config/timeouts.go

@@ -4,45 +4,66 @@ import (
 	"time"
 
 	"github.com/caarlos0/env/v11"
+	"github.com/go-logr/logr"
 )
 
-// Timeouts is the configuration for connection and HTTP timeouts
+// Timeouts is the configuration for request handling and connection timeouts.
 type Timeouts struct {
-	// Connect is the per-attempt TCP dial timeout (net.Dialer.Timeout)
-	Connect time.Duration `env:"KEDA_HTTP_CONNECT_TIMEOUT" envDefault:"500ms"`
-	// KeepAlive is the interval between keepalive probes
-	KeepAlive time.Duration `env:"KEDA_HTTP_KEEP_ALIVE" envDefault:"1s"`
-	// ResponseHeaderTimeout is how long to wait between when the HTTP request
-	// is sent to the backing app and when response headers need to arrive
-	ResponseHeader time.Duration `env:"KEDA_RESPONSE_HEADER_TIMEOUT" envDefault:"500ms"`
-	// WorkloadReplicas is how long to wait for the backing workload
+	// Request is the total wall-clock deadline from request arrival to response completion.
+	// When 0 (the default), there is no total request deadline.
+	Request time.Duration `env:"KEDA_HTTP_REQUEST_TIMEOUT" envDefault:"0s"`
+	// ResponseHeader is how long to wait between when the HTTP request
+	// is sent to the backing app and when response headers need to arrive.
+	// Defaults to 300s as a safety net against hung backends. Set to 0 to disable.
+	ResponseHeader time.Duration `env:"KEDA_HTTP_RESPONSE_HEADER_TIMEOUT" envDefault:"300s"`
+	// Readiness is how long to wait for the backing workload
 	// to have 1 or more replicas before connecting and sending the HTTP request.
-	WorkloadReplicas time.Duration `env:"KEDA_CONDITION_WAIT_TIMEOUT" envDefault:"20s"`
-	// ForceHTTP2 toggles whether to try to force HTTP2 for all requests
-	ForceHTTP2 bool `env:"KEDA_HTTP_FORCE_HTTP2" envDefault:"false"`
+	// When 0 (the default), the readiness wait is bounded only by the request
+	// timeout, giving the full request budget to cold starts. When a fallback
+	// service is configured and this is 0, a 30s default is applied.
+	Readiness time.Duration `env:"KEDA_HTTP_READINESS_TIMEOUT" envDefault:"0s"`
+	// Connect is the per-attempt TCP dial timeout (net.Dialer.Timeout).
+	// Bounded by the request context deadline.
+	Connect time.Duration `env:"KEDA_HTTP_CONNECT_TIMEOUT" envDefault:"500ms"`
+
 	// MaxIdleConns is the max number of idle connections to keep in the
 	// interceptor's internal connection pool across all backend services.
 	// Increase this if you proxy to many unique backend services.
 	MaxIdleConns int `env:"KEDA_HTTP_MAX_IDLE_CONNS" envDefault:"1000"`
 	// MaxIdleConnsPerHost is the max number of idle connections to keep per backend service.
 	// Increase this if you observe many new connection establishments under load.
 	MaxIdleConnsPerHost int `env:"KEDA_HTTP_MAX_IDLE_CONNS_PER_HOST" envDefault:"200"`
-	// IdleConnTimeout is the timeout after which a connection in the interceptor's
-	// internal connection pool will be closed
-	IdleConnTimeout time.Duration `env:"KEDA_HTTP_IDLE_CONN_TIMEOUT" envDefault:"90s"`
-	// TLSHandshakeTimeout is the max amount of time the interceptor will
-	// wait to establish a TLS connection
-	TLSHandshakeTimeout time.Duration `env:"KEDA_HTTP_TLS_HANDSHAKE_TIMEOUT" envDefault:"10s"`
-	// ExpectContinueTimeout is the max amount of time the interceptor will wait
-	// for a 100 Continue response from the backend after sending request headers
-	// with Expect: 100-continue
-	ExpectContinueTimeout time.Duration `env:"KEDA_HTTP_EXPECT_CONTINUE_TIMEOUT" envDefault:"1s"`
-	// DialRetryTimeout caps the total time spent retrying failed dial attempts.
-	DialRetryTimeout time.Duration `env:"KEDA_HTTP_DIAL_RETRY_TIMEOUT" envDefault:"15s"`
+	// ForceHTTP2 toggles whether to try to force HTTP2 for all requests.
+	ForceHTTP2 bool `env:"KEDA_HTTP_FORCE_HTTP2" envDefault:"false"`
+}
+
+// deprecatedTimeouts holds deprecated env vars that take precedence when set.
+type deprecatedTimeouts struct {
+	// ResponseHeader is how long to wait between when the HTTP request
+	// is sent to the backing app and when response headers need to arrive.
+	ResponseHeader time.Duration `env:"KEDA_RESPONSE_HEADER_TIMEOUT"`
+	// WorkloadReplicas is how long to wait for the backing workload
+	// to have 1 or more replicas before connecting and sending the HTTP request.
+	WorkloadReplicas time.Duration `env:"KEDA_CONDITION_WAIT_TIMEOUT"`
 }
 
-// MustParseTimeouts parses standard configs and returns the
-// newly created config. It panics if parsing fails.
-func MustParseTimeouts() Timeouts {
-	return env.Must(env.ParseAs[Timeouts]())
+// MustParseTimeouts parses timeout configuration from environment variables.
+// Deprecated env vars take precedence over new ones when set, to preserve
+// existing behavior for users who haven't migrated yet.
+func MustParseTimeouts(log logr.Logger) Timeouts {
+	cfg := env.Must(env.ParseAs[Timeouts]())
+
+	deprecated := env.Must(env.ParseAs[deprecatedTimeouts]())
+
+	if deprecated.WorkloadReplicas > 0 {
+		log.Info("WARNING: KEDA_CONDITION_WAIT_TIMEOUT is deprecated, use KEDA_HTTP_READINESS_TIMEOUT instead")
+		cfg.Readiness = deprecated.WorkloadReplicas
+	}
+
+	if deprecated.ResponseHeader > 0 {
+		log.Info("WARNING: KEDA_RESPONSE_HEADER_TIMEOUT is deprecated, use KEDA_HTTP_RESPONSE_HEADER_TIMEOUT instead")
+		cfg.ResponseHeader = deprecated.ResponseHeader
+	}
+
+	return cfg
 }

interceptor/config/timeouts_test.go

@@ -0,0 +1,23 @@
+package config
+
+import (
+	"testing"
+	"time"
+
+	"github.com/go-logr/logr"
+)
+
+func TestMustParseTimeouts_DeprecatedOverride(t *testing.T) {
+	t.Setenv("KEDA_HTTP_READINESS_TIMEOUT", "25s")
+	t.Setenv("KEDA_CONDITION_WAIT_TIMEOUT", "31s")
+	t.Setenv("KEDA_RESPONSE_HEADER_TIMEOUT", "7s")
+
+	cfg := MustParseTimeouts(logr.Discard())
+
+	if got, want := cfg.Readiness, 31*time.Second; got != want {
+		t.Errorf("Readiness = %v, want %v (deprecated var should take precedence)", got, want)
+	}
+	if got, want := cfg.ResponseHeader, 7*time.Second; got != want {
+		t.Errorf("ResponseHeader = %v, want %v (deprecated var should take precedence)", got, want)
+	}
+}

interceptor/handler/upstream.go

@@ -1,7 +1,9 @@
 package handler
 
 import (
+	"context"
 	"errors"
+	"net"
 	"net/http"
 	"net/http/httputil"
 	"time"
@@ -12,7 +14,6 @@ import (
 
 	"github.com/kedacore/http-add-on/interceptor/config"
 	kedahttp "github.com/kedacore/http-add-on/pkg/http"
-	"github.com/kedacore/http-add-on/pkg/k8s"
 	"github.com/kedacore/http-add-on/pkg/util"
 )
 
@@ -23,16 +24,16 @@ var (
 )
 
 type Upstream struct {
-	transportPool     *kedahttp.TransportPool
-	tracingCfg        config.Tracing
-	respHeaderTimeout time.Duration
+	transportPool         *kedahttp.TransportPool
+	tracingCfg            config.Tracing
+	responseHeaderTimeout time.Duration
 }
 
-func NewUpstream(baseTransport *http.Transport, tracingCfg config.Tracing, respHeaderTimeout time.Duration) *Upstream {
+func NewUpstream(baseTransport *http.Transport, tracingCfg config.Tracing, responseHeaderTimeout time.Duration) *Upstream {
 	return &Upstream{
-		transportPool:     kedahttp.NewTransportPool(baseTransport),
-		tracingCfg:        tracingCfg,
-		respHeaderTimeout: respHeaderTimeout,
+		transportPool:         kedahttp.NewTransportPool(baseTransport),
+		tracingCfg:            tracingCfg,
+		responseHeaderTimeout: responseHeaderTimeout,
 	}
 }
 
@@ -59,16 +60,15 @@ func (uh *Upstream) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	respHeaderTimeout := uh.respHeaderTimeout
-	// TODO(v1): remove timeout compatibility fallback for HTTPSO before v1 release
+	// Select transport with per-route or global response header timeout.
+	responseHeaderTimeout := uh.responseHeaderTimeout
 	if ir := util.InterceptorRouteFromContext(ctx); ir != nil {
-		if v, ok := ir.Annotations[k8s.AnnotationResponseHeaderTimeout]; ok {
-			if d, err := time.ParseDuration(v); err == nil && d > 0 {
-				respHeaderTimeout = d
-			}
+		if ir.Spec.Timeouts.ResponseHeader != nil {
+			responseHeaderTimeout = ir.Spec.Timeouts.ResponseHeader.Duration
 		}
 	}
-	transport := uh.transportPool.Get(respHeaderTimeout)
+
+	transport := uh.transportPool.Get(responseHeaderTimeout)
 
 	var rt http.RoundTripper = transport
 	if uh.tracingCfg.Enabled {
@@ -99,7 +99,15 @@ func (uh *Upstream) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		BufferPool: bufferPool,
 		Transport:  rt,
 		ErrorHandler: func(w http.ResponseWriter, r *http.Request, err error) {
-			sh := NewStatic(http.StatusBadGateway, err)
+			code := http.StatusBadGateway
+			var netErr net.Error
+			if errors.As(err, &netErr) && netErr.Timeout() {
+				// Respond with 504 Gateway Timeout on timeouts to differentiate from general server errors
+				code = http.StatusGatewayTimeout
+			} else if errors.Is(err, context.DeadlineExceeded) {
+				code = http.StatusGatewayTimeout
+			}
+			sh := NewStatic(code, err)
 			sh.ServeHTTP(w, r)
 		},
 	}