Skip to content

SIM_MODEL override + optional HTTP request/response logging (log-http)#447

Open
jland-redhat wants to merge 4 commits intollm-d:mainfrom
jland-redhat:capture-request
Open

SIM_MODEL override + optional HTTP request/response logging (log-http)#447
jland-redhat wants to merge 4 commits intollm-d:mainfrom
jland-redhat:capture-request

Conversation

@jland-redhat
Copy link
Copy Markdown

Description

Adds two operational knobs for deployments and debugging.

SIM_MODEL — If this environment variable is non-empty, it overrides the configured/CLI model after the config file and flags are applied. That makes it easier to point the same image or manifest at different model names (for example in Kubernetes) without editing command-line arguments or regenerating config.

log-http / --log-http — When enabled, logs each HTTP request and response at INFO: method, URI, remote address, headers, and body when it is buffered in memory. Bodies are truncated after 512 KiB. Streamed responses (for example SSE) log status and headers only, not the stream body. Documented in docs/configuration.md with a warning that logs may include sensitive data such as Authorization headers, so this should be used only in trusted environments.

Tests — Ginkgo tests cover SIM_MODEL overriding --model, overriding a value from a config file, and leaving the model unchanged when the variable is unset.

@irar2
Copy link
Copy Markdown
Collaborator

irar2 commented Apr 26, 2026

@jland-redhat Thanks a lot for the PR.

Could you please split this into two separate PRs, since it introduces two distinct features? That will make it easier to review and track.

Regarding the SIM_MODEL part: after looking into it, the commonly accepted precedence order is:
command-line flags (highest priority), then environment variables, then config files, and finally hardcoded defaults.

This ordering should address the issue you’re tackling. We also have an existing environment variable, PYTHONHASHSEED, that doesn’t follow this pattern - so it would be great if you could align its behavior as part of this change.

Please also add a short explanation of the precedence order to the configuration.md.

The HTTP logging part looks fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants