Skip to content

Move HTTP healthcheck from shared image to API service only #521

@kht2199

Description

@kht2199

Problem description

Honcho currently defines an HTTP-based HEALTHCHECK in the shared Docker image (Dockerfile) that probes http://localhost:8000/openapi.json.

That works for the api service, but the same image is also used for the deriver service. deriver is a background worker (python -m src.deriver), not an HTTP server, so inheriting the image-level HTTP healthcheck makes Docker report the worker as unhealthy even when the process is running normally.

In local verification:

  • deriver started successfully and logged:
    • Starting deriver queue processor
    • Running main loop
    • ReconcilerScheduler started ...
  • but container health failed with:
    • urllib.error.URLError: <urlopen error [Errno 111] Connection refused>
    • because the healthcheck still probes localhost:8000/openapi.json

This makes it look like deriver is broken, when the real issue is that the healthcheck assumes every consumer of the image exposes the API server.

Desired solution

Move the HTTP healthcheck out of the shared Dockerfile and define it only on the api service in docker-compose.yml (or equivalent service-specific config).

That would let:

  • api keep its HTTP/OpenAPI healthcheck
  • deriver run without inheriting an irrelevant API probe
  • future worker-style services reuse the image without false unhealthy states

Alternatives considered

  • Disable healthcheck only for deriver in compose: works as a local workaround, but still leaves the image with an API-specific assumption.
  • Create a worker-specific process healthcheck for deriver: possible, but still does not solve the more general design issue that the base image encodes API behavior.

Why this matters

This is easy to misdiagnose as an LLM / local-model / embedding-server failure, even though the root cause is only healthcheck placement. Moving the healthcheck to the api service makes the container role boundaries clearer and avoids false alarms during deployment and debugging.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions