πβ Enhancement Request
Is your enhancement request related to a problem? Please describe.
Honcho currently assumes embedding providers are OpenAI/Gemini/OpenRouter-style integrations, which makes it awkward to use lightweight local embedding servers that expose their own HTTP API.
A practical example is a local MLX embedding server on macOS that serves native endpoints such as:
POST /embed
POST /embed_batch
rather than OpenAI-compatible /v1/embeddings.
Today, integrating these local servers requires either:
- pretending they are OpenAI-compatible when they are not, or
- patching Honcho locally.
This makes local/self-hosted embedding setups harder than necessary.
Describe the solution you'd like
Add first-class support for a local embedding provider with configurable settings:
LLM_EMBEDDING_PROVIDER=local
LLM_EMBEDDING_BASE_URL=...
LLM_EMBEDDING_MODEL=...
LLM_EMBEDDING_API_KEY=...
Behavior:
local provider calls native local HTTP endpoints such as:
POST {base_url}/embed
POST {base_url}/embed_batch
EMBEDDING_MODEL should also work as an override for existing providers where applicable.
.env.template should document these settings.
This would make Honcho easier to use with self-hosted/local embedding backends without affecting existing users.
Describe alternatives you've considered
-
OpenAI-compatible shim
- Wrap the local embedding server behind a fake
/v1/embeddings API.
- Works, but adds another layer and complexity.
-
Reuse an existing provider setting
- For example, overload
openrouter or openai-compatible behavior.
- This is less clear than a dedicated
local provider and can be confusing for users.
-
Keep the current behavior
- Possible, but makes local/self-hosted embeddings unnecessarily difficult to adopt.
Additional context
I have a working local implementation using a native HTTP embedding server with:
- single-text embedding endpoint
- batch embedding endpoint
- configurable model name
- optional bearer auth
This change is additive and opt-in:
- existing providers keep working as before
- users only opt in when they explicitly choose
LLM_EMBEDDING_PROVIDER=local
I intentionally want to separate this request from two larger follow-up topics:
- configurable vector dimensions / DB migration strategy
- fallback behavior when embeddings fail but document persistence should continue
Those feel like separate design discussions from basic local-provider support.
πβ Enhancement Request
Is your enhancement request related to a problem? Please describe.
Honcho currently assumes embedding providers are OpenAI/Gemini/OpenRouter-style integrations, which makes it awkward to use lightweight local embedding servers that expose their own HTTP API.
A practical example is a local MLX embedding server on macOS that serves native endpoints such as:
POST /embedPOST /embed_batchrather than OpenAI-compatible
/v1/embeddings.Today, integrating these local servers requires either:
This makes local/self-hosted embedding setups harder than necessary.
Describe the solution you'd like
Add first-class support for a
localembedding provider with configurable settings:LLM_EMBEDDING_PROVIDER=localLLM_EMBEDDING_BASE_URL=...LLM_EMBEDDING_MODEL=...LLM_EMBEDDING_API_KEY=...Behavior:
localprovider calls native local HTTP endpoints such as:POST {base_url}/embedPOST {base_url}/embed_batchEMBEDDING_MODELshould also work as an override for existing providers where applicable..env.templateshould document these settings.This would make Honcho easier to use with self-hosted/local embedding backends without affecting existing users.
Describe alternatives you've considered
OpenAI-compatible shim
/v1/embeddingsAPI.Reuse an existing provider setting
openrouteroropenai-compatiblebehavior.localprovider and can be confusing for users.Keep the current behavior
Additional context
I have a working local implementation using a native HTTP embedding server with:
This change is additive and opt-in:
LLM_EMBEDDING_PROVIDER=localI intentionally want to separate this request from two larger follow-up topics:
Those feel like separate design discussions from basic local-provider support.