Skip to content

Commit cfe1439

Browse files
authored
feat(tools): add tool dispatch and agentic docs (#2063)
* Add max tool rounds * feat(tools): add http tool dispatch * feat(tools): add CLI agentic flags, secure tool dispatch, Python SDK support - Remove tool_dispatch_url from HTTP request body to prevent SSRF from untrusted API clients. It is now server-level only via --tool-dispatch-url CLI flag, while the Rust and Python SDKs can still set it per-request. - Add --max-tool-rounds and --tool-dispatch-url CLI flags to `serve` command, wired through AgenticDefaults axum Extension. - Expose max_tool_rounds and tool_dispatch_url in Python SDK ChatCompletionRequest for trusted per-request control. - Rewrite TOOL_CALLING.md for cohesive structure: two modes (basic vs agentic), unified examples section, clear dispatch order. - Add examples: tool_dispatch.py, agentic_tool_rounds.py, agentic_tools.py. - Update README agentic section and HTTP.md with security note. * Add agents docs * Simplify docs * Simplify docs
1 parent 1e3c73b commit cfe1439

41 files changed

Lines changed: 1846 additions & 577 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ Fast, flexible LLM inference.
3737
- **Built-in web UI**: `mistralrs serve --ui` gives you a web interface instantly.
3838
- **Hardware-aware**: `mistralrs tune` benchmarks your system and picks optimal quantization + device mapping.
3939
- **Flexible SDKs**: Python package and Rust crate to build your projects.
40-
- **Agentic features** — tool calling, web search, and MCP client built in
40+
- **[Agentic features](docs/AGENTS.md)**server-side tool loop, web search, MCP client, and HTTP tool dispatch
4141

4242
## Quick Start
4343

@@ -120,9 +120,12 @@ mistralrs doctor
120120
- [Multiple models](docs/multi_model/README.md): Load/unload at runtime
121121

122122
**Agentic Features**
123-
- Integrated [tool calling](docs/TOOL_CALLING.md) with Python/Rust callbacks
124-
-[Web search integration](docs/WEB_SEARCH.md)
125-
-[MCP client](docs/MCP/client.md): Connect to external tools automatically
123+
- Integrated [tool calling](docs/TOOL_CALLING.md) with grammar enforcement and strict schema mode
124+
- ⭐ Server-side [agentic loop](docs/TOOL_CALLING.md#agentic-loop): auto-execute tools and feed results back
125+
-[Tool dispatch URL](docs/TOOL_CALLING.md#tool-dispatch-url): POST tool calls to your own endpoint
126+
-[Web search integration](docs/WEB_SEARCH.md) with embedding-based ranking
127+
-[MCP client](docs/MCP/client.md): Connect to external tools via Process, HTTP, or WebSocket
128+
- Python/Rust [tool callbacks](docs/TOOL_CALLING.md#tool-callbacks) for custom execution
126129

127130
[Full feature documentation](docs/README.md)
128131

0 commit comments

Comments
 (0)