feat(go): added common middleware (e.g. tool approval, retry, fallback)

[apascal07]

Adds a middleware plugin that bundles five production-ready implementations of the ai.Middleware interface introduced in #4464: Retry with exponential backoff, Fallback across models, ToolApproval for human-in-the-loop gating, Filesystem for scoped file access, and Skills for loadable SKILL.md personas.

Examples

Composing Retry and Fallback

Register the middleware plugin during Init to expose the built-ins to the Dev UI, then attach them with ai.WithUse. Middleware composes outer-to-inner, so ai.WithUse(&Retry{...}, &Fallback{...}) expands to Retry { Fallback { model } }: each model in the fallback list is retried with exponential backoff before the next fallback is tried.

import "github.com/firebase/genkit/go/plugins/middleware"

g := genkit.Init(ctx, genkit.WithPlugins(
    &googlegenai.GoogleAI{},
    &middleware.Middleware{},
))

response, _ := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithPrompt("Explain quantum computing."),
    ai.WithUse(
        &middleware.Retry{MaxRetries: 3},
        &middleware.Fallback{Models: []ai.ModelRef{
            googlegenai.ModelRef("googleai/gemini-3.1-flash", nil),
        }},
    ),
)

---

Tool Approval

ToolApproval interrupts any tool call outside its AllowedTools list. The caller approves (or rejects) and resumes with ai.WithToolRestarts, reusing the existing interrupt/restart machinery:

response, _ := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithPrompt("Transfer $500 to Alice."),
    ai.WithTools(transferTool, lookupTool),
    ai.WithUse(&middleware.ToolApproval{
        AllowedTools: []string{lookupTool.Name()}, // transfer triggers an interrupt
    }),
)

---

Filesystem

Filesystem registers list_files, read_file, and optionally write_file and search_and_replace, all confined to RootDir. Path safety is enforced by os.Root (Go 1.25+), which rejects any path that resolves outside the root, including via .., absolute paths, or symlinks:

response, _ := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithPrompt("Summarise docs/ and save the summary to summary.md"),
    ai.WithUse(&middleware.Filesystem{
        RootDir:          "./workspace",
        AllowWriteAccess: true,
    }),
)

---

Skills

Skills exposes a local library of loadable instructions. Each skill is a directory with a SKILL.md file; the middleware advertises available skills (name plus optional description) in the system prompt and registers a use_skill tool that loads the chosen skill's full body into the conversation on demand:

response, _ := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithPrompt("Explain how rainbows form, in the voice of a pirate."),
    ai.WithUse(&middleware.Skills{SkillPaths: []string{"skills"}}),
)

---

API Reference

Built-in middleware (plugins/middleware)

// Middleware is the plugin that registers the built-ins with the Dev UI.
type Middleware struct{}

// Retry retries failed model calls with exponential backoff and jitter.
// Non-GenkitError errors are always retried; GenkitErrors only if their status is in Statuses.
type Retry struct {
    MaxRetries     int               `json:"maxRetries,omitempty"`      // default 3
    Statuses       []core.StatusName `json:"statuses,omitempty"`        // default: UNAVAILABLE, DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, ABORTED, INTERNAL
    InitialDelayMs int               `json:"initialDelayMs,omitempty"`  // default 1000
    MaxDelayMs     int               `json:"maxDelayMs,omitempty"`      // default 60000
    BackoffFactor  float64           `json:"backoffFactor,omitempty"`   // default 2
    NoJitter       bool              `json:"noJitter,omitempty"`
}

// Fallback tries alternative models when the primary fails with a retryable status.
// Each model's config comes from its ModelRef; the original request config is not inherited.
type Fallback struct {
    Models   []ai.ModelRef     `json:"models,omitempty"`
    Statuses []core.StatusName `json:"statuses,omitempty"` // default also includes NOT_FOUND, UNIMPLEMENTED
}

// ToolApproval interrupts any tool call not present in AllowedTools.
// An empty list interrupts all tools.
type ToolApproval struct {
    AllowedTools []string `json:"allowedTools,omitempty"`
}

// Filesystem grants the model scoped file access under RootDir.
// Registers list_files and read_file; adds write_file and search_and_replace
// when AllowWriteAccess is true.
type Filesystem struct {
    RootDir          string `json:"rootDirectory,omitempty"`
    AllowWriteAccess bool   `json:"allowWriteAccess,omitempty"`
    ToolNamePrefix   string `json:"toolNamePrefix,omitempty"` // prefix tool names when attaching multiple Filesystem middlewares
}

// Skills exposes a local library of SKILL.md files as loadable system instructions.
// Injects a system prompt listing available skills and registers a use_skill tool.
type Skills struct {
    SkillPaths []string `json:"skillPaths,omitempty"` // default []string{"skills"}
}

Each type satisfies the simplified ai.Middleware contract from #4464 by implementing Name() string and New(ctx) (*ai.Hooks, error). New returns a per-call hook bundle (Tools, WrapGenerate, WrapModel, WrapTool); per-call state — like Filesystem's message queue and resolved os.Root, or Skills' scanned skill set — lives in closures captured by New so concurrent calls stay isolated.

---

Notes

• Builds on the simplified Middleware interface from feat(go/ai): added DefineMiddleware (Middleware V2) #4464 (config struct + New(ctx) (*ai.Hooks, error); tools surfaced via Hooks.Tools).
• Requires Go 1.25+ (Filesystem depends on os.Root).
• Samples added under go/samples/basic-middleware/: retry-fallback, filesystem, and skills.
• Also included in this branch: tighter error wrapping in plugins/googlegenai, a streaming ordering fix that preserves generate > model > tool turn order when resuming restarted tools, a refactor of the Fallback/Retry split into separate middlewares, and a streaming-format-handler fix so chunks emitted from WrapGenerate accumulate alongside model-emitted chunks.