Skip to content

Add Flows + realtime (S2S) compatibility docs#741

Open
jamsea wants to merge 4 commits intomainfrom
jh/flows-realtime-models
Open

Add Flows + realtime (S2S) compatibility docs#741
jamsea wants to merge 4 commits intomainfrom
jh/flows-realtime-models

Conversation

@jamsea
Copy link
Copy Markdown
Contributor

@jamsea jamsea commented Apr 17, 2026

Summary

  • Documents that Pipecat Flows requires a cascade LLM service (STT → LLM → TTS) and is not compatible with realtime speech-to-speech services (Gemini Live, OpenAI Realtime, AWS Nova Sonic, Grok, Inworld, Ultravox). Native S2S support is currently being developed.
  • New guide at pipecat-flows/guides/realtime-models.mdx with a compatibility table and a concrete cascade example (Deepgram STT + Gemini Flash LLM + Cartesia TTS).
  • Cross-linked from the Flows introduction, the Flows API overview, and each of the 7 S2S service pages so users land on the guide regardless of entry point.

Closes a Kapa coverage gap where users across English, Russian, and Dutch conversations asked the same question (Flows + Gemini Live / OpenAI Realtime) and only got a "not supported" answer with no workaround.

Test plan

  • mint dev renders the new guide page without errors
  • mint broken-links passes in CI
  • New guide appears under Pipecat Flows → Guides in the sidebar
  • Every S2S service page shows the "Not compatible with Pipecat Flows" callout and links back to the new guide
  • Flows introduction and API overview both link to the new guide

Documents that Pipecat Flows requires a cascade LLM service (STT → LLM
→ TTS) and is not compatible with realtime speech-to-speech services
(Gemini Live, OpenAI Realtime, AWS Nova Sonic, Grok, Inworld, Ultravox).
Native S2S support is currently being developed.

Closes a Kapa coverage gap where users across English, Russian, and Dutch
conversations asked the same question and got only a "not supported" answer
with no workaround.

- New guide: pipecat-flows/guides/realtime-models.mdx with a compatibility
  table and a concrete cascade example (Deepgram + Gemini Flash + Cartesia)
- Cross-linked from the Flows introduction, the Flows API overview, and
  each of the 7 S2S service pages so users land on the guide regardless
  of entry point
@mintlify
Copy link
Copy Markdown

mintlify bot commented Apr 17, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
daily 🟢 Ready View Preview Apr 17, 2026, 7:58 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds documentation clarifying that Pipecat Flows currently requires a cascade (STT → LLM → TTS) pipeline and is not compatible with realtime speech-to-speech (S2S) services, plus a central guide page that’s linked from Flows entry points and S2S service docs.

Changes:

  • Adds a new Flows guide with an S2S compatibility table and a recommended cascade configuration example.
  • Cross-links the new guide from the Flows introduction + Flows API overview.
  • Adds a “Not compatible with Pipecat Flows” callout + link to the guide across all S2S service pages.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pipecat-flows/introduction.mdx Adds a callout linking Flows users looking for realtime S2S to the new compatibility guide.
pipecat-flows/guides/realtime-models.mdx New guide explaining incompatibility and recommending a cascade workaround with an example.
docs.json Adds the new guide to the Pipecat Flows → Guides navigation.
api-reference/server/services/s2s/aws.mdx Adds Flows incompatibility callout and links to the new guide.
api-reference/server/services/s2s/gemini-live.mdx Adds Flows incompatibility callout and links to the new guide.
api-reference/server/services/s2s/gemini-live-vertex.mdx Adds Flows incompatibility callout and links to the new guide.
api-reference/server/services/s2s/grok.mdx Adds Flows incompatibility callout and links to the new guide.
api-reference/server/services/s2s/inworld.mdx Adds Flows incompatibility callout and links to the new guide.
api-reference/server/services/s2s/openai.mdx Adds Flows incompatibility callout and links to the new guide.
api-reference/server/services/s2s/ultravox.mdx Adds Flows incompatibility callout and links to the new guide.
api-reference/pipecat-flows/overview.mdx Adds a “Realtime (S2S) models” note linking to the new guide.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pipecat-flows/guides/realtime-models.mdx
Comment thread pipecat-flows/guides/realtime-models.mdx
Comment thread pipecat-flows/guides/realtime-models.mdx
Addresses Copilot review feedback: readers seeing just the `FlowManager`
construction may think setup is complete. Adds the `on_client_connected`
event handler that calls `flow_manager.initialize(create_initial_node())`,
matching the pattern in the Flows Quickstart.

Any service that extends Pipecat's `LLMService` base class is supported. This includes OpenAI-compatible services like Groq, Together, Cerebras, DeepSeek, and others.

### Realtime (S2S) models
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we only need this small section in overview. Can you update this PR?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants