feat: add opt-in content capture to GenerateContent spans#648
feat: add opt-in content capture to GenerateContent spans#648brucearctor wants to merge 3 commits intogoogle:mainfrom
Conversation
Add support for capturing system instructions, input messages, and output messages as span attributes on generate_content spans, behind the existing SetGenAICaptureMessageContent opt-in flag. When enabled, the following OTel semantic convention attributes are populated: - gen_ai.system_instructions: system instructions text - gen_ai.input.messages: JSON-serialized input contents - gen_ai.output.messages: JSON-serialized model response content This aligns with the OpenTelemetry Semantic Conventions for GenAI: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/\#capturing-instructions-inputs-and-outputs Fixes google#608
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request reintroduces the capability to capture detailed LLM request and response content within OpenTelemetry spans for GenAI operations. This feature is opt-in, ensuring that existing behavior remains unchanged by default, and provides valuable debugging and observability insights when activated. The changes align the telemetry with OpenTelemetry's semantic conventions for GenAI, enhancing the traceability of LLM interactions. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request correctly implements opt-in content capture for GenerateContent spans, aligning with OpenTelemetry semantic conventions. The changes are well-structured and include appropriate tests. My review includes a few suggestions to enhance code efficiency and conciseness, primarily by batching OpenTelemetry attribute updates and removing some redundant checks.
- Batch SetAttributes calls into single invocation for better performance - Remove redundant params.Response != nil check (already guarded by early return) - Remove redundant len(texts) == 0 check (strings.Join handles empty slices)
|
Thanks for working on this, @brucearctor! Any idea how we can move this forward? Looks like the Python ADK (which is the "source of truth" as per I wonder if it is still possible to get partial support for this via this PR, rather than waiting for the Python ADK to promote this out of the experimental stage? |
…support Add fallback to the standard OTel env var for content capture configuration, aligning with the Python ADK's configuration surface. The programmatic SetGenAICaptureMessageContent() API takes precedence; the env var acts as a fallback. Any non-empty value enables content capture, with a path toward supporting the full mode set (EVENT_ONLY, SPAN_ONLY, SPAN_AND_EVENT) in the future.
|
Thanks for the context, @lukasmalkmus! The Python ADK's implementation has a 2-layer system:
I think we can move this forward incrementally: This PR: Land the current span content capture, but also read Would that work as a path forward? Once out of experimental phase, then can follow up with extending in the go sdk. |
2 participants
Description
Fixes #608
In the migration from v0.4.0 to v0.5.0, the telemetry package moved to standard OpenTelemetry Semantic Conventions for GenAI, but dropped the ability to capture LLM request/response content in spans. This PR restores that capability as an opt-in feature, aligned with the OTel spec recommendation.
When
SetGenAICaptureMessageContent(true)is called (the same flag already used by the OTel log events inlogger.go), the following span attributes are populated ongenerate_contentspans:gen_ai.system_instructionsLLMRequest.Config.SystemInstructiongen_ai.input.messagesLLMRequest.Contentsgen_ai.output.messagesLLMResponse.ContentContent is not captured by default — the existing behavior is preserved.
Changes
internal/telemetry/telemetry.go: Added content capture attribute keys, extendedStartGenerateContentSpanParamswithLLMRequest, and conditionally populate content attributes inStartGenerateContentSpanandTraceGenerateContentResult.internal/llminternal/base_flow.go: PassLLMRequesttoStartGenerateContentSpanParams.internal/telemetry/telemetry_test.go: AddedTestGenerateContentContentCapturewith sub-tests for disabled (default) and enabled content capture.Testing Plan
Unit Tests
Full Verification
go test ./internal/telemetry/...— all tests passgo vet ./internal/telemetry/... ./internal/llminternal/...— cleangofmt -l— cleango build ./...— clean