@effect/ai-amazon-bedrock: reasoningTokens never populated in GenerateTextResponse usage

[Chaoran-Huang]

What version of Effect is running?

• @effect/ai: ^0.33.0
• @effect/ai-amazon-bedrock: ^0.13.0

Description

When extended thinking is enabled via additionalModelRequestFields.thinking, the Bedrock Converse API returns reasoning content in output.message.content (as reasoningContent blocks), and the @effect/ai-amazon-bedrock provider correctly extracts this into ReasoningPart objects (accessible via response.reasoningText).

However, response.usage.reasoningTokens is always undefined. The @effect/ai library defines reasoningTokens in Response.Usage (line 154 of LanguageModel.ts):

reasoningTokens: Schema.optionalWith(Schema.Number, { as: "Option" })

But the Bedrock provider's makeResponse function never maps a value for it:

// AmazonBedrockLanguageModel.ts makeResponse (lines 607-622)
usage: {
  inputTokens: raw.usage!.inputTokens!,
  outputTokens: raw.usage!.outputTokens!,
  cacheCreationInputTokens: (raw.usage as any).cacheCreationInputTokens ?? 0,
  cacheReadInputTokens: (raw.usage as any).cacheReadInputTokens ?? 0,
  // reasoningTokens is never set
},

The Bedrock Converse API does not return reasoning tokens as a separate field in its usage metrics — reasoning tokens are bundled into outputTokens. The Anthropic direct API does return them separately (as cache_creation_input_tokens and cache_read_input_tokens for caching, and model-specific fields for thinking).

Root Cause

Two related issues:

1. Bedrock Converse API limitation: The Converse API's TokenUsage response only includes inputTokens, outputTokens, totalTokens, cacheReadInputTokens, and cacheWriteInputTokens. There is no reasoningTokens or thinkingTokens field. Reasoning tokens are included in outputTokens.

2. Provider gap: The @effect/ai-amazon-bedrock provider doesn't attempt to derive reasoning tokens. A possible heuristic would be to count the tokens in the returned reasoningContent blocks, but this would require a tokenizer and wouldn't be precise.

Reproduction

import { Chat } from "@effect/ai"
import { AmazonBedrockLanguageModel } from "@effect/ai-amazon-bedrock"
import { Effect, Option } from "effect"

const program = Effect.gen(function* () {
  const chat = yield* Chat.fromPrompt([{ role: "system", content: "Think carefully." }])

  const response = yield* chat.generateText({
    prompt: "What is 2+2? Show your reasoning.",
  }).pipe(
    AmazonBedrockLanguageModel.withConfigOverride({
      additionalModelRequestFields: {
        thinking: { type: "enabled", budget_tokens: 4096 },
      },
    })
  )

  console.log("reasoningText:", response.reasoningText)
  // => "Let me think about this..." (populated correctly)

  console.log("reasoningTokens:", Option.getOrUndefined(response.usage.reasoningTokens))
  // => undefined (never populated)
})

Expected Behavior

Either:

1. Populate reasoningTokens by deriving it from the Bedrock response (e.g., if the direct Anthropic API provides it via the x-amzn-bedrock-* headers or if Bedrock adds the field in the future), or
2. Document that reasoningTokens is not available when using the Bedrock Converse API and that reasoning tokens are included in outputTokens, so consumers know not to rely on it for cost calculations.

Note: The Anthropic direct API (@effect/ai-anthropic) may have access to this data and could serve as a reference implementation.