Fix/token usage dto cached write tool use 3586576

Pravesh Pooniarequested to merge
issue/ai-3586576:fix/token-usage-dto-cached-write-tool-use-3586576 into 1.x

Summary

  • Added support for two additional AI token usage types:

    • Cache write tokens (cachedWrite)
    • Tool use tokens (toolUse)

  • This ensures token usage data from AI providers is captured more completely for observability.

Changes

  • Added cachedWrite: ?int and toolUse: ?int fields to TokenUsageDto with NULL defaults to maintain backward compatibility.

  • Updated StreamedChatMessageInterface by adding:

    • setCachedWriteTokenUsage()
    • getCachedWriteTokenUsage()
    • setToolUseTokenUsage()
    • getToolUseTokenUsage()

  • Implemented all four new methods in StreamedChatMessage with corresponding private properties.

  • Added $cachedWriteTokenUsage and $toolUseTokenUsage properties to StreamedChatMessageIterator.

  • Updated StreamedChatMessageIterator to propagate the new token types through:

    • setTokenUsageFromChunk()
    • setTokenUsageOnChatOutput()

  • Updated OpenAiBasedProviderClientBase::setChatTokenUsage() to populate:

    • cachedWrite from prompt_tokens_details.cache_write_tokens
    • toolUse from completion_tokens_details.tool_use_tokens

  • Updated the cached field documentation to clarify that it represents cache read tokens.

  • No changes were required in ai_observability, as DtoBaseMethodsTrait::toArray() automatically includes the new DTO properties via get_object_vars().

Merge request reports