>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3572955. --> Reported by: [jibla](https://www.drupal.org/user/1013184) Related to !29 >>> <p>[Tracker]<br> <strong>Update Summary: </strong>New - audio transcription via Gemini multimodal<br> <strong>Check-in Date: </strong>02/11/2026<br> <strong>Additional Collaborators: </strong><br> <em>Metadata is used by the <a href="https://www.drupalstarforge.ai/" title="AI Tracker">AI Tracker.</a> Docs and additional fields <a href="https://www.drupalstarforge.ai/ai-dashboard/docs" title="AI Issue Tracker Documentation">here</a>.</em><br> [/Tracker]</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Gemini can transcribe audio files (WAV, MP3, AIFF, AAC, OGG, FLAC) up to 9.5 hours via its multimodal <code>generateContent</code> endpoint. Audio is sent as a Blob or uploaded via the Files API, with a transcription prompt. The PHP library supports audio input via Blob with audio MIME types.</p> <p>The OpenAI provider already implements <code>SpeechToTextInterface</code> (via Whisper). We should reach parity.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <ul> <li>Implement <code>SpeechToTextInterface</code> in GeminiProvider</li> <li>Add <code>speech_to_text</code> to <code>getSupportedOperationTypes()</code></li> <li>Accept <code>SpeechToTextInput</code> (<code>AudioFile</code> binary data), convert to Gemini Blob with appropriate audio MIME type</li> <li>Send via <code>generateContent</code> with a transcription system prompt</li> <li>Return text response as <code>SpeechToTextOutput</code></li> <li>Add STT configuration to <code>definitions/api_defaults.yml</code></li> <li>Add model filtering in <code>getConfiguredModels()</code> for STT-capable models</li> <li>Update <code>ai_provider_info.yml</code> with <code>speech_to_text</code></li> <li>Add tests</li> </ul> <p><strong>Note:</strong> This uses Gemini's multimodal understanding rather than a dedicated STT endpoint. The approach differs from OpenAI's Whisper (dedicated transcription model) but achieves the same result.</p> <p><strong>References:</strong></p> <ul> <li><a href="https://ai.google.dev/gemini-api/docs/audio">Gemini audio understanding documentation</a></li> </ul> <h3 id="summary-remaining-tasks">Target date or deadline</h3> <p>Before stable release.</p> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <ul> <li>Implement <code>SpeechToTextInterface</code></li> <li>Handle audio file MIME type detection</li> <li>Design transcription prompt (or make it configurable)</li> <li>Add model filtering for speech_to_text operation type</li> <li>Write tests</li> <li>phpcs clean</li> </ul> <h3 id="summary-ai-usage">AI usage (if applicable)</h3> <p>[x] AI Assisted Issue<br> This issue was generated with AI assistance, but was reviewed and refined by the creator.</p> <p>[ ] AI Assisted Code</p> <p>[ ] AI Generated Code</p> <p>[ ] Vibe Coded</p> > Related issue: [Issue #3572958](https://www.drupal.org/node/3572958)