Multiple Speaker Detection on audio transcription with end-user audio uploads (Microsoft VibeVoice)

Feature Requests
1

Add native speech-to-text transcription as a platform capability. This means two things working together:

  1. Let end users upload audio/video files (mp3, m4a, wav, mp4) directly through the chat or form interface
  2. Transcribe those files with a built-in engine such as OpenAI Whisper or Microsoft VibeVoice, then pass the transcript to the agent as context

Why the current options fall short

The Knowledge Base already accepts mp3 uploads on the builder side. End users have no equivalent. Audio files fail to upload through the chat interface, even at modest sizes (see this thread on 30-minute meeting recordings: Generating transcription from video files (not YouTube) ).

The AssemblyAI action is a good start but requires a publicly accessible URL. Most end users do not have public direct links to their audio. Dropbox and Google Drive share links need manual conversion before they work. That friction kills the use case for non-technical users.

Use cases this unlocks

  • Transcribe and summarize meeting recordings and Zoom calls
  • Turn podcast episodes into show notes, blog posts, and social content
  • Analyze recorded interviews, coaching calls, and sales calls
  • Let users dictate long-form input instead of typing it

These are among the most requested AI workflows anywhere right now. Builders who need them currently leave the platform or duct-tape solutions together with Make webhooks.

Suggested engines

  • Whisper is the obvious default. It is battle-tested, cheap to run, and widely supported by inference providers.
  • VibeVoice-ASR (Microsoft, open source) is worth evaluating as a premium option. It processes up to 60 minutes of audio in a single pass, includes speaker diarization and timestamps, and supports custom hotwords so users can pass in names and technical terms for correct spelling. Repo: GitHub - microsoft/VibeVoice: Open-Source Frontier Voice AI · GitHub

Either engine could be exposed the same way image generation is today: a capability toggle in the Prompt tab, with the transcript injected into the conversation as context.

Implementation notes

  • Speaker labels and timestamps should be optional flags. Some tools want clean prose, others want a structured transcript.
  • This also pairs naturally with OpenClaw, which could process the audio in its sandboxed environment.
2

After talking with @lindsay_support during the office hours I did some more testing, it looks like a lot of this is already there. It looks like Pickaxe has Whisper support already.

So the main missing piece is multiple speaker detection, which Microsoft’s VibeVoice can handle.