AssemblyAI

The API returned detailed confidence scores per word which helped us filter uncertain segments but processing time scaled unpredictably with file length.

Their real-time WebSocket endpoint emits partial transcripts with word-level timestamps before finalization, making live captioning trivial.

Transcription accuracy on clear audio is excellent but background noise in call center recordings dropped the word error rate noticeably.

Their TypeScript SDK ships with full type definitions for transcript objects, sentiment enums, and webhook payloads out of the box.

Timestamps align well with video files for subtitle generation, but the API doesn't expose word-level confidence scores in the default response, only when you set an undocumented query param.

Entity extraction tagged brands and people accurately across 50 podcasts, yet the API returned a 200 with partial results when one file was corrupt instead of failing explicitly.

The lemur endpoint bundles question answering and action-item extraction over transcripts, though it requires a separate API call after transcription completes.

Auto-chapters split long podcasts into titled segments with start timestamps, which beats manual chunking for summarization pipelines.

Their audio intelligence models tag PII entities like credit card numbers and SSNs inline, which saved a compliance sprint on call recordings.

Uploading audio via URL worked smoothly for public S3 links, but the error message for a 403 presigned URL just said "download failed" with no hint about auth or expiry.

The `/v2/transcript/:id` GET includes an `error` field with human-readable messages when audio quality blocks transcription, no cryptic codes.

Auto-detect language correctly identified Japanese and Spanish in our test set, but it bills per audio minute even when detection fails, and the error comes only after the file is fully processed.

Real-time streaming worked smoothly for live calls though the sentiment analysis sometimes labeled neutral customer service language as negative.

The transcription endpoint handled podcast files reliably but the speaker diarization often merged two voices in overlapping speech segments.

Speaker labels in the utterances array stay consistent across retries of the same file, making deterministic test assertions possible.

Webhook signature validation uses HMAC-SHA256 with a secret in headers, and their guide includes line-by-line verification snippets for Flask and Express.

The Python SDK makes uploads straightforward and the polling helper is convenient but there's no built-in chunking for files over 2GB.

The SDK raises a typed `AssemblyAIError` on failures and surfaces HTTP status codes, making retry logic straightforward in agent loops.

The API handled background noise in call center recordings well, but a snippet with overlapping speech transcribed both voices into one run-on sentence with no indication that utterances were concurrent.

IAB category tagging is a nice addition for ad insertion, yet it returned "News & Politics" for a gaming livestream because the streamer mentioned an election once in 90 minutes.

The `/v2/transcript` POST accepts a public URL or base64 audio blob, then polls via GET until `status: "completed"` with zero boilerplate.

Their entity detection catches names, organizations, and locations with confidence scores, which improved knowledge-graph extraction over regex approaches.

Auto-highlights pulled key quotes from a webinar accurately, but it returned 12 highlights for a 15-minute video, which is too dense for a summary view, and there's no top-k parameter to limit the count.

Auto-highlights extract key phrases from transcripts with ranking scores, which agents pipe into meeting summaries without LLM post-processing.

Dual-channel processing preserved left-right speaker separation in our stereo court depositions, but it costs double the single-channel rate with no warning in the API request, and we only noticed after the bill came.