# Omi Medical Speech-to-Text API Omi provides one direct-upload transcription front door plus a presigned large-upload path, encounter vocabulary, structured-token Patterns, per-utterance automatic language detection, speaker diarization, and acoustic word timestamps. Realtime streaming is available on every plan with optional speaker activity. All audio is processed in the EU (eu-central-1) on both `api.omi.health` and `api.eu.omi.health`. ## Authentication Base URL: `https://api.omi.health` Send every HTTP request with: ```http Authorization: Bearer ``` Create and rotate keys at `https://console.omi.health/dashboard`. Do not place long-lived API keys in public browser bundles. ## First transcription ```bash curl https://api.omi.health/v1/audio/transcriptions \ -H "Authorization: Bearer $OMI_API_KEY" \ -F file=@consultation.wav \ -F model=omi-medical-1 \ -F language=en \ -F response_format=verbose_json ``` The direct-upload front door accepts exactly up to 100,000,000 bytes. Audio up to 60.000 seconds without a webhook returns an OpenAI-compatible 200 response. Audio above 60.000 seconds, or any request with `webhook_url`, returns a 202 transcription job. Accepted formats: WAV, MP3, M4A/MP4, WebM, OGG, FLAC, and AAC. Files longer than 60 seconds use the asynchronous-optimized pipeline. For the Python or JavaScript quickstarts in the documentation site, install the client once with `python -m pip install openai` or `npm install openai`. Response formats: - `text`: final transcript as `text/plain`. - `json`: minimal `{"text":"..."}`. - `verbose_json`: text, language, segments, duration, model, request ID, metadata, and words when alignment is available. - `diarized_json`: historical speaker-segment response shape. ### Processing profile The flagship has one authoritative processing profile: `default`. Normally omit `profile`, or send `profile=default`. The 60.000-second boundary changes response delivery—inline versus a durable job—not transcription behavior. Historical `standard` and `turbo` values remain accepted as deprecated compatibility aliases to `default` and behave identically. Alias responses include deprecation and warning headers. The same contract applies to `POST /v1/jobs`. ## Models - `omi-medical-1`: flagship medical speech-to-text model. The downloadable local runtime is a separate deployment option, not a hosted API model. See [Open-source local runtime](/edge) for supported hardware and installation details. Published benchmark claims apply to English unless the claim states otherwise. ## Languages Omi Medical STT is an English-first medical speech model. English is the benchmarked and medically evaluated language. Spanish, Portuguese, French, German, Dutch, Arabic, and Hindi were included in training and are available for testing and evaluation; they have not been medically evaluated and should not be relied on for medical accuracy. | Language | Family | Accepted examples | Medical evaluation | |---|---|---|---| | English | `en` | `en-AU`, `en-GB`, `en-US` | Benchmarked | | Spanish | `es` | `es-ES`, `es-MX`, `es-419` | Testing only | | Portuguese | `pt` | `pt-BR`, `pt-PT` | Testing only | | French | `fr` | `fr-FR`, `fr-CA` | Testing only | | German | `de` | `de-DE`, `de-AT` | Testing only | | Dutch | `nl` | `nl-NL`, `nl-BE` | Testing only | | Arabic | `ar` | `ar-SA`, `ar-AE` | Testing only | | Hindi | `hi` | `hi-IN` | Testing only | Use an explicit BCP-47 tag when known. If `language` is omitted, Omi detects the dominant language from the first speech-bearing 30-second window (checking up to four windows when a recording starts with silence), then transcribes the original full recording in that language. Mixed probes escalate to the full per-utterance path. Explicit `language=auto` remains the deliberate mode for code-switching encounters and optional `language_hints`. ### Language feature matrix Transcription and word-timing availability are separate capabilities. All eight language families can receive transcription, encounter vocabulary, and real acoustic speaker segments. Word-level timing is currently available only for the language families marked below: | Language | Transcription | Vocabulary | Segment speakers | Word timing | |---|---|---|---|---| | `en` / `es` / `pt` / `fr` / `de` | Yes | Yes | Yes | Yes | | `nl` | Yes | Yes | Yes | Yes | | `ar` / `hi` | Yes | Yes | Yes | No — use `speaker_segments` | Dutch can return word timestamps while preserving the exact Omi transcript. If word timing is unavailable or ambiguous, the API retains real segment-level speaker intervals. For Arabic and Hindi, a diarized verbose response returns real acoustic intervals in `speaker_segments[]` and sets `speaker_timing.word_timing` to `false`; it does not guess speaker ownership for transcript words. Arabic and Hindi remain pending language-specific quality validation. ### Automatic language detection Set `language=auto` on `omi-medical-1` requests. Omi detects each closed utterance, collapses consecutive same-language utterances into a run, and concatenates the result in time order. `language_hints` is an optional JSON array string containing 1–8 candidate BCP-47 tags: ```bash --form-string 'language_hints=["en","es-MX","pt-BR"]' ``` Without hints, all eight families are candidates. An unsupported detected language fails explicitly; Omi never silently falls back to English. Token-level code switching is outside V1. The current detector does not expose a calibrated confidence value. Automatic detection is available on every plan for synchronous and async requests when the detector is healthy. English is the benchmarked, medically evaluated language. The seven additional languages carry the same serving safety rails and are available for testing and evaluation; they are not medically evaluated. ## Encounter vocabulary Pass exact expected terms as a JSON array: ```bash --form-string 'vocabulary=["Tinel","Hepcludex","tirzepatide"]' ``` Use 5–50 terms likely to occur in the encounter. Lists of 50 terms or fewer are considered in full. Larger lists (up to 1,000) select up to 50 relevant terms for that recording. Arabic and Hindi dictionaries above 50 terms use a second transcription pass; that higher-latency path is preview and not medically evaluated. Verbose responses use `schema: "omi-vocabulary-audit-v3"`. Read `delivered_terms` / `delivered` for requested terms present in the returned transcript, `corrected_terms` / `corrections_applied` for conservative caller-owned surface corrections, and `unresolved_terms` / `unresolved` for terms still absent. Read `prompt_status`, `prompted_terms`, `prompted_chunks` and `fallback_chunks` for request coverage. Deprecated `applied_terms` / `applied` are aliases of the delivered outcome, never prompt coverage. `dropped` is reserved for terms rejected before decoding. Request terms are merged above the API key’s stored language-specific dictionary. Set `dictionary=false` to suppress stored terms for one request. Structural or numeric safety failures fall back to the no-context transcript. Long recordings are checked chunk by chunk, so a failing chunk returns its plain text while safe chunks retain vocabulary. Prompt coverage and delivered outcome are reported separately, so a partial prompt fallback cannot be mistaken for either total success or total failure. ## Patterns Patterns are request-only hints for expected structured tokens: ```bash --form-string 'patterns=[0-9]{1,3}(mg|ml|mcg),[A-Z]{2}-[0-9]{4}' ``` Only set patterns you expect in the recording. A pattern can format a token only when the transcript independently supports it. Up to 32 validated expressions are accepted. Patterns are never stored or applied globally. Patterns are enabled per key on request; other keys receive `403 entitlement_denied`. ## Speaker diarization and timestamps Set `diarize=true` and use `response_format=verbose_json`. The response may contain `segments[].speaker`, `words[].speaker`, `speaker_segments`, `start`, `end`, and `speakers`. When word timing is unavailable for the language, `segments[]` remains transcript-only and speaker ownership is exposed only in `speaker_segments[]`; `speaker_timing.word_timing` is then `false`. Speaker labels are anonymous, session-local clusters, not names or roles. Speaker labels and timestamps cannot rewrite transcript wording. Very short interjections, overlap, reverberation, and background speech can reduce attribution quality. `max_speakers` is a public labelling ceiling, not permission to merge an extra person into another speaker. Omi still checks for up to four voices. If more voices are detected than the requested ceiling, excess passages keep their timestamps but are left unlabelled and marked `review_required`. The response then sets `speaker_timing.speaker_ceiling_reached=true` and includes `requested_max_speakers`, `detected_speaker_count`, and `speaker_ceiling_action`. Re-run with the correct `max_speakers` value; do not treat review-required passages as belonging to the preceding speaker. ## Open-source local runtime Omi Med STT v1 can run locally without an Omi API key. The released `omi-med-stt` package supports Apple Silicon, CUDA-capable GPUs, and Linux/Windows CPU runtimes. ```bash pip install -U omi-med-stt pip install -U "omi-med-stt[mlx]" # Apple Silicon omi-med-stt consultation.m4a --json omi-med-stt check ``` The local runtime is an English whole-file transcription product. It does not inherit the hosted API's jobs, webhooks, automatic language detection, speaker decoration, or encounter-vocabulary contract. Runtime code is MIT licensed; model weights are CC-BY-4.0. Source and full platform instructions: `https://github.com/Omi-Health/omi-med-stt-runtime`. ## Asynchronous transcription For files up to 100,000,000 bytes, the same direct-upload front door can dispatch recordings longer than 60.000 seconds through 2 hours to a job and return `Location` plus `Retry-After: 5`. It must receive, validate, and durably store the complete multipart body before it can return that job envelope. The accepted duration window is 1 second to 7,200 seconds; the `min_audio_seconds` and `max_audio_seconds` values in the job envelope are authoritative. For long recordings, prefer the presigned upload path—even below 100 MB. It returns the job handle before upload, sends bytes directly to EU object storage, and makes interrupted uploads easier to retry. It supports files up to 1 GiB: 1. `POST /v1/jobs` with filename, content type, exact byte length, model, and options. 2. POST the file to every field and URL returned in `upload`. 3. `POST /v1/jobs/{job_id}/complete`. 4. Long-poll `GET /v1/jobs/{job_id}?wait=20&include_result=true`. The request returns as soon as state changes, so completion is not hidden behind a client-side sleep. 5. When status is `succeeded`, read JSON up to 4 MiB from `result.content`. If `result.inline` is false, make the authenticated GET to `result.download_url` instead. The complete Python example uses Requests; install it once with `python -m pip install requests`. Job states: `awaiting_upload → accepted → running → succeeded`, or `failed`. Send the same `Authorization: Bearer $OMI_API_KEY` header when the returned download URL is on an Omi API hostname. Storage-backed result URLs expire after 15 minutes. Uploaded audio is deleted when processing completes or the job terminally fails. Results and job metadata are retained until your configured expiry: default 24 hours, configurable from 1 to 72 hours in the console (both `api.omi.health` and `api.eu.omi.health` are served from eu-central-1). Jobs with a `webhook_url` have an effective 9-hour minimum so the final delivery retry can still be picked up. Polling a succeeded job may mint a fresh 15-minute URL while the result is retained. After the configured expiry, polling reports `result: {"expired": true}`. An already expired presigned URL returns the storage provider's native expiry response. Optional `Idempotency-Key` applies to asynchronous job creation and is retained for 24 hours. Inline 200 requests accept and ignore it. ## Webhooks Create a separate webhook signing secret in the developer console before sending `webhook_url`. The secret is shown once and is not your API key. Completion events are transcript-free. They include an event id, event type, created time, job id, status, and authenticated poll URL. Verify the raw body using: ```text webhook-signature = v1,base64(HMAC-SHA256( base64decode(whsec_...), "webhook-id.webhook-timestamp.raw-body" )) ``` Reject timestamps outside ±5 minutes and deduplicate by `webhook-id`. Delivery is at least once. Omi attempts immediately, then after 1 minute, 5 minutes, 30 minutes, 2 hours, and 8 hours. Each attempt has a fresh timestamp/signature, a 10-second timeout, and never follows redirects. Rotation keeps the previous version valid for a 24-hour overlap; in-flight events remain pinned to their original version. ## Realtime Connect to `wss://api.omi.health/v1/stream`. Send a JSON `start` message with the API key, `streaming_mode: "realtime"`, PCM16/16 kHz mono audio settings, `language: "en"`, and optional `diarization: {"enabled": true}`. Set `event_schema: "omi-agent-v1"` and a unique `session_id`. Send binary audio frames after `ready`, then `{"type":"stop"}`. Browsers authenticate in the start message because they cannot set a custom header during WebSocket upgrade. Realtime supports explicit `en`, `es`, `pt`, `fr`, `de`, `nl`, `ar`, and `hi` sessions on every plan. It shares the 25-hour monthly allowance and is $0.45/audio-hour afterward on Pay as you go. Draft text appears quickly and may change. Each `partial` carries a `turn_id`, strictly increasing `sequence`, `revision_id`, and, after the first version, `replaces_revision_id`. Replace the previous preview for that turn and ignore stale revisions; never append revisions as duplicate text. `turn_end_candidate` is presentation-only and cannot authorize a response. `turn_continuation` cancels work based on that candidate. `turn_final` is immutable and is the only realtime text event that may authorize an agent response. The closing `final` event contains the immutable assembled record. Live-speaker labels are provisional and independently revisioned. They cannot rewrite `turn_final` or the final record. If optional speaker processing is unavailable, the service reports it explicitly rather than silently dropping audio. Confirmed `turn_final` events append completed immutable utterances. The closing `final` event contains the assembled record text. Live speaker activity is off by default, provisional, and independently revisioned from transcript text. Upload the complete recording through the batch endpoint when you need word timestamps and canonical speaker labels. Realtime sessions and live speaker labels are admitted from separate bounded pools. Builder allows 2 live sessions including 1 speaker room; Pay as you go allows 10 including 2 speaker rooms. Limits apply per project. Excess starts fail before audio with WebSocket close code `1013`: `TOO_MANY_STREAMS` means text capacity is full; `SPEAKER_CAPACITY_FULL` means text capacity is available but the optional live-speaker pool is full. Reconnect without diarization or retry with exponential backoff and jitter. Enterprise offers custom maximums and reserved concurrency. ## Production checklist - Keep the one-time-reveal API key in a server-side secret manager. Never put it in browser or mobile code. - Send a stable `Idempotency-Key` on job-creating uploads and preserve it across retries. - Retry 429 and transient 5xx responses with bounded exponential backoff, jitter, and `Retry-After`. Do not retry 4xx requests unchanged. - Store `request_id`, job id, effective language, `language_source`, and `metadata.model_build` with your integration record. Avoid logging clinical content. - Choose the shortest 1–72 hour result-retention window your workflow needs. - Sign a BAA before sending PHI, subscribe to `https://status.omi.health/`, and use `https://omi.health/trust` for security and compliance information. ## Errors Every error response uses exactly this envelope, on every route: ```json {"error": {"code": "entitlement_denied", "message": "…"}} ``` No other top-level keys are returned, and the message never reflects request content. Branch on `error.code`, never on message text. | Status | Code | Meaning | Action | |---|---|---|---| | 400 | `invalid_request` | Malformed request, unsupported model option, mismatched media type | Fix; do not retry unchanged | | 401 | `unauthorized` | Missing or invalid key | Verify the bearer token and replace or rotate the credential if needed | | 402 | `billing_blocked` | Usage is paused for billing: promotional credit and the monthly included allowance are used up, a spend cap was reached, or a payment failed | Add or fix a card in the console, or wait for the monthly reset. You are never silently billed; result retrieval keeps working | | 403 | `entitlement_denied` | The key is valid but not entitled to something the request asked for (for example Patterns or automatic language detection) | Remove the field or upgrade/check the plan in the console; do not retry unchanged | | 404 | `not_found` | Unknown or non-owned job | Check job ID and owner key | | 408 | `upload_timeout`, `upload_too_slow` | The upload timed out or remained below the minimum transfer rate | Retry as a new request; preserve the idempotency key only for the same request identity | | 409 | `idempotency_conflict`, `idempotency_in_progress` | The idempotency key is in progress or belongs to different request content | Retry/poll the original, or use a new key only for different content | | 413 | `upload_too_large` | File or duration exceeds the route | Use the presigned path or split at silence | | 415 | `unsupported_media_type` | Multipart input or audio container is unsupported | Send a supported audio file and multipart content type | | 422 | `invalid_request` | Unsupported language/detection, bad vocabulary/pattern/hints | Correct field or candidate set | | 429 | `rate_limited` | Rate, queue, or capacity limit | Honor `Retry-After` with jitter | | 503 | `capacity_unavailable`, `service_unavailable` | Processing capacity is unavailable, or a temporary backend/storage/detector issue | Honor `Retry-After`; never silently change models, language, or profile | ## Data handling and support Uploaded audio is deleted when processing completes or fails — inline and asynchronous alike. Async results and job metadata are owner-scoped and retained until your configured expiry: default 24 hours, configurable 1–72 hours in the console (both `api.omi.health` and `api.eu.omi.health` are served from eu-central-1); webhook jobs have an effective 9-hour minimum. Operational telemetry stores non-content usage and safety counters. Status: `https://status.omi.health/` Support: `hello@omi.health`. Include the response `request_id`.