Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 190 additions & 0 deletions communication/api.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
---
title: "API messages"
description: "Talk to your assistant from your own code"
---

Every other channel puts a person at one end — a phone, an inbox, a chat
window. This one puts your code there instead. Send your assistant a message
with an HTTP request, poll for its reply, and wire it into whatever you're
building. No browser, no phone number, no connected account — just your API
key.

It's the same assistant either way. Something you ask for over the API is
something you can follow up on in [Chat](/communication/console-chat) an hour
later, and it'll know what you mean.

## What you need

- **Your API key** and **your assistant's ID**, both from the
[Console](https://console.unify.ai).
- The base URL: `https://api.unify.ai/v0`
- A Bearer token header on every request:

```
Authorization: Bearer YOUR_API_KEY
```

<Note>
Assistants running in a [local deployment](/local-deployment/overview) don't
receive API messages — this channel is for assistants hosted by Unify.
</Note>

## Sending a message

`POST /messages` with the assistant's ID and your message:

```bash
curl -X POST https://api.unify.ai/v0/messages \
-H "Authorization: Bearer $UNIFY_KEY" \
-H "Content-Type: application/json" \
-d '{
"assistant_id": <your-assistant-id>,
"message": "Add milk to my shopping list."
}'
```

You get back a `message_id` straight away, before your assistant has done
anything with it:

```json
{
"info": {
"message_id": "msg_abc123",
"assistant_id": 42,
"message": "Add milk to my shopping list.",
"status": "processing",
"response": null,
"tags": [],
"attachments": [],
"response_tags": null,
"response_attachments": null,
"created_at": "2026-08-10T12:00:00Z",
"completed_at": null
}
}
```

## Polling for the reply

Your assistant works on the message in the background — it might answer in a
second, or go off and actually do the thing first. Poll
`GET /messages/{message_id}` until `status` flips from `processing` to
`completed`:

```bash
curl https://api.unify.ai/v0/messages/msg_abc123 \
-H "Authorization: Bearer $UNIFY_KEY"
```

```json
{
"info": {
"message_id": "msg_abc123",
"assistant_id": 42,
"message": "Add milk to my shopping list.",
"status": "completed",
"response": "Done! I've added milk to your shopping list.",
"tags": [],
"attachments": [],
"response_tags": [],
"response_attachments": null,
"created_at": "2026-08-10T12:00:00Z",
"completed_at": "2026-08-10T12:00:05Z"
}
}
```

`response` can come back `null` on a completed message. That's not an error —
your assistant decides whether a reply is warranted, the same way it does on
any other channel, and sometimes doing the task quietly is the right answer.

## From Python

The SDK wraps both calls:

```python
import unisdk

status = unisdk.agent.send_message(
assistant_id=<your-assistant-id>,
message="Add milk to my shopping list.",
)

status = unisdk.agent.get_message_status(status["message_id"])
print(status["status"], status["response"])
```

## Sending files

Upload each file first, then reference it in the message.

<Steps>
<Step title="Upload the file">
```bash
curl -X POST https://api.unify.ai/v0/messages/attachments \
-H "Authorization: Bearer $UNIFY_KEY" \
-F "file=@report.pdf" \
-F "assistant_id=<your-assistant-id>"
```

You get back the file's metadata:

```json
{
"id": "att_xyz789",
"filename": "report.pdf",
"gs_url": "gs://bucket/path/report.pdf",
"content_type": "application/pdf",
"size_bytes": 204800
}
```
</Step>
<Step title="Send the message with the attachment">
```bash
curl -X POST https://api.unify.ai/v0/messages \
-H "Authorization: Bearer $UNIFY_KEY" \
-H "Content-Type: application/json" \
-d '{
"assistant_id": <your-assistant-id>,
"message": "Please summarise this report.",
"attachments": [
{
"id": "att_xyz789",
"filename": "report.pdf",
"gs_url": "gs://bucket/path/report.pdf"
}
]
}'
```
</Step>
</Steps>

Files are capped at **25 MB** each. Your assistant can send files back too —
they arrive as `response_attachments` on the completed message, each with a
download URL.

## Tags

Tags are arbitrary strings you can hang off a message. Your assistant treats
them as opaque routing labels — it doesn't read anything into them — and
echoes them back on its reply as `response_tags`.

```bash
curl -X POST https://api.unify.ai/v0/messages \
-H "Authorization: Bearer $UNIFY_KEY" \
-H "Content-Type: application/json" \
-d '{
"assistant_id": <your-assistant-id>,
"message": "Summarise today'\''s sales figures.",
"tags": ["source:slack", "channel:#analytics"]
}'
```

That's what makes them useful for bridging: if you're relaying messages from
somewhere else, tag the inbound message with wherever it came from and the
reply tells you where to send it back.

<Tip>
Tags are how you keep one assistant serving several surfaces at once without
losing track of which reply belongs to which request.
</Tip>
17 changes: 13 additions & 4 deletions communication/behavior.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@ It won't send you play-by-play updates unless you want them, and in group
settings — Slack channels, meetings, calls with several people — it speaks
when addressed or when it genuinely has something to add.

In a room with several AI teammates — a [group or team
chat](/communication/group-chats), a Slack or Teams channel — exactly one of
them should answer any given message. Name the teammate you want and the
rest stay out of it; name nobody and whoever the work belongs to picks it
up. Assistants also mostly don't reply to *each other*: acknowledging a
teammate is noise, and two assistants trading acknowledgements is a loop
that costs real money.

## It can reach out first

Your assistant isn't purely reactive:
Expand All @@ -28,10 +36,11 @@ Your assistant isn't purely reactive:
tells you on an appropriate channel.
- **Scheduled work** — [tasks](/tasks/overview) you've scheduled can end
with a message, an email, or a call at the time you chose.
- **Gentle follow-ups** — if a conversation with your T-W1N goes quiet while
something needs your input, it may send a friendly follow-up email. This
re-engagement is specific to your T-W1N; hired teammates don't do it. Ask it
to stop and it stops.
- **Gentle follow-ups** — if you've been away from the platform for a while,
your T-W1N sends a short check-in email. At most three per quiet spell, and
then it leaves you alone until you're active again. This re-engagement is
specific to your T-W1N; hired teammates don't do it. Ask it to stop and it
stops, until you ask it to resume.
- **Ringing you** — when a live conversation beats a wall of text, it can
ring you on [Unify Meet](/communication/unify-meet), and falls back to chat
if you don't pick up.
Expand Down
7 changes: 7 additions & 0 deletions communication/console-chat.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,13 @@ brings you back to now.
For history *beyond* this one thread — email, SMS, WhatsApp, and every
other channel — see [Transcripts](/communication/recordings-and-transcripts).

## Beyond your own thread

This page is your one-to-one thread with one assistant. In an organization
the same composer also runs [group chats](/communication/group-chats) —
rooms shared with colleagues, teams, and several assistants at once, where
naming the teammate you want decides who answers.

## Always on, and honest when it isn't

Chat is your assistant's home channel: updates, links, and deliverables
Expand Down
51 changes: 40 additions & 11 deletions communication/developers/conversation-manager.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,7 @@ plus call-lifecycle events (`PhoneCallReceived`, `PhoneCallAnswered`,
`PhoneCallEnded`, `UnifyMeetStarted`…), per-utterance voice events
(`InboundPhoneUtterance`, `OutboundUnifyMeetUtterance`…), reactions,
voice-control events (`FastBrainNotification`, `VoiceInterrupt`), and
lifecycle events (`StartupEvent`, `TaskDue`, `InactivityFollowup`,
`PreHireMessage`).
lifecycle events (`StartupEvent`, `TaskDue`, `PreHireMessage`).

Two mapping layers connect the world to handlers:

Expand All @@ -56,7 +55,7 @@ Two mapping layers connect the world to handlers:
defines the `Medium` enum — the single source of truth for channel types
(`UNIFY_MESSAGE`, `EMAIL`, `SMS_MESSAGE`, `WHATSAPP_MESSAGE`,
`WHATSAPP_CALL`, `PHONE_CALL`, `UNIFY_MEET`, `GOOGLE_MEET`, `TEAMS_MEET`,
Slack/Discord variants, and both the delegated-Graph
`API_MESSAGE`, Slack/Discord variants, and both the delegated-Graph
(`TEAMS_MESSAGE`, `TEAMS_CHANNEL_MESSAGE`) and org-installed bot
(`MS_TEAMS_BOT_MESSAGE`, `MS_TEAMS_BOT_CHANNEL_MESSAGE`) Teams mediums…).
The two org-installed bot send tools are gated behind the assistant's
Expand Down Expand Up @@ -121,9 +120,8 @@ concern:
| [`renderer.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/renderer.py) | State → prompt snapshots and diffs |
| [`contact_index.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/contact_index.py) | Live per-medium message threads |
| [`notifications.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/notifications.py) | The `NotificationBar` surfaced in prompts |
| [`task_activation.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/task_activation.py) | `TaskDue` wake-ups and live task execution |
| [`inactivity.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/inactivity.py) | Re-engagement follow-ups |
| [`proactive_speech.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/proactive_speech.py) / [`speech_urgency.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/speech_urgency.py) | Breaking silence on calls / preempting the slow brain for urgent speech |
| [`task_execution.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/task_execution.py) | `TaskDue` wake-ups and live task execution |
| [`proactive_speech.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/proactive_speech.py) | Breaking silence on calls |
| [`comms_utils.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/comms_utils.py) | HTTP calls to the gateway's channel endpoints |
| [`managers_utils.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/managers_utils.py) | Manager init and `log_message()` → transcripts |

Expand Down Expand Up @@ -154,6 +152,32 @@ Outbound events published by the brain's own sends set
`suppress_slow_brain_wake` so the assistant isn't re-woken by its own
confirmations.

### The API message round trip

The [developer API channel](/communication/api) is the one send path that
doesn't originate a conversation — it completes a caller that is already
waiting, so it runs request/response rather than fire-and-forget:

1. **Inbound.** Orchestra's `POST /v0/messages` persists an `ApiMessage`
row and dispatches it to the adapters, which publish an `api_message`
envelope. `CommsManager.events_map` maps that thread to
`ApiMessageReceived`, whose handler parks the `api_message_id` and the
caller's tags on the `ConversationManager` as
`_pending_api_message_id` / `_pending_api_message_tags`.
2. **Outbound.** `send_api_response` reads that pending id, so it is only
meaningful while a call is in flight — with nothing pending it returns
`{"status": "ok", "note": "no pending api message"}` rather than opening
a new outbound conversation. It uploads any attachments, then
`comms_utils.complete_api_message()` issues
`PUT {ORCHESTRA_URL}/messages/{id}/complete`, which is what flips the
caller's poll from `processing` to `completed`.
3. **Record.** An `api_message_sent` event writes the transcript row,
anchored to the boss contact — the API exchange lands in the same
durable history as every other medium (`Medium.API_MESSAGE`).

Tags are opaque to the runtime; the prompt tells the brain to echo them
back by default so the developer can route the reply on their own side.

## Transcripts

[`unify/transcript_manager/transcript_manager.py`](https://github.com/unifyai/unify/blob/main/unify/transcript_manager/transcript_manager.py)
Expand All @@ -175,8 +199,13 @@ machinery](https://github.com/unifyai/unify/blob/main/unify/conversation_manager
in two directions. Downward: every in-flight `act(...)` is a steerable
handle the brain can `ask`, `interject`, `pause`, `resume`, or `stop` via
dynamically generated tools. Upward: when a user speaks while the slow
brain is mid-turn, `SpeechUrgencyEvaluator`
([`domains/speech_urgency.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/speech_urgency.py))
classifies the utterance and may cancel the running turn so the new input
takes priority — the runtime-level implementation of "you can always
interrupt."
brain is mid-turn, the input does **not** cancel the running turn. The
`Debouncer`
([`domains/utils.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/utils.py))
holds one running turn plus at most one pending turn — a new submission
replaces whatever was pending and starts when the current turn finishes.
Preemption used to exist (a `SpeechUrgencyEvaluator` that classified the
utterance and cancelled the running turn) and was removed: the queue of two
is the whole mechanism. What makes interruption feel immediate on a call is
the fast brain — barge-in and the interim turn — not turn cancellation; see
[Voice calls](/communication/developers/voice).
47 changes: 33 additions & 14 deletions communication/developers/voice.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -71,25 +71,44 @@ browser automation rather than SIP.
records the `spoken_prefix` actually delivered and the
`unheard_remainder`, so the slow brain knows exactly what the user did
and didn't hear and can re-weave the rest.
- **Urgency preemption.** If the user says something urgent while the slow
brain is mid-turn, `SpeechUrgencyEvaluator`
([`domains/speech_urgency.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/speech_urgency.py))
can cancel the running turn in favor of the new input.
- **No turn preemption.** Speaking mid-turn does not cancel the running
slow-brain turn. The new input queues as the single pending turn (see
[the conversation runtime](/communication/developers/conversation-manager)),
and the fast brain covers the gap. Urgency-based preemption was tried and
removed.
- **Proactive speech.** During long silences while work runs,
`ProactiveSpeech`
([`domains/proactive_speech.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/domains/proactive_speech.py))
decides whether the assistant should say something unprompted.

## Speakers and enrollment

Diarized speakers who aren't engaged contacts are transcribed as context
but don't get replies; the slow brain's `engage_speaker` /
`disengage_speaker` tools flip that, mirrored in the call manager's
engagement state. Voice profiles (`VoiceEnrollmentCaptured`,
`VoiceEnrollmentSuggested`, and `speaker_id.SpeakerTracker` in
[`medium_scripts/`](https://github.com/unifyai/unify/tree/main/unify/conversation_manager/medium_scripts))
let known voices be pinned across calls — enrolled embeddings ride along
in the dispatch metadata.
## Who said what

Attribution comes from the meeting roster and the platform's own
participant signals — never from matching voices.

In a browser meeting the two halves arrive separately: the transcriber
tags each finalised utterance with an anonymous diarization id (`S0`,
`S1`), while the meeting platform reports spans of who was speaking over
the Recall relay.
[`meet_speaker_map.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/meet_speaker_map.py)
pairs them by **overlap of time spans** rather than by sampling who is
speaking when a final lands — by then the speaker has stopped and the
platform has already sent `speech_off`, so instantaneous sampling fires
only when someone talks over the previous speaker's tail, exactly when
it's most likely to name the wrong person. Votes accumulate rather than
binding on first sight, because diarization ids are per-call and drift.
Elsewhere, an unresolved turn is attributed to the call contact and the
slow brain infers the real speaker from the conversation.

Enrolled voice embeddings are **not** compared against live audio to
decide who is speaking. That path was removed after production audio
showed the cosines conflated distinct speakers — including the
assistant's own TTS voice — about as often as they separated them, so a
matched label was as likely to be wrong as right. `SpeakerTracker`
([`speaker_id.py`](https://github.com/unifyai/unify/blob/main/unify/conversation_manager/speaker_id.py))
remains as capture-only machinery: clustering keeps auto-enrollment
single-voice pure and counts distinct voices behind
`VoiceEnrollmentCaptured` / `VoiceEnrollmentSuggested`.

## Hang-up semantics

Expand Down
Loading
Loading