This repository has no description
tools.hermes-voice: let Hermes end the call (end_call)
gpt-live-1 with client delegation has no client-side function tools: the
only model-to-page signal is session.delegation.created, which carries no
name or arguments. So the voice model does not hang up itself: goodbye,
"that's all", "hang up" and "I'm leaving" are delegated like any request,
and Hermes decides. BACKEND_NOTE tells it to call end_call once requested
work has been started or handed off, never while it still needs an answer
such as an approval.
end_call lives in the gateway plugin. Plugin handlers get the calling
session_id (model_tools.py passes it to registry.dispatch), and the
session context names the platform. The tool succeeds only on the API
server in a session whose lineage starts at a voice-* id: context
compression continues a session under a new id with the old one as
parent_session_id, so the prefix alone would not survive a long call.
Elsewhere it returns an error and changes nothing.
hermes-voice sees the tool in the run's events (tool.completed of
end_call without error) and passes an end_call item to the page instead
of a tool item. The page hands Hermes's reply to the voice model as
usual and closes the session once the voice has spoken it and been quiet
for 800 ms, at the latest after 30 s; close() now runs only once. A run
no page listens to (hang-up flush, page gone) only logs it.
author
Hermes
committer
lassulus
date
(Oct 17, 2026, 9:13 AM CEST)
commit
307216c4
307216c42c3f479721b03639f174836a99c6d5d8
parent
a2be883f
a2be883fa7fa81c18ba19648db9d66ede7182522
+109
-7
+22
-4
tools/hermes-voice/hermes_voice.py
+22
-4
tools/hermes-voice/hermes_voice.py
···
19
19
through its webhook platform), and the hermes-voice plugin's
20
20
`/assist approve|deny <code>` command answers it.
21
21
22
+
Hermes ends a call itself: when the user says goodbye, the voice model
23
+
delegates that like any request, and Hermes calls the plugin's end_call tool.
24
+
This server sees the call in the run's events and tells a listening page to
25
+
close the session once the voice has spoken the reply.
26
+
22
27
This server exists so neither secret reaches the browser: it exchanges the
23
28
page's SDP offer for a Live session with the OpenAI key, and it calls Hermes
24
29
with the API server's bearer token. That token is root-equivalent for the
···
100
105
# went away while a run is quiet (e.g. waiting for an approval).
101
106
PAGE_KEEPALIVE_SECONDS = 10
102
107
SESSION_ID_RE = re.compile(r"^[A-Za-z0-9_-]{1,64}$")
108
+
# Hermes's tool that ends the call (hermes-plugin/__init__.py).
109
+
END_CALL_TOOL = "end_call"
103
110
104
111
# Frontend persona, adapted from Hermes's own (tools/voice_live.py): role,
105
112
# style and a labelled delegation policy. The backend carries the real
···
119
126
- The user asks you to do, check, find, make, fix, run or remember anything.
120
127
- A correction changes work already requested.
121
128
129
+
- The user says goodbye, says that is all, asks you to hang up, or says they are leaving. Hermes ends the call after your reply; you may still say a short goodbye, but never end the call yourself.
130
+
122
131
Do not delegate to the backend when:
123
132
- The user greets you, makes small talk, or asks you to repeat a result already provided.
124
133
- You need a brief clarification to understand the request.
···
130
139
131
140
# Ephemeral system prompt for every delegated Hermes turn (Hermes layers system
132
141
# messages on top of its core prompt without storing them).
133
-
BACKEND_NOTE = """This turn is a delegation from a live spoken conversation. The user message is a speech transcript: it may contain mis-hearings, phonetic spellings of technical terms, hesitations and later corrections; use the latest intent. Your reply will be spoken aloud by a friendly voice model that paraphrases it: answer in warm, plain conversational sentences, keep it short (a few sentences unless the user asked for detail), no markdown, no lists, no code blocks, no URLs read out character by character. Do the work with your tools as usual; only the final facts need to be spoken. Report an action as done only after it actually succeeded. The user may hang up before your reply is spoken: when they asked for a result or a question somewhere else, such as their Matrix chat (matrix_dm_send), send it there yourself."""
142
+
BACKEND_NOTE = """This turn is a delegation from a live spoken conversation. The user message is a speech transcript: it may contain mis-hearings, phonetic spellings of technical terms, hesitations and later corrections; use the latest intent. Your reply will be spoken aloud by a friendly voice model that paraphrases it: answer in warm, plain conversational sentences, keep it short (a few sentences unless the user asked for detail), no markdown, no lists, no code blocks, no URLs read out character by character. Do the work with your tools as usual; only the final facts need to be spoken. Report an action as done only after it actually succeeded. The user may hang up before your reply is spoken: when they asked for a result or a question somewhere else, such as their Matrix chat (matrix_dm_send), send it there yourself. When the user says goodbye, says that is all, asks to hang up or says they are leaving, call end_call: only after any work they asked for has been started or handed off (for example a result promised to their Matrix chat), and never while you still need an answer from them, such as an approval. Your reply is still spoken before the call ends, so make it a short confirmation or goodbye."""
134
143
135
144
# Instead of BACKEND_NOTE for what the user said right before hanging up, when
136
145
# the voice model had not delegated it yet.
137
-
HUNG_UP_NOTE = """This turn comes from a live spoken conversation the user has already hung up. The user message is a speech transcript of what they said since the last request: it may contain mis-hearings, phonetic spellings of technical terms, hesitations and later corrections; use the latest intent. Nobody hears your reply. If the transcript asks for something, do it with your tools; when the user asked for a result somewhere, such as their Matrix chat (matrix_dm_send), send it there, and if you need something from them, ask in their Matrix chat. If it only says goodbye or thanks, or repeats a request already handled in this conversation, do nothing and reply with a short note saying so."""
146
+
HUNG_UP_NOTE = """This turn comes from a live spoken conversation the user has already hung up. The user message is a speech transcript of what they said since the last request: it may contain mis-hearings, phonetic spellings of technical terms, hesitations and later corrections; use the latest intent. Nobody hears your reply. If the transcript asks for something, do it with your tools; when the user asked for a result somewhere, such as their Matrix chat (matrix_dm_send), send it there, and if you need something from them, ask in their Matrix chat. If it only says goodbye or thanks, or repeats a request already handled in this conversation, do nothing and reply with a short note saying so. The call is over already: do not call end_call."""
138
147
139
148
140
149
def glossary_text(glossary):
···
318
327
319
328
def hermes_follow_run(config, run_id):
320
329
"""Follow a run's events; yield NDJSON-ready dicts (tool, delta, approval,
321
-
done, error). Leaving early only detaches: the run keeps going."""
330
+
end_call, done, error). Leaving early only detaches: the run keeps going."""
322
331
with hermes_request(
323
332
config, "GET", f"/v1/runs/{run_id}/events", timeout=EVENTS_READ_TIMEOUT
324
333
) as response:
325
334
for _, data in iter_sse(response):
326
335
payload = json.loads(data)
327
336
event = payload.get("event", "")
328
-
if event in ("tool.started", "tool.completed"):
337
+
if payload.get("tool") == END_CALL_TOOL:
338
+
# The plugin's tool fails outside a voice session; the page
339
+
# closes the call after speaking the reply.
340
+
if event == "tool.completed" and not payload.get("error"):
341
+
yield {"type": "end_call"}
342
+
elif event in ("tool.started", "tool.completed"):
329
343
status = "running" if event == "tool.started" else "completed"
330
344
if payload.get("error"):
331
345
status = "error"
···
457
471
with self.cond:
458
472
self.items.append(item)
459
473
self.cond.notify_all()
474
+
unheard = not self.pages or self.hung_up
475
+
if item["type"] == "end_call":
476
+
# Without a page the call has ended already.
477
+
log.info("run %s ends the call%s", self.run_id, " (no call)" if unheard else "")
460
478
self.ask_unheard()
461
479
462
480
def attach(self):
+60
tools/hermes-voice/hermes-plugin/__init__.py
+60
tools/hermes-voice/hermes-plugin/__init__.py
···
8
8
Matrix adapter. Sent messages are mirrored into the DM's session, so the
9
9
Matrix side sees what was asked when the user answers there.
10
10
11
+
end_call ends the voice call the session belongs to. The tool itself only
12
+
checks that it runs in a voice session and says so; hermes-voice sees the
13
+
call in the run's event stream and lets the page close the call once the
14
+
current reply has been spoken.
15
+
11
16
/assist approve|deny [code] answers an approval that hermes-voice asked in the
12
17
DM because no call could hear it. The gateway only dispatches commands of
13
18
allowed users (MATRIX_ALLOWED_USERS), so the answer is the user's own; the
···
38
43
},
39
44
}
40
45
46
+
END_CALL_SCHEMA = {
47
+
"name": "end_call",
48
+
"description": (
49
+
"End the current voice call after the current reply has been spoken. Only works in a "
50
+
"voice call (assist.lassul.us). Call it when the user says goodbye, says that is all, "
51
+
"asks you to hang up or says they are leaving, once any work they asked for has been "
52
+
"started or handed off; never while you still need an answer from them, such as an "
53
+
"approval. Your reply is still spoken before the call ends: keep it to a short "
54
+
"confirmation or goodbye."
55
+
),
56
+
"parameters": {"type": "object", "properties": {}},
57
+
}
58
+
59
+
# hermes-voice names the Hermes session of a call voice-<date>-<random>.
60
+
VOICE_SESSION_PREFIX = "voice-"
61
+
41
62
CHOICES = {"approve": "once", "deny": "deny"}
42
63
43
64
···
47
68
return send_message_tool(
48
69
{"action": "send", "target": "matrix", "message": args.get("message", "")}
49
70
)
71
+
72
+
73
+
def voice_session(session_id):
74
+
"""Whether the calling session is a hermes-voice call: an API server
75
+
session whose lineage starts at a voice-* session (context compression
76
+
continues a session under a new id, with the old one as parent)."""
77
+
from gateway.session_context import get_session_env
78
+
from hermes_state import SessionDB
79
+
80
+
if get_session_env("HERMES_SESSION_PLATFORM", "") != "api_server":
81
+
return False
82
+
session_id = session_id or get_session_env("HERMES_SESSION_ID", "")
83
+
db = SessionDB(read_only=True)
84
+
try:
85
+
seen = set()
86
+
while session_id and session_id not in seen and len(seen) < 100:
87
+
if session_id.startswith(VOICE_SESSION_PREFIX):
88
+
return True
89
+
seen.add(session_id)
90
+
session = db.get_session(session_id)
91
+
session_id = session and session.get("parent_session_id")
92
+
finally:
93
+
db.close()
94
+
return False
95
+
96
+
97
+
def end_call(args, session_id=None, **_kw):
98
+
from tools.registry import tool_error
99
+
100
+
if not voice_session(session_id):
101
+
return tool_error("end_call only works in a voice call; there is no call to end here.")
102
+
return json.dumps({"status": "The call ends once your reply has been spoken."})
50
103
51
104
52
105
def assist(raw_args):
···
91
144
schema=SEND_SCHEMA,
92
145
handler=send,
93
146
emoji="💬",
147
+
)
148
+
ctx.register_tool(
149
+
name="end_call",
150
+
toolset="voice_call",
151
+
schema=END_CALL_SCHEMA,
152
+
handler=end_call,
153
+
emoji="📴",
94
154
)
95
155
ctx.register_command(
96
156
"assist",
+2
-1
tools/hermes-voice/hermes-plugin/plugin.yaml
+2
-1
tools/hermes-voice/hermes-plugin/plugin.yaml
···
1
1
name: hermes-voice
2
2
version: 1.0.0
3
-
description: "hermes-voice's side in the gateway: sessions may write to the owner's Matrix DM (the Matrix home room), and /assist answers approvals of voice-call runs from Matrix."
3
+
description: "hermes-voice's side in the gateway: sessions may write to the owner's Matrix DM (the Matrix home room), voice calls can be ended by the agent, and /assist answers approvals of voice-call runs from Matrix."
4
4
author: lassulus
5
5
provides_tools:
6
6
- matrix_dm_send
7
+
- end_call
+25
-2
tools/hermes-voice/static/app.js
+25
-2
tools/hermes-voice/static/app.js
···
7
7
const APPEND_CHAR_LIMIT = 1400;
8
8
// Transcript deltas can trail the delegation event; wait for the last words.
9
9
const DELEGATION_SETTLE_MS = 400;
10
+
// Once Hermes ended the call (end_call item), the call closes when the voice
11
+
// has been quiet this long after speaking the reply, at the latest after
12
+
// END_CALL_MAX_MS.
13
+
const END_CALL_QUIET_MS = 800;
14
+
const END_CALL_MAX_MS = 30000;
10
15
const STORAGE_KEY = "hermes-voice";
11
16
12
17
const talk = document.getElementById("talk");
···
169
174
this.lastSpeaker = null;
170
175
this.delegation = null;
171
176
this.closed = false;
177
+
this.closing = false;
178
+
this.ending = null;
172
179
this.eventCounter = 0;
173
180
}
174
181
···
263
270
this.speaking = speaking;
264
271
talk.classList.toggle("speaking", speaking);
265
272
}
273
+
if (this.ending?.spoken && quiet * 100 >= END_CALL_QUIET_MS) this.close();
266
274
}, 100);
267
275
}
268
276
···
287
295
case "session.output_transcript.delta":
288
296
this.lastActivity = Date.now();
289
297
addTranscript("assistant", event.delta || "");
298
+
if (this.ending) this.ending.spoken = true;
290
299
break;
291
300
case "session.delegation.created":
292
301
this.delegate(event.delegation.id);
···
340
349
341
350
let answer = "";
342
351
let failure = null;
352
+
let endCall = false;
343
353
try {
344
354
const response = await api("/delegate", {
345
355
method: "POST",
···
389
399
APPEND_CHAR_LIMIT,
390
400
),
391
401
});
402
+
} else if (item.type === "end_call") {
403
+
endCall = true;
392
404
} else if (item.type === "delta") {
393
405
answer += item.text;
394
406
} else if (item.type === "done" && item.error) {
···
427
439
content: chunk,
428
440
});
429
441
}
430
-
setStatus("Listening");
442
+
if (endCall) this.endAfterReply();
443
+
else setStatus("Listening");
444
+
}
445
+
446
+
// Hermes ended the call: close it once the voice has spoken the reply just
447
+
// handed over.
448
+
endAfterReply() {
449
+
this.ending = { spoken: false };
450
+
this.endTimer = setTimeout(() => this.close(), END_CALL_MAX_MS);
451
+
setStatus("Ending the call…");
431
452
}
432
453
433
454
askApproval(id, item) {
···
522
543
}
523
544
524
545
close() {
525
-
if (this.closed) return;
546
+
if (this.closed || this.closing) return;
547
+
this.closing = true;
526
548
if (!this.send({ type: "session.close" })) {
527
549
this.finish("closed");
528
550
return;
···
554
576
if (this.closed) return;
555
577
this.closed = true;
556
578
clearTimeout(this.closeTimer);
579
+
clearTimeout(this.endTimer);
557
580
clearInterval(this.idleTimer);
558
581
clearInterval(this.probe);
559
582
this.hangUp();