Voice¶
Voice lets roamd tell you when something on your computer needs you, and lets you speak a command instead of typing it. It's off until you turn it on, one computer at a time.
Private mode: everything stays on your device¶
your computer our website YOUR BROWSER
------------- ----------- ------------
terminal output --[ LOCKED ]--> passes --[ LOCKED ]--> unlocked here
through |
v
"is this worth saying?"
no -> stays quiet
yes -> shown + spoken
(voice on this
device only)
What it says:
- Errors, like "Error: permission denied".
- Questions waiting for you, like a password prompt or "Continue? [Y/n]".
- Long commands finishing, like "cargo finished after 4 minutes" (you choose how long counts as long).
- Your own words: any line containing a word you listed on the Account page.
Everything else (ordinary output, the command you just typed, full-screen programs like
vim or top) stays quiet.
Speaking commands. Tap Dictate and the Compose box slides up. Say what you want: your words appear in the box as you talk, a moment behind you. The last few words are a first guess and can change as more is heard; each phrase settles when you pause. Tap Dictate again (in the box) to finish. Closing the box (X or Escape) also stops listening, and what you'd already said stays in it for next time. Sending empties it: words still being worked out when you tap Send are dropped, so the box is empty next time. Nothing reaches the terminal until you press Send or Send and run, so a misheard word can't run by itself, and you can fix it first. A pause doesn't end it, so you can take your time. Your phone keyboard's own microphone works in the box too.
tap Dictate --> Compose opens --> beep --> you talk (a pause does not end it)
|
v
YOUR BROWSER turns it into words
(the sound never leaves your device)
|
Compose box <-------- words as they come ------+
(last words may change; each settles at a pause)
|
tap Dictate again to stop, check it, then Send (no Enter) or Send and run
The first time you tap Dictate, a short note explains that it uses a very small speech-to-text model running in your browser, that no recordings are sent anywhere, and that each one is deleted as soon as the model has turned it into words. Nothing listens until you press OK, and you won't see the note again.
Between Dictate recordings, roamd keeps the granted microphone access but disables its audio: it does not listen while idle. This avoids asking again on every tap while the app stays open. Switching away or closing the app releases that access (unless Hands-free is on). Your browser may ask again when you return or reopen it.
Allow -> Dictate -> Stop -> muted -> Dictate again
|
leave app -> release access
Why it's private: your browser already has the unlocked terminal text, so it does all the deciding and speaking itself. We never receive the text or your voice. Turning your voice into words happens in your browser too, with a small speech model that runs right there, so your voice isn't sent anywhere. roamd only uses voices that run on your device. Some browsers offer "online" voices that send the words away to be spoken; roamd never uses those.
Limits¶
- Deciding what's "important" uses simple rules, so it can miss things or speak up for something harmless.
- Dictate works in any browser that can use a microphone, Brave included, but hasn't been tried on real phones yet. The first time, your browser downloads a small speech model and keeps it. It copes best with common commands: uncommon words and spelling can be misheard.
- Some browsers have no on-device voice. Then messages are shown under the terminal but not spoken.
There is no AI in Voice¶
Voice used to have an optional Smart mode that sent recent terminal text to an AI service. It has been removed: the AI summaries that matter now happen on your own computer, in the History view, using your own AI app, and nothing from them passes through us. Voice speaks only what its simple rules pick out, and all of that stays in your browser. A Smart setting saved in your vault long ago is simply ignored.
Hands-free: speak commands through your headset¶
Turn on Hands-free under the terminal, put your phone in your pocket, and work by voice:
you press the headset button
|
v
beep - you say "git status" - beep
|
v
YOUR BROWSER turns the sound into words
(the sound never leaves your device)
|
v
your phone says: "I heard: git status. Say run it, or cancel."
|
+-- "run it" --> the command runs, and Voice tells you
| what happened, as usual
+-- "cancel" --> nothing runs
- Nothing runs without "run it". Every command is read back first, so a misheard word can't do anything by itself. If roamd doesn't understand your answer, it asks once more, then cancels.
- Your voice never leaves your device. A small speech model runs inside your browser, so the sound isn't sent to us, to your computer, or anywhere else.
- The first time, your browser downloads the speech model from our website, checks that roamd signed it and that the file is exactly the one that was signed, and keeps it. Until that's finished, roamd says it's getting speech ready. After that it works without downloading again.
- It waits for you to finish. Once you stop talking, a small bar under the terminal counts down before it takes what you said; carry on talking and it keeps listening.
- It learns the room. It keeps track of how noisy the background is as it goes, so it follows you between a quiet room and a noisy street, and isn't fooled by steady noise. If nobody speaks for a while, it gives up and says it didn't catch that.
- No headset button? Tap Hands-free again instead of pressing it. Double-tap to turn it off. It also switches off when you leave that computer's terminal.
- On a computer with a keyboard, press Ctrl and Alt together and let go (with no other key) and it listens, just like a headset button. Under Account → Hands-free shortcut you can pick Ctrl + Shift instead, or turn the shortcut off. The choice is kept on that computer only, so your phone and your laptop can differ; phones and tablets use the headset button and don't show the setting.
- It keeps the screen awake while it's on, and asks for the microphone the first time.
Limits: the model is small so it runs on a phone: it's good with common commands but can mishear unusual words, file names and spelling. It's slower on old phones, and a sentence can take a few seconds. Headset buttons depend on the phone and browser, and this hasn't been tried on real phones yet.
On a phone¶
To keep the screen for the terminal, the voice buttons sit behind the More button (three dots), the first key in the row of keys under the terminal. Tap it and they open as icons right after it, with the keys moving along to make room:
closed: (...) Esc Tab Ctrl ...
open: (...) ( speaker ) ( headset ) ( mic ) ( pencil ) ( <> ) Esc Tab ...
More Voice Hands-free Dictate Compose Snippets
Compose opens a big box at the bottom (on a touch screen it fades in, already clear of the keyboard, and closing it or sending puts the keyboard away), for writing a longer prompt with a cursor you can tap into; it has its own Dictate, and Send and Send and run buttons (see Compose in the manual).
Tap More again to close them. Opening or closing them never moves the terminal. A lit icon is switched on. If you've turned the key row off, More and the buttons sit in a row of their own instead. On a computer the buttons show their names as well.
Private mode¶
All processing is in the web app, after SSH decryption. No terminal text, audio or transcript is sent to the hub or any third party.
SSH channel data (decrypted in the browser)
|
+--> terminal (display)
+--> narrator (only while Voice is on for this machine)
- strip control sequences (colours, cursor, titles),
honouring carriage-return redraws
- drop the echo of the command the user typed
- wait for a short quiet period, then judge the burst
- full-screen (alternate buffer) programs: ignored
|
v
messages --> on-screen log (ARIA live region)
--> speechSynthesis, local voices only
| Rule | Trigger |
|---|---|
| Error | A new line matching common failure words (error, fatal, failed, denied, panic, traceback…); one message per burst with a count of the rest |
| Waiting | The unfinished last line looks like a prompt ([y/N], (yes/no), password:, a trailing ?); said once |
| Finished | The shell prompt returns after a command that ran longer than the user's threshold |
| Keyword | A new line (not already reported as an error) containing one of the user's keywords; a few per burst |
Messages are length-capped. Input the narrator sees is limited to what the user typed into this terminal, and is only used to recognise the echoed command.
Voices and speech recognition¶
- Speech output: only voices the browser marks as local (
localService) are used. If there are none, messages are shown but not spoken. - First use: the first press of Dictate opens a note instead of recording. The microphone isn't opened and the model isn't fetched until OK; OK records that you've seen it (kept with your other settings, so it follows you to your other browsers) and starts the recording the press asked for. Closing the note any other way records nothing and shows it again next time.
- Beeps: a higher beep when listening starts and a lower one when it stops, each about a tenth of a second, fading in and out so they don't click. The ear, and a phone's small speaker even more, hears a higher note as louder, so the start beep is played more softly and the two sound as loud as each other. Both go through one sound output.
- Speech input: Dictate records in the page (Web Audio) until you tap again; a pause
doesn't end it, and a long limit only guards against a forgotten microphone. It resamples
to 16 kHz and writes out each phrase as it ends (after about 0.6 seconds of quiet, or after
15 seconds of continuous speech), handing each to a
speech engine running in a Web Worker, as hands-free does, so the page never freezes. The audio
never leaves the browser, and the working copies of each recording are wiped as soon as
the words are out, in the page and in the engine's memory. The engine favours common shell words plus your listed words,
and the clip gets a little silence either side, because very short commands are heard
worse when they start and stop sharply. The browser's own recogniser
(
SpeechRecognition) is never used, so behaviour is the same in every browser. - Words as they come: Dictate opens the Compose sheet and the words go there, never straight to the terminal. Each finished phrase is shown as soon as it is written out. While a phrase is still being spoken, a copy of it so far is written out again now and then (under a second apart, and only when the engine has nothing else to do, so finished phrases are never held up) and shown as a guess; the finished phrase replaces it. A guess that comes back after its phrase has ended is thrown away. The sheet owns only the stretch of text this dictation wrote, starting at the cursor, and replaces just that each time, so the text around it is left alone. On a touch screen the box isn't focused when Dictate opens it, so the on-screen keyboard stays down. Opened any other way on a touch screen, the box takes the cursor during the tap itself (the only time the on-screen keyboard will open for it), so the sheet fades in where it sits rather than sliding up: a box still below the screen at that moment is left under the keyboard.
tap Compose --> sheet drawn in place (fading in) --> box takes the cursor
|
v
keyboard opens and moves the display up to show the box
On a touch screen, closing or sending lets go of the cursor instead of giving it back to the terminal (a pop-up on a computer gives it back to whatever opened it), so the keyboard goes away with the sheet.
Close / Send --> touch screen? --yes--> cursor let go ---------> keyboard goes away
|
no --> cursor back to the terminal
Closing the sheet ends Dictate as a tap would (or, if speech is still
getting ready, stops it before the microphone opens); the last phrase is still written out
into the kept draft, and never onto the terminal. Sending while Dictate is on drops what is still on its way
(dropDictation: the words in flight reach neither the cleared draft nor the terminal), and each opening forgets where a
cut-short dictation was writing. Sending is the usual Compose path (line breaks joined in a plain shell,
kept for an AI app); Enter is always yours. Completed Dictate recordings keep the
stream with its tracks disabled, so idle audio is silence and the next tap can reuse
the grant without another getUserMedia call. A hidden page or pagehide stops Dictate
and releases its stream unless Hands-free owns it. Failed or cancelled starts release
it too. Ended tracks are replaced on the next tap. Permission persistence across
app restarts remains a browser decision; automated tests do not verify real iOS prompts. Speech is made ready before listening starts, so
loading it never costs the first words.
- Enter after "run it": Run on a snippet (and "run it") types the command into the terminal, then
sends Enter as a separate keystroke a moment later. Text that arrives together with its
Enter looks like a paste to full-screen programs such as Claude Code, which would keep
the Enter as a new line instead of running the command.
- Microphone: the site's permissions policy allows the microphone for roamd's own page
only, and turns off camera, location, USB and payment features entirely.
Settings¶
Per-machine Voice on/off, keywords, the "long command" threshold and whether the first-use Dictate note has been accepted are one vault item: encrypted in the browser like the rest of the vault and synced to the account's other browsers. The hands-free keyboard shortcut is the exception: it is kept in this browser only, per device (see Hands-free).
Limits¶
- The rules are heuristics: they can miss or over-report. History recall and tab completion hide the exact command, so the next line is assumed to be its echo.
- On-device recognition is a new browser feature: not yet tested on real devices.
Hands-free¶
headset button (Media Session play, pause or next track) or a tap on Hands-free
-> browser records one sentence (loudness-based end of speech:
a stretch of quiet after speech, shown as a countdown),
16 kHz mono
-> speech engine in a Web Worker (WebAssembly, on the CPU),
favouring common shell words + the user's Voice keywords
-> text, still in the browser
-> read back: "I heard: ... Say run it, or cancel."
-> second sentence, same path
matches "run it" (or a close synonym) -> typed into the
terminal + Enter
matches "cancel" -> nothing sent
anything else -> asked once more, then cancelled
- Confirmation is mandatory: no transcript is ever sent to the terminal without a spoken confirmation that is itself transcribed in the browser.
- Audio and transcripts never leave the browser: no machine, hub or third party receives them. Nothing is stored.
- The engine and the model are separate. The engine is an open-source WebAssembly speech runtime shipped with the app. It runs a model file, and the page reaches it through one small interface (samples in, words out), so a different model, or a different engine, is an entry in one table and not a rewrite. The model is an open speech model.
- Noise learning: the page keeps the loudness of the last few seconds; the background is the quietest recent level, so it follows a change of place, and talking (which has gaps) doesn't drag it up. Speech is anything well above that background. The sentence ends after a stretch of quiet following speech, and the countdown bar shows how much of that stretch is left (it only appears once the quiet has lasted a moment, so the gaps between words don't flicker it). If no speech starts within a limit, or the sentence runs past another, the turn ends and an empty one is reported as not caught.
- Model supply and signing: the model is downloaded from roamd's own download site the first time speech is used, and kept in the browser's cache storage. The site also publishes a short list (version, file name, SHA-256) signed with the roamd release key, the same key every machine trusts for updates. The page checks that signature against the key built into the app, then the file against the list, before it uses anything. A model that fails either check is refused with a message saying which, and nothing is cached. If the site can't be reached, the last list that passed is used. A list older than one the browser has already used is refused even if it was once signed, so an old list can't be replayed to bring back an old model. Because the list is signed, a new or better model ships with a new list and no app release.
- Who can use it: any browser that can use a microphone. Nothing here depends on the machine's roamd, so an older roamd there doesn't matter.
- Keyboard shortcut: on devices without a touch screen, the page watches for the chosen modifier pair (Ctrl+Alt by default, or Ctrl+Shift) pressed and released together. It fires on the first release, only if no other key was touched meanwhile, so shortcuts like Ctrl+Alt+Delete and the AltGr key (which keyboards report as Ctrl+Alt) never trigger it. It takes a turn only while hands-free is on, as the headset button does. The pair is stored in this browser's local storage, not the account, so each device keeps its own.
- Browser side: microphone via
getUserMedia(echo cancellation and noise suppression on), a looping silent audio element so headset buttons are delivered to the page, a screen wake lock while on. The local voice reads back; if none exists, the read-back is shown only. - Limits: the model is small and trades accuracy for speed; uncommon words and spelling are unreliable, short commands most of all. The first use needs a download and storage for it. Headset button delivery varies by platform and is untested on real devices.
Phone layout¶
width <= 600px (matchMedia, follows rotation)
|
+-- key row shown? --yes--> "More" is the key row's first key
| open: Voice, Hands-free, Dictate, Compose, Snippets drawn
| right after it, before Esc (same scrolling row)
|
+-- no -----------------> "More" and the buttons in the voice row
- The buttons are drawn in one place only, so their ids and names never repeat; closed, they aren't in the page at all. Wider screens always draw them in the voice row.
-
With the buttons in the key row, the voice row stays empty, so it gets no
more-openmargins: toggling More leaves the key row, the footer and the terminal the same height. -
Each button keeps its word in a visually hidden span, so its accessible name is unchanged; wider screens show the icon and the word.
- On/off is
aria-pressed, drawn as the accent border and fill.