Jarvis: turn speech into text on a NUC, then let a model think

The starting point: my thoughts are ahead of my typing

When I use AI, I often know what I want to ask before I have finished typing the prompt. Jarvis starts with that friction: could I speak naturally, let the NUC on my desk transcribe my Chinese, and hand the text to a model without breaking my flow?

I want a voice input I would actually choose in everyday work. Capture what I say, pass the text to a suitable model, and bring the answer back to the desktop. Whether this saves time over typing depends on recognition errors, corrections, and waiting as well as speaking speed.

The decisions: separate transcription from reasoning

I want speech-to-text to run on the local hardware. Reasoning can happen elsewhere: use Ollama when a local model can handle the task, and Claude when cloud capability is useful. The experiment is about whether that division works in daily use, with model choice kept flexible.

Local transcription does not mean the whole conversation stays on the device. Choosing a cloud model sends the transcript and relevant context to the cloud, and the current edge-tts speech output also requires a network. The user should be able to understand and choose where their information goes.

The implementation: three steps from speech to response

The Python and Electron prototype connects speech recognition, model responses, and a desktop interface. First, Breeze / faster-whisper transcribes speech locally. Next, the text goes to the selected Ollama local model or Claude. Finally, the HUD displays progress and the reply, with speech output connected. Model switching is implemented; automatically choosing the best model for a task remains a research direction.

I define the use cases, priorities, and acceptance goals, work with AI coding tools on implementation, and use my experience with the prototype to guide changes. Interface feedback matters: I need to know whether it is listening, thinking, or waiting for me to intervene.

Next: measure the path from speaking to a usable prompt

This is a private prototype without a public walkthrough yet. The code connects local transcription with local or cloud reasoning. End-to-end stability, latency, and any time saved over typing still need comparable hardware tests. The cover illustrates the intended use; it is not a screenshot of the running application.

My next experiment will use the same Chinese tasks for typed and spoken input: time to a usable prompt, number of corrections, and model response time. Recording the NUC configuration, model versions, and recording conditions will help explain where voice is useful and where it still falls short.