JARVIS

A voice assistant in the spirit of Tony Stark’s J.A.R.V.I.S. — it listens, thinks with any model on build.nvidia.com/models, and talks back using free text-to-speech.

Any NVIDIA model – one nvapi- key unlocks the whole catalog (Llama, Nemotron, DeepSeek, GPT-OSS, Qwen, Mistral, Gemma, Kimi …). Switch models by typing an id.
Free TTS, three ways – browser voices (offline, zero setup), Microsoft Edge neural voices (edge-tts, no key), or NVIDIA’s own Magpie TTS (same key).
Free speech-to-text – the browser’s Web Speech API, with an optional “Jarvis” wake word.
Tools – time, weather (Open-Meteo, no key), a safe calculator and system diagnostics, called automatically via function calling.
Streaming – replies stream in and are spoken sentence-by-sentence while still generating.
Web UI and a terminal CLI. No openai SDK dependency – plain HTTPS.

1. Get a free NVIDIA API key

Go to https://build.nvidia.com and sign in (a free NVIDIA Developer account – email only, no card).
Open https://build.nvidia.com/settings/api-keys → Generate API Key. (Or open any model page, e.g. https://build.nvidia.com/meta/llama-3_3-70b-instruct, and click Get API Key.)
Copy the key. It starts with nvapi-.

That single key works for every model in the catalog and for NVIDIA’s speech models.

2. Install

git clone https://github.com/Onlinecodestimming/JARVIS.git
cd JARVIS
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env               # Windows: copy .env.example .env

Edit .env and paste your key:

NVIDIA_API_KEY=nvapi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
JARVIS_MODEL=meta/llama-3.3-70b-instruct
JARVIS_TTS_ENGINE=browser

3. Run

Web UI (recommended – voice in and out)

python -m jarvis --serve

Open http://localhost:8000. Click the microphone (or press Space) and talk, or type. Use Chrome or Edge for the best speech recognition support.

Terminal

python -m jarvis                          # interactive chat, spoken replies via edge-tts
python -m jarvis --ask "status report"    # one question, then exit
python -m jarvis --quiet                  # text only
python -m jarvis --model deepseek-ai/deepseek-r1

Free text-to-speech: how it works and how to set it up

JARVIS never needs a paid voice service. Pick an engine with JARVIS_TTS_ENGINE in .env (or the Engine dropdown in the web UI – changes apply instantly).

Engine	Cost	Needs internet	Needs API key	Quality	Where it runs
`browser`	free	no	no	depends on OS voices	in your browser (web UI only)
`edge`	free	yes	no	excellent, very natural	on the server via `edge-tts`
`nvidia`	free tier	yes	yes (same key)	excellent	NVIDIA cloud (Magpie TTS)

`browser` – zero setup (default)

Uses the Web Speech API built into Chrome, Edge, Safari and Firefox. Nothing is sent anywhere; it uses the voices installed on your computer.

The UI lists every voice your browser offers and pre-selects a British male voice if one exists (e.g. Google UK English Male, Daniel on macOS, Ryan on Windows).
Want better voices? On Windows: Settings → Time & Language → Speech → Add voices. On macOS: System Settings → Accessibility → Spoken Content → System voice → Manage voices (the “Enhanced”/”Premium” voices are excellent and free). Chrome’s Google … voices are cloud-backed but also free.

`edge` – best free quality

Uses edge-tts, which talks to the same neural voice service Microsoft Edge’s “Read aloud” uses. No account, no key.

JARVIS_TTS_ENGINE=edge
JARVIS_VOICE=en-GB-RyanNeural      # a JARVIS-like British voice

Other good picks: en-GB-ThomasNeural, en-GB-SoniaNeural, en-US-GuyNeural, en-US-AriaNeural, en-AU-WilliamNeural. List all 300+ voices (in 70+ languages) with:

python -m jarvis --list-voices --tts edge
python -m jarvis --say "Good evening, sir." --tts edge          # test it
python -m jarvis --say "Hello" --tts edge --output hello.mp3    # save to a file

In terminal mode JARVIS plays the audio with whatever is installed: ffplay (ffmpeg), mpv, mpg123, vlc, afplay (macOS) or PowerShell (Windows). If none is found it falls back to the OS speech engine (say / espeak-ng / Windows SAPI). The web UI plays audio directly in the browser and needs nothing extra.

`nvidia` – Magpie TTS from build.nvidia.com

Uses NVIDIA’s magpie-tts-multilingual model with the same NVIDIA_API_KEY. Nothing else to install.

JARVIS_TTS_ENGINE=nvidia
JARVIS_VOICE=Magpie-Multilingual.EN-US.Aria

python -m jarvis --list-voices --tts nvidia

NVIDIA sometimes rotates speech endpoints. If you get a 404, open the model page above, click Try API, copy the function id from the URL shown there and set NVIDIA_TTS_FUNCTION_ID=<id> in .env.

Speech recognition (voice input)

Voice input uses the browser’s free Web Speech API (Chrome/Edge/Safari). No key needed.

Push to talk: click the mic or press Space while not typing.
Hands-free: enable Hands-free in the sidebar. With wake word on, only sentences starting with “Jarvis” (or “Hey Jarvis”) are sent; the rest is ignored.
JARVIS stops listening while it is speaking so it does not hear itself, then resumes.
Set the language with JARVIS_TTS_LANGUAGE (e.g. en-GB, de-DE, es-ES).
Firefox does not ship speech recognition; typing and speech output still work.

Using any model from build.nvidia.com

Every model on https://build.nvidia.com/models is served from one OpenAI-compatible endpoint, https://integrate.api.nvidia.com/v1. JARVIS only needs the model id shown on the model’s page (it is the model= value in the page’s code sample).

JARVIS_MODEL=deepseek-ai/deepseek-v3.1

Or change it on the fly: type/pick an id in the Model box in the web UI (applies to your next message) or use /model <id> in the terminal.

python -m jarvis --list-models      # prints every model id your key can use

Some ids to try:

Model	Notes
`meta/llama-3.3-70b-instruct`	fast, reliable, great default; supports tools
`nvidia/llama-3.1-nemotron-70b-instruct`	NVIDIA-tuned Llama, very good conversationalist
`nvidia/llama-3.3-nemotron-super-49b-v1.5`	strong reasoning + tools
`deepseek-ai/deepseek-v3.1`	top-tier general model
`deepseek-ai/deepseek-r1`	reasoning model – its thinking shows in a collapsible panel
`openai/gpt-oss-120b` / `openai/gpt-oss-20b`	OpenAI’s open-weight models
`qwen/qwen3-235b-a22b`	excellent multilingual
`mistralai/mistral-small-24b-instruct`	quick and light
`google/gemma-3-27b-it`	compact and capable

Tool calling (weather, time, calculator, diagnostics) works with any model that supports OpenAI-style function calling; with models that don’t, JARVIS still chats normally. Disable tools with JARVIS_TOOLS=false.

Model-specific switches can be passed through as JSON, e.g. to turn off Nemotron’s thinking mode:

JARVIS_EXTRA_BODY={"chat_template_kwargs": {"thinking": false}}

Configuration reference

All settings live in .env (see .env.example for full comments).

Variable	Default	Meaning
`NVIDIA_API_KEY`	–	Required. Your `nvapi-…` key
`JARVIS_MODEL`	`meta/llama-3.3-70b-instruct`	Any id from build.nvidia.com/models
`JARVIS_TEMPERATURE` / `JARVIS_MAX_TOKENS`	`0.6` / `1024`	Sampling
`JARVIS_TOOLS`	`true`	Allow tool calls
`JARVIS_EXTRA_BODY`	–	Extra JSON merged into each request
`NVIDIA_BASE_URL`	`https://integrate.api.nvidia.com/v1`	Change only for self-hosted NIMs
`JARVIS_TTS_ENGINE`	`browser`	`browser`, `edge` or `nvidia`
`JARVIS_VOICE`	engine default	Voice name for the engine
`JARVIS_TTS_LANGUAGE`	`en-US`	Language for STT/TTS
`JARVIS_TTS_RATE`	`1.0`	Speaking speed
`NVIDIA_TTS_FUNCTION_ID`	Magpie multilingual	Override NVIDIA TTS endpoint
`JARVIS_NAME` / `JARVIS_USER_TITLE` / `JARVIS_USER_NAME`	`JARVIS` / `sir` / –	Persona
`JARVIS_HOST` / `JARVIS_PORT`	`0.0.0.0` / `8000`	Web server

HTTP API

The web UI talks to a small JSON API you can use from anything else:

Endpoint	Purpose
`POST /api/chat` `{message, session_id?, model?}`	Server-sent events: `token`, `thinking`, `tool`, `done`, `error`
`POST /api/speak` `{text, engine?, voice?}`	Returns MP3 (edge) / WAV (nvidia) audio, or cleaned text for `browser`
`GET /api/voices?engine=edge`	Available voices
`GET /api/models`	Model ids your key can use
`GET /api/config`, `GET /api/health`	Status
`POST /api/reset` `{session_id}`	Forget a conversation

curl -N localhost:8000/api/chat -H 'Content-Type: application/json' -d '{"message":"status report"}'
curl localhost:8000/api/speak -H 'Content-Type: application/json' -d '{"text":"Good evening, sir.","engine":"edge"}' -o hi.mp3

Project layout

jarvis/
  assistant.py   conversation loop: streaming, history, tool calls
  nvidia.py      OpenAI-compatible client for integrate.api.nvidia.com
  tts.py         browser / edge-tts / NVIDIA Magpie engines
  tools.py       time, weather, calculator, system status
  persona.py     the JARVIS system prompt
  server.py      FastAPI app + SSE endpoints
  cli.py         terminal interface
  static/        single-page web UI (no build step)
tests/           47 tests, run with:  pip install -r requirements-dev.txt && pytest

Troubleshooting

“NVIDIA rejected the API key (401)” – the key must start with nvapi-; regenerate at https://build.nvidia.com/settings/api-keys. Make sure .env is in the folder you run from.
“model not found (404)” – copy the id exactly from the model page (ids are case-sensitive and contain a /). python -m jarvis --list-models shows valid ids.
429 rate limit – the free tier has per-minute limits; wait a moment or pick another model.
No sound in the browser – browsers require a click on the page before audio can play; click Test voice once. Check the Speak replies switch is on.
Microphone not working – allow mic access in the address bar; on non-localhost hosts the page must be served over HTTPS for the Web Speech API to work.
Edge TTS errors – it needs outbound internet to speech.platform.bing.com; on locked-down networks use browser or nvidia instead.

License

MIT