Nova Bench: Benchmarking 11 LLMs
on NVIDIA DGX Spark

When NVIDIA released the DGX Spark — a compact desktop powered by the GB10 Grace Blackwell Superchip — one question stood out: how fast can it actually run large language models?

Nobody had published comprehensive benchmark data. Reviews showed one or two models. Marketing materials quoted theoretical TOPS. But no one had systematically loaded a dozen models and measured real-world performance across different tasks.

So I did. Over the course of a week, I loaded 14 models onto my DGX Spark, built a benchmarking framework, and ran 4 systematic benchmark suites covering general inference, code generation, context scaling, and vision.

The Hardware

The DGX Spark measures just 150mm × 150mm × 50.5mm — smaller than most routers — yet packs serious specifications under the hood.

The key differentiator is the unified memory architecture. CPU and GPU share the same 128GB pool connected via NVLink-C2C at 5x PCIe Gen5 bandwidth. Models that wouldn't fit on a discrete GPU's VRAM can load entirely into this shared memory space. That's why a 73GB model like Mistral Large 123B runs on hardware that costs less than a used car.

The Results

I tested 11 models with the same prompt using Ollama 0.18.3. Here's the full ranking by generation speed:

The sweet spot? 27–32B models. Fast enough to feel interactive at 10–12 tok/s, large enough to produce quality output. If I could only keep three models, it would be Gemma3 27B, Qwen2.5-Coder 32B, and Llama 3.1 70B.

🔬 Explore the Interactive Dashboard

Sort models by speed, compare code generation benchmarks, explore context scaling data, and examine vision model results — all with real data from my DGX Spark:

The Code Benchmark

I asked each model to write a Python function for finding the longest palindromic substring — a common interview problem that tests reasoning, code quality, and documentation.

The most striking result: DeepSeek-R1 generated 10,710 tokens of step-by-step reasoning for this single question. It spent 39 minutes thinking through the problem before producing its answer. Qwen3 produced 8,180 tokens. These reasoning models treat every question as a deep thinking exercise.

For practical daily coding, Qwen2.5-Coder 32B at 10.33 tok/s is the winner — purpose-built for code, fast enough to be interactive, and produces clean solutions with proper type hints and docstrings.

Context Scaling — The Surprise

The most interesting finding came from comparing short versus long prompts. With a 130-token complex business analysis prompt versus a 19-token simple prompt:

This means the DGX Spark gets relatively faster with more complex inputs — exactly what you want for research and production workloads with long contexts.

Vision — 90B on a Desktop

Llama3.2-Vision 90B — a 54GB multimodal model — loaded in 16.8 seconds and processed images at 3.47 tok/s for generation and 6.02 tok/s for prompt evaluation.

A 90 billion parameter vision model running on a desktop computer. Not fast enough for production serving, but absolutely viable for development, prototyping, and research. Try doing that on a consumer GPU.

Key Findings

Should You Buy a DGX Spark?

Yes, if you need to run 70B+ models locally for development, prototyping, or research. No consumer GPU can fit these models in VRAM. The Spark handles them natively.

Maybe, if you primarily work with 8B–32B models. An RTX 4090 or Mac Studio with 192GB RAM can match or exceed the Spark's speed for smaller models at a lower price.

No, if you need production serving throughput. At 4–5 tok/s for 70B models through Ollama, this is a development platform, not a serving platform.

Reproduce These Results

Everything is open source. Clone the repo and run on your own hardware:

Full dataset: huggingface.co/datasets/G3nadh/dgx-spark-benchmarks