NPU benchmark

The NPU benchmark measures the performance of your system's neural processing unit, a dedicated chip designed to accelerate machine learning and AI workloads. The NPU score appears in your Novabench results only when compatible hardware is detected.

What is an NPU?

A neural processing unit (NPU) is a specialized processor optimized for machine learning inference, the process of running trained models to produce predictions or outputs. Unlike CPUs and GPUs, which handle general-purpose computation, NPUs are designed specifically for the matrix math and tensor operations that machine learning models rely on.

NPUs appear in an increasing number of devices, including recent laptops (marketed as "AI PCs" or "Copilot+ PCs"), and Apple Silicon Macs (as the Neural Engine). An NPU is often used to run AI-powered application functionality such as image recognition, natural language processing, background noise removal, or camera effects. The NPU is not necessarily more capable than the system's CPU or GPU at running these models, but it is often more power-efficient.

What the NPU test measures

The NPU benchmark tests two aspects of neural processing performance:

Throughput (TOPS)

TOPS stands for Tera Operations Per Second, the number of trillion operations the NPU can perform each second. Higher TOPS indicates a faster NPU that can process AI workloads more quickly. This metric reflects the raw computational capacity of the neural processor.

Novabench measures throughput using SSD-ResNet50, a realistic object detection model. Parallel inference sessions run simultaneously, stressing the NPU's ability to sustain concurrent workloads rather than just single-stream performance.

TOPS recorded by Novabench are likely to be lower than TOPS values advertised by vendors, which are often theoretical maximum values that cannot be attained by real models.

Inference latency

Inference latency measures how long the NPU takes to process a single input through a neural network model. Lower latency means faster response times for AI features that need real-time results, such as live camera effects, voice recognition, and gesture detection.

Novabench measures latency using Selfie Segmentation, modelling the real-world scenario of segmenting every frame of 60 fps video. The result reflects average per-frame inference time; lower is better.

Score calculation

The NPU score combines throughput and latency into a single number. Both metrics must be successfully measured for a score to appear. If either measurement fails (for example, if the NPU is not supported or not available), the NPU score is omitted from your results.

Supported hardware

The NPU benchmark runs automatically when Novabench detects a supported neural processing unit. Novabench supports NPUs through several runtime providers:

Systems without an NPU

If your system does not have a supported NPU, the benchmark skips the NPU test entirely. Your overall Novabench Score is calculated from the four core components (CPU, GPU, Memory, Storage) without any penalty for the missing NPU score.

The NPU score is a supplemental metric. Systems with and without NPUs are still fully comparable on the core benchmark components.

When the NPU score matters

NPU performance is relevant when you use (or plan to use) AI-powered features that run locally on your device:

• Windows AI features: Copilot, Windows Studio Effects (background blur, eye contact, auto framing), live captions, and other AI-powered Windows features use the NPU when available
• macOS AI features: on-device Siri processing, image analysis, and Core ML-based applications may use the Apple Neural Engine
• Creative applications: photo and video editing tools with AI-powered features (noise reduction, subject selection, upscaling) can offload work to the NPU

For most general computing tasks (web browsing, office work, gaming), the NPU does not play a significant role. The CPU and GPU remain the primary performance determinants for these workloads.

Factors affecting NPU scores

Driver versions

NPU drivers are newer and less mature than CPU or GPU drivers. Performance improvements between driver versions can be significant. Keep NPU drivers up to date through your system manufacturer's update tool or Windows Update.

Thermal conditions

NPUs share the thermal envelope with the CPU (and possibly GPU) in most systems.

Related pages

• Understanding your scores: how the NPU score fits into the overall Novabench Score
• Benchmark methodology: how all benchmark tests work at a conceptual level