High idle CPU usage on Windows (push-to-talk mode)

[pszemraj]

Hi, first off great project, love the design philsophy!

After using for a few days, I noticed: With the app sitting idle, GUI minimized, not recording, wake word disabled, CPU usage sits at 10-20% about 60-70% of the time. It cycles down to ~0% briefly then climbs back up. After an hour of doing nothing. Confirmed in both btop and PowerShell's Get-Process

Environment/specs

app details:

• version: v1.8.0
• Mode: Push-to-talk
• Voice Mode/Wake Word: OFF

machine:

• OS: Windows 11 Pro (25H2) x86_64
• CPU: AMD Ryzen 9 8945HS (16) @ 5.25 GHz
• RAM: 16.87 GiB / 31.29 GiB (54%)
• GPU 1: NVIDIA GeForce RTX 4070 Laptop GPU @ 3.10 GHz (7.76 GiB) [Discrete]
• GPU 2: AMD Radeon 780M Graphics (418.48 MiB) [Integrated]

The main reason I'm sharing the GPU info is because I noticed ort is hard-locked to CPU runtime and it would be nice to opt-in use what I have (this prob should be its own issue thread, i can create if you like)

attempt to be useful

I poked around the code a bit and noticed that the three ONNX sessions created at startup (encoder, decoder_joint, preprocessor) all pass None for intra_threads. afaik that means ONNX Runtime falls back to using all CPU cores for its thread pool, and on Windows it apparently uses a spin-then-sleep strategy that can eat CPU even when nothing is actually running.

Not sure if that's the full story or if I'm reading this wrong, but from a quick look that's what I can see. Specifically in my case, I've got a bunch of CPUs (above) so maybe for me its worse than normal. Would passing a small explicit thread count there help, or is there a reason it's left as None?

Also: while looking for things that could cause the CPU use I noticed the overlay window seems to always get created at startup even when overlay mode is off. figured I'd mention it.

• if either of the two things are useful ^ I can draft a PR for them

[Kieirra]

Hi, and thank you for opening this issue.

The main reason I'm sharing the GPU info is because I noticed ort is hard-locked to CPU runtime and it would be nice to opt-in use what I have

Yes, this is intentional, for three reasons:

• Not everyone has a GPU, and supporting CUDA + other GPU architectures (and fallbacks) would significantly complicate the codebase
• In practice, it’s not really necessary: CPU is already fast enough for near-instant transcription since audio inputs are short
• It leaves the GPU fully available for LLM integrations, which are much more demanding. LLMs perform poorly on CPU, so prioritizing GPU for them gives better overall results

Not sure if that's the full story or if I'm reading this wrong, but from a quick look that's what I can see. Specifically in my case, I've got a bunch of CPUs (above) so maybe for me its worse than normal. Would passing a small explicit thread count there help, or is there a reason it's left as None?

I think that ONNX Runtime and the overlay are not the cause here.
For example, on Linux, idle CPU usage is around 0.08 - 0.12% on my machine, and both parts use the exact same code across platforms.

What does differ between platforms is the shortcut listener:

• On Linux, it’s event-driven → near zero cost
• On Windows, I have to poll GetAsyncKeyState every 32ms in a dedicated thread

This polling is very likely what you’re seeing. Global shortcuts are honestly a nightmare across platforms: very OS-specific, fragile, and a lot of software struggles with them. That’s why I’m a bit cautious about trying to "optimize" this part.

If you want to test this hypothesis, you can increase the polling interval to something like 1000ms (it won’t be usable 😄), but I think you should see CPU usage drop.

[Kieirra]

Code is here if you want to play with it:

• https://github.com/Kieirra/murmure/blob/main/src-tauri/src/shortcuts/platform_windows.rs#L25
• https://github.com/Kieirra/murmure/blob/main/src-tauri/src/shortcuts/platform_windows.rs#L88

You can even remove the loop entirely to confirm it's the culprit.

[pszemraj]

Hey I'll give you a better response in a day or so! Quick version:

I tried both: first manually setting all ONNX creation instances to 2 threads/workers/etc (from None) and then setting polling time to 100ms (see comment below) in addition. Neither seems to have solved it. Unsure if it's the numbers that need tweaking or it's something else.. but if it were polling rate you'd think 3x more room would make a difference, no?

Additionally, I am very impressed by the choice of 32ms default polling when human reaction time to visual stimuli is ~200ms 😉

Jain, A., Bansal, R., Kumar, A., & Singh, K. D. (2015). A comparative study of visual and auditory reaction times on the basis of gender and physical activity levels of medical first year students. International Journal of Applied and Basic Medical Research, 5(2), 124–127. https://pmc.ncbi.nlm.nih.gov/articles/PMC4456887/