Can BenchmarkDotNet generate a standalone benchmark executable without requiring the .NET SDK on the target machine?

[takla21]

Hi,

I want to run BenchmarkDotNet benchmarks on devices where installing the .NET SDK is not allowed. These devices can run .NET runtime applications, but they cannot build anything.

My goal is to:

• Build the benchmark project on a machine that has the .NET SDK
• Export or generate a standalone benchmark executable
• Copy that executable to another device and run the benchmarks there (runtime only, no SDK)

Is there a supported way to produce such a standalone benchmark runner? Or does BenchmarkDotNet always require the SDK on the machine where the benchmarks are executed?

Thanks!

[timcassell]

The only way to do it today is to use InProcessEmitToolchain for JIT runtimes, or InProcessNoEmitToolchain for AOT runtimes. It will run the benchmarks in the same process, so you lose process-level isolation.

[timcassell]

• Ensuring all of my benchmark test are annoted with [MethodImpl(MethodImplOptions.NoInlining)]

That really shouldn't be required, BDN should be auto-applying it during the build. I'd like to get my hands on a minimal repro to debug it if possible.

One last question: do you know when we can expect a stable release that includes the fix for the timeout issue I encountered earlier?

I'm not sure, I still have a couple more breaking changes to get in. Hopefully not more than a month, but no promises. I can only work on this in my free time.

[takla21]

That really shouldn't be required, BDN should be auto-applying it during the build. I'd like to get my hands on a minimal repro to debug it if possible.

Let me check again. Maybe I've added this unnecessarily.

I'm not sure, I still have a couple more breaking changes to get in. Hopefully not more than a month, but no promises. I can only work on this in my free time.

Fair enough. In the meantime, we'll try to optimize our tests so it can work with the latest stable version.

[takla21]

While I was trying to reproduce the issue above, I've figured out why I had the other issues earlier which seems to be a regression on the nigthly build. I've created a new issue here #3102

[timcassell]

@takla21 Are you still able to repro the NoInlining issue with the latest?

[takla21]

Right I totally forgot about that part. I'll try to test tomorrow if I can gain access to a linux machine