On Thu, Apr 10, 2025 at 01:44:55PM +0200, Toon Claes wrote: > Patrick Steinhardt <ps@xxxxxx> writes: > > > Wire up benchmarks in Meson. The setup is mostly the same as how we wire > > up our tests. The only difference is that benchmarks get wired up via > > the `benchmark()` option instead of via `test()`, which gives them a bit > > of special treatment: > > > > - Benchmarks never run in parallel. > > > > - Benchmarks aren't run by default when tests are executed. > > > > - Meson does not inject the `MALLOC_PERTURB` environment variable. > > > > Using benchmarks is quite simple: > > > > ``` > > $ meson setup build > > # Run all benchmarks. > > $ meson test -C build --benchmark > > # Run a specific benchmark. > > $ meson test -C build --benchmark p0000-* > > ``` > > I really appreciate the efforts, and I'd love to run the perf tests with > Meson, but at the moment I don't know how these changes are useful > because this doesn't print anything relevant: > > $ meson test -C build --benchmark 'p0005-*' > ninja: Entering directory `/home/toon/devel/git/build' > [1/28] Generating GIT-VERSION-FILE with a custom command (wrapped by meson to set env) > 1/1 p0005-status OK 7.39s > > Ok: 1 > Expected Fail: 0 > Fail: 0 > Unexpected Pass: 0 > Skipped: 0 > Timeout: 0 > > Full log written to /home/toon/devel/git/build/meson-logs/testlog.txt You can execute with `meson test -i`, which will cause Meson to print the benchmark's output. > > Other than that the usual command line arguments accepted when running > > tests are also accepted when running benchmarks. > > > > Note that the benchmarking target is somewhat limited because it will > > only run benchmarks for the current build. Other usecases, like running > > benchmarks against multiple different versions of Git, are not currently > > supported. Users should continue to use "t/perf/run" for those usecases. > > The script should get extended at one point in time to support Meson, > > but this is outside of the scope of this series. > > Yeah, this is unfortunate, but totally understandable. I've been digging > in `t/perf/run` and `f/perf/aggregate.perl` and it doesn't look easy to > adapt to Meson. > > But I was wondering, instead of trying to fully integrate Meson into > those scripts, could we modify the scripts so they work with binaries > built by Meson? I mean, if we could run > `cd t/perf && ./run ../../build1 ../../build2 p0005*` and it would > simply run the benchmarks in those directories (without trying to check > out code and build the sources). I think this would help a lot already. This is another thing we could (and eventually should) do, but it's orthogonal to this patch series from my point of view. Patrick
