On Thu, Aug 07, 2025 at 03:04:46PM +0200, Phil Sutter wrote: > On Thu, Aug 07, 2025 at 02:14:13PM +0200, Pablo Neira Ayuso wrote: > > On Thu, Aug 07, 2025 at 01:51:09PM +0200, Phil Sutter wrote: > > > On Wed, Aug 06, 2025 at 07:05:02PM +0200, Pablo Neira Ayuso wrote: > > > > On Fri, Aug 01, 2025 at 06:11:05PM +0200, Phil Sutter wrote: > > > > > Add the various testsuite runners to TESTS variable and have make call > > > > > them with RUN_FULL_TESTSUITE=1 env var. > > > > > > > > > > Signed-off-by: Phil Sutter <phil@xxxxxx> > > > > > --- > > > > > Makefile.am | 6 ++++++ > > > > > 1 file changed, 6 insertions(+) > > > > > > > > > > diff --git a/Makefile.am b/Makefile.am > > > > > index ba09e7f0953d5..4fb75b85a5d59 100644 > > > > > --- a/Makefile.am > > > > > +++ b/Makefile.am > > > > > @@ -409,5 +409,11 @@ EXTRA_DIST += \ > > > > > tests \ > > > > > $(NULL) > > > > > > > > > > +AM_TESTS_ENVIRONMENT = RUN_FULL_TESTSUITE=1; export RUN_FULL_TESTSUITE; > > > > > > > > I use make distcheck to build the tarballs. > > > > > > > > I would prefer not to run the tests at the time of the release > > > > process, I always do this before release, but I prefer not to inline > > > > this to the release process. > > > > > > Oh, good to know. Running just 'make dist' is no option for you? > > > > I can just modify the script to do so, no idea on the implications. > > There is more to distcheck than just the 'make check' call, so it's > definitely worth doing it. The best option might be to run 'make > distcheck' before the release for a complete test run and only 'make > dist' during the release process. Though this requires to run 'make > distcheck' as root, not sure if that is a good idea. Hm, I prefer not to run make distcheck as root. > > Or wait for one more hour for the test run to finish during the > > release process. > > Does not seem feasible, especially for a redundant test run you're not > interested in. It also implies that you're creating the distribution on > a system which is able to pass (or skip) all tests, which may not be the > case. Yes. > > > BTW: There is the same situation with iptables, though if called as > > > unprivileged user there is only the xlate test suite which runs (and > > > quickly finishes). > > > > > > > Maybe we can make this work this way? > > > > > > > > export RUN_FULL_TESTSUITE=1; make check > > > > > > > > so make check is no-op without this variable? > > > > > > > > Does this make sense to you? > > > > > > It seems odd to enable 'make check' only to disable it again, but > > > there's still added value in it. > > > > > > I'm currently looking into distcheck-hook and DISTCHECK_CONFIGURE_FLAGS > > > in order to identify the caller to 'make check' call. > > > > > > An alternative would be to drop fake root functionality from shell > > > test suite, then it would skip just like all the other test suites if run > > > as non-root (assuming you don't run 'make distcheck' as root). > > > > Looking at the current release script I have, it all runs as non-root. > > > > Maybe simply as a run-all.sh under nftables/tests/? > > Also an option, yes. Or a custom 'make testrun' or so. 'make testrun' sounds nicer than my run-it-all shell script proposal, it would be nice to have a short summary at the test run not to scroll up to find each individual test result. And I think 'make testrun' should continue on errors so it is also useful for testing patches under development. Thanks.
