On Thu, Aug 07, 2025 at 01:51:09PM +0200, Phil Sutter wrote: > On Wed, Aug 06, 2025 at 07:05:02PM +0200, Pablo Neira Ayuso wrote: > > On Fri, Aug 01, 2025 at 06:11:05PM +0200, Phil Sutter wrote: > > > Add the various testsuite runners to TESTS variable and have make call > > > them with RUN_FULL_TESTSUITE=1 env var. > > > > > > Signed-off-by: Phil Sutter <phil@xxxxxx> > > > --- > > > Makefile.am | 6 ++++++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/Makefile.am b/Makefile.am > > > index ba09e7f0953d5..4fb75b85a5d59 100644 > > > --- a/Makefile.am > > > +++ b/Makefile.am > > > @@ -409,5 +409,11 @@ EXTRA_DIST += \ > > > tests \ > > > $(NULL) > > > > > > +AM_TESTS_ENVIRONMENT = RUN_FULL_TESTSUITE=1; export RUN_FULL_TESTSUITE; > > > > I use make distcheck to build the tarballs. > > > > I would prefer not to run the tests at the time of the release > > process, I always do this before release, but I prefer not to inline > > this to the release process. > > Oh, good to know. Running just 'make dist' is no option for you? I can just modify the script to do so, no idea on the implications. Or wait for one more hour for the test run to finish during the release process. > BTW: There is the same situation with iptables, though if called as > unprivileged user there is only the xlate test suite which runs (and > quickly finishes). > > > Maybe we can make this work this way? > > > > export RUN_FULL_TESTSUITE=1; make check > > > > so make check is no-op without this variable? > > > > Does this make sense to you? > > It seems odd to enable 'make check' only to disable it again, but > there's still added value in it. > > I'm currently looking into distcheck-hook and DISTCHECK_CONFIGURE_FLAGS > in order to identify the caller to 'make check' call. > > An alternative would be to drop fake root functionality from shell > test suite, then it would skip just like all the other test suites if run > as non-root (assuming you don't run 'make distcheck' as root). Looking at the current release script I have, it all runs as non-root. Maybe simply as a run-all.sh under nftables/tests/? It is not so elegant as make check, but still allows for running _all_ tests with minimal changes. tests/build is slow but very useful. > Another side-quest extracted from this mail: There's an odd failure from > shell test suite when run by 'make distcheck': > > | E: cannot execute nft command: ../../tests/shell/../../src/nft > | ERROR tests/shell/run-tests.sh (exit status: 99) > > > > +TESTS = tests/json_echo/run-test.py \ > > > + tests/monitor/run-tests.sh \ > > > + tests/py/nft-test.py \ > > > + tests/shell/run-tests.sh > > > > BTW, there are also tests/build/ that are slow but useful, that helped > > me find this: > > > > https://git.netfilter.org/nftables/commit/?id=0584f1c1c2073ff082badc7b49ed667de41002d9 > > Ah, I just "discovered" that bug as well, it surfaces from the build > performed by 'make distcheck', too. > > Thanks, Phil
