On Wed, 10 Sep 2025, Mauro Carvalho Chehab <mchehab+huawei@xxxxxxxxxx> wrote: > Em Wed, 10 Sep 2025 13:46:17 +0300 > Jani Nikula <jani.nikula@xxxxxxxxxxxxxxx> escreveu: > >> On Thu, 04 Sep 2025, Mauro Carvalho Chehab <mchehab+huawei@xxxxxxxxxx> wrote: >> > There are too much magic inside docs Makefile to properly run >> > sphinx-build. Create an ancillary script that contains all >> > kernel-related sphinx-build call logic currently at Makefile. >> > >> > Such script is designed to work both as an standalone command >> > and as part of a Makefile. As such, it properly handles POSIX >> > jobserver used by GNU make. >> > >> > On a side note, there was a line number increase due to the >> > conversion: >> > >> > Documentation/Makefile | 131 +++---------- >> > tools/docs/sphinx-build-wrapper | 293 +++++++++++++++++++++++++++++++ >> > 2 files changed, 323 insertions(+), 101 deletions(-) >> > >> > This is because some things are more verbosed on Python and because >> > it requires reading env vars from Makefile. Besides it, this script >> > has some extra features that don't exist at the Makefile: >> > >> > - It can be called directly from command line; >> > - It properly return PDF build errors. >> > >> > When running the script alone, it will only take handle sphinx-build >> > targets. On other words, it won't runn make rustdoc after building >> > htmlfiles, nor it will run the extra check scripts. >> >> I've always strongly believed we should aim to make it possible to build >> the documentation by running sphinx-build directly on the >> command-line. Not that it would be the common way to run it, but to not >> accumulate things in the Makefile that need to happen before or >> after. To promote handling the documentation build in Sphinx. To be able >> to debug issues and try new Sphinx versions without all the hacks. > > That would be the better, but, unfortunately, this is not possible, for > several reasons: > > 1. SPHINXDIRS. It needs a lot of magic to work, both before running > sphinx-build and after (inside conf.py); Makes you wonder if that's the right solution to the original problem. It was added as a kind of hack, and it stuck. > 2. Several extensions require kernel-specific environment variables to > work. Calling sphinx-build directly breaks them; The extensions shouldn't be using environment variables for configuration anyway. Add config options and set them in conf.py like everything else? > 3. Sphinx itself doesn't build several targets alone. Instead, they create > a Makefile, and an extra step is needed to finish the build. That's > the case for pdf and texinfo, for instance; That's not true for the Makefile currently generated by sphinx-quickstart. Granted, I haven't used Sphinx much for pdf output. > 4. Man pages generation. Sphinx support to generate it is very poor; In what way? > 5. Rust integration adds more complexity to the table; > > I'm not seeing sphinx-build supporting the above needs anytime soon, > and, even if we push our needs to Sphinx and it gets accepted there, > we'll still need to wait for quite a while until LTS distros merge > them. I'm not suggesting to add anything to Sphinx upstream. >> This patch moves a bunch of that logic into a Python wrapper, and I feel >> like it complicates matters. You can no longer rely on 'make V=1' to get >> the build commands, for instance. > > Quite the opposite. if you try using "make V=1", it won't show the > command line used to call sphinx-build anymore. > > This series restore it. > > See, if you build with this series with V=1, you will see exactly > what commands are used on the build: > > $ make V=1 htmldocs > ... > python3 ./tools/docs/sphinx-build-wrapper htmldocs \ > --sphinxdirs="." --conf="conf.py" \ > --builddir="Documentation/output" \ > --theme= --css= --paper= > python3 /new_devel/docs/sphinx_latest/bin/sphinx-build -j25 -b html -c /new_devel/docs/Documentation -d /new_devel/docs/Documentation/output/.doctrees -D kerneldoc_bin=scripts/kernel-doc.py -D version=6.17.0-rc1 -D release=6.17.0-rc1+ -D kerneldoc_srctree=. /new_devel/docs/Documentation /new_devel/docs/Documentation/output > ... > > > >> Newer Sphinx versions have the -M option for "make mode". The Makefiles >> produced by sphinx-quickstart only have one build target: >> >> # Catch-all target: route all unknown targets to Sphinx using the new >> # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). > > I didn't know about this, but from [1] it sounds it covers just two > targets: "latexpdf" and "info". sphinx-build -M help gives a list of 24 targets. > The most complex scenario is still not covered: SPHINXDIRS. > > [1] https://www.sphinx-doc.org/en/master/man/sphinx-build.html > >> %: Makefile >> @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) >> >> That's all. > > Try doing such change on your makefile. it will break: > > - SPHINXDIRS; > - V=1; > - rustdoc I know it does. That's the problem. > and will still be dependent on variables that are passed via > env from Kernel makefile. So, stil you can't run from command > line. Also, if you call sphinx-build from command line: > > $ sphinx-build -j25 -b html Documentation Documentation/output > ... > File "<frozen os>", line 717, in __getitem__ > KeyError: 'srctree' > > It won't work, as several parameters that are required by conf.py and by > Sphinx extensions would be missing (the most important one is srctree, but > there are others in the line too). > >> The proposed wrapper duplicates loads of code that's supposed to be >> handled by sphinx-build directly. > > Once we get the wrapper, we can work to simplify it, but still I > can't see how to get rid of it. I just don't understand the mentality of first adding something complex, and then working to simplify it. Don't make it a Rube Goldberg machine in the first place. >> Including the target/builder names. > > True, but this was a design decision taken lots of years ago: instead > of: > make html > > we're using: > > make htmldocs > > This series doesn't change that: either makefile or the script need > to tho the namespace conversion. In the above Makefile snippet that conversion would be $(@:docs=) The clean Makefile way of checking for having Sphinx and the required versions of Python and dependencies etc. would be a .PHONY target that just checks, and doesn't do *anything* else. It shouldn't be part of the sphinx-build rules. PHONY += check-versions check-versions: sphinx-pre-install --version-check htmldocs: check-versions ... Or something like that. >> Seems to me the goal should be to figure out *generic* wrappers for >> handling parallelism, not Sphinx aware/specific. >> >> >> BR, >> Jani. -- Jani Nikula, Intel
