Hi Mauro,
On Fri, 15 Aug 2025 13:50:32 +0200, Mauro Carvalho Chehab wrote:
> There are too much magic inside docs Makefile to properly run
> sphinx-build. Create an ancillary script that contains all
> kernel-related sphinx-build call logic currently at Makefile.
>
> Such script is designed to work both as an standalone command
> and as part of a Makefile. As such, it properly handles POSIX
> jobserver used by GNU make.
>
> It should be noticed that, when running the script alone,
> it will only take care of sphinx-build and cleandocs target.
> As such:
>
> - it won't run "make rustdoc";
> - no extra checks.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@xxxxxxxxxx>
> ---
> .pylintrc | 2 +-
> scripts/sphinx-build-wrapper | 627 +++++++++++++++++++++++++++++++++++
> 2 files changed, 628 insertions(+), 1 deletion(-)
> create mode 100755 scripts/sphinx-build-wrapper
>
[...]
> diff --git a/scripts/sphinx-build-wrapper b/scripts/sphinx-build-wrapper
> new file mode 100755
> index 000000000000..5c728956b53c
> --- /dev/null
> +++ b/scripts/sphinx-build-wrapper
> @@ -0,0 +1,627 @@
[...]
> + def handle_pdf(self, output_dirs):
> + """
> + Extra steps for PDF output.
> +
> + As PDF is handled via a LaTeX output, after building the .tex file,
> + a new build is needed to create the PDF output from the latex
> + directory.
> + """
> + builds = {}
> + max_len = 0
> +
> + for from_dir in output_dirs:
> + pdf_dir = os.path.join(from_dir, "../pdf")
> + os.makedirs(pdf_dir, exist_ok=True)
> +
> + if self.latexmk_cmd:
> + latex_cmd = [self.latexmk_cmd, f"-{self.pdflatex}"]
> + else:
> + latex_cmd = [self.pdflatex]
> +
> + latex_cmd.extend(shlex.split(self.latexopts))
> +
> + tex_suffix = ".tex"
> +
> + # Process each .tex file
> + has_tex = False
> + build_failed = False
> + with os.scandir(from_dir) as it:
> + for entry in it:
> + if not entry.name.endswith(tex_suffix):
> + continue
> +
> + name = entry.name[:-len(tex_suffix)]
> + has_tex = True
> +
> + try:
> + subprocess.run(latex_cmd + [entry.path],
> + cwd=from_dir, check=True)
So, runs of latexmk (or xelatex) would be serialized, wouldn't they?
That would be a *huge* performance regression when I say:
"make -j10 -O pdfdocs"
Current Makefile delegates .tex --> .pdf part of pdfdocs to sub make
of .../output/Makefile, which is generated on-the-fly by Sphinx's
latex builder. That "-j10 -O" flag is passed to the sub make.
Another issue is that you are not deny-listing variable Noto CJK
fonts for latexmk/xelatex. So this version of wrapper won't be able
to build translations.pdf if you have such variable fonts installed.
That failuer is not caught by your ad-hoc logic below.
> + except subprocess.CalledProcessError:
> + # LaTeX PDF error code is almost useless: it returns
> + # error codes even when build succeeds but has warnings.
> + pass
> +
> + # Instead of checking errors, let's do the next best thing:
> + # check if the PDF file was actually created.
I've seen cases where a corrupt .pdf file is left after premature crashes
of xdvipdfmx. So, checking .pdf is not good enough for determining
success/failure.
One way to see if a .pdf file is properly formatted would be to run
"pdffonts" against the resulting .pdf.
For example, if I run "pdffonts translations.pdf" against the corrupted
one, I get:
Syntax Error: Couldn't find trailer dictionary
Syntax Error: Couldn't find trailer dictionary
Syntax Error: Couldn't read xref table
, with the exit code of 1.
Against a successfully built translations.pdf, I get something like:
name type encoding emb sub uni object ID
------------------------------------ ----------------- ---------------- --- --- --- ---------
JPRCQB+DejaVuSans-Bold CID TrueType Identity-H yes yes yes 4 0
QFNXFP+DejaVuSerif-Bold CID TrueType Identity-H yes yes yes 13 0
NMFBZR+NotoSerifCJKjp-Bold-Identity-H CID Type 0C Identity-H yes yes yes 15 0
WYMCYC+NotoSansCJKjp-Black-Identity-H CID Type 0C Identity-H yes yes yes 32 0
[...]
So I have to say this version of your wrapper does not look quite
ready to replace what you call "too much magic inside docs Makefile".
Regards,
Akira
