On Tue, Sep 02, 2025 at 02:00:43PM +0100, Mark Brown wrote:
> On Tue, Sep 02, 2025 at 02:14:34PM +0200, Mauro Carvalho Chehab wrote:
> > Mauro Carvalho Chehab <mchehab+huawei@xxxxxxxxxx> escreveu:
>
> > > > That takes about 90s for me.
>
> > > I wander why here is 3 times faster... disk cache? python version?
> > > faster ssd?
>
> > > What python version are you using?
>
> > Heh, after running twice or three times to avoid cache issues, I tested
> > running it on my machine with two different python versions:
>
> > $ time python3.13 scripts/kernel-doc.py . --none
>
> > real 0m31,660s
> > user 0m30,911s
> > sys 0m0,588s
>
> For me it's fairly consistent with python 3.11.2, I get some variation
> depending on what else is going on with the system but nothing hugely
> surprising. It should mostly be running from cache, the underlying disk
> is a reasonable SSD.
Yeah, it sounds that the huge performance increment was indeed on 3.11:
$ time python3.6 ./scripts/kernel-doc --none .
real 1m17,854s
user 1m16,651s
sys 0m0,707s
$ time python3.9 ./scripts/kernel-doc --none .
real 1m0,193s
user 0m58,942s
sys 0m0,614s
$ time python3.10 ./scripts/kernel-doc --none .
real 0m55,376s
user 0m54,168s
sys 0m0,636s
$ time python3.11 ./scripts/kernel-doc --none .
real 0m34,878s
user 0m33,665s
sys 0m0,661s
$ time python3.12 ./scripts/kernel-doc --none .
real 0m34,590s
user 0m33,844s
sys 0m0,613s
$ time python3.13 ./scripts/kernel-doc --none .
real 0m31,751s
user 0m30,951s
sys 0m0,640s
============== ============= ============= ================================
Python Version Real Time (s) User Time (s) Performance Increase (Real Time)
============== ============= ============= ================================
3.6 77.854 s 76.651 s (baseline)
3.9 60.193 s 58.942 s 22.7% faster
3.10 55.376 s 54.168 s 28.9% faster
3.11 34.878 s 33.665 s 55.2% faster
3.12 34.590 s 33.844 s 55.6% faster
3.13 31.751 s 30.951 s 59.2% faster
============== ============= ============= ================================
> The single core performance on this machine isn't
> amazing, it's more getting it's speed from having a bunch of cores.
As kernel-doc is currently single threaded, performance for a single
core is what matters most. I suspect that most of kernel-doc time is
spent handling regular expressions, specially when I/O is fast.
-
To sum-up those discussions, I can propose a patchset for the next
merge window that would:
1. change kernel-doc exec to re-run using the latest available python
version if version < 3.11, on a similar same way to what
scripts-pre-install and scripts-build-wrapper does(*);
2. add a command line parameter for kernel-doc to make it pick only
files that have a .. kernel-doc markup;
3. add a build logic to run it with make all, perhaps inside a Kconfig
symbol like "config DOC_WARNINGS", disabled by default, but enabled
with allyesconfig/allmodconfig.
4. with time, add more validations there, like checking for
EXPORT_SYMBOL without documentation and other neat features.
This needs to be coordinated with some efforts to cleanup warnings,
to avoid having hundreds of new warnings at build time.
This way, even on LTS, we'll have fast kernel-doc check, and will likely
take lot less than 32 seconds, as it will only validate a small set of
files that are pointed by a kernel-doc markup notation inside
Documentation, specially if Python >= 3.11 is installed.
It should be noticed that can generate out of blue lots of new
warnings that are currently there by currently hidden, specially
if we add "-Wall" to the build target.
Comments?
(*) The logic there checks if python version is below a minimal
version. If it is, it seeks for python 3.x exec files, picking
the latest one if found, and re-running it with such version.
--
Thanks,
Mauro
