On Tue, Aug 26, 2025 at 09:45:48PM +0200, Anders Roxell wrote: > Regressions found while booting the Linux next-20250826 on the > qemu-arm64, qemu-armv7 due to following kernel log. > > Bisection identified this commit as the cause of the regression. > > Regression Analysis: > - New regression? Yes > - Reproducible? Yes > > First seen on the next-20250826 > Good: next-20250825 > Bad: next-20250826 > > Test regression: next-20250826 gcc-13 boot failed on qemu-arm64 and > qemu-armv7. > > Expected behavior: System should boot normally and virtio block devices > should be detected and initialized immediately. > > Actual behavior: System hangs for ~30 seconds during virtio block device > initialization before showing scheduler deadline replenish errors and > failing to complete boot. > > Reported-by: Linux Kernel Functional Testing <lkft@xxxxxxxxxx> > > [...] > <6>[ 1.369038] virtio-pci 0000:00:01.0: enabling device (0000 -> > 0003) > <6>[ 1.420097] Serial: 8250/16550 driver, 4 ports, IRQ sharing > enabled > <6>[ 1.450858] msm_serial: driver initialized > <6>[ 1.454489] SuperH (H)SCI(F) driver initialized > <6>[ 1.456056] STM32 USART driver initialized > <6>[ 1.513325] loop: module loaded > <6>[ 1.515744] virtio_blk virtio0: 2/0/0 default/read/poll queues > <5>[ 1.527859] virtio_blk virtio0: [vda] 5397504 512-byte logical > blocks (2.76 GB/2.57 GiB) > <4>[ 29.761219] sched: DL replenish lagged too much > [here it hangs] FWIW, I am also seeing this on real arm64 hardware (an LX2160A board and an Ampere Altra one) but with my NVMe drives failing to be recognized. In somewhat ironic fashion, I am seeing the message from cover letter repeating. nvme nvme0: I/O tag 8 (1008) QID 0 timeout, completion polled [ 125.810062] dracut-initqueue[640]: Timed out while waiting for udev queue to empty. nvme nvme0: I/O tag 9 (1009) QID 0 timeout, completion polled I am happy to test patches or provide information. Cheers, Nathan # bad: [d0630b758e593506126e8eda6c3d56097d1847c5] Add linux-next specific files for 20250826 # good: [b6add54ba61890450fa54fd9327d10fdfd653439] Merge tag 'pinctrl-v6.17-2' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-pinctrl git bisect start 'd0630b758e593506126e8eda6c3d56097d1847c5' 'b6add54ba61890450fa54fd9327d10fdfd653439' # good: [968d16786392f6e047329f5eff66acc131636019] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git git bisect good 968d16786392f6e047329f5eff66acc131636019 # good: [042e9f528d5362c499b5d8e2716cf6f64ca53add] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/ieee1394/linux1394.git git bisect good 042e9f528d5362c499b5d8e2716cf6f64ca53add # bad: [beebb75399dc36e7c244db0a08426053b4581ecc] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git git bisect bad beebb75399dc36e7c244db0a08426053b4581ecc # good: [62df8fb299358a45a915381de09025cf5e6a4a8f] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git git bisect good 62df8fb299358a45a915381de09025cf5e6a4a8f # bad: [1e6d2dcb13c8d94b56de1eff60235ca90587046b] Merge branch 'master' of https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git git bisect bad 1e6d2dcb13c8d94b56de1eff60235ca90587046b # bad: [a0daa9e939dbcd7767090151771d94ade75a4fd5] Merge branch into tip/master: 'x86/build' git bisect bad a0daa9e939dbcd7767090151771d94ade75a4fd5 # bad: [d147a3db0dfa15c8e460f007128bd0fe2e1b877f] Merge branch into tip/master: 'perf/core' git bisect bad d147a3db0dfa15c8e460f007128bd0fe2e1b877f # good: [be5697d7136525a91e7f30fdca2e7de737d9a8ed] Merge branch into tip/master: 'irq/core' git bisect good be5697d7136525a91e7f30fdca2e7de737d9a8ed # good: [5d299897f1e36025400ca84fd36c15925a383b03] perf: Split out the RB allocation git bisect good 5d299897f1e36025400ca84fd36c15925a383b03 # bad: [7fb83eb664e9b3a0438dd28859e9f0fd49d4c165] irqchip/loongson-eiointc: Route interrupt parsed from bios table git bisect bad 7fb83eb664e9b3a0438dd28859e9f0fd49d4c165 # bad: [7ee4a5a2ec3748facfb4ca96e4cce6cabbdecab2] irqchip/sg2042-msi: Set MSI_FLAG_MULTI_PCI_MSI flags for SG2044 git bisect bad 7ee4a5a2ec3748facfb4ca96e4cce6cabbdecab2 # bad: [9d8c41816bac518b4824f83b346ae30a1be83f68] irqchip/sg2042-msi: Fix broken affinity setting git bisect bad 9d8c41816bac518b4824f83b346ae30a1be83f68 # bad: [54f45a30c0d0153d2be091ba2d683ab6db6d1d5b] PCI/MSI: Add startup/shutdown for per device domains git bisect bad 54f45a30c0d0153d2be091ba2d683ab6db6d1d5b # first bad commit: [54f45a30c0d0153d2be091ba2d683ab6db6d1d5b] PCI/MSI: Add startup/shutdown for per device domains
