Re: Large(ish) variance induced by SCHED_FIFO

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 05/09/2025 16:03, John Ogness wrote:

> On 2025-09-05, Marc Gonzalez wrote:
>
>> (I used CLOCK_MONOTONIC_RAW.)
> 
> Not really relevant to your issue, but you should always use CLOCK_MONOTONIC.

Could you provide some insight as to why CLOCK_MONOTONIC is preferable?

https://www.man7.org/linux/man-pages/man3/clock_gettime.3.html

CLOCK_MONOTONIC

A non-settable system-wide clock that represents monotonic time since
(as described by POSIX) "some unspecified point in the past". On Linux,
that point corresponds to the number of seconds that the system has been
running since it was booted.

The CLOCK_MONOTONIC clock is not affected by discontinuous jumps in the
system time (e.g., if the system administrator manually changes the clock),
but is affected by the incremental adjustments performed by adjtime(3) and NTP.
This clock does not count time that the system is suspended.
All CLOCK_MONOTONIC variants guarantee that the time returned by
consecutive calls will not go backwards, but successive calls may
(depending on the architecture) return identical (not-increased)
time values.


CLOCK_MONOTONIC_RAW (since Linux 2.6.28; Linux-specific)

Similar to CLOCK_MONOTONIC, but provides access to a raw hardware-based time
that is not subject to NTP adjustments or the incremental adjustments performed
by adjtime(3). This clock does not count time that the system is suspended.


CLOCK_MONOTONIC_RAW seemed less susceptible to "interference" :)

>> It looks like there is some kind of 50 millisecond quantum
>> involved somewhere somehow.
> 
> Did you forget to disable RT throttling?
> 
> $ sudo sysctl -w kernel.sched_rt_runtime_us=-1
> 
> If you check the kernel logs, you would see a message about throttling:
> 
> "sched: RT throttling activated"

YES!!!

I didn't think to look at dmesg :(

And I had looked at sched-related proc nodes...

$ grep "" /proc/sys/kernel/sched_*
/proc/sys/kernel/sched_autogroup_enabled:1
/proc/sys/kernel/sched_cfs_bandwidth_slice_us:5000
/proc/sys/kernel/sched_deadline_period_max_us:4194304
/proc/sys/kernel/sched_deadline_period_min_us:100
/proc/sys/kernel/sched_rr_timeslice_ms:100
/proc/sys/kernel/sched_rt_period_us:1000000
/proc/sys/kernel/sched_rt_runtime_us:950000
/proc/sys/kernel/sched_schedstats:0
/proc/sys/kernel/sched_util_clamp_max:1024
/proc/sys/kernel/sched_util_clamp_min:1024
/proc/sys/kernel/sched_util_clamp_min_rt_default:1024

... but my brain didn't register the obvious clue,
although it was right there: 1000000 - 950000 = 50 ms !!

Now everything works as expected, even in multi-user mode:

$ sudo ./run_benchmark.sh fifo
2155899243
2155930350
2155925344
2155545153
2155520262
2155520647
2155531815
2155586328
2155501088
2155887042
2155937889
2155990740
2155939605
2155509468
2155898441
2155615316
2155498933
2155968150
2155493295
2155496287
2155949449
2155488499
2155932900
2155498135
2155531294
2156047123
2156035893
2155926392
2155917489
2155482792

EXCEPT... that my mouse is sluggish as molasses when the script
runs on CPU1 (but not when it runs on other cores).

I'm probably starving an important ISR :)

[migration/0] pid 18's current affinity list: 0
[migration/1] pid 23's current affinity list: 1
[migration/2] pid 29's current affinity list: 2
[migration/3] pid 35's current affinity list: 3
[idle_inject/0] pid 19's current affinity list: 0
[idle_inject/1] pid 22's current affinity list: 1
[idle_inject/2] pid 28's current affinity list: 2
[idle_inject/3] pid 34's current affinity list: 3
[irq/9-acpi] pid 54's current affinity list: 1
[watchdogd] pid 62's current affinity list: 0-3
[irq/24-PCIe PME] pid 68's current affinity list: 2
[irq/24-pciehp] pid 69's current affinity list: 2
[irq/24-s-pciehp] pid 70's current affinity list: 2
[irq/25-PCIe PME] pid 71's current affinity list: 3
[irq/8-rtc0] pid 74's current affinity list: 0
[irq/23-ehci_hcd:usb1] pid 75's current affinity list: 1
[irq/16-ehci_hcd:usb2] pid 81's current affinity list: 2
[irq/26-xhci_hcd] pid 159's current affinity list: 1
[irq/27-ahci[0000:00:1f.2]] pid 162's current affinity list: 1
[irq/18-i801_smbus] pid 388's current affinity list: 1
[irq/29-mei_me] pid 411's current affinity list: 2
[card1-crtc0] pid 461's current affinity list: 0-3
[card1-crtc1] pid 462's current affinity list: 0-3
[irq/30-i915] pid 463's current affinity list: 3
[irq/31-snd_hda_intel:card0] pid 514's current affinity list: 0
[irq/32-snd_hda_intel:card1] pid 518's current affinity list: 1
[psimon] pid 334's current affinity list: 0-3
[psimon] pid 1075's current affinity list: 0-3

Most of the USB ISRs run on core 1, no wonder my USB mouse feels limp :)

Thanks David & John for pointing me in the right direction!

Regards






[Index of Archives]     [RT Stable]     [Kernel Newbies]     [IDE]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux ATA RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux