Re: Latency spikes on V6.15.1 Preempt RT and maybe related to intel? IGB

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Sebastian,

On 14.06.25 10:52, Marc Strämke wrote:

I do not really understand where the IRQ/Preemption disabling is happening. What would the next thing be to do? Function (graph?) tracing on all the functions visible in the backtrace?


I tried to look at the event race output starting with the call to igb_set_rx_mode. I have attached the trace with all events and a function filter on igb on only the cpu executing ip.  I cannot understand what is happening between timestasmp 700.149995 and the IRQ disable event on 700.150795....


Thanks for your help,

Marc


I am still trying to figure that puzzle out: Please see the following function trace snippet:

ip-690530  [000] ..... 178636.460435: rtnl_is_locked <-__dev_change_flags
ip-690530  [000] ..... 178636.460435: __local_bh_disable_ip <-__dev_change_flags ip-690530  [000] ..... 178636.460435: migrate_disable <-__local_bh_disable_ip ip-690530  [000] ...1. 178636.460435: preempt_disable: caller=__local_bh_disable_ip+0x76/0xe0 parent=__local_bh_disable_ip+0x76/0xe0 ip-690530  [000] ...11 178636.460435: preempt_enable: caller=__local_bh_disable_ip+0x76/0xe0 parent=__local_bh_disable_ip+0x76/0xe0
ip-690530  [000] ....1 178636.460435: rt_spin_lock <-__local_bh_disable_ip
ip-690530  [000] ....1 178636.460436: __rcu_read_lock <-rt_spin_lock
ip-690530  [000] ....1 178636.460436: migrate_disable <-__local_bh_disable_ip ip-690530  [000] ....2 178636.460436: __rcu_read_lock <-__local_bh_disable_ip
ip-690530  [000] b...2 178636.460436: rt_spin_lock <-__dev_change_flags
ip-690530  [000] b...2 178636.460436: __rcu_read_lock <-rt_spin_lock
ip-690530  [000] b...2 178636.460436: migrate_disable <-__dev_change_flags
ip-690530  [000] b...3 178636.460436: __dev_set_rx_mode <-__dev_change_flags
ip-690530  [000] b...3 178636.460437: igb_set_rx_mode <-__dev_change_flags

It is my understanding that __local_bh_disable_ip called from netif_addr_lock_bh seems to be the origin of my latency.

What i do not understand is, even if the bottom halves are disabled. Shouldn't I see the interrupt arriving in the trace?

If i understood it correctly, your talk "Unblocking the softirq lock for PREEMPT_RT" during the plumbers conference 2023 is exactly about that case, right?

Probably fixing this issue is out of my abilities for now ;-) The wast variety of locking concepts inside the kernel is quite intimidating for a newcomer in kernel land...


Regards,


Marc





[Index of Archives]     [RT Stable]     [Kernel Newbies]     [IDE]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux ATA RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux