The current value of SMC_WR_BUF_CNT is 16 which leads to heavy contention on the wr_tx_wait workqueue of the SMC-R linkgroup and its spinlock when many connections are competing for the work request buffers. Currently up to 256 connections per linkgroup are supported. To make things worse when finally a buffer becomes available and smc_wr_tx_put_slot() signals the linkgroup's wr_tx_wait wq, because WQ_FLAG_EXCLUSIVE is not used all the waiters get woken up, most of the time a single one can proceed, and the rest is contending on the spinlock of the wq to go to sleep again. Addressing this by simply bumping SMC_WR_BUF_CNT to 256 was deemed risky, because the large-ish physically continuous allocation could fail and lead to TCP fall-backs. For reference see this discussion thread on "[PATCH net-next] net/smc: increase SMC_WR_BUF_CNT" (in archive https://lists.openwall.net/netdev/2024/11/05/186), which concludes with the agreement to try to come up with something smarter, which is what this series aims for. Additionally if for some reason it is known that heavy contention is not to be expected going with something like 256 work request buffers is wasteful. To address these concerns make the number of work requests configurable, and introduce a back-off logic with handles -ENOMEM form smc_wr_alloc_link_mem() gracefully. Changelog: v2: 1) Fixed https://lore-kernel.gnuweeb.org/linux-doc/202509070225.pVKkaaCr-lkp@xxxxxxxxx/ 2) Inspired by 1) made the sysctls namespaced and moved the backing variables into struct netns_smc 3) Use tabs instead of spaces for alignment in (Dust Li) 4) Renamed the sysctls form smcr_max_*_wr to smcr_pref_*_wr (along with the affiliated fields/variables based on the discussion with Dust Li (although maximal is mathematically accurate preferred is more fitting IMHO). Should the community decide tha max was better I can roll this change back very quickly 5) Removed Wenjia's r-b form patch 1 as quite a few things changed 4) Add r-b from Mahanta I forgot to add in v1 (which remains the same moduo rename. v1: https://lore.kernel.org/all/20250904211254.1057445-1-pasic@xxxxxxxxxxxxx/ Halil Pasic (2): net/smc: make wr buffer count configurable net/smc: handle -ENOMEM from smc_wr_alloc_link_mem gracefully Documentation/networking/smc-sysctl.rst | 40 +++++++++++++++++++++++++ include/net/netns/smc.h | 2 ++ net/smc/smc_core.c | 34 ++++++++++++++------- net/smc/smc_core.h | 8 +++++ net/smc/smc_ib.c | 7 ++--- net/smc/smc_llc.c | 2 ++ net/smc/smc_sysctl.c | 22 ++++++++++++++ net/smc/smc_sysctl.h | 2 ++ net/smc/smc_wr.c | 32 ++++++++++---------- net/smc/smc_wr.h | 2 -- 10 files changed, 119 insertions(+), 32 deletions(-) base-commit: f3883b1ea5a8f31df4eba1c2cb5196e3a249a09e -- 2.48.1
