Hi,
On 4/1/25 2:15 PM, Rik Theys wrote:
Hi Daniel,
On 4/1/25 2:05 PM, Daniel Kobras wrote:
Hi Rik!
Am 15.12.24 um 13:38 schrieb Rik Theys:
Suddenly, a number of clients start to send an abnormal amount of
NFS traffic to the server that saturates their link and never seems
to stop. Running iotop on the clients shows
kworker-{rpciod,nfsiod,xprtiod} processes generating the write
traffic. On the server side, the system seems to process the traffic
as the disks are processing the write requests.
This behavior continues even after stopping all user processes on
the clients and unmounting the NFS mount on the client. Is this
normal? I was under the impression that once the NFS mount is
unmounted no further traffic to the server should be visible?
I'm currently looking at an issue that resembles your description
above (excess traffic to the server for data that was already written
and committed), and part of the packet capture also looks roughly
similar to what you've sent in a followup. Before I dig any deeper:
Did you manage to pinpoint or resolve the problem in the meantime?
Our server is currently running the 6.12 LTS kernel and we haven't had
this specific issue any more. But we were never able to reproduce it,
so unfortunately I can't say for sure if it's fixed, or what fixed it
:-/.
If I remember correctly the issue came back almost immediately if we
rebooted the server. Also rebooting the clients that seemed to take part
in the flood was not sufficient. It would almost immediately start again.
In the end, I believe I:
1. stopped NFS on the server,
2. removed the state files from /var/lib/nfs on the server and
3. then rebooted the server. I waited 100s in the GRUB boot loader
before starting the OS to make sure this was longer than the 90s lease
delegation timeout.
Afterwards the issue stopped. But I'm not sure removing the state files
has any impact.
Regards,
Rik
--
Rik Theys
System Engineer
KU Leuven - Dept. Elektrotechniek (ESAT)
Kasteelpark Arenberg 10 bus 2440 - B-3001 Leuven-Heverlee
+32(0)16/32.11.07
----------------------------------------------------------------
<<Any errors in spelling, tact or fact are transmission errors>>
