On 9/2/25 11:56 AM, Chuck Lever wrote:
> On 8/30/25 1:38 PM, Mike Snitzer wrote:
>> dt (j:1 t:1): File System Information:
>> dt (j:1 t:1): Mounted from device: 192.168.0.105:/hs_test
>> dt (j:1 t:1): Mounted on directory: /mnt/hs_test
>> dt (j:1 t:1): Filesystem type: nfs4
>> dt (j:1 t:1): Filesystem options: rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,fatal_neterrors=none,proto=tcp,nconnect=16,port=20491,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.106,local_lock=none,addr=192.168.0.105
>
> I haven't been able to reproduce a similar failure in my lab with
> NFSv4.2 over RDMA with FDR InfiniBand. I've run dt 6-7 times, all
> successful. Also, for shit giggles, I tried the fsx-based subtests in
> fstests, no new failures there either. The export is xfs on an NVMe
> add-on card; server uses direct I/O for READ and page cache for WRITE.
>
> Notice the mount options for your test run: "proto=tcp" and
> "nconnect=16". Even if your network fabric is RoCE, "proto=tcp" will
> not use RDMA at all; it will use bog standard TCP/IP on your ultra
> fast Ethernet network.
>
> What should I try next? I can apply 2/2 or add "nconnect" or move the
> testing to my RoCE fabric after lunch and keep poking at it.
>
> Or, I could switch to TCP. Suggestions welcome.
The client is not sending any READ procedures/operations to the server.
The following is NFSv3 for clarity, but NFSv4.x results are similar:
nfsd-1669 [003] 1466.634816: svc_process:
addr=192.168.2.67 xid=0x7b2a6274 service=nfsd vers=3 proc=NULL
nfsd-1669 [003] 1466.635389: svc_process:
addr=192.168.2.67 xid=0x7d2a6274 service=nfsd vers=3 proc=FSINFO
nfsd-1669 [003] 1466.635420: svc_process:
addr=192.168.2.67 xid=0x7e2a6274 service=nfsd vers=3 proc=PATHCONF
nfsd-1669 [003] 1466.635451: svc_process:
addr=192.168.2.67 xid=0x7f2a6274 service=nfsd vers=3 proc=GETATTR
nfsd-1669 [003] 1466.635486: svc_process:
addr=192.168.2.67 xid=0x802a6274 service=nfsacl vers=3 proc=NULL
nfsd-1669 [003] 1466.635558: svc_process:
addr=192.168.2.67 xid=0x812a6274 service=nfsd vers=3 proc=FSINFO
nfsd-1669 [003] 1466.635585: svc_process:
addr=192.168.2.67 xid=0x822a6274 service=nfsd vers=3 proc=GETATTR
nfsd-1669 [003] 1470.029208: svc_process:
addr=192.168.2.67 xid=0x832a6274 service=nfsd vers=3 proc=ACCESS
nfsd-1669 [003] 1470.029255: svc_process:
addr=192.168.2.67 xid=0x842a6274 service=nfsd vers=3 proc=LOOKUP
nfsd-1669 [003] 1470.029296: svc_process:
addr=192.168.2.67 xid=0x852a6274 service=nfsd vers=3 proc=FSSTAT
nfsd-1669 [003] 1470.039715: svc_process:
addr=192.168.2.67 xid=0x862a6274 service=nfsacl vers=3 proc=GETACL
nfsd-1669 [003] 1470.039758: svc_process:
addr=192.168.2.67 xid=0x872a6274 service=nfsd vers=3 proc=CREATE
nfsd-1669 [003] 1470.040091: svc_process:
addr=192.168.2.67 xid=0x882a6274 service=nfsd vers=3 proc=WRITE
nfsd-1669 [003] 1470.040469: svc_process:
addr=192.168.2.67 xid=0x892a6274 service=nfsd vers=3 proc=GETATTR
nfsd-1669 [003] 1470.040503: svc_process:
addr=192.168.2.67 xid=0x8a2a6274 service=nfsd vers=3 proc=ACCESS
nfsd-1669 [003] 1470.041867: svc_process:
addr=192.168.2.67 xid=0x8b2a6274 service=nfsd vers=3 proc=FSSTAT
nfsd-1669 [003] 1470.042109: svc_process:
addr=192.168.2.67 xid=0x8c2a6274 service=nfsd vers=3 proc=REMOVE
So I'm probably missing some setting on the reproducer/client.
/mnt from klimt.ib.1015granger.net:/export/fast
Flags: rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,
fatal_neterrors=none,proto=rdma,port=20049,timeo=600,retrans=2,
sec=sys,mountaddr=192.168.2.55,mountvers=3,mountproto=tcp,
local_lock=none,addr=192.168.2.55
Linux morisot.1015granger.net 6.15.10-100.fc41.x86_64 #1 SMP
PREEMPT_DYNAMIC Fri Aug 15 14:55:12 UTC 2025 x86_64 GNU/Linux
--
Chuck Lever
