> But for simple loopback NFS mount that
> doesn't use pnfs flexfiles it _should_ work, e.g.:
>
> # cat /sys/fs/nfs/0\:46/localio
> 1
Yes, it works, thank you. For NFS localio single thread sequential
read with enabled localio_O_DIRECT_semantics, performance improved 3.5
times!
I'm also looking for non-localio NFS performance improvements,
interested in end-to-end O_DIRECT.
How to enable non-localio end-to-end O_DIRECT ?
Anton
чт, 31 июл. 2025 г. в 20:58, Mike Snitzer <snitzer@xxxxxxxxxx>:
>
> On Wed, Jul 30, 2025 at 03:43:00PM +0300, Anton Gavriliuk wrote:
> > Hi
> >
> > How to mount NFS with localio on Fedora Server 42 (6.15.8 kernel) ?
> >
> > Localio enabled in kernel
> >
> > [root@23-127-77-5 ~]# cat /boot/config-6.15.8-200.fc42.x86_64 | grep -i localio
> > CONFIG_NFS_COMMON_LOCALIO_SUPPORT=m
> > CONFIG_NFS_LOCALIO=y
> >
> > [root@23-127-77-5 ~]# lsmod | grep -i localio
> > nfs_localio 36864 2 nfsd,nfs
> > sunrpc 925696 30
> > nfs_localio,nfsd,rpcrdma,nfsv4,auth_rpcgss,lockd,rpcsec_gss_krb5,nfs_acl,nfs
> >
> > [root@23-127-77-5 ~]# mount -t nfs 127.0.0.1:/mnt /mnt1
> > [root@23-127-77-5 ~]#
> > [root@23-127-77-5 ~]# mount | grep -i mnt1
> > 127.0.0.1:/mnt on /mnt1 type nfs4
> > (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,fatal_neterrors=none,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=127.0.0.1,local_lock=none,addr=127.0.0.1)
> >
> > So /mnt1 is mounted with localio ?
>
> Should be.
>
> One way to tell, albeit less ironclad, is that performance improves
> (reduced use of network/sunrpc is also a tell-tale).
>
> But see commit 62d2cde203def ("NFS: add localio to sysfs")
> It adds semi-useful exposure of whether LOCALIO is active for a given
> rpc client. I say "semi-useful" because I'm left wanting on the
> reliability of this interface (particularly when pnfs flexfiles is
> used, the separate NFS client that is established to connect to the
> local NFSD over v3 isn't getting added to sysfs for some reason.. not
> put time to fixing that yet). But for simple loopback NFS mount that
> doesn't use pnfs flexfiles it _should_ work, e.g.:
>
> # cat /sys/fs/nfs/0\:46/localio
> 1
>
> I've found that the best way to _know_ LOCALIO enabled is to use trace
> points, but yeah that is kind of obscure and certainly not common in
> production.
>
> These tracepoints really showcase LOCALIO is being used:
>
> echo 1 > /sys/kernel/tracing/events/sunrpc/svc_process/enable
> echo 1 > /sys/kernel/tracing/events/nfs_localio/nfs_localio_enable_client/enable
> echo 1 > /sys/kernel/tracing/events/nfs_localio/nfs_localio_disable_client/enable
> echo 1 > /sys/kernel/tracing/events/nfs/nfs_local_open_fh/enable
>
> (NOTE: it is only with this patch applied that the nfs_local_open_fh
> tracepoint is made much more useful, otherwise it'd only show if the
> function failed to open the fh.. unfortunately not yet picked up for
> upstream:
> https://lore.kernel.org/linux-nfs/20250724193102.65111-9-snitzer@xxxxxxxxxx/
> )
>
> echo nop > /sys/kernel/debug/tracing/current_tracer
> echo 1 > /sys/kernel/debug/tracing/tracing_on
>
> With these enabled, mounting NFS via loopback and doing a simple dd
> shows the following in: cat /sys/kernel/debug/tracing/trace
>
> nfsd-10448 [024] ..... 4316.520916: svc_process: addr=192.168.1.106 xid=0xcf2cdbdf service=nfslocalio vers=1 proc=UUID_IS_LOCAL
> kworker/u194:0-9772 [042] ..... 4316.520951: nfs_localio_enable_client: server=192.168.1.106 NFSv3
> kworker/u194:0-9772 [042] ..... 4316.647334: nfs_local_open_fh: fhandle=0x4d34e6c1 mode=READ|WRITE result=0
>
> Also, enabling various nfsd tracepoints and seeing the absence of them
> is telling. Similarly, enabling tracepoints for NFSD's underlying
> filesystem (e.g. xfs) and seeing the process that is triggering the
> trace isn't nfsd showcases LOCALIO being used, e.g.:
>
> with LOCALIO:
>
> kworker/u194:3-9540 [027] ..... 5155.011380: xfs_file_direct_write: dev 259:16 ino 0x3e00008f disize 0xb7a0 pos 0xc000 bytecount 0xa000
>
> without LOCALIO:
>
> nfsd-10448 [034] ..... 5730.314274: xfs_file_direct_write: dev 259:16 ino 0x3e00008f disize 0xb7a0 pos 0xc000 bytecount 0xa000
>
> Hope this helps,
> Mike
