Hi Alex, Thank you for detailed explanation regarding the lock isolation between multiple libcephfs clients. I would like to follow up with a few questions: 1. Would it be advisable to switch to kernel mount in order to ensure consistent lock coordination between multiple clients? 2. Are there lock/cache issues if I use a hybrid setup(Samba uses kernel client and NFS uses libcephfs, for example)? Or more generally, when running multiple protocols over the same CephFS backend, is there a recommended approach regarding libcephfs vs. kernel client usage? Thanks, Frank ________________________________________ 寄件者: Alex Markuze <amarkuze@xxxxxxxxxx> 寄件日期: 2025年7月7日 下午 07:42 收件者: Frank Hsiao 蕭法宣 副本: Ceph Development 主旨: Re: [RFC] Different flock/fcntl locking behaviors via libcephfs and cephfs kernel client Subject: Re: File locking semantics with CephFS via Samba and NFS (libcephfs) ________________________________ Hi Frank, To your core questions: Yes, the observed behavior is expected when using libcephfs. Unlike the kernel CephFS client, libcephfs does not provide full POSIX lock coordination across multiple client instances. Samba and NFS-Ganesha are each using their own libcephfs context and thus maintain separate lock states, and locks acquired on one side are not visible or enforced on the other. There are indeed known limitations when mixing protocols over libcephfs. In particular, file locking, delegation, and caching semantics are not shared across libcephfs consumers, and the Ceph MDS doesn't currently merge lock states across independent user-space clients unless mediated through the kernel client or coordinated explicitly. On Mon, Jul 7, 2025 at 12:59 PM Frank Hsiao 蕭法宣 <frankhsiao@xxxxxxxx> wrote: > > Hi all, > > I’m encountering unexpected behavior when testing file locking semantics on > CephFS accessed concurrently via Samba and NFS, both backed by libcephfs. > > I’m running a CephFS cluster with both Samba and NFS-Ganesha services. > All clients (Samba and NFS) use libcephfs as the underlying CephFS mount > mechanism. My goal is to verify how file locking behaves across protocols > (SMB and NFS) and whether POSIX lock interoperability is maintained. > > Ceph version: 19.2.2 > Samba version: 4.20.7 > Ganesha version: V5.5 > "posix locking = yes" is toggled in Samba > > For each combination of mount type and posix locking setting in Samba, we test: > 1. flock()/fcntl() from NFS and Samba clients. > 2. Write attempts from the other side (e.g., NFS flock, then Samba write). > > A. Samba with libcephfs + posix locking = yes > - samba flock()/fcntl(), samba write() -> permission denied (expected) > - nfs flock()/fcntl(), samba write() -> write succeeds (unexpected) > - samba flock()/fcntl(), nfs write() -> write succeeds (expected) > - nfs flock()/fcntl(), nfs write() -> write succeeds (expected) > > B. Samba with libcephfs + posix locking = no > - samba flock()/fcntl(), samba write() -> permission denied (expected) > - nfs flock()/fcntl(), samba write() -> write succeeds (expected) > - samba flock()/fcntl(), nfs write() -> write succeeds (expected) > - nfs flock()/fcntl(), nfs write() -> write succeeds (expected) > > We expected that if "posix locking = yes", locks obtained via NFS (via fcntl > or flock) would be respected by Samba as POSIX locks, but in case A, they are > seemingly ignored. > > On the other hand, using ceph kernel mount for Samba (instead of libcephfs) > respect NFS-side locks when "posix locking = yes". (deny write when NFS obtain > flock/fcntl locks) > > My questions are: > > 1. Is this behavior expected with libcephfs? > 2. Are there known limitations when mixing cephfs use across protocols? > 3. Is there a recommended practice for exporting cephfs volumes by Samba and > NFS simultaneously? > > Any comments would be greatly appreciated, and please let me know if any > additional logs or test code would be helpful. > > Thanks, > Frank
