On 5/2/25 11:29, Orion Poplawski wrote:
> One of our users is struggling with multiple kerberos ticket caches impacting
> access to NFS sec=krb5 mounts.
>
> Because home directories are NFS mounted, we use GSSAPI auth to forward a
> ticket. But then we need to kinit to have a long-term renewable ticket.
>
> But we seem to be seeing that new ssh connections which create a new ticket
> cache break access to the NFS mounts, resulting in "permission denied" or
> "Stale file handle" messages. Switching back to a renewable ticket cache
> seems to resolve the issue.
>
> Any suggestions? Is this expected? I would have thought that the nfs access
> would work with any valid ticket.
>
> NAME="AlmaLinux"
> VERSION="8.10 (Cerulean Leopard)"
> nfs-utils-2.3.3-59.el8.x86_64
> 4.18.0-553.50.1.el8_10.x86_64
>
More details. The issue seems to arise when doing gssapi delegation from a
macOS client to the Linux box. If I have ticket on macOS:
Ticket cache: API:427671FC-DB63-442F-ACA4-13A9194F4398
Default principal: user@xxxxxxxxxxx
Valid starting Expires Service principal
05/06/25 13:29:54 05/06/25 23:29:54 krbtgt/AD.NWRA.COM@xxxxxxxxxxx
renew until 05/13/25 13:29:50
05/06/25 13:30:30 05/06/25 23:29:54 krbtgt/NWRA.COM@xxxxxxxxxxx
05/06/25 13:30:30 05/06/25 23:29:54 host/host@xxxxxxxx
and ssh to the Linux box, I can't access the nfs mount:
-bash: /home/user/.bash_profile: Permission denied
Ticket cache: KEYRING:persistent:30657:krb_ccache_efgrZpc
Default principal: user@xxxxxxxxxxx
Valid starting Expires Service principal
05/06/25 13:30:30 05/06/25 23:29:54 krbtgt/AD.NWRA.COM@xxxxxxxxxxx
I also notice that the ticket is non-renewable.
If I then kinit I can access the home directory fine. Other than the new
ticket being renewable I don't see any difference:
Ticket cache: KEYRING:persistent:30657:krb_ccache_efgrZpc
Default principal: user@xxxxxxxxxxx
Valid starting Expires Service principal
05/06/25 13:31:27 05/06/25 23:31:27 nfs/server@xxxxxxxx
renew until 05/13/25 13:31:22
05/06/25 13:31:27 05/06/25 23:31:27 krbtgt/NWRA.COM@xxxxxxxxxxx
renew until 05/13/25 13:31:22
05/06/25 13:31:27 05/06/25 23:31:27 krbtgt/AD.NWRA.COM@xxxxxxxxxxx
renew until 05/13/25 13:31:22
Actually, I also notice now that there is a krbtgt/NWRA.COM principal as well.
I wonder if that is the difference.
On the initial connection I see:
May 06 13:16:22 rpc.gssd[1539533]:
handle_gssd_upcall(0x7f9c73b2fc80):
'mech=krb5 uid=30657 enctypes=18,17,16,23,3,1,2' (nfs/clnt2c)
May 06 13:16:22 rpc.gssd[1539533]: start_upcall_thread(0x7f9c73b2fc80):
created thread id 0x7f9c69d07700
May 06 13:16:22 rpc.gssd[1539533]: krb5_not_machine_creds(0x7f9c69d07700): uid
30657 tgtname (null)
May 06 13:16:22 rpc.gssd[1539533]: create_auth_rpc_client(0x7f9c69d07700):
creating tcp client for server server
May 06 13:16:22 rpc.gssd[1539533]: DEBUG: port already set to 2049
May 06 13:16:22 rpc.gssd[1539533]: create_auth_rpc_client(0x7f9c69d07700):
creating context with server nfs@server
May 06 13:16:22 rpc.gssd[1539533]: WARNING: Failed to create krb5 context for
user with uid 30657 for server nfs@server
May 06 13:16:22 rpc.gssd[1539533]: looking for client creds with uid 30657 for
server serverin /tmp
May 06 13:16:22 rpc.gssd[1539533]: CC '/tmp/krb5ccmachine_NWRA.COM' being
considered, with preferred realm 'NWRA.COM'
May 06 13:16:22 rpc.gssd[1539533]: CC '/tmp/krb5ccmachine_NWRA.COM' owned by
0, not 30657
May 06 13:16:22 rpc.gssd[1539533]: looking for client creds with uid 30657 for
server server in /run/user/%U
May 06 13:16:22 rpc.gssd[1539533]: do_error_downcall(0x7f9c69d07700): uid
30657 err -13
after kinit:
May 06 13:31:27 rpc.gssd[1539533]:
handle_gssd_upcall(0x7f9c73b2fc80):
'mech=krb5 uid=30657 enctypes=18,17,16,23,3,1,2' (nfs/
clnt2c)
May 06 13:31:27 rpc.gssd[1539533]: start_upcall_thread(0x7f9c73b2fc80):
created thread id 0x7f9c63fff700
May 06 13:31:27 rpc.gssd[1539533]: krb5_not_machine_creds(0x7f9c63fff700): uid
30657 tgtname (null)
May 06 13:31:27 rpc.gssd[1539533]: create_auth_rpc_client(0x7f9c63fff700):
creating tcp client for server server
May 06 13:31:27 rpc.gssd[1539533]: DEBUG: port already set to 2049
May 06 13:31:27 rpc.gssd[1539533]: create_auth_rpc_client(0x7f9c63fff700):
creating context with server nfs@server
May 06 13:31:27 rpc.gssd[1539533]: DEBUG: serialize_krb5_ctx: lucid version!
May 06 13:31:27 rpc.gssd[1539533]: prepare_krb5_rfc4121_buffer: protocol 1
May 06 13:31:27 rpc.gssd[1539533]: prepare_krb5_rfc4121_buffer: serializing
key with enctype 18 and size 32
May 06 13:31:27 rpc.gssd[1539533]: do_downcall(0x7f9c63fff700):
lifetime_rec=10h:0m:0s acceptor=nfs@server
--
Orion Poplawski
he/him/his - surely the least important thing about me
Manager of IT Systems 720-772-5637
NWRA, Boulder Office FAX: 303-415-9702
3380 Mitchell Lane orion@xxxxxxxx
Boulder, CO 80301 https://www.nwra.com/
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
