Hi,
this is absolutely the right place to ask.
First, I would separate the issues (dashboard and crash). And there's
no need to redeploy anything, it's just a mgr issue (and crash),
nothing critical regarding the operational state of your cluster.
Can you paste the output of 'ceph health detail'? And then you should
look into the active mgr log and search for stack traces or errors in
general.
I would ignore crash for now, fix the mgr first.
Regards,
Eugen
Zitat von info@xxxxxxxxxxx:
Hi all,
not sure it's ok to ask questions here as we are only a little
Theater where I manage the data of our sound and video files.
I set up a ceph cluster as second backup location on bullseye with
quincy which worked flawless.
All runs smooth but I thought I would have to do a step forward to
get on track.
Yes I know: never change a running system. :-D
After updating all three nodes I forget to set the ceph packages on
hold so they were updated to reef with trixie. No way to downgrade
...I tried a lot....! So I went forward.
After all all 3 nodes are back online running reef my cluster seems
to work, my cephfs share is reachable by my client machine, backups
are working.
This is my ceph status output:
cluster:
id: c5986871-ab9f-4365-8113-a1854fa07786
health: HEALTH_WARN
15 mgr modules have failed dependencies
services:
mon: 3 daemons, quorum ceph1,ceph2,ceph3 (age 19h)
mgr: ceph1(active, since 18h), standbys: ceph2, ceph3
mds: 1/1 daemons up, 2 standby
osd: 24 osds: 24 up (since 19h), 24 in (since 22M)
data:
volumes: 1/1 healthy
pools: 3 pools, 1041 pgs
objects: 3.34M objects, 11 TiB
usage: 32 TiB used, 56 TiB / 87 TiB avail
pgs: 1041 active+clean
Problem: dashboard is not working any more and I have this erre/warn:
15 mgr modules have failed dependencies
The cluster was made without podman/containers.
ceph-crash gives me a flood of this errors:
...
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T11:51:45.411885Z_738ee242-f7bd-4f6c-bd90-d96a08e06bf6 as client.admin failed: 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff728064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff728068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff72cea8410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash.ceph1 failed: 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81ac0640c8) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81ac0686c0) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81b3ea8410) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash failed: 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5b9c064068) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5b9c068540) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5ba2ea8410) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.admin failed: 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc844064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc844068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc8494fd410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash.ceph1 failed: 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d40640c8) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d40686c0) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d9aa8410) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash failed: 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a30064068) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a30068540) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a37aa8410) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.admin failed: 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa800064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa800068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa805ea8410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
......
In /etc/ceph I have
ceph.client.admin.keyring
ceph.conf
rbdmap
Maye there is a way to fix this but if it's to complicated I will
think about a complete re-install.
I'm not an admin but an amateur with some linux knowledge.
If there is hope and someone could support to solev this I would be
very happy!
Regards,
Christian Peters
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
