Hi all,
not sure it's ok to ask questions here as we are only a little
Theater where I manage the data of our sound and video files.
I set up a ceph cluster as second backup location on bullseye with
quincy which worked flawless.
All runs smooth but I thought I would have to do a step forward to
get on track.
Yes I know: never change a running system. :-D
After updating all three nodes I forget to set the ceph packages on
hold so they were updated to reef with trixie. No way to downgrade
...I tried a lot....! So I went forward.
After all all 3 nodes are back online running reef my cluster seems
to work, my cephfs share is reachable by my client machine, backups
are working.
This is my ceph status output:
cluster:
id: c5986871-ab9f-4365-8113-a1854fa07786
health: HEALTH_WARN
15 mgr modules have failed dependencies
services:
mon: 3 daemons, quorum ceph1,ceph2,ceph3 (age 19h)
mgr: ceph1(active, since 18h), standbys: ceph2, ceph3
mds: 1/1 daemons up, 2 standby
osd: 24 osds: 24 up (since 19h), 24 in (since 22M)
data:
volumes: 1/1 healthy
pools: 3 pools, 1041 pgs
objects: 3.34M objects, 11 TiB
usage: 32 TiB used, 56 TiB / 87 TiB avail
pgs: 1041 active+clean
Problem: dashboard is not working any more and I have this erre/warn:
15 mgr modules have failed dependencies
The cluster was made without podman/containers.
ceph-crash gives me a flood of this errors:
...
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T11:51:45.411885Z_738ee242-f7bd-4f6c-bd90-d96a08e06bf6 as client.admin failed: 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff728064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff728068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff72cea8410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash.ceph1 failed: 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81ac0640c8) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81ac0686c0) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81b3ea8410) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash failed: 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5b9c064068) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5b9c068540) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5ba2ea8410) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.admin failed: 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc844064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc844068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc8494fd410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash.ceph1 failed: 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d40640c8) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d40686c0) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d9aa8410) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash failed: 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a30064068) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a30068540) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a37aa8410) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
WARNING:ceph-crash:post
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.admin failed: 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa800064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa800068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa805ea8410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
......
In /etc/ceph I have
ceph.client.admin.keyring
ceph.conf
rbdmap
Maye there is a way to fix this but if it's to complicated I will
think about a complete re-install.
I'm not an admin but an amateur with some linux knowledge.
If there is hope and someone could support to solev this I would be
very happy!
Regards,
Christian Peters
_______________________________________________
ceph-users mailing list --ceph-users@xxxxxxx
To unsubscribe send an email toceph-users-leave@xxxxxxx