Hi EUgen
great, that sounds not too bad.
Here is the output:
ceph health detail
HEALTH_WARN 15 mgr modules have failed dependencies
[WRN] MGR_MODULE_DEPENDENCY: 15 mgr modules have failed dependencies
Module 'balancer' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'crash' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'dashboard' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'devicehealth' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'iostat' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'nfs' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'orchestrator' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'pg_autoscaler' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'progress' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'prometheus' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'rbd_support' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'restful' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'status' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'telemetry' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Module 'volumes' has failed dependency: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
Regards,
Christian
> Eugen Block <eblock@xxxxxx> hat am 22.08.2025 12:49 CEST geschrieben:
>
>
> Hi,
>
> this is absolutely the right place to ask.
>
> First, I would separate the issues (dashboard and crash). And there's
> no need to redeploy anything, it's just a mgr issue (and crash),
> nothing critical regarding the operational state of your cluster.
>
> Can you paste the output of 'ceph health detail'? And then you should
> look into the active mgr log and search for stack traces or errors in
> general.
>
> I would ignore crash for now, fix the mgr first.
>
> Regards,
> Eugen
>
> Zitat von info@xxxxxxxxxxx:
>
> > Hi all,
> >
> > not sure it's ok to ask questions here as we are only a little
> > Theater where I manage the data of our sound and video files.
> > I set up a ceph cluster as second backup location on bullseye with
> > quincy which worked flawless.
> > All runs smooth but I thought I would have to do a step forward to
> > get on track.
> > Yes I know: never change a running system. :-D
> > After updating all three nodes I forget to set the ceph packages on
> > hold so they were updated to reef with trixie. No way to downgrade
> > ...I tried a lot....! So I went forward.
> > After all all 3 nodes are back online running reef my cluster seems
> > to work, my cephfs share is reachable by my client machine, backups
> > are working.
> > This is my ceph status output:
> >
> > cluster:
> > id: c5986871-ab9f-4365-8113-a1854fa07786
> > health: HEALTH_WARN
> > 15 mgr modules have failed dependencies
> >
> > services:
> > mon: 3 daemons, quorum ceph1,ceph2,ceph3 (age 19h)
> > mgr: ceph1(active, since 18h), standbys: ceph2, ceph3
> > mds: 1/1 daemons up, 2 standby
> > osd: 24 osds: 24 up (since 19h), 24 in (since 22M)
> >
> > data:
> > volumes: 1/1 healthy
> > pools: 3 pools, 1041 pgs
> > objects: 3.34M objects, 11 TiB
> > usage: 32 TiB used, 56 TiB / 87 TiB avail
> > pgs: 1041 active+clean
> >
> > Problem: dashboard is not working any more and I have this erre/warn:
> >
> > 15 mgr modules have failed dependencies
> >
> >
> > The cluster was made without podman/containers.
> >
> > ceph-crash gives me a flood of this errors:
> >
> > ...
> > WARNING:ceph-crash:post
> > /var/lib/ceph/crash/2025-08-21T11:51:45.411885Z_738ee242-f7bd-4f6c-bd90-d96a08e06bf6 as client.admin failed: 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1
> > AuthRegistry(0x7ff728064068) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
> > AuthRegistry(0x7ff728068540) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
> > AuthRegistry(0x7ff72cea8410) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > [errno 2] RADOS object not found (error connecting to the cluster)
> > WARNING:ceph-crash:post
> > /var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash.ceph1 failed: 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
> > AuthRegistry(0x7f81ac0640c8) no keyring found at
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
> > AuthRegistry(0x7f81ac0686c0) no keyring found at
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
> > AuthRegistry(0x7f81b3ea8410) no keyring found at
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > [errno 2] RADOS object not found (error connecting to the cluster)
> > WARNING:ceph-crash:post
> > /var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash failed: 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
> > AuthRegistry(0x7f5b9c064068) no keyring found at
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
> > AuthRegistry(0x7f5b9c068540) no keyring found at
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
> > AuthRegistry(0x7f5ba2ea8410) no keyring found at
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > [errno 2] RADOS object not found (error connecting to the cluster)
> > WARNING:ceph-crash:post
> > /var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.admin failed: 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1
> > AuthRegistry(0x7fc844064068) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
> > AuthRegistry(0x7fc844068540) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
> > AuthRegistry(0x7fc8494fd410) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > [errno 2] RADOS object not found (error connecting to the cluster)
> > WARNING:ceph-crash:post
> > /var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash.ceph1 failed: 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
> > AuthRegistry(0x7f83d40640c8) no keyring found at
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
> > AuthRegistry(0x7f83d40686c0) no keyring found at
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
> > AuthRegistry(0x7f83d9aa8410) no keyring found at
> > /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > [errno 2] RADOS object not found (error connecting to the cluster)
> > WARNING:ceph-crash:post
> > /var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash failed: 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1
> > AuthRegistry(0x7f2a30064068) no keyring found at
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
> > AuthRegistry(0x7f2a30068540) no keyring found at
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
> > AuthRegistry(0x7f2a37aa8410) no keyring found at
> > /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > [errno 2] RADOS object not found (error connecting to the cluster)
> > WARNING:ceph-crash:post
> > /var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.admin failed: 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
> > AuthRegistry(0x7fa800064068) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
> > AuthRegistry(0x7fa800068540) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
> > keyring on
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or
> > directory
> > 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
> > AuthRegistry(0x7fa805ea8410) no keyring found at
> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling
> > cephx
> > [errno 2] RADOS object not found (error connecting to the cluster)
> > ......
> >
> > In /etc/ceph I have
> >
> > ceph.client.admin.keyring
> > ceph.conf
> > rbdmap
> >
> >
> > Maye there is a way to fix this but if it's to complicated I will
> > think about a complete re-install.
> > I'm not an admin but an amateur with some linux knowledge.
> > If there is hope and someone could support to solev this I would be
> > very happy!
> >
> > Regards,
> >
> > Christian Peters
> >
> >
> >
> > _______________________________________________
> > ceph-users mailing list -- ceph-users@xxxxxxx
> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
>
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
