Matthew Darwin <bugs@xxxxxxxxxx> hat am 22.08.2025 13:56 CEST geschrieben:
Please follow this tracker: https://tracker.ceph.com/issues/64213
(basically dashboard does not work on reef/debian)
On 2025-08-22 07:26, info@xxxxxxxxxxx wrote:
> Hi EUgen
>
> great, that sounds not too bad.
>
> Here is the output:
> ceph health detail
>
> HEALTH_WARN 15 mgr modules have failed dependencies
> [WRN] MGR_MODULE_DEPENDENCY: 15 mgr modules have failed dependencies
> Module 'balancer' has failed dependency: PyO3 modules do not
yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'crash' has failed dependency: PyO3 modules do not
yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'dashboard' has failed dependency: PyO3 modules do
not yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'devicehealth' has failed dependency: PyO3 modules do
not yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'iostat' has failed dependency: PyO3 modules do not
yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'nfs' has failed dependency: PyO3 modules do not yet
support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
> Module 'orchestrator' has failed dependency: PyO3 modules do
not yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'pg_autoscaler' has failed dependency: PyO3 modules
do not yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'progress' has failed dependency: PyO3 modules do not
yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'prometheus' has failed dependency: PyO3 modules do
not yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'rbd_support' has failed dependency: PyO3 modules do
not yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'restful' has failed dependency: PyO3 modules do not
yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'status' has failed dependency: PyO3 modules do not
yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'telemetry' has failed dependency: PyO3 modules do
not yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
> Module 'volumes' has failed dependency: PyO3 modules do not
yet support subinterpreters,
seehttps://github.com/PyO3/pyo3/issues/576
>
> Regards,
>
> Christian
>
>> Eugen Block<eblock@xxxxxx> hat am 22.08.2025 12:49 CEST geschrieben:
>>
>>
>> Hi,
>>
>> this is absolutely the right place to ask.
>>
>> First, I would separate the issues (dashboard and crash). And there's
>> no need to redeploy anything, it's just a mgr issue (and crash),
>> nothing critical regarding the operational state of your cluster.
>>
>> Can you paste the output of 'ceph health detail'? And then you should
>> look into the active mgr log and search for stack traces or errors in
>> general.
>>
>> I would ignore crash for now, fix the mgr first.
>>
>> Regards,
>> Eugen
>>
>> Zitat voninfo@xxxxxxxxxxx:
>>
>>> Hi all,
>>>
>>> not sure it's ok to ask questions here as we are only a little
>>> Theater where I manage the data of our sound and video files.
>>> I set up a ceph cluster as second backup location on bullseye with
>>> quincy which worked flawless.
>>> All runs smooth but I thought I would have to do a step forward to
>>> get on track.
>>> Yes I know: never change a running system. :-D
>>> After updating all three nodes I forget to set the ceph packages on
>>> hold so they were updated to reef with trixie. No way to downgrade
>>> ...I tried a lot....! So I went forward.
>>> After all all 3 nodes are back online running reef my cluster seems
>>> to work, my cephfs share is reachable by my client machine, backups
>>> are working.
>>> This is my ceph status output:
>>>
>>> cluster:
>>> id: c5986871-ab9f-4365-8113-a1854fa07786
>>> health: HEALTH_WARN
>>> 15 mgr modules have failed dependencies
>>>
>>> services:
>>> mon: 3 daemons, quorum ceph1,ceph2,ceph3 (age 19h)
>>> mgr: ceph1(active, since 18h), standbys: ceph2, ceph3
>>> mds: 1/1 daemons up, 2 standby
>>> osd: 24 osds: 24 up (since 19h), 24 in (since 22M)
>>>
>>> data:
>>> volumes: 1/1 healthy
>>> pools: 3 pools, 1041 pgs
>>> objects: 3.34M objects, 11 TiB
>>> usage: 32 TiB used, 56 TiB / 87 TiB avail
>>> pgs: 1041 active+clean
>>>
>>> Problem: dashboard is not working any more and I have this erre/warn:
>>>
>>> 15 mgr modules have failed dependencies
>>>
>>>
>>> The cluster was made without podman/containers.
>>>
>>> ceph-crash gives me a flood of this errors:
>>>
>>> ...
>>> WARNING:ceph-crash:post
>>>
/var/lib/ceph/crash/2025-08-21T11:51:45.411885Z_738ee242-f7bd-4f6c-bd90-d96a08e06bf6 as client.admin failed: 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1
>>> AuthRegistry(0x7ff728064068) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
>>> AuthRegistry(0x7ff728068540) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
>>> AuthRegistry(0x7ff72cea8410) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> [errno 2] RADOS object not found (error connecting to the cluster)
>>> WARNING:ceph-crash:post
>>>
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash.ceph1 failed: 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
>>> AuthRegistry(0x7f81ac0640c8) no keyring found at
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
>>> AuthRegistry(0x7f81ac0686c0) no keyring found at
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
>>> AuthRegistry(0x7f81b3ea8410) no keyring found at
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> [errno 2] RADOS object not found (error connecting to the cluster)
>>> WARNING:ceph-crash:post
>>>
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash failed: 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
>>> AuthRegistry(0x7f5b9c064068) no keyring found at
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
>>> AuthRegistry(0x7f5b9c068540) no keyring found at
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
>>> AuthRegistry(0x7f5ba2ea8410) no keyring found at
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> [errno 2] RADOS object not found (error connecting to the cluster)
>>> WARNING:ceph-crash:post
>>>
/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.admin failed: 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1
>>> AuthRegistry(0x7fc844064068) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
>>> AuthRegistry(0x7fc844068540) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
>>> AuthRegistry(0x7fc8494fd410) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> [errno 2] RADOS object not found (error connecting to the cluster)
>>> WARNING:ceph-crash:post
>>>
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash.ceph1 failed: 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
>>> AuthRegistry(0x7f83d40640c8) no keyring found at
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
>>> AuthRegistry(0x7f83d40686c0) no keyring found at
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
>>> AuthRegistry(0x7f83d9aa8410) no keyring found at
>>>
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> [errno 2] RADOS object not found (error connecting to the cluster)
>>> WARNING:ceph-crash:post
>>>
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash failed: 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1
>>> AuthRegistry(0x7f2a30064068) no keyring found at
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
>>> AuthRegistry(0x7f2a30068540) no keyring found at
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
>>> AuthRegistry(0x7f2a37aa8410) no keyring found at
>>>
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> [errno 2] RADOS object not found (error connecting to the cluster)
>>> WARNING:ceph-crash:post
>>>
/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.admin failed: 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
>>> AuthRegistry(0x7fa800064068) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
>>> AuthRegistry(0x7fa800068540) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
>>> keyring on
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file
or
>>> directory
>>> 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
>>> AuthRegistry(0x7fa805ea8410) no keyring found at
>>>
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
disabling
>>> cephx
>>> [errno 2] RADOS object not found (error connecting to the cluster)
>>> ......
>>>
>>> In /etc/ceph I have
>>>
>>> ceph.client.admin.keyring
>>> ceph.conf
>>> rbdmap
>>>
>>>
>>> Maye there is a way to fix this but if it's to complicated I will
>>> think about a complete re-install.
>>> I'm not an admin but an amateur with some linux knowledge.
>>> If there is hope and someone could support to solev this I would be
>>> very happy!
>>>
>>> Regards,
>>>
>>> Christian Peters
>>>
>>>
>>>
>>> _______________________________________________
>>> ceph-users mailing list --ceph-users@xxxxxxx
>>> To unsubscribe send an email toceph-users-leave@xxxxxxx
>>
>> _______________________________________________
>> ceph-users mailing list --ceph-users@xxxxxxx
>> To unsubscribe send an email toceph-users-leave@xxxxxxx
> _______________________________________________
> ceph-users mailing list --ceph-users@xxxxxxx
> To unsubscribe send an email toceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
