Re: Cephadm upgrade from 16.2.15 -> 17.2.0

[Eugen Block <eblock@xxxxxx>]

It would help if you only pasted the relevant parts. Anyway, these two  
sections stand out:

---snip---
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
debug 2025-04-05T20:33:48.909+0000 7f26f0200700 0 [balancer INFO root]  
Some PGs (1.000000) are unknown; try again later
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
debug 2025-04-05T20:33:48.917+0000 7f2663400700 -1 mgr load Failed to  
construct class in 'cephadm'
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
debug 2025-04-05T20:33:48.917+0000 7f2663400700 -1 mgr load Traceback  
(most recent call last):
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
File "/usr/share/ceph/mgr/cephadm/module.py", line 470, in __init__
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
self.upgrade = CephadmUpgrade(self)
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
File "/usr/share/ceph/mgr/cephadm/upgrade.py", line 112, in __init__
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
self.upgrade_state: Optional[UpgradeState] =  
UpgradeState.from_json(json.loads(t))
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
File "/usr/share/ceph/mgr/cephadm/upgrade.py", line 93, in from_json
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
return cls(**c)
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
TypeError: __init__() got an unexpected keyword argument 'daemon_types'
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
Apr 05 20:33:48 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
debug 2025-04-05T20:33:48.918+0000 7f2663400700 -1 mgr operator()  
Failed to run module in active mode ('cephadm')

Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
debug 2025-04-05T20:33:49.273+0000 7f2663400700 -1 mgr load Failed to  
construct class in 'snap_schedule'
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
debug 2025-04-05T20:33:49.273+0000 7f2663400700 -1 mgr load Traceback  
(most recent call last):
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
File "/usr/share/ceph/mgr/snap_schedule/module.py", line 38, in __init__
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
self.client = SnapSchedClient(self)
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line  
158, in __init__
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
with self.get_schedule_db(fs_name) as conn_mgr:
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line  
192, in get_schedule_db
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
db.executescript(dump)
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
sqlite3.OperationalError: table schedules already exists
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
Apr 05 20:33:49 cn03.ceph.xyz.corp  
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
debug 2025-04-05T20:33:49.274+0000 7f2663400700 -1 mgr operator()  
Failed to run module in active mode ('snap_schedule')
---snip---

Your cluster seems to be in an error state (ceph -s) because of an  
unknown PG. It's recommended to have a healthy cluster before  
attemping an upgrade. It's possible that these errors come from the  
not upgraded MGR, I'm not sure.

Since the upgrade was only successful for two MGRs, I am thinking  
about downgrading both MGRs back to 16.2.15, then retry an upgrade to  
a newer version, either 17.2.8 or 18.2.4. I haven't checked the  
snap_schedule error yet, though. Maybe someone else knows that already.

Zitat von Jeremy Hansen <jeremy@xxxxxxxxxx>:

> This is a small three node cluster. Here’s logs from all three nodes  
> during the failover. The current manager was cn02 and it failed over  
> to cn03.
>
> cn01
> ——
> Apr 05 20:33:35 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:35.823+0000 7f2aace00700 0  
> log_channel(cluster) log [DBG] : pgmap v821: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 22 KiB/s wr, 5 op/s
> Apr 05 20:33:37 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:37.170+0000 7f2aaec00700 0 [insights INFO  
> root] Received health check update 0 pending
> Apr 05 20:33:37 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:37.215+0000 7f2aab000700 0 [insights INFO  
> root] Applying 1 health updates to slot key  
> health_history/2025-04-05_20 next flush None checks check count 0
> Apr 05 20:33:37 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:37.215+0000 7f2aab000700 0 [insights INFO  
> root] Maybe flushing slot key health_history/2025-04-05_20 next  
> flush None checks check count 0 needed False
> Apr 05 20:33:37 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:37.823+0000 7f2aace00700 0  
> log_channel(cluster) log [DBG] : pgmap v822: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 22 KiB/s wr, 5 op/s
> Apr 05 20:33:39 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:39] "GET /metrics  
> HTTP/1.1" 200 267241 "" "Prometheus/2.33.4"
> Apr 05 20:33:39 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:39.825+0000 7f2aace00700 0  
> log_channel(cluster) log [DBG] : pgmap v823: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 97 KiB/s wr, 12 op/s
> Apr 05 20:33:41 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:41.826+0000 7f2aace00700 0  
> log_channel(cluster) log [DBG] : pgmap v824: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 85 KiB/s wr, 10 op/s
> Apr 05 20:33:42 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:42.170+0000 7f2aaec00700 0 [insights INFO  
> root] Received health check update 0 pending
> Apr 05 20:33:42 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:42.191+0000 7f2aab000700 0 [insights INFO  
> root] Applying 1 health updates to slot key  
> health_history/2025-04-05_20 next flush None checks check count 0
> Apr 05 20:33:42 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:42.191+0000 7f2aab000700 0 [insights INFO  
> root] Maybe flushing slot key health_history/2025-04-05_20 next  
> flush None checks check count 0 needed False
> Apr 05 20:33:42 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:42] "GET /metrics  
> HTTP/1.1" 200 267249 "" "Prometheus/2.33.4"
> Apr 05 20:33:43 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:43] "GET /metrics  
> HTTP/1.1" 200 267249 "" "Prometheus/2.33.4"
> Apr 05 20:33:43 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:43.827+0000 7f2aace00700 0  
> log_channel(cluster) log [DBG] : pgmap v825: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 511 B/s rd, 133 KiB/s wr, 18 op/s
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 -1 mgr  
> handle_mgr_map I was active but no longer am
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn e:  
> '/usr/bin/ceph-mgr'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 0:  
> '/usr/bin/ceph-mgr'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 1: '-n'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 2:  
> 'mgr.cn01.tjmtph'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 3: '-f'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 4:  
> '--setuser'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 5:  
> 'ceph'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 6:  
> '--setgroup'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 7:  
> 'ceph'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 8:  
> '--default-log-to-file=false'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 9:  
> '--default-log-to-stderr=true'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 10:  
> '--default-log-stderr-prefix=debug '
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn  
> respawning with exe /usr/bin/ceph-mgr
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn  
> exe_path /proc/self/exe
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ignoring --setuser ceph since I am not root
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ignoring --setgroup ceph since I am not root
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.449+0000 7fb814447000 0 ceph version  
> 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable),  
> process ceph-mgr, pid 2
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.449+0000 7fb814447000 0 pidfile_write:  
> ignore empty --pid-file
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.477+0000 7fb814447000 1 mgr[py] Loading  
> python module 'alerts'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.612+0000 7fb814447000 -1 mgr[py] Module  
> alerts has missing NOTIFY_TYPES member
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.612+0000 7fb814447000 1 mgr[py] Loading  
> python module 'balancer'
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.726+0000 7fb814447000 -1 mgr[py] Module  
> balancer has missing NOTIFY_TYPES member
> Apr 05 20:33:45 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:45.726+0000 7fb814447000 1 mgr[py] Loading  
> python module 'cephadm'
> Apr 05 20:33:46 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:46.417+0000 7fb814447000 1 mgr[py] Loading  
> python module 'crash'
> Apr 05 20:33:46 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:46.540+0000 7fb814447000 -1 mgr[py] Module  
> crash has missing NOTIFY_TYPES member
> Apr 05 20:33:46 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:46.540+0000 7fb814447000 1 mgr[py] Loading  
> python module 'dashboard'
> Apr 05 20:33:47 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:47.210+0000 7fb814447000 1 mgr[py] Loading  
> python module 'devicehealth'
> Apr 05 20:33:47 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:47.331+0000 7fb814447000 -1 mgr[py] Module  
> devicehealth has missing NOTIFY_TYPES member
> Apr 05 20:33:47 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:47.331+0000 7fb814447000 1 mgr[py] Loading  
> python module 'diskprediction_local'
> Apr 05 20:33:47 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:47.653+0000 7fb814447000 -1 mgr[py] Module  
> diskprediction_local has missing NOTIFY_TYPES member
> Apr 05 20:33:47 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:47.654+0000 7fb814447000 1 mgr[py] Loading  
> python module 'influx'
> Apr 05 20:33:47 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:47.769+0000 7fb814447000 -1 mgr[py] Module  
> influx has missing NOTIFY_TYPES member
> Apr 05 20:33:47 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:47.769+0000 7fb814447000 1 mgr[py] Loading  
> python module 'insights'
> Apr 05 20:33:47 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:47.884+0000 7fb814447000 1 mgr[py] Loading  
> python module 'iostat'
> Apr 05 20:33:48 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:48.082+0000 7fb814447000 -1 mgr[py] Module  
> iostat has missing NOTIFY_TYPES member
> Apr 05 20:33:48 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:48.082+0000 7fb814447000 1 mgr[py] Loading  
> python module 'k8sevents'
> Apr 05 20:33:48 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:48.679+0000 7fb814447000 1 mgr[py] Loading  
> python module 'localpool'
> Apr 05 20:33:48 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:48.796+0000 7fb814447000 1 mgr[py] Loading  
> python module 'mds_autoscaler'
> Apr 05 20:33:48 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:48.983+0000 7fb814447000 1 mgr[py] Loading  
> python module 'mirroring'
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.118+0000 7fb814447000 1 mgr[py] Loading  
> python module 'nfs'
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.315+0000 7fb814447000 -1 mgr[py] Module  
> nfs has missing NOTIFY_TYPES member
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.315+0000 7fb814447000 1 mgr[py] Loading  
> python module 'orchestrator'
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.607+0000 7fb814447000 -1 mgr[py] Module  
> orchestrator has missing NOTIFY_TYPES member
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.607+0000 7fb814447000 1 mgr[py] Loading  
> python module 'osd_perf_query'
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.723+0000 7fb814447000 -1 mgr[py] Module  
> osd_perf_query has missing NOTIFY_TYPES member
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.723+0000 7fb814447000 1 mgr[py] Loading  
> python module 'osd_support'
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.837+0000 7fb814447000 -1 mgr[py] Module  
> osd_support has missing NOTIFY_TYPES member
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.837+0000 7fb814447000 1 mgr[py] Loading  
> python module 'pg_autoscaler'
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.970+0000 7fb814447000 -1 mgr[py] Module  
> pg_autoscaler has missing NOTIFY_TYPES member
> Apr 05 20:33:49 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:49.970+0000 7fb814447000 1 mgr[py] Loading  
> python module 'progress'
> Apr 05 20:33:50 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:50.082+0000 7fb814447000 -1 mgr[py] Module  
> progress has missing NOTIFY_TYPES member
> Apr 05 20:33:50 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:50.082+0000 7fb814447000 1 mgr[py] Loading  
> python module 'prometheus'
> Apr 05 20:33:50 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:50.617+0000 7fb814447000 -1 mgr[py] Module  
> prometheus has missing NOTIFY_TYPES member
> Apr 05 20:33:50 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:50.617+0000 7fb814447000 1 mgr[py] Loading  
> python module 'rbd_support'
> Apr 05 20:33:50 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:50.761+0000 7fb814447000 -1 mgr[py] Module  
> rbd_support has missing NOTIFY_TYPES member
> Apr 05 20:33:50 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:50.761+0000 7fb814447000 1 mgr[py] Loading  
> python module 'restful'
> Apr 05 20:33:51 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:51.023+0000 7fb814447000 1 mgr[py] Loading  
> python module 'rook'
> Apr 05 20:33:51 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:51.788+0000 7fb814447000 -1 mgr[py] Module  
> rook has missing NOTIFY_TYPES member
> Apr 05 20:33:51 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:51.788+0000 7fb814447000 1 mgr[py] Loading  
> python module 'selftest'
> Apr 05 20:33:51 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:51.912+0000 7fb814447000 -1 mgr[py] Module  
> selftest has missing NOTIFY_TYPES member
> Apr 05 20:33:51 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:51.912+0000 7fb814447000 1 mgr[py] Loading  
> python module 'snap_schedule'
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.038+0000 7fb814447000 -1 mgr[py] Module  
> snap_schedule has missing NOTIFY_TYPES member
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.038+0000 7fb814447000 1 mgr[py] Loading  
> python module 'stats'
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.155+0000 7fb814447000 1 mgr[py] Loading  
> python module 'status'
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.280+0000 7fb814447000 -1 mgr[py] Module  
> status has missing NOTIFY_TYPES member
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.280+0000 7fb814447000 1 mgr[py] Loading  
> python module 'telegraf'
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.398+0000 7fb814447000 -1 mgr[py] Module  
> telegraf has missing NOTIFY_TYPES member
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.398+0000 7fb814447000 1 mgr[py] Loading  
> python module 'telemetry'
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.594+0000 7fb814447000 -1 mgr[py] Module  
> telemetry has missing NOTIFY_TYPES member
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.594+0000 7fb814447000 1 mgr[py] Loading  
> python module 'test_orchestrator'
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.771+0000 7fb814447000 -1 mgr[py] Module  
> test_orchestrator has missing NOTIFY_TYPES member
> Apr 05 20:33:52 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:52.771+0000 7fb814447000 1 mgr[py] Loading  
> python module 'volumes'
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.225+0000 7fb814447000 -1 mgr[py] Module  
> volumes has missing NOTIFY_TYPES member
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.225+0000 7fb814447000 1 mgr[py] Loading  
> python module 'zabbix'
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.342+0000 7fb814447000 -1 mgr[py] Module  
> zabbix has missing NOTIFY_TYPES member
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.344+0000 7fb807c00700 0  
> ms_deliver_dispatch: unhandled message 0x555c78fe1080 mon_map magic:  
> 0 v1 from mon.0 v2:192.168.47.71:3300/0
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.344+0000 7fb808600700 0 [dashboard DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.345+0000 7fb808600700 1 mgr load  
> Constructed class from module: dashboard
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.345+0000 7fb808600700 0 [prometheus DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.345+0000 7fb808600700 1 mgr load  
> Constructed class from module: prometheus
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.345+0000 7fb6abe00700 0 [dashboard INFO  
> root] server: ssl=yes host=:: port=8443
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.346+0000 7fb6b4a00700 0 [prometheus INFO  
> root] server_addr: :: server_port: 9283
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.346+0000 7fb6b4a00700 0 [prometheus INFO  
> root] Starting engine...
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> [05/Apr/2025:20:33:53] ENGINE Bus STARTING
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.346+0000 7fb6b4a00700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:53] ENGINE Bus STARTING
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.377+0000 7fb6abe00700 0 [dashboard INFO  
> root] Configured CherryPy, starting engine...
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.377+0000 7fb6abe00700 0 [dashboard INFO  
> root] Starting engine...
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> CherryPy Checker:
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> The Application mounted at '' has an empty config.
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> [05/Apr/2025:20:33:53] ENGINE Serving on http://:::9283
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.450+0000 7fb6b4a00700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:53] ENGINE Serving on  
> http://:::9283
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> [05/Apr/2025:20:33:53] ENGINE Bus STARTED
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.450+0000 7fb6b4a00700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:53] ENGINE Bus STARTED
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.451+0000 7fb6b4a00700 0 [prometheus INFO  
> root] Engine started.
> Apr 05 20:33:53 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:53.489+0000 7fb6abe00700 0 [dashboard INFO  
> root] Engine started...
> Apr 05 20:33:59 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:59] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:33:59 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:33:59.603+0000 7fb76c600700 0 [prometheus INFO  
> cherrypy.access.140425911062880] ::ffff:192.168.47.72 - -  
> [05/Apr/2025:20:33:59] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:02 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:02] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:34:02 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:02.476+0000 7fb763a00700 0 [prometheus INFO  
> cherrypy.access.140425911062880] ::ffff:192.168.47.71 - -  
> [05/Apr/2025:20:34:02] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:03 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:03] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:34:03 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:03.061+0000 7fb75ae00700 0 [prometheus INFO  
> cherrypy.access.140425911062880] ::ffff:192.168.47.73 - -  
> [05/Apr/2025:20:34:03] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:09 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:09] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:34:09 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:09.602+0000 7fb752200700 0 [prometheus INFO  
> cherrypy.access.140425911062880] ::ffff:192.168.47.72 - -  
> [05/Apr/2025:20:34:09] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:12 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:12] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:34:12 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:12.476+0000 7fb749600700 0 [prometheus INFO  
> cherrypy.access.140425911062880] ::ffff:192.168.47.71 - -  
> [05/Apr/2025:20:34:12] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:13 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:13] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:34:13 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:13.060+0000 7fb740a00700 0 [prometheus INFO  
> cherrypy.access.140425911062880] ::ffff:192.168.47.73 - -  
> [05/Apr/2025:20:34:13] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.493+0000 7fb714e00700 0 [dashboard INFO  
> root] Redirecting to active 'https://192.168.47.73:8443/'
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.494+0000 7fb714e00700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52368] [GET] [303] [0.003s] [96.0B]  
> [0031fbdd-947f-4c6c-943d-55cf256edfba] /api/summary
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.512+0000 7fb70c200700 0 [dashboard INFO  
> root] Redirecting to active 'https://192.168.47.73:8443/'
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.513+0000 7fb70c200700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52369] [GET] [303] [0.001s] [96.0B]  
> [76c9a6ab-aae0-4ef9-815e-dbd0ac8f8b49] /api/feature_toggles
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.527+0000 7fb703600700 0 [dashboard INFO  
> root] Redirecting to active 'https://192.168.47.73:8443/'
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.528+0000 7fb703600700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52370] [GET] [303] [0.001s] [96.0B]  
> [db017da3-e6f6-45fc-9d99-7b9f3b3859f2] /api/health/minimal
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.541+0000 7fb6faa00700 0 [dashboard INFO  
> root] Redirecting to active 'https://192.168.47.73:8443/'
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.542+0000 7fb6faa00700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52371] [GET] [303] [0.001s] [96.0B]  
> [564ba322-b99c-49de-a3a3-e0ed422bfae7] /api/prometheus
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.616+0000 7fb6f1e00700 0 [dashboard INFO  
> root] Redirecting to active 'https://192.168.47.73:8443/'
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.617+0000 7fb6f1e00700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52372] [GET] [303] [0.001s] [96.0B]  
> [1f349e52-9a10-4fda-b13f-621785208ea9] /api/prometheus/rules
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.630+0000 7fb6e0600700 0 [dashboard INFO  
> root] Redirecting to active 'https://192.168.47.73:8443/'
> Apr 05 20:34:15 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:15.630+0000 7fb6e0600700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52368] [GET] [303] [0.001s] [96.0B]  
> [30d749d7-c972-44f0-88c7-b70078e2385c] /api/prometheus/notifications
> Apr 05 20:34:19 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:19] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:34:19 cn01.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:  
> debug 2025-04-05T20:34:19.601+0000 7fb737e00700 0 [prometheus INFO  
> cherrypy.access.140425911062880] ::ffff:192.168.47.72 - -  
> [05/Apr/2025:20:34:19] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
>
> cn02
> ——
> Apr 05 20:33:40 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:40] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:40 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:40.701+0000 7ff710400700 0 [prometheus INFO cherrypy.access.140700134177256] ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:40] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:43 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:43] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:43 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:43.287+0000 7ff707800700 0 [prometheus INFO cherrypy.access.140700134177256] ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:43] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.370+0000 7ff7e7c00700 1 mgr handle_mgr_map  
> Activating!
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.371+0000 7ff7e7c00700 1 mgr handle_mgr_map I am now  
> activating
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.382+0000 7ff67aa00700 0 [dashboard INFO root] Stopping  
> engine...
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.382+0000 7ff67aa00700 0 [dashboard INFO root] Stopped  
> engine...
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.382+0000 7ff67aa00700 0 [prometheus INFO root] Stopping  
> engine...
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.382+0000 7ff67aa00700 0 [prometheus INFO root] Stopped  
> engine
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE Bus  
> STOPPING
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.383+0000 7ff695c00700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE Bus  
> STOPPING
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.394+0000 7ff68d000700 0 [dashboard INFO root] Engine  
> stopped.
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut  
> down
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.395+0000 7ff695c00700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut  
> down
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE Bus  
> STOPPED
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.396+0000 7ff695c00700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE Bus  
> STOPPED
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.396+0000 7ff695c00700 0 [prometheus INFO root] Engine  
> stopped.
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.398+0000 7ff67aa00700 0 [alerts DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.398+0000 7ff67aa00700 0 [alerts INFO root]  
> Init
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.398+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> alerts
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.399+0000 7ff67aa00700 0 [balancer DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.399+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> balancer
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.399+0000 7ff74d800700 0 [alerts INFO root]  
> Starting
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.401+0000 7ff67aa00700 0 [cephadm DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.402+0000 7ff710400700 0 [balancer INFO root]  
> Starting
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.408+0000 7ff710400700 0 [balancer INFO root] Optimize plan  
> auto_2025-04-05_20:33:45
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.408+0000 7ff710400700 0 [balancer INFO root] Mode upmap, max misplaced  
> 0.050000
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.409+0000 7ff710400700 0 [balancer INFO root] Some PGs (1.000000) are unknown; try again  
> later
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.551+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> cephadm
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.552+0000 7ff67aa00700 0 [crash DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.552+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> crash
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.554+0000 7ff67aa00700 0 [dashboard DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.554+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> dashboard
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.555+0000 7ff67aa00700 0 [devicehealth DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.555+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> devicehealth
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.556+0000 7ff676400700 0 [dashboard INFO access_control] Loading user roles DB  
> version=2
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.556+0000 7ff676400700 0 [dashboard INFO sso] Loading SSO DB  
> version=1
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.556+0000 7ff676400700 0 [dashboard INFO root] server: ssl=yes host=::  
> port=8443
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.557+0000 7ff674600700 0 [devicehealth INFO root]  
> Starting
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.558+0000 7ff67aa00700 0 [insights DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.559+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> insights
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.561+0000 7ff674600700 0 [devicehealth INFO root] Check  
> health
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.566+0000 7ff671e00700 0 [insights INFO root] Reset curr health slot key health_history/2025-04-05_20 next flush None checks check count Apr 05 20:33:45 cn02.ceph.xyz.corp ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.567+0000 7ff67aa00700 0 [iostat DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.567+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> iostat
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.572+0000 7ff67aa00700 0 [mirroring DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.579+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> mirroring
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.604+0000 7ff67aa00700 0 [nfs DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.604+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> nfs
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.622+0000 7ff67aa00700 0 [orchestrator DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.622+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> orchestrator
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.623+0000 7ff67aa00700 0 [pg_autoscaler DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.623+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> pg_autoscaler
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.625+0000 7ff662a00700 0 [pg_autoscaler INFO root]  
> _maybe_adjust
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.631+0000 7ff67aa00700 0 [progress DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.631+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> progress
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: Warning: Permanently added '192.168.47.72' (ECDSA) to the list of known  
> hosts.
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: Warning: Permanently added '192.168.47.73' (ECDSA) to the list of known  
> hosts.
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.637+0000 7ff67aa00700 0 [prometheus DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.637+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> prometheus
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.638+0000 7ff661600700 0 [progress INFO root]  
> Loading...
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.638+0000 7ff660200700 0 [prometheus INFO root] server_addr: :: server_port:  
> 9283
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.638+0000 7ff660200700 0 [prometheus INFO root] Cache  
> enabled
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.638+0000 7ff65ee00700 0 [prometheus INFO root] starting metric collection  
> thread
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.640+0000 7ff67aa00700 0 [rbd_support DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.642+0000 7ff661600700 0 [progress INFO root] Loaded [<progress.module.GhostEvent object at 0x7ff7443993c8>, <progress.module.GhostEvent object at 0x7ff744399400>, <progress.module.GhostEvent object at 0x7ff744399438>, <progress.module.GhostEvent object at 0x7ff744399470>, <progress.module.GhostEvent object at 0x7ff7443994a8>, <progress.module.GhostEvent object at 0x7ff7443994e0>, <progress.module.GhostEvent object at 0x7ff744399518>, <progress.module.GhostEvent object at 0x7ff744399550>, <progress.module.GhostEvent object at 0x7ff744399588>, <progress.module.GhostEvent object at 0x7ff7443995c0>, <progress.module.GhostEvent object at 0x7ff7443995f8>, <progress.module.GhostEvent object at 0x7ff744399630>, <progress.module.GhostEvent object at 0x7ff744399668>, <progress.module.GhostEvent object at 0x7ff7443996a0>, <progress.module.GhostEvent object at 0x7ff7443996d8>, <progress.module.GhostEvent object at 0x7ff744399710>, <progress.module.GhostEvent object at 0x7ff744399748>, <progress.module.GhostEvent object at 0x7ff744399780>, <progress.module.GhostEvent object at 0x7ff7443997b8>, <progress.module.GhostEvent object at 0x7ff7443997f0>, <progress.module.GhostEvent object at 0x7ff744399828>, <progress.module.GhostEvent object at 0x7ff744399860>, <progress.module.GhostEvent object at 0x7ff744399898>, <progress.module.GhostEvent object at 0x7ff7443998d0>, <progress.module.GhostEvent object at 0x7ff744399908>, <progress.module.GhostEvent object at 0x7ff744399940>, <progress.module.GhostEvent object at 0x7ff744399978>, <progress.module.GhostEvent object at 0x7ff7443999b0>, <progress.module.GhostEvent object at 0x7ff7443999e8>, <progress.module.GhostEvent object at 0x7ff744399a20>, <progress.module.GhostEvent object at 0x7ff744399a58>, <progress.module.GhostEvent object at 0x7ff744399a90>, <progress.module.GhostEvent object at 0x7ff744399ac8>, <progress.module.GhostEvent object at 0x7ff744399b00>, <progress.modul
e.GhostEvent object at 0x7ff744399b38>, <progress.module.GhostEvent object at 0x7ff744399b70>, <progress.module.GhostEvent object at 0x7ff744399ba8>, <progress.module.GhostEvent object at 0x7ff744399be0>, <progress.module.GhostEvent object at 0x7ff744399c18>, <progress.module.GhostEvent object at 0x7ff744399c50>, <progress.module.GhostEvent object at 0x7ff744399c88>, <progress.module.GhostEvent object at 0x7ff744399cc0>, <progress.module.GhostEvent object at 0x7ff744399cf8>, <progress.module.GhostEvent object at 0x7ff744399d30>, <progress.module.GhostEvent object at 0x7ff744399d68>, <progress.module.GhostEvent object at 0x7ff744399da0>, <progress.module.GhostEvent object at 0x7ff744399dd8>, <progress.module.GhostEvent object at 0x7ff744399e10>, <progress.module.GhostEvent object at 0x7ff744399e48>, <progress.module.GhostEvent object at 0x7ff744399e80>] historic  
> events
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.642+0000 7ff661600700 0 [progress INFO root] Loaded OSDMap,  
> ready.
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.642+0000 7ff661600700 0 [progress INFO root] Processing OSDMap change  
> 27185..27185
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.646+0000 7ff676400700 0 [dashboard INFO root] Configured CherryPy, starting  
> engine...
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.646+0000 7ff660200700 0 [prometheus INFO root] Starting  
> engine...
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE Bus  
> STARTING
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.647+0000 7ff660200700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE Bus  
> STARTING
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.656+0000 7ff654e00700 0 [rbd_support INFO root] recovery thread  
> starting
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.656+0000 7ff654e00700 0 [rbd_support INFO root] starting  
> setup
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.656+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> rbd_support
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: Warning: Permanently added '192.168.47.71' (ECDSA) to the list of known  
> hosts.
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.659+0000 7ff67aa00700 0 [restful DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.659+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> restful
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.660+0000 7ff651c00700 0 [restful INFO root] server_addr: :: server_port:  
> 8003
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.661+0000 7ff654e00700 0 [rbd_support INFO root] MirrorSnapshotScheduleHandler:  
> load_schedules
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.697+0000 7ff651c00700 0 [restful WARNING root] server not running: no certificate  
> configured
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.705+0000 7ff654e00700 0 [rbd_support INFO root] load_schedules: csxyz,  
> start_after=
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.706+0000 7ff67aa00700 0 [snap_schedule DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.707+0000 7ff64f400700 0 [rbd_support INFO root] MirrorSnapshotScheduleHandler:  
> starting
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.708+0000 7ff64ea00700 0 [rbd_support INFO root] PerfHandler:  
> starting
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.709+0000 7ff654e00700 0 [rbd_support INFO root] load_task_task: csxyz,  
> start_after=
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.718+0000 7ff649a00700 0 [rbd_support INFO root] TaskHandler:  
> starting
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.719+0000 7ff654e00700 0 [rbd_support INFO root] TrashPurgeScheduleHandler:  
> load_schedules
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.721+0000 7ff654e00700 0 [rbd_support INFO root] load_schedules: csxyz,  
> start_after=
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.723+0000 7ff649000700 0 [rbd_support INFO root] TrashPurgeScheduleHandler:  
> starting
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.723+0000 7ff654e00700 0 [rbd_support INFO root] setup  
> complete
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.726+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> snap_schedule
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.727+0000 7ff67aa00700 0 [status DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.727+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> status
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.728+0000 7ff67aa00700 0 [telemetry DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.729+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> telemetry
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.729+0000 7ff67aa00700 0 [volumes DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.742+0000 7ff67aa00700 0 [volumes INFO volumes.fs.async_job] queuing job for volume  
> 'cssnfsxyz'
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.742+0000 7ff67aa00700 0 [volumes INFO volumes.fs.async_job] queuing job for volume  
> 'cssnfsxyz'
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.742+0000 7ff67aa00700 1 mgr load Constructed class from module:  
> volumes
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17) File  
> exists
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: CherryPy  
> Checker:
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: The Application mounted at '' has an empty  
> config.
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE Serving on  
> http://:::9283
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.754+0000 7ff660200700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE Serving on  
> http://:::9283
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE Bus  
> STARTED
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.754+0000 7ff660200700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE Bus  
> STARTED
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.754+0000 7ff660200700 0 [prometheus INFO root] Engine  
> started.
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.785+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: CephFS ->  
> /api/cephfs
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.787+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: CephFsUi ->  
> /ui-api/cephfs
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.789+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: IscsiUi ->  
> /ui-api/iscsi
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.789+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Iscsi ->  
> /api/iscsi
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.789+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: IscsiTarget ->  
> /api/iscsi/target
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.791+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: NFSGaneshaCluster ->  
> /api/nfs-ganesha/cluster
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.791+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: NFSGaneshaExports ->  
> /api/nfs-ganesha/export
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.792+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: NFSGaneshaUi ->  
> /ui-api/nfs-ganesha
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.792+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Rbd ->  
> /api/block/image
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.795+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdStatus ->  
> /ui-api/block/rbd
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.795+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdSnapshot ->  
> /api/block/image/{image_spec}/snap
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.796+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdTrash ->  
> /api/block/image/trash
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.797+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdNamespace ->  
> /api/block/pool/{pool_name}/namespace
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.797+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Pool ->  
> /api/pool
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.799+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: PoolUi ->  
> /ui-api/pool
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.800+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RBDPool ->  
> /api/pool
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.801+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Orchestrator ->  
> /ui-api/orchestrator
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.801+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Service ->  
> /api/service
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.802+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroring ->  
> /api/block/mirroring
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.802+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringSummary ->  
> /api/block/mirroring/summary
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.803+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringPoolMode ->  
> /api/block/mirroring/pool
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.803+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringPoolBootstrap ->  
> /api/block/mirroring/pool/{pool_name}/bootstrap
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.803+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringPoolPeer ->  
> /api/block/mirroring/pool/{pool_name}/peer
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.804+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringStatus ->  
> /ui-api/block/mirroring
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.805+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Rgw ->  
> /ui-api/rgw
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.805+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwDaemon ->  
> /api/rgw/daemon
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.806+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwSite ->  
> /api/rgw/site
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.806+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwBucket ->  
> /api/rgw/bucket
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.807+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwUser ->  
> /api/rgw/user
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.810+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Auth ->  
> /api/auth
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.811+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Cluster ->  
> /api/cluster
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.811+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: ClusterConfiguration ->  
> /api/cluster_conf
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.813+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: CrushRule ->  
> /api/crush_rule
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.814+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: CrushRuleUi ->  
> /ui-api/crush_rule
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.814+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Daemon ->  
> /api/daemon
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.815+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Docs ->  
> /docs
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.815+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: ErasureCodeProfile ->  
> /api/erasure_code_profile
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.816+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: ErasureCodeProfileUi ->  
> /ui-api/erasure_code_profile
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.817+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: FrontendLogging ->  
> /ui-api/logging
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.817+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Grafana ->  
> /api/grafana
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.817+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Host ->  
> /api/host
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.819+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: HostUi ->  
> /ui-api/host
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.820+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Health ->  
> /api/health
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.820+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: HomeController ->  
> /
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.822+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: LangsController ->  
> /ui-api/langs
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.824+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: LoginController ->  
> /ui-api/login
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.824+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Logs ->  
> /api/logs
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.824+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MgrModules ->  
> /api/mgr/module
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.826+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Monitor ->  
> /api/monitor
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.826+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Osd ->  
> /api/osd
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.828+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: OsdUi ->  
> /ui-api/osd
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.830+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: OsdFlagsController ->  
> /api/osd/flags
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.831+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MdsPerfCounter ->  
> /api/perf_counters/mds
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.831+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MonPerfCounter ->  
> /api/perf_counters/mon
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.832+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: OsdPerfCounter ->  
> /api/perf_counters/osd
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.832+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwPerfCounter ->  
> /api/perf_counters/rgw
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.833+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirrorPerfCounter ->  
> /api/perf_counters/rbd-mirror
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.833+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MgrPerfCounter ->  
> /api/perf_counters/mgr
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.834+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: TcmuRunnerPerfCounter ->  
> /api/perf_counters/tcmu-runner
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.834+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: PerfCounters ->  
> /api/perf_counters
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.835+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: PrometheusReceiver ->  
> /api/prometheus_receiver
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.835+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Prometheus ->  
> /api/prometheus
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.836+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: PrometheusNotifications ->  
> /api/prometheus/notifications
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.836+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Role ->  
> /api/role
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.838+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Scope ->  
> /ui-api/scope
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.838+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Saml2 ->  
> /auth/saml2
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.838+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Settings ->  
> /api/settings
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.840+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: StandardSettings ->  
> /ui-api/standard_settings
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.840+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Summary ->  
> /api/summary
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.841+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Task ->  
> /api/task
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.841+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Telemetry ->  
> /api/telemetry
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.842+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: User ->  
> /api/user
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.843+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: UserPasswordPolicy ->  
> /api/user
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.843+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: UserChangePassword ->  
> /api/user/{username}
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.843+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: FeatureTogglesEndpoint ->  
> /api/feature_toggles
> Apr 05 20:33:45 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.844+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MessageOfTheDay ->  
> /ui-api/motd
> Apr 05 20:33:46 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:46.050+0000 7ff676400700 0 [dashboard INFO dashboard.module] Engine  
> started.
> Apr 05 20:33:46 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:46.378+0000 7ff67a000700 0 log_channel(cluster) log [DBG] : pgmap v3: 610 pgs: 610 active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB  
> avail
> Apr 05 20:33:47 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:47.379+0000 7ff678c00700 0 log_channel(cluster) log [DBG] : pgmap v4: 610 pgs: 610 active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB  
> avail
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:48] "GET /metrics HTTP/1.1" 200 42104 ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.568+0000 7ff65b200700 0 [prometheus INFO cherrypy.access.140699978276536] ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:48] "GET /metrics HTTP/1.1" 200 42104 ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff721c00700 0 [cephadm INFO cephadm.serve] Updating  
> cn01.ceph.xyz.corp:/etc/ceph/ceph.conf
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff721c00700 0 log_channel(cephadm) log [INF] : Updating  
> cn01.ceph.xyz.corp:/etc/ceph/ceph.conf
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff719000700 0 [cephadm INFO cephadm.serve] Updating  
> cn02.ceph.xyz.corp:/etc/ceph/ceph.conf
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff6fec00700 0 [cephadm INFO cephadm.serve] Updating  
> cn03.ceph.xyz.corp:/etc/ceph/ceph.conf
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff719000700 0 log_channel(cephadm) log [INF] : Updating  
> cn02.ceph.xyz.corp:/etc/ceph/ceph.conf
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff6fec00700 0 log_channel(cephadm) log [INF] : Updating  
> cn03.ceph.xyz.corp:/etc/ceph/ceph.conf
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff721c00700 0 [cephadm INFO cephadm.serve] Updating  
> cn01.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff721c00700 0 log_channel(cephadm) log [INF] : Updating  
> cn01.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff719000700 0 [cephadm INFO cephadm.serve] Updating  
> cn02.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff719000700 0 log_channel(cephadm) log [INF] : Updating  
> cn02.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff6fec00700 0 [cephadm INFO cephadm.serve] Updating  
> cn03.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff6fec00700 0 log_channel(cephadm) log [INF] : Updating  
> cn03.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.812+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 824b8fde-5a37-4436-af45-a7d8255e2c66 does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.813+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 621cb660-d584-45e4-8033-4a476e804bba does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.813+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 007a4dd2-46d3-4655-9c93-c02f97e555cb does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.814+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 6b0fbff2-881e-4203-9149-f0977bc13eab does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.814+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 8a1a90b8-1f2a-48c8-924f-6e74032b6319 does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.815+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 3c322d26-9edb-4d26-b4bc-97c330f5a1d8 does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.825+0000 7ff6a7400700 0 [progress WARNING root] complete: ev f3e5a517-d643-4fa5-b127-57716264e4e3 does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.825+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 3fb020c7-92e8-4eb1-a884-2d04e5bca889 does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.840+0000 7ff6a7400700 0 [progress WARNING root] complete: ev bf9e0705-075e-4134-b77f-aa64fe92112a does not  
> exist
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.848+0000 7ff6a7400700 0 [cephadm INFO cephadm.upgrade] Upgrade: Need to upgrade myself  
> (mgr.cn02.ceph.xyz.corp.ggixgj)
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.848+0000 7ff6a7400700 0 log_channel(cephadm) log [INF] : Upgrade: Need to upgrade myself  
> (mgr.cn02.ceph.xyz.corp.ggixgj)
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.848+0000 7ff6a7400700 0 [progress INFO root] update: starting ev e7e1a809-558d-43a7-842a-c6229fdc57af (Upgrade to  
> 17.2.0)
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.851+0000 7ff6a7400700 0 [cephadm INFO cephadm.services.cephadmservice] Failing over to other  
> MGR
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.851+0000 7ff6a7400700 0 log_channel(cephadm) log [INF] : Failing over to other  
> MGR
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 -1 mgr handle_mgr_map I was active but no longer  
> am
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn e:  
> '/usr/bin/ceph-mgr'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 0:  
> '/usr/bin/ceph-mgr'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 1:  
> '-n'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 2:  
> 'mgr.cn02.ceph.xyz.corp.ggixgj'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 3:  
> '-f'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 4:  
> '--setuser'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 5:  
> 'ceph'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 6:  
> '--setgroup'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 7:  
> 'ceph'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 8:  
> '--default-log-to-file=false'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 9:  
> '--default-log-to-stderr=true'
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 10: '--default-log-stderr-prefix=debug  
> '
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn respawning with exe  
> /usr/bin/ceph-mgr
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn exe_path  
> /proc/self/exe
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ignoring --setuser ceph since I am not  
> root
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ignoring --setgroup ceph since I am not  
> root
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.949+0000 7f5a66492680 0 ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516) pacific (stable), process ceph-mgr, pid  
> 2
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.949+0000 7f5a66492680 0 pidfile_write: ignore empty  
> --pid-file
> Apr 05 20:33:48 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.976+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'alerts'
> Apr 05 20:33:49 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.105+0000 7f5a66492680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:49 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.105+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'balancer'
> Apr 05 20:33:49 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.212+0000 7f5a66492680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:49 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.212+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'cephadm'
> Apr 05 20:33:49 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.568+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'crash'
> Apr 05 20:33:49 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.677+0000 7f5a66492680 -1 mgr[py] Module crash has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:49 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.677+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'dashboard'
> Apr 05 20:33:50 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.276+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'devicehealth'
> Apr 05 20:33:50 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.415+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'diskprediction_local'
> Apr 05 20:33:50 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.640+0000 7f5a66492680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:50 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.640+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'influx'
> Apr 05 20:33:50 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.750+0000 7f5a66492680 -1 mgr[py] Module influx has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:50 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.750+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'insights'
> Apr 05 20:33:50 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.919+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'iostat'
> Apr 05 20:33:51 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.020+0000 7f5a66492680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:51 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.020+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'k8sevents'
> Apr 05 20:33:51 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.533+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'localpool'
> Apr 05 20:33:51 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.634+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'mds_autoscaler'
> Apr 05 20:33:51 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.894+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'mirroring'
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.019+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'nfs'
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.209+0000 7f5a66492680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.209+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'orchestrator'
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.386+0000 7f5a66492680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.386+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'osd_support'
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.494+0000 7f5a66492680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.494+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'pg_autoscaler'
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.760+0000 7f5a66492680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.760+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'progress'
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.870+0000 7f5a66492680 -1 mgr[py] Module progress has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:52 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.870+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'prometheus'
> Apr 05 20:33:53 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.257+0000 7f5a66492680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:53 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.257+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'rbd_support'
> Apr 05 20:33:53 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.388+0000 7f5a66492680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:53 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.388+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'restful'
> Apr 05 20:33:53 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.799+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'rook'
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.318+0000 7f5a66492680 -1 mgr[py] Module rook has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.318+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'selftest'
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.428+0000 7f5a66492680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.428+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'snap_schedule'
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.549+0000 7f5a66492680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.549+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'stats'
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.660+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'status'
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.972+0000 7f5a66492680 -1 mgr[py] Module status has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:54 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.972+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'telegraf'
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.089+0000 7f5a66492680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.089+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'telemetry'
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.268+0000 7f5a66492680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.268+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'test_orchestrator'
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.448+0000 7f5a66492680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.448+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'volumes'
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.676+0000 7f5a66492680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.676+0000 7f5a66492680 1 mgr[py] Loading python module  
> 'zabbix'
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.792+0000 7f5a66492680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES  
> member
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.793+0000 7f5a52600700 0 ms_deliver_dispatch: unhandled message 0x556eb85f26e0 mon_map magic: 0 v1 from mon.2  
> v2:192.168.47.72:3300/0
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f5a53000700 0 [dashboard DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f5a53000700 1 mgr load Constructed class from module:  
> dashboard
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f5a53000700 0 [prometheus DEBUG root] setting log level based on debug_mgr: INFO  
> (2/5)
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f5a53000700 1 mgr load Constructed class from module:  
> prometheus
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f58f7800700 0 [dashboard INFO root] server: ssl=yes host=::  
> port=8443
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.795+0000 7f5900400700 0 [prometheus INFO root] server_addr: :: server_port:  
> 9283
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.795+0000 7f5900400700 0 [prometheus INFO root] Starting  
> engine...
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:55] ENGINE Bus  
> STARTING
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.795+0000 7f5900400700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:55] ENGINE Bus  
> STARTING
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.797+0000 7f5a52600700 0 client.0 ms_handle_reset on  
> v2:192.168.47.73:6842/917991990
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.825+0000 7f58f7800700 0 [dashboard INFO root] Configured CherryPy, starting  
> engine...
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.826+0000 7f58f7800700 0 [dashboard INFO root] Starting  
> engine...
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: CherryPy  
> Checker:
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: The Application mounted at '' has an empty  
> config.
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:55] ENGINE Serving on  
> http://:::9283
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.900+0000 7f5900400700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:55] ENGINE Serving on  
> http://:::9283
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:55] ENGINE Bus  
> STARTED
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.900+0000 7f5900400700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:55] ENGINE Bus  
> STARTED
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.900+0000 7f5900400700 0 [prometheus INFO root] Engine  
> started.
> Apr 05 20:33:55 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.939+0000 7f58f7800700 0 [dashboard INFO root] Engine  
> started...
> Apr 05 20:33:58 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:58] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:58 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:58.569+0000 7f59b8000700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:58] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:00 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:00] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:00 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:00.702+0000 7f59af400700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:00] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:03 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:03] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:03 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:03.286+0000 7f59a6800700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:03] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:08 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:08] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:08 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:08.569+0000 7f599dc00700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:08] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:10 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:10] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:10 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:10.702+0000 7f5995000700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:10] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:13 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:13] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:13 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:13.286+0000 7f598c400700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:13] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:18 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:18] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:18 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:18.569+0000 7f5983800700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:18] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:20 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:20] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:20 cn02.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:20.702+0000 7f597ac00700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:20] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
>
> cn03
> ——
> Apr 05 20:33:42 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:42] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:33:42 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:42.753+0000 7f2724a00700 0 [prometheus INFO  
> cherrypy.access.139806526902960] ::ffff:192.168.47.72 - -  
> [05/Apr/2025:20:33:42] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:43 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:43] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:33:43 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:43.772+0000 7f271be00700 0 [prometheus INFO  
> cherrypy.access.139806526902960] ::ffff:192.168.47.71 - -  
> [05/Apr/2025:20:33:43] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:47 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:47] "GET /metrics  
> HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> Apr 05 20:33:47 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:47.037+0000 7f2713200700 0 [prometheus INFO  
> cherrypy.access.139806526902960] ::ffff:192.168.47.73 - -  
> [05/Apr/2025:20:33:47] "GET /metrics HTTP/1.1" 200 - ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.867+0000 7f27d1a00700 1 mgr handle_mgr_map  
> Activating!
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.868+0000 7f27d1a00700 1 mgr handle_mgr_map  
> I am now activating
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.878+0000 7f2663400700 0 [dashboard INFO  
> root] Stopping engine...
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.878+0000 7f2663400700 0 [dashboard INFO  
> root] Stopped engine...
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.878+0000 7f2663400700 0 [prometheus INFO  
> root] Stopping engine...
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.878+0000 7f2663400700 0 [prometheus INFO  
> root] Stopped engine
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> [05/Apr/2025:20:33:48] ENGINE Bus STOPPING
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.879+0000 7f267e600700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:48] ENGINE Bus STOPPING
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.890+0000 7f2675a00700 0 [dashboard INFO  
> root] Engine stopped.
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> [05/Apr/2025:20:33:48] ENGINE HTTP Server  
> cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.891+0000 7f267e600700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:48] ENGINE HTTP Server  
> cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> [05/Apr/2025:20:33:48] ENGINE Bus STOPPED
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.892+0000 7f267e600700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:48] ENGINE Bus STOPPED
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.892+0000 7f267e600700 0 [prometheus INFO  
> root] Engine stopped.
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.898+0000 7f2663400700 0 [alerts DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.898+0000 7f2663400700 0 [alerts INFO root]  
> Init
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.898+0000 7f2663400700 1 mgr load  
> Constructed class from module: alerts
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.899+0000 7f2663400700 0 [balancer DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.899+0000 7f2663400700 1 mgr load  
> Constructed class from module: balancer
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.899+0000 7f26f8e00700 0 [alerts INFO root]  
> Starting
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.901+0000 7f2663400700 0 [cephadm DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.901+0000 7f26f0200700 0 [balancer INFO  
> root] Starting
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.908+0000 7f26f0200700 0 [balancer INFO  
> root] Optimize plan auto_2025-04-05_20:33:48
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.908+0000 7f26f0200700 0 [balancer INFO  
> root] Mode upmap, max misplaced 0.050000
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.909+0000 7f26f0200700 0 [balancer INFO  
> root] Some PGs (1.000000) are unknown; try again later
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.917+0000 7f2663400700 -1 mgr load Failed  
> to construct class in 'cephadm'
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.917+0000 7f2663400700 -1 mgr load  
> Traceback (most recent call last):
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> File "/usr/share/ceph/mgr/cephadm/module.py", line 470, in __init__
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> self.upgrade = CephadmUpgrade(self)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> File "/usr/share/ceph/mgr/cephadm/upgrade.py", line 112, in __init__
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> self.upgrade_state: Optional[UpgradeState] =  
> UpgradeState.from_json(json.loads(t))
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> File "/usr/share/ceph/mgr/cephadm/upgrade.py", line 93, in from_json
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> return cls(**c)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> TypeError: __init__() got an unexpected keyword argument  
> 'daemon_types'
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.918+0000 7f2663400700 -1 mgr operator()  
> Failed to run module in active mode ('cephadm')
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.919+0000 7f2663400700 0 [crash DEBUG root]  
> setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.919+0000 7f2663400700 1 mgr load  
> Constructed class from module: crash
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.919+0000 7f2663400700 0 [dashboard DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.919+0000 7f2663400700 1 mgr load  
> Constructed class from module: dashboard
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.920+0000 7f2660c00700 0 [dashboard INFO  
> access_control] Loading user roles DB version=2
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.921+0000 7f2660c00700 0 [dashboard INFO  
> sso] Loading SSO DB version=1
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.921+0000 7f2660c00700 0 [dashboard INFO  
> root] server: ssl=yes host=:: port=8443
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.922+0000 7f2663400700 0 [devicehealth  
> DEBUG root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.922+0000 7f2663400700 1 mgr load  
> Constructed class from module: devicehealth
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.922+0000 7f2663400700 0 [insights DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.922+0000 7f2663400700 1 mgr load  
> Constructed class from module: insights
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.923+0000 7f2660200700 0 [devicehealth INFO  
> root] Starting
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.925+0000 7f2663400700 0 [iostat DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.925+0000 7f2663400700 1 mgr load  
> Constructed class from module: iostat
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.925+0000 7f265f800700 0 [insights INFO  
> root] Reset curr health slot key health_history/2025-04-05_20 next  
> flush None checks check count 0
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.926+0000 7f2663400700 0 [mirroring DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.931+0000 7f2663400700 1 mgr load  
> Constructed class from module: mirroring
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.932+0000 7f2663400700 0 [nfs DEBUG root]  
> setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.932+0000 7f2663400700 1 mgr load  
> Constructed class from module: nfs
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.932+0000 7f2663400700 0 [orchestrator  
> DEBUG root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.932+0000 7f2663400700 1 mgr load  
> Constructed class from module: orchestrator
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.933+0000 7f2663400700 0 [pg_autoscaler  
> DEBUG root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.933+0000 7f2663400700 1 mgr load  
> Constructed class from module: pg_autoscaler
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.934+0000 7f2663400700 0 [progress DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.934+0000 7f2663400700 1 mgr load  
> Constructed class from module: progress
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.934+0000 7f2657600700 0 [pg_autoscaler  
> INFO root] _maybe_adjust
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.940+0000 7f2656c00700 0 [progress INFO  
> root] Loading...
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.940+0000 7f2663400700 0 [prometheus DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.944+0000 7f2663400700 1 mgr load  
> Constructed class from module: prometheus
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.944+0000 7f2656200700 0 [prometheus INFO  
> root] server_addr: :: server_port: 9283
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.944+0000 7f2656200700 0 [prometheus INFO  
> root] Cache enabled
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.945+0000 7f2656c00700 0 [progress INFO  
> root] Loaded [<progress.module.GhostEvent object at 0x7f27363ac400>,  
> <progress.module.GhostEvent object at 0x7f27363ac438>,  
> <progress.module.GhostEvent object at 0x7f27363ac3c8>,  
> <progress.module.GhostEvent object at 0x7f27363ac5c0>,  
> <progress.module.GhostEvent object at 0x7f27363ac4e0>,  
> <progress.module.GhostEvent object at 0x7f27363ac4a8>,  
> <progress.module.GhostEvent object at 0x7f278d317e10>,  
> <progress.module.GhostEvent object at 0x7f278d4397b8>,  
> <progress.module.GhostEvent object at 0x7f278d439908>,  
> <progress.module.GhostEvent object at 0x7f27363ed2e8>,  
> <progress.module.GhostEvent object at 0x7f27363ed4e0>,  
> <progress.module.GhostEvent object at 0x7f27363ed358>,  
> <progress.module.GhostEvent object at 0x7f27363ed278>,  
> <progress.module.GhostEvent object at 0x7f27363edeb8>,  
> <progress.module.GhostEvent object at 0x7f27363ed438>,  
> <progress.module.GhostEvent object at 0x7f27363ed400>,  
> <progress.module.GhostEvent object at 0x7f27363edb00>,  
> <progress.module.GhostEvent object at 0x7f27363ed780>,  
> <progress.module.GhostEvent object at 0x7f27363ed240>,  
> <progress.module.GhostEvent object at 0x7f27363edba8>,  
> <progress.module.GhostEvent object at 0x7f27363ed2b0>,  
> <progress.module.GhostEvent object at 0x7f27363ed0b8>,  
> <progress.module.GhostEvent object at 0x7f27363edcf8>,  
> <progress.module.GhostEvent object at 0x7f27363ed128>,  
> <progress.module.GhostEvent object at 0x7f27363ed080>,  
> <progress.module.GhostEvent object at 0x7f27363edc18>,  
> <progress.module.GhostEvent object at 0x7f27363ed860>,  
> <progress.module.GhostEvent object at 0x7f27363eda58>,  
> <progress.module.GhostEvent object at 0x7f27363eda90>,  
> <progress.module.GhostEvent object at 0x7f27363ed160>,  
> <progress.module.GhostEvent object at 0x7f27363ed908>,  
> <progress.module.GhostEvent object at 0x7f27363ed940>,  
> <progress.module.GhostEvent object at 0x7f27363ed8d0>,  
> <progress.module.GhostEvent object at 0x7f27363ed978>,  
> <progress.module.GhostEvent object at 0x7f27363ed748>,  
> <progress.module.GhostEvent object at 0x7f27363ed898>,  
> <progress.module.GhostEvent object at 0x7f27363eda20>,  
> <progress.module.GhostEvent object at 0x7f27363ed9b0>,  
> <progress.module.GhostEvent object at 0x7f27363ed9e8>,  
> <progress.module.GhostEvent object at 0x7f27363ed828>,  
> <progress.module.GhostEvent object at 0x7f27363ed7f0>,  
> <progress.module.GhostEvent object at 0x7f27363ed7b8>,  
> <progress.module.GhostEvent object at 0x7f27363ed668>,  
> <progress.module.GhostEvent object at 0x7f27363ed320>,  
> <progress.module.GhostEvent object at 0x7f27363ede48>,  
> <progress.module.GhostEvent object at 0x7f27363edc50>,  
> <progress.module.GhostEvent object at 0x7f27363ede80>,  
> <progress.module.GhostEvent object at 0x7f27363edf60>,  
> <progress.module.GhostEvent object at 0x7f27363edd68>,  
> <progress.module.GhostEvent object at 0x7f27363edda0>] historic events
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.945+0000 7f2656c00700 0 [progress INFO  
> root] Loaded OSDMap, ready.
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.946+0000 7f2663400700 0 [rbd_support DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.946+0000 7f2655800700 0 [prometheus INFO  
> root] starting metric collection thread
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.947+0000 7f2656200700 0 [prometheus INFO  
> root] Starting engine...
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> [05/Apr/2025:20:33:48] ENGINE Bus STARTING
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.948+0000 7f2656200700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:48] ENGINE Bus STARTING
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.948+0000 7f2660c00700 0 [dashboard INFO  
> root] Configured CherryPy, starting engine...
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.953+0000 7f2663400700 0 [rbd_support INFO  
> root] MirrorSnapshotScheduleHandler: load_schedules
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.957+0000 7f2663400700 0 [rbd_support INFO  
> root] load_schedules: csxyz, start_after=
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.989+0000 7f2649a00700 0 [rbd_support INFO  
> root] MirrorSnapshotScheduleHandler: starting
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.990+0000 7f2649000700 0 [rbd_support INFO  
> root] PerfHandler: starting
> Apr 05 20:33:48 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:48.991+0000 7f2663400700 0 [rbd_support INFO  
> root] load_task_task: csxyz, start_after=
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.019+0000 7f2648600700 0 [rbd_support INFO  
> root] TaskHandler: starting
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.020+0000 7f2663400700 0 [rbd_support INFO  
> root] TrashPurgeScheduleHandler: load_schedules
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.022+0000 7f2663400700 0 [rbd_support INFO  
> root] load_schedules: csxyz, start_after=
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.024+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> CephFS -> /api/cephfs
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.026+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> CephFsUi -> /ui-api/cephfs
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.028+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> IscsiUi -> /ui-api/iscsi
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.028+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Iscsi -> /api/iscsi
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.028+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> IscsiTarget -> /api/iscsi/target
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.030+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> NFSGanesha -> /api/nfs-ganesha
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.030+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> NFSGaneshaCluster -> /api/nfs-ganesha/cluster
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.030+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> NFSGaneshaExports -> /api/nfs-ganesha/export
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.031+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> NFSGaneshaUi -> /ui-api/nfs-ganesha
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.031+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller: Rbd  
> -> /api/block/image
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.034+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdSnapshot -> /api/block/image/{image_spec}/snap
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.035+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdTrash -> /api/block/image/trash
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.035+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdNamespace -> /api/block/pool/{pool_name}/namespace
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.036+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdMirroring -> /api/block/mirroring
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.036+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdMirroringSummary -> /api/block/mirroring/summary
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.037+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdMirroringPoolMode -> /api/block/mirroring/pool
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.037+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdMirroringPoolBootstrap ->  
> /api/block/mirroring/pool/{pool_name}/bootstrap
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.037+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdMirroringPoolPeer -> /api/block/mirroring/pool/{pool_name}/peer
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.038+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller: Rgw  
> -> /api/rgw
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.039+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RgwDaemon -> /api/rgw/daemon
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.039+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RgwSite -> /api/rgw/site
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.040+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RgwBucket -> /api/rgw/bucket
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.041+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RgwUser -> /api/rgw/user
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.044+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Auth -> /api/auth
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.044+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Cluster -> /api/cluster
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.045+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> ClusterConfiguration -> /api/cluster_conf
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.046+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> CrushRule -> /api/crush_rule
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.047+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> CrushRuleUi -> /ui-api/crush_rule
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.048+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Orchestrator -> /api/orchestrator
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.048+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Daemon -> /api/daemon
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.049+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Docs -> /docs
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.049+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> ErasureCodeProfile -> /api/erasure_code_profile
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.050+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> ErasureCodeProfileUi -> /ui-api/erasure_code_profile
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.050+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> FeedbackController -> /api/feedback
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.051+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> FeedbackApiController -> /api/feedback/api_key
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.052+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> FeedbackUiController -> /ui-api/feedback/api_key
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.052+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> FrontendLogging -> /ui-api/logging
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.052+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Grafana -> /api/grafana
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.052+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Host -> /api/host
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.054+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> HostUi -> /ui-api/host
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.054+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Health -> /api/health
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.054+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> HomeController -> /
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> [05/Apr/2025:20:33:49] ENGINE Serving on http://:::9283
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.055+0000 7f2656200700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:49] ENGINE Serving on  
> http://:::9283
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> [05/Apr/2025:20:33:49] ENGINE Bus STARTED
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.055+0000 7f2656200700 0 [prometheus INFO  
> cherrypy.error] [05/Apr/2025:20:33:49] ENGINE Bus STARTED
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.055+0000 7f2656200700 0 [prometheus INFO  
> root] Engine started.
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.056+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> LangsController -> /ui-api/langs
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.056+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Logs -> /api/logs
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.056+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> MgrModules -> /api/mgr/module
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.058+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Monitor -> /api/monitor
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.058+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller: Osd  
> -> /api/osd
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.060+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> OsdFlagsController -> /api/osd/flags
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.061+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> MdsPerfCounter -> /api/perf_counters/mds
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.061+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> MonPerfCounter -> /api/perf_counters/mon
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.062+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> OsdPerfCounter -> /api/perf_counters/osd
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.062+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RgwPerfCounter -> /api/perf_counters/rgw
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.063+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> RbdMirrorPerfCounter -> /api/perf_counters/rbd-mirror
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.063+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> MgrPerfCounter -> /api/perf_counters/mgr
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.064+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> TcmuRunnerPerfCounter -> /api/perf_counters/tcmu-runner
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.064+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> PerfCounters -> /api/perf_counters
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.065+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Pool -> /api/pool
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.066+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> PoolUi -> /ui-api/pool
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.067+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> PrometheusReceiver -> /api/prometheus_receiver
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.067+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Prometheus -> /api/prometheus
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.068+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> PrometheusNotifications -> /api/prometheus/notifications
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.069+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Role -> /api/role
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.070+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Scope -> /ui-api/scope
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.070+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Saml2 -> /auth/saml2
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.070+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Service -> /api/service
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.071+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Settings -> /api/settings
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.073+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> StandardSettings -> /ui-api/standard_settings
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.073+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Summary -> /api/summary
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.073+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Task -> /api/task
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.074+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> Telemetry -> /api/telemetry
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.074+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> User -> /api/user
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.076+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> UserPasswordPolicy -> /api/user
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.076+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> UserChangePassword -> /api/user/{username}
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.076+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> FeatureTogglesEndpoint -> /api/feature_toggles
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.076+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.controllers._base_controller] Initializing controller:  
> MessageOfTheDay -> /ui-api/motd
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.090+0000 7f2647200700 0 [rbd_support INFO  
> root] TrashPurgeScheduleHandler: starting
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.096+0000 7f2663400700 1 mgr load  
> Constructed class from module: rbd_support
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.102+0000 7f2663400700 0 [restful DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.102+0000 7f2663400700 1 mgr load  
> Constructed class from module: restful
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.108+0000 7f2645e00700 0 [restful INFO  
> root] server_addr: :: server_port: 8003
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.109+0000 7f2663400700 0 [snap_schedule  
> DEBUG root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.116+0000 7f2645e00700 0 [restful WARNING  
> root] server not running: no certificate configured
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.257+0000 7f2660c00700 0 [dashboard INFO  
> dashboard.module] Engine started.
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.273+0000 7f2663400700 -1 mgr load Failed  
> to construct class in 'snap_schedule'
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.273+0000 7f2663400700 -1 mgr load  
> Traceback (most recent call last):
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> File "/usr/share/ceph/mgr/snap_schedule/module.py", line 38, in  
> __init__
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> self.client = SnapSchedClient(self)
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line  
> 158, in __init__
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> with self.get_schedule_db(fs_name) as conn_mgr:
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line  
> 192, in get_schedule_db
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> db.executescript(dump)
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> sqlite3.OperationalError: table schedules already exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.274+0000 7f2663400700 -1 mgr operator()  
> Failed to run module in active mode ('snap_schedule')
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.274+0000 7f2663400700 0 [status DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.274+0000 7f2663400700 1 mgr load  
> Constructed class from module: status
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.275+0000 7f2663400700 0 [telemetry DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.276+0000 7f2663400700 1 mgr load  
> Constructed class from module: telemetry
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.276+0000 7f2663400700 0 [volumes DEBUG  
> root] setting log level based on debug_mgr: INFO (2/5)
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.284+0000 7f2660200700 0 [devicehealth INFO  
> root] Check health
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.290+0000 7f2663400700 0 [volumes INFO  
> volumes.fs.async_job] queuing job for volume 'cssnfsxyz'
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.290+0000 7f2663400700 0 [volumes INFO  
> volumes.fs.async_job] queuing job for volume 'cssnfsxyz'
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.290+0000 7f2663400700 1 mgr load  
> Constructed class from module: volumes
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error  
> registering admin socket command: (17) File exists
> Apr 05 20:33:49 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:49.883+0000 7f2662a00700 0  
> log_channel(cluster) log [DBG] : pgmap v3: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail
> Apr 05 20:33:50 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:50.875+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v4: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail
> Apr 05 20:33:52 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:52] "GET /metrics  
> HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> Apr 05 20:33:52 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:52.753+0000 7f264f400700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.72 - -  
> [05/Apr/2025:20:33:52] "GET /metrics HTTP/1.1" 200 33671 ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:52 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:52.876+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v5: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail
> Apr 05 20:33:53 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:53] "GET /metrics  
> HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> Apr 05 20:33:53 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:53.772+0000 7f264ea00700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.71 - -  
> [05/Apr/2025:20:33:53] "GET /metrics HTTP/1.1" 200 33671 ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:53 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:53.870+0000 7f2663400700 0 [insights INFO  
> root] Received health check update 0 pending
> Apr 05 20:33:53 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:53.871+0000 7f265f800700 0 [insights INFO  
> root] Applying 1 health updates to slot key  
> health_history/2025-04-05_20 next flush None checks check count 0
> Apr 05 20:33:53 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:53.871+0000 7f265f800700 0 [insights INFO  
> root] Maybe flushing slot key health_history/2025-04-05_20 next  
> flush None checks check count 0 needed False
> Apr 05 20:33:54 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:54.877+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v6: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail; 0  
> B/s rd, 12 KiB/s wr, 2 op/s
> Apr 05 20:33:55 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:55.796+0000 7f2662a00700 2 mgr.server  
> handle_open ignoring open from mgr.cn02.ceph.xyz.corp.ggixgj  
> 192.168.47.72:0/2580067884; not ready for session (expect reconnect)
> Apr 05 20:33:56 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:56.878+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v7: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 439 B/s rd, 27 KiB/s wr, 6 op/s
> Apr 05 20:33:57 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:57] "GET /metrics  
> HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> Apr 05 20:33:57 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:57.038+0000 7f264e000700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.73 - -  
> [05/Apr/2025:20:33:57] "GET /metrics HTTP/1.1" 200 33671 ""  
> "Prometheus/2.33.4"
> Apr 05 20:33:58 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:58.870+0000 7f2663400700 0 [insights INFO  
> root] Received health check update 0 pending
> Apr 05 20:33:58 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:58.878+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v8: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 341 B/s rd, 21 KiB/s wr, 4 op/s
> Apr 05 20:33:58 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:58.897+0000 7f265f800700 0 [insights INFO  
> root] Applying 1 health updates to slot key  
> health_history/2025-04-05_20 next flush None checks check count 0
> Apr 05 20:33:58 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:33:58.897+0000 7f265f800700 0 [insights INFO  
> root] Maybe flushing slot key health_history/2025-04-05_20 next  
> flush None checks check count 0 needed False
> Apr 05 20:34:00 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:00.880+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v9: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 279 B/s rd, 24 KiB/s wr, 6 op/s
> Apr 05 20:34:02 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:02] "GET /metrics  
> HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> Apr 05 20:34:02 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:02.753+0000 7f264d600700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.72 - -  
> [05/Apr/2025:20:34:02] "GET /metrics HTTP/1.1" 200 33671 ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:02 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:02.880+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v10: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 22 KiB/s wr, 5 op/s
> Apr 05 20:34:03 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:03] "GET /metrics  
> HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> Apr 05 20:34:03 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:03.772+0000 7f264cc00700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.71 - -  
> [05/Apr/2025:20:34:03] "GET /metrics HTTP/1.1" 200 33671 ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:03 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:03.870+0000 7f2663400700 0 [insights INFO  
> root] Received health check update 0 pending
> Apr 05 20:34:03 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:03.873+0000 7f265f800700 0 [insights INFO  
> root] Applying 1 health updates to slot key  
> health_history/2025-04-05_20 next flush None checks check count 0
> Apr 05 20:34:03 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:03.874+0000 7f265f800700 0 [insights INFO  
> root] Maybe flushing slot key health_history/2025-04-05_20 next  
> flush None checks check count 0 needed False
> Apr 05 20:34:04 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:04.881+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v11: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 59 KiB/s wr, 11 op/s
> Apr 05 20:34:06 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:06.882+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v12: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 511 B/s rd, 66 KiB/s wr, 13 op/s
> Apr 05 20:34:07 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:07] "GET /metrics  
> HTTP/1.1" 200 267239 "" "Prometheus/2.33.4"
> Apr 05 20:34:07 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:07.037+0000 7f264c200700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.73 - -  
> [05/Apr/2025:20:34:07] "GET /metrics HTTP/1.1" 200 267239 ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:08 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:08.871+0000 7f2663400700 0 [insights INFO  
> root] Received health check update 0 pending
> Apr 05 20:34:08 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:08.883+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v13: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 55 KiB/s wr, 11 op/s
> Apr 05 20:34:08 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:08.907+0000 7f265f800700 0 [insights INFO  
> root] Applying 1 health updates to slot key  
> health_history/2025-04-05_20 next flush None checks check count 0
> Apr 05 20:34:08 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:08.907+0000 7f265f800700 0 [insights INFO  
> root] Maybe flushing slot key health_history/2025-04-05_20 next  
> flush None checks check count 0 needed False
> Apr 05 20:34:10 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:10.884+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v14: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 79 KiB/s wr, 16 op/s
> Apr 05 20:34:12 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:12] "GET /metrics  
> HTTP/1.1" 200 267239 "" "Prometheus/2.33.4"
> Apr 05 20:34:12 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:12.753+0000 7f264b800700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.72 - -  
> [05/Apr/2025:20:34:12] "GET /metrics HTTP/1.1" 200 267239 ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:12 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:12.884+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v15: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 255 B/s rd, 73 KiB/s wr, 14 op/s
> Apr 05 20:34:13 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:13] "GET /metrics  
> HTTP/1.1" 200 267239 "" "Prometheus/2.33.4"
> Apr 05 20:34:13 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:13.772+0000 7f264ae00700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.71 - -  
> [05/Apr/2025:20:34:13] "GET /metrics HTTP/1.1" 200 267239 ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:13 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:13.871+0000 7f2663400700 0 [insights INFO  
> root] Received health check update 0 pending
> Apr 05 20:34:13 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:13.883+0000 7f265f800700 0 [insights INFO  
> root] Applying 1 health updates to slot key  
> health_history/2025-04-05_20 next flush None checks check count 0
> Apr 05 20:34:13 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:13.883+0000 7f265f800700 0 [insights INFO  
> root] Maybe flushing slot key health_history/2025-04-05_20 next  
> flush None checks check count 0 needed False
> Apr 05 20:34:14 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:14.886+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v16: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 82 KiB/s rd, 103 KiB/s wr, 113 op/s
> Apr 05 20:34:15 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:15.754+0000 7f2644000700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52375] [GET] [200] [0.089s] [1.7K]  
> [6c8ffcbf-e0d6-4f01-a24a-693d11b5fcdb] /
> Apr 05 20:34:15 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:15.769+0000 7f2643600700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52375] [GET] [200] [0.003s] [1.7K]  
> [d20dd549-caaf-4b35-9784-465556871a83] /
> Apr 05 20:34:15 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:15.787+0000 7f2642c00700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52375] [GET] [200] [0.002s] [1.7K]  
> [2efb9ce6-0ec0-4525-afcf-5d04356c0f18] /
> Apr 05 20:34:15 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:15.803+0000 7f2642200700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52375] [GET] [200] [0.002s] [1.7K]  
> [2f0b6080-a99d-4fdf-b7f9-c013a4cf2b78] /
> Apr 05 20:34:15 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:15.825+0000 7f2641800700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52375] [GET] [200] [0.002s] [1.7K]  
> [5c67930f-511b-4e2a-993e-13bc0c73fe2c] /
> Apr 05 20:34:15 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:15.842+0000 7f2640e00700 0 [dashboard INFO  
> request] [::ffff:192.168.10.101:52375] [GET] [200] [0.002s] [1.7K]  
> [043cc2d4-1863-4072-a15e-91a04422c2d3] /
> Apr 05 20:34:16 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:16.887+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v17: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 82 KiB/s rd, 66 KiB/s wr, 106 op/s
> Apr 05 20:34:17 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:17] "GET /metrics  
> HTTP/1.1" 200 267239 "" "Prometheus/2.33.4"
> Apr 05 20:34:17 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:17.037+0000 7f264a400700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.73 - -  
> [05/Apr/2025:20:34:17] "GET /metrics HTTP/1.1" 200 267239 ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:18 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:18.871+0000 7f2663400700 0 [insights INFO  
> root] Received health check update 0 pending
> Apr 05 20:34:18 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:18.887+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v18: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 82 KiB/s rd, 54 KiB/s wr, 103 op/s
> Apr 05 20:34:18 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:18.916+0000 7f265f800700 0 [insights INFO  
> root] Applying 1 health updates to slot key  
> health_history/2025-04-05_20 next flush None checks check count 0
> Apr 05 20:34:18 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:18.916+0000 7f265f800700 0 [insights INFO  
> root] Maybe flushing slot key health_history/2025-04-05_20 next  
> flush None checks check count 0 needed False
> Apr 05 20:34:18 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:18.931+0000 7f2659e00700 0 [mirroring INFO  
> mgr_util] scanning for idle connections..
> Apr 05 20:34:18 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:18.931+0000 7f2659e00700 0 [mirroring INFO  
> mgr_util] cleaning up connections: []
> Apr 05 20:34:19 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:19.109+0000 7f2645400700 0 [snap_schedule  
> INFO mgr_util] scanning for idle connections..
> Apr 05 20:34:19 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:19.109+0000 7f2645400700 0 [snap_schedule  
> INFO mgr_util] cleaning up connections: []
> Apr 05 20:34:19 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:19.277+0000 7f2632800700 0 [volumes INFO  
> mgr_util] scanning for idle connections..
> Apr 05 20:34:19 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:19.277+0000 7f2632800700 0 [volumes INFO  
> mgr_util] cleaning up connections: []
> Apr 05 20:34:19 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:19.282+0000 7f262d800700 0 [volumes INFO  
> mgr_util] scanning for idle connections..
> Apr 05 20:34:19 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:19.282+0000 7f262d800700 0 [volumes INFO  
> mgr_util] cleaning up connections: []
> Apr 05 20:34:19 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:19.287+0000 7f2629c00700 0 [volumes INFO  
> mgr_util] scanning for idle connections..
> Apr 05 20:34:19 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:19.287+0000 7f2629c00700 0 [volumes INFO  
> mgr_util] cleaning up connections: []
> Apr 05 20:34:20 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:20.889+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v19: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 82 KiB/s rd, 70 KiB/s wr, 107 op/s
> Apr 05 20:34:22 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:22] "GET /metrics  
> HTTP/1.1" 200 267240 "" "Prometheus/2.33.4"
> Apr 05 20:34:22 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:22.753+0000 7f264fe00700 0 [prometheus INFO  
> cherrypy.access.139806389938328] ::ffff:192.168.47.72 - -  
> [05/Apr/2025:20:34:22] "GET /metrics HTTP/1.1" 200 267240 ""  
> "Prometheus/2.33.4"
> Apr 05 20:34:22 cn03.ceph.xyz.corp  
> ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:  
> debug 2025-04-05T20:34:22.889+0000 7f2661600700 0  
> log_channel(cluster) log [DBG] : pgmap v20: 610 pgs: 610  
> active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;  
> 82 KiB/s rd, 45 KiB/s wr, 102 op/s
>
> Thank you
> -jeremy
>
> > On Saturday, Apr 05, 2025 at 1:32 PM, Eugen Block <eblock@xxxxxx  
> > (mailto:eblock@xxxxxx)> wrote:
> > The mgr logs should contain a stack trace, can you check again?
> >
> >
> > Zitat von Jeremy Hansen <jeremy@xxxxxxxxxx>:
> >
> > > No. Same issue unfortunately.
> > >
> > > > On Saturday, Apr 05, 2025 at 1:27 PM, Eugen Block <eblock@xxxxxx
> > > > (mailto:eblock@xxxxxx)> wrote:
> > > > I don't see the module error in the logs you provided. Has it cleared?
> > > > A mgr failover is often helpful in such cases, basically it's the
> > > > first thing I suggest since two or three years.
> > > >
> > > > Zitat von Jeremy Hansen <jeremy@xxxxxxxxxx>:
> > > >
> > > > > Thank you so much for the detailed instructions. Here’s logs from
> > > > > the failover to a new node.
> > > > >
> > > > > Apr 05 20:06:08 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:06:08] "GET /metrics HTTP/1.1" 200  
> > -
> > > > ""
> > > > > "Prometheus/2.33.4"
> > > > > Apr 05 20:06:08 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:08.568+0000 7efcbd800700 0 [prometheus INFO cherrypy.access.139625005087208] ::ffff:192.168.47.72 - - [05/Apr/2025:20:06:08] "GET /metrics HTTP/1.1" 200  
> > -
> > > > ""
> > > > > "Prometheus/2.33.4"
> > > > > Apr 05 20:06:10 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:06:10] "GET /metrics HTTP/1.1" 200  
> > -
> > > > ""
> > > > > "Prometheus/2.33.4"
> > > > > Apr 05 20:06:10 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:10.702+0000 7efcb4c00700 0 [prometheus INFO cherrypy.access.139625005087208] ::ffff:192.168.47.71 - - [05/Apr/2025:20:06:10] "GET /metrics HTTP/1.1" 200  
> > -
> > > > ""
> > > > > "Prometheus/2.33.4"
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.027+0000 7efd95200700 1  
> > mgr
> > > > handle_mgr_map
> > > > > Activating!
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.028+0000 7efd95200700 1 mgr handle_mgr_map I  
> > am
> > > > now
> > > > > activating
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.040+0000 7efc27e00700 0 [dashboard INFO  
> > root]
> > > > Stopping
> > > > > engine...
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.040+0000 7efc27e00700 0 [dashboard INFO  
> > root]
> > > > Stopped
> > > > > engine...
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.041+0000 7efc27e00700 0 [prometheus INFO  
> > root]
> > > > Stopping
> > > > > engine...
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.041+0000 7efc27e00700 0 [prometheus INFO  
> > root]
> > > > Stopped
> > > > > engine
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:06:12]  
> > ENGINE
> > > > Bus
> > > > > STOPPING
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.041+0000 7efc43000700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:06:12]  
> > ENGINE
> > > > Bus
> > > > > STOPPING
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.053+0000 7efc3a400700 0 [dashboard INFO  
> > root]
> > > > Engine
> > > > > stopped.
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:06:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::',  
> > 9283))
> > > > shut
> > > > > down
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.054+0000 7efc43000700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:06:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::',  
> > 9283))
> > > > shut
> > > > > down
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:06:12]  
> > ENGINE
> > > > Bus
> > > > > STOPPED
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.054+0000 7efc43000700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:06:12]  
> > ENGINE
> > > > Bus
> > > > > STOPPED
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.054+0000 7efc43000700 0 [prometheus INFO  
> > root]
> > > > Engine
> > > > > stopped.
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.056+0000 7efc27e00700 0 [alerts DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.057+0000 7efc27e00700 0 [alerts  
> > INFO
> > > > root]
> > > > > Init
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.057+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > alerts
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.058+0000 7efc27e00700 0 [balancer DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.058+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > balancer
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.058+0000 7efcfac00700 0 [alerts  
> > INFO
> > > > root]
> > > > > Starting
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.060+0000 7efc27e00700 0 [cephadm DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.060+0000 7efcb4c00700 0 [balancer  
> > INFO
> > > > root]
> > > > > Starting
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.067+0000 7efcb4c00700 0 [balancer INFO root]  
> > Optimize
> > > > plan
> > > > > auto_2025-04-05_20:06:12
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.067+0000 7efcb4c00700 0 [balancer INFO root] Mode upmap,  
> > max
> > > > misplaced
> > > > > 0.050000
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.067+0000 7efcb4c00700 0 [balancer INFO root] Some PGs (1.000000) are unknown;  
> > try
> > > > again
> > > > > later
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.212+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > cephadm
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.213+0000 7efc27e00700 0 [crash DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.213+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > crash
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.215+0000 7efc27e00700 0 [dashboard DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.215+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > dashboard
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.216+0000 7efc23800700 0 [dashboard INFO access_control] Loading user  
> > roles
> > > > DB
> > > > > version=2
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.217+0000 7efc23800700 0 [dashboard INFO sso] Loading  
> > SSO
> > > > DB
> > > > > version=1
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.217+0000 7efc23800700 0 [dashboard INFO root] server:  
> > ssl=yes
> > > > host=::
> > > > > port=8443
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.218+0000 7efc27e00700 0 [devicehealth DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.218+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > devicehealth
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.219+0000 7efc27e00700 0 [insights DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.219+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > insights
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.219+0000 7efc20600700 0 [insights INFO root] Reset curr health slot key health_history/2025-04-05_20 next flush None checks check count Apr 05 20:06:12 cn02.ceph.xyz.corp ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.219+0000 7efc21a00700 0 [devicehealth  
> > INFO
> > > > root]
> > > > > Starting
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.222+0000 7efc27e00700 0 [iostat DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.222+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > iostat
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.224+0000 7efc27e00700 0 [mirroring DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.227+0000 7efc21a00700 0 [devicehealth INFO  
> > root]
> > > > Check
> > > > > health
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.235+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > mirroring
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.236+0000 7efc27e00700 0 [nfs DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.236+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > nfs
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.237+0000 7efc27e00700 0 [orchestrator DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.237+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > orchestrator
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.239+0000 7efc27e00700 0 [pg_autoscaler DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.239+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > pg_autoscaler
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.239+0000 7efc10200700 0 [pg_autoscaler  
> > INFO
> > > > root]
> > > > > _maybe_adjust
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.240+0000 7efc27e00700 0 [progress DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.240+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > progress
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.247+0000 7efc27e00700 0 [prometheus DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.248+0000 7efc27e00700 1 mgr load Constructed class  
> > from
> > > > module:
> > > > > prometheus
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.252+0000 7efc0ee00700 0 [progress  
> > INFO
> > > > root]
> > > > > Loading...
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.252+0000 7efc27e00700 0 [rbd_support DEBUG root] setting log level based on  
> > debug_mgr:
> > > > INFO
> > > > > (2/5)
> > > > > Apr 05 20:06:12 cn02.ceph.xyz.corp
> > > > >
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:06:12.253+0000 7efc0ee00700 0 [progress INFO root] Loaded [<progress.module.GhostEvent object at 0x7efcf2127710>, <progress.module.GhostEvent object at 0x7efcf2127cc0>, <progress.module.GhostEvent object at 0x7efcf2127208>, <progress.module.GhostEvent object at 0x7efcf2127f28>, <progress.module.GhostEvent object at 0x7efcf21270f0>, <progress.module.GhostEvent object at 0x7efcf2127eb8>, <progress.module.GhostEvent object at 0x7efcf2127b38>, <progress.module.GhostEvent object at 0x7efcf2127a20>, <progress.module.GhostEvent object at 0x7efcf2127da0>, <progress.module.GhostEvent object at 0x7efcf2127f98>, <progress.module.GhostEvent object at 0x7efcf2127780>, <progress.module.GhostEvent object at 0x7efcf2127320>, <progress.module.GhostEvent object at 0x7efcf2127588>, <progress.module.GhostEvent object at 0x7efcf2127518>, <progress.module.GhostEvent object at 0x7efcf2127c50>, <progress.module.GhostEvent object at 0x7efcf21279e8>, <progress.module.GhostEvent object at 0x7efcf2127d68>, <progress.module.GhostEvent object at 0x7efcf2127470>, <progress.module.GhostEvent object at  
> > 0x7efcf
> [Text gekürzt]


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx