Re: Cephadm upgrade from 16.2.15 -> 17.2.0

[Eugen Block <eblock@xxxxxx>]

No, that's not necessary, just edit the unit.run file for the MGRs to  
use a different image. See Frédéric's instructions:

https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/message/32APKOXKRAIZ7IDCNI25KVYFCCCF6RJG/

But I'm not entirely sure if you need to clear some config-keys first  
in order to reset the upgrade state. If I have time, I'll try to check  
tomorrow, or on Monday.

Zitat von Jeremy Hansen <jeremy@xxxxxxxxxx>:

> Would I follow this process to downgrade?
>
> https://docs.ceph.com/en/quincy/cephadm/troubleshooting/#manually-deploying-a-manager-daemon
>
> Thank you
>
> > On Saturday, Apr 05, 2025 at 2:04 PM, Jeremy Hansen  
> > <jeremy@xxxxxxxxxx (mailto:jeremy@xxxxxxxxxx)> wrote:
> > ceph -s claims things are healthy:
> >
> > ceph -s
> > cluster:
> > id: 95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1
> > health: HEALTH_OK
> >
> > services:
> > mon: 3 daemons, quorum cn01,cn03,cn02 (age 20h)
> > mgr: cn03.negzvb(active, since 26m), standbys: cn01.tjmtph,  
> > cn02.ceph.xyz.corp.ggixgj
> > mds: 1/1 daemons up, 2 standby
> > osd: 15 osds: 15 up (since 19h), 15 in (since 14M)
> >
> > data:
> > volumes: 1/1 healthy
> > pools: 6 pools, 610 pgs
> > objects: 284.59k objects, 1.1 TiB
> > usage: 3.3 TiB used, 106 TiB / 109 TiB avail
> > pgs: 610 active+clean
> >
> > io:
> > client: 255 B/s rd, 1.2 MiB/s wr, 10 op/s rd, 16 op/s wr
> >
> >
> >
> > —
> > How do I downgrade if the orch is down?
> >
> > Thank you
> > -jeremy
> >
> >
> >
> > > On Saturday, Apr 05, 2025 at 1:56 PM, Eugen Block <eblock@xxxxxx  
> > (mailto:eblock@xxxxxx)> wrote:
> > > It would help if you only pasted the relevant parts. Anyway, these two
> > > sections stand out:
> > >
> > > ---snip---
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > debug 2025-04-05T20:33:48.909+0000 7f26f0200700 0 [balancer INFO root]
> > > Some PGs (1.000000) are unknown; try again later
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > debug 2025-04-05T20:33:48.917+0000 7f2663400700 -1 mgr load Failed to
> > > construct class in 'cephadm'
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > debug 2025-04-05T20:33:48.917+0000 7f2663400700 -1 mgr load Traceback
> > > (most recent call last):
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > File "/usr/share/ceph/mgr/cephadm/module.py", line 470, in __init__
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > self.upgrade = CephadmUpgrade(self)
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > File "/usr/share/ceph/mgr/cephadm/upgrade.py", line 112, in __init__
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > self.upgrade_state: Optional[UpgradeState] =
> > > UpgradeState.from_json(json.loads(t))
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > File "/usr/share/ceph/mgr/cephadm/upgrade.py", line 93, in from_json
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > return cls(**c)
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > TypeError: __init__() got an unexpected keyword argument 'daemon_types'
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > debug 2025-04-05T20:33:48.918+0000 7f2663400700 -1 mgr operator()
> > > Failed to run module in active mode ('cephadm')
> > >
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > debug 2025-04-05T20:33:49.273+0000 7f2663400700 -1 mgr load Failed to
> > > construct class in 'snap_schedule'
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > debug 2025-04-05T20:33:49.273+0000 7f2663400700 -1 mgr load Traceback
> > > (most recent call last):
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > File "/usr/share/ceph/mgr/snap_schedule/module.py", line 38, in __init__
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > self.client = SnapSchedClient(self)
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line
> > > 158, in __init__
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > with self.get_schedule_db(fs_name) as conn_mgr:
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line
> > > 192, in get_schedule_db
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > db.executescript(dump)
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > sqlite3.OperationalError: table schedules already exists
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > debug 2025-04-05T20:33:49.274+0000 7f2663400700 -1 mgr operator()
> > > Failed to run module in active mode ('snap_schedule')
> > > ---snip---
> > >
> > > Your cluster seems to be in an error state (ceph -s) because of an
> > > unknown PG. It's recommended to have a healthy cluster before
> > > attemping an upgrade. It's possible that these errors come from the
> > > not upgraded MGR, I'm not sure.
> > >
> > > Since the upgrade was only successful for two MGRs, I am thinking
> > > about downgrading both MGRs back to 16.2.15, then retry an upgrade to
> > > a newer version, either 17.2.8 or 18.2.4. I haven't checked the
> > > snap_schedule error yet, though. Maybe someone else knows that already.
> > >
> > > Zitat von Jeremy Hansen <jeremy@xxxxxxxxxx>:
> > >
> > > > This is a small three node cluster. Here’s logs from all three nodes
> > > > during the failover. The current manager was cn02 and it failed over
> > > > to cn03.
> > > >
> > > > cn01
> > > > ——
> > > > Apr 05 20:33:35 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:35.823+0000 7f2aace00700 0
> > > > log_channel(cluster) log [DBG] : pgmap v821: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 22 KiB/s wr, 5 op/s
> > > > Apr 05 20:33:37 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:37.170+0000 7f2aaec00700 0 [insights INFO
> > > > root] Received health check update 0 pending
> > > > Apr 05 20:33:37 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:37.215+0000 7f2aab000700 0 [insights INFO
> > > > root] Applying 1 health updates to slot key
> > > > health_history/2025-04-05_20 next flush None checks check count 0
> > > > Apr 05 20:33:37 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:37.215+0000 7f2aab000700 0 [insights INFO
> > > > root] Maybe flushing slot key health_history/2025-04-05_20 next
> > > > flush None checks check count 0 needed False
> > > > Apr 05 20:33:37 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:37.823+0000 7f2aace00700 0
> > > > log_channel(cluster) log [DBG] : pgmap v822: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 22 KiB/s wr, 5 op/s
> > > > Apr 05 20:33:39 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:39] "GET /metrics
> > > > HTTP/1.1" 200 267241 "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:39 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:39.825+0000 7f2aace00700 0
> > > > log_channel(cluster) log [DBG] : pgmap v823: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 97 KiB/s wr, 12 op/s
> > > > Apr 05 20:33:41 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:41.826+0000 7f2aace00700 0
> > > > log_channel(cluster) log [DBG] : pgmap v824: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 85 KiB/s wr, 10 op/s
> > > > Apr 05 20:33:42 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:42.170+0000 7f2aaec00700 0 [insights INFO
> > > > root] Received health check update 0 pending
> > > > Apr 05 20:33:42 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:42.191+0000 7f2aab000700 0 [insights INFO
> > > > root] Applying 1 health updates to slot key
> > > > health_history/2025-04-05_20 next flush None checks check count 0
> > > > Apr 05 20:33:42 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:42.191+0000 7f2aab000700 0 [insights INFO
> > > > root] Maybe flushing slot key health_history/2025-04-05_20 next
> > > > flush None checks check count 0 needed False
> > > > Apr 05 20:33:42 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:42] "GET /metrics
> > > > HTTP/1.1" 200 267249 "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:43 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:43] "GET /metrics
> > > > HTTP/1.1" 200 267249 "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:43 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:43.827+0000 7f2aace00700 0
> > > > log_channel(cluster) log [DBG] : pgmap v825: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 511 B/s rd, 133 KiB/s wr, 18 op/s
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 -1 mgr
> > > > handle_mgr_map I was active but no longer am
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn e:
> > > > '/usr/bin/ceph-mgr'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 0:
> > > > '/usr/bin/ceph-mgr'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 1: '-n'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 2:
> > > > 'mgr.cn01.tjmtph'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 3: '-f'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 4:
> > > > '--setuser'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 5:
> > > > 'ceph'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 6:
> > > > '--setgroup'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 7:
> > > > 'ceph'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 8:
> > > > '--default-log-to-file=false'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 9:
> > > > '--default-log-to-stderr=true'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn 10:
> > > > '--default-log-stderr-prefix=debug '
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn
> > > > respawning with exe /usr/bin/ceph-mgr
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.364+0000 7f2c1d200700 1 mgr respawn
> > > > exe_path /proc/self/exe
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ignoring --setuser ceph since I am not root
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ignoring --setgroup ceph since I am not root
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.449+0000 7fb814447000 0 ceph version
> > > > 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e) quincy (stable),
> > > > process ceph-mgr, pid 2
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.449+0000 7fb814447000 0 pidfile_write:
> > > > ignore empty --pid-file
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.477+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'alerts'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.612+0000 7fb814447000 -1 mgr[py] Module
> > > > alerts has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.612+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'balancer'
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.726+0000 7fb814447000 -1 mgr[py] Module
> > > > balancer has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:45 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:45.726+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'cephadm'
> > > > Apr 05 20:33:46 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:46.417+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'crash'
> > > > Apr 05 20:33:46 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:46.540+0000 7fb814447000 -1 mgr[py] Module
> > > > crash has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:46 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:46.540+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'dashboard'
> > > > Apr 05 20:33:47 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:47.210+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'devicehealth'
> > > > Apr 05 20:33:47 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:47.331+0000 7fb814447000 -1 mgr[py] Module
> > > > devicehealth has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:47 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:47.331+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'diskprediction_local'
> > > > Apr 05 20:33:47 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:47.653+0000 7fb814447000 -1 mgr[py] Module
> > > > diskprediction_local has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:47 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:47.654+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'influx'
> > > > Apr 05 20:33:47 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:47.769+0000 7fb814447000 -1 mgr[py] Module
> > > > influx has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:47 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:47.769+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'insights'
> > > > Apr 05 20:33:47 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:47.884+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'iostat'
> > > > Apr 05 20:33:48 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:48.082+0000 7fb814447000 -1 mgr[py] Module
> > > > iostat has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:48 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:48.082+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'k8sevents'
> > > > Apr 05 20:33:48 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:48.679+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'localpool'
> > > > Apr 05 20:33:48 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:48.796+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'mds_autoscaler'
> > > > Apr 05 20:33:48 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:48.983+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'mirroring'
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.118+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'nfs'
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.315+0000 7fb814447000 -1 mgr[py] Module
> > > > nfs has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.315+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'orchestrator'
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.607+0000 7fb814447000 -1 mgr[py] Module
> > > > orchestrator has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.607+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'osd_perf_query'
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.723+0000 7fb814447000 -1 mgr[py] Module
> > > > osd_perf_query has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.723+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'osd_support'
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.837+0000 7fb814447000 -1 mgr[py] Module
> > > > osd_support has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.837+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'pg_autoscaler'
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.970+0000 7fb814447000 -1 mgr[py] Module
> > > > pg_autoscaler has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:49 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:49.970+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'progress'
> > > > Apr 05 20:33:50 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:50.082+0000 7fb814447000 -1 mgr[py] Module
> > > > progress has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:50 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:50.082+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'prometheus'
> > > > Apr 05 20:33:50 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:50.617+0000 7fb814447000 -1 mgr[py] Module
> > > > prometheus has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:50 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:50.617+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'rbd_support'
> > > > Apr 05 20:33:50 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:50.761+0000 7fb814447000 -1 mgr[py] Module
> > > > rbd_support has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:50 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:50.761+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'restful'
> > > > Apr 05 20:33:51 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:51.023+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'rook'
> > > > Apr 05 20:33:51 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:51.788+0000 7fb814447000 -1 mgr[py] Module
> > > > rook has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:51 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:51.788+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'selftest'
> > > > Apr 05 20:33:51 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:51.912+0000 7fb814447000 -1 mgr[py] Module
> > > > selftest has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:51 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:51.912+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'snap_schedule'
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.038+0000 7fb814447000 -1 mgr[py] Module
> > > > snap_schedule has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.038+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'stats'
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.155+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'status'
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.280+0000 7fb814447000 -1 mgr[py] Module
> > > > status has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.280+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'telegraf'
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.398+0000 7fb814447000 -1 mgr[py] Module
> > > > telegraf has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.398+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'telemetry'
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.594+0000 7fb814447000 -1 mgr[py] Module
> > > > telemetry has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.594+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'test_orchestrator'
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.771+0000 7fb814447000 -1 mgr[py] Module
> > > > test_orchestrator has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:52 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:52.771+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'volumes'
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.225+0000 7fb814447000 -1 mgr[py] Module
> > > > volumes has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.225+0000 7fb814447000 1 mgr[py] Loading
> > > > python module 'zabbix'
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.342+0000 7fb814447000 -1 mgr[py] Module
> > > > zabbix has missing NOTIFY_TYPES member
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.344+0000 7fb807c00700 0
> > > > ms_deliver_dispatch: unhandled message 0x555c78fe1080 mon_map magic:
> > > > 0 v1 from mon.0 v2:192.168.47.71:3300/0
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.344+0000 7fb808600700 0 [dashboard DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.345+0000 7fb808600700 1 mgr load
> > > > Constructed class from module: dashboard
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.345+0000 7fb808600700 0 [prometheus DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.345+0000 7fb808600700 1 mgr load
> > > > Constructed class from module: prometheus
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.345+0000 7fb6abe00700 0 [dashboard INFO
> > > > root] server: ssl=yes host=:: port=8443
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.346+0000 7fb6b4a00700 0 [prometheus INFO
> > > > root] server_addr: :: server_port: 9283
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.346+0000 7fb6b4a00700 0 [prometheus INFO
> > > > root] Starting engine...
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > [05/Apr/2025:20:33:53] ENGINE Bus STARTING
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.346+0000 7fb6b4a00700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:53] ENGINE Bus STARTING
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.377+0000 7fb6abe00700 0 [dashboard INFO
> > > > root] Configured CherryPy, starting engine...
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.377+0000 7fb6abe00700 0 [dashboard INFO
> > > > root] Starting engine...
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > CherryPy Checker:
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > The Application mounted at '' has an empty config.
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > [05/Apr/2025:20:33:53] ENGINE Serving on http://:::9283
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.450+0000 7fb6b4a00700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:53] ENGINE Serving on
> > > > http://:::9283
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > [05/Apr/2025:20:33:53] ENGINE Bus STARTED
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.450+0000 7fb6b4a00700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:53] ENGINE Bus STARTED
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.451+0000 7fb6b4a00700 0 [prometheus INFO
> > > > root] Engine started.
> > > > Apr 05 20:33:53 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:53.489+0000 7fb6abe00700 0 [dashboard INFO
> > > > root] Engine started...
> > > > Apr 05 20:33:59 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:59] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:59 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:33:59.603+0000 7fb76c600700 0 [prometheus INFO
> > > > cherrypy.access.140425911062880] ::ffff:192.168.47.72 - -
> > > > [05/Apr/2025:20:33:59] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:02 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:02] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:02 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:02.476+0000 7fb763a00700 0 [prometheus INFO
> > > > cherrypy.access.140425911062880] ::ffff:192.168.47.71 - -
> > > > [05/Apr/2025:20:34:02] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:03 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:03] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:03 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:03.061+0000 7fb75ae00700 0 [prometheus INFO
> > > > cherrypy.access.140425911062880] ::ffff:192.168.47.73 - -
> > > > [05/Apr/2025:20:34:03] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:09 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:09] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:09 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:09.602+0000 7fb752200700 0 [prometheus INFO
> > > > cherrypy.access.140425911062880] ::ffff:192.168.47.72 - -
> > > > [05/Apr/2025:20:34:09] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:12 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:12] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:12 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:12.476+0000 7fb749600700 0 [prometheus INFO
> > > > cherrypy.access.140425911062880] ::ffff:192.168.47.71 - -
> > > > [05/Apr/2025:20:34:12] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:13 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:13] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:13 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:13.060+0000 7fb740a00700 0 [prometheus INFO
> > > > cherrypy.access.140425911062880] ::ffff:192.168.47.73 - -
> > > > [05/Apr/2025:20:34:13] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.493+0000 7fb714e00700 0 [dashboard INFO
> > > > root] Redirecting to active 'https://192.168.47.73:8443/'
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.494+0000 7fb714e00700 0 [dashboard INFO
> > > > request] [::ffff:192.168.10.101:52368] [GET] [303] [0.003s] [96.0B]
> > > > [0031fbdd-947f-4c6c-943d-55cf256edfba] /api/summary
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.512+0000 7fb70c200700 0 [dashboard INFO
> > > > root] Redirecting to active 'https://192.168.47.73:8443/'
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.513+0000 7fb70c200700 0 [dashboard INFO
> > > > request] [::ffff:192.168.10.101:52369] [GET] [303] [0.001s] [96.0B]
> > > > [76c9a6ab-aae0-4ef9-815e-dbd0ac8f8b49] /api/feature_toggles
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.527+0000 7fb703600700 0 [dashboard INFO
> > > > root] Redirecting to active 'https://192.168.47.73:8443/'
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.528+0000 7fb703600700 0 [dashboard INFO
> > > > request] [::ffff:192.168.10.101:52370] [GET] [303] [0.001s] [96.0B]
> > > > [db017da3-e6f6-45fc-9d99-7b9f3b3859f2] /api/health/minimal
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.541+0000 7fb6faa00700 0 [dashboard INFO
> > > > root] Redirecting to active 'https://192.168.47.73:8443/'
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.542+0000 7fb6faa00700 0 [dashboard INFO
> > > > request] [::ffff:192.168.10.101:52371] [GET] [303] [0.001s] [96.0B]
> > > > [564ba322-b99c-49de-a3a3-e0ed422bfae7] /api/prometheus
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.616+0000 7fb6f1e00700 0 [dashboard INFO
> > > > root] Redirecting to active 'https://192.168.47.73:8443/'
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.617+0000 7fb6f1e00700 0 [dashboard INFO
> > > > request] [::ffff:192.168.10.101:52372] [GET] [303] [0.001s] [96.0B]
> > > > [1f349e52-9a10-4fda-b13f-621785208ea9] /api/prometheus/rules
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.630+0000 7fb6e0600700 0 [dashboard INFO
> > > > root] Redirecting to active 'https://192.168.47.73:8443/'
> > > > Apr 05 20:34:15 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:15.630+0000 7fb6e0600700 0 [dashboard INFO
> > > > request] [::ffff:192.168.10.101:52368] [GET] [303] [0.001s] [96.0B]
> > > > [30d749d7-c972-44f0-88c7-b70078e2385c] /api/prometheus/notifications
> > > > Apr 05 20:34:19 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:19] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:19 cn01.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn01-tjmtph[771594]:
> > > > debug 2025-04-05T20:34:19.601+0000 7fb737e00700 0 [prometheus INFO
> > > > cherrypy.access.140425911062880] ::ffff:192.168.47.72 - -
> > > > [05/Apr/2025:20:34:19] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > >
> > > > cn02
> > > > ——
> > > > Apr 05 20:33:40 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:40] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:40 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:40.701+0000 7ff710400700 0 [prometheus INFO cherrypy.access.140700134177256] ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:40] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:43 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:43] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:43 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:43.287+0000 7ff707800700 0 [prometheus INFO cherrypy.access.140700134177256] ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:43] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.370+0000 7ff7e7c00700 1 mgr  
> > handle_mgr_map
> > > > Activating!
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.371+0000 7ff7e7c00700 1 mgr handle_mgr_map I am  
> > now
> > > > activating
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.382+0000 7ff67aa00700 0 [dashboard INFO root]  
> > Stopping
> > > > engine...
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.382+0000 7ff67aa00700 0 [dashboard INFO root]  
> > Stopped
> > > > engine...
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.382+0000 7ff67aa00700 0 [prometheus INFO root]  
> > Stopping
> > > > engine...
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.382+0000 7ff67aa00700 0 [prometheus INFO root]  
> > Stopped
> > > > engine
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE  
> > Bus
> > > > STOPPING
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.383+0000 7ff695c00700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE  
> > Bus
> > > > STOPPING
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.394+0000 7ff68d000700 0 [dashboard INFO root]  
> > Engine
> > > > stopped.
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283))  
> > shut
> > > > down
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.395+0000 7ff695c00700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283))  
> > shut
> > > > down
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE  
> > Bus
> > > > STOPPED
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.396+0000 7ff695c00700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE  
> > Bus
> > > > STOPPED
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.396+0000 7ff695c00700 0 [prometheus INFO root]  
> > Engine
> > > > stopped.
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.398+0000 7ff67aa00700 0 [alerts DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.398+0000 7ff67aa00700 0 [alerts INFO  
> > root]
> > > > Init
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.398+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > alerts
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.399+0000 7ff67aa00700 0 [balancer DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.399+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > balancer
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.399+0000 7ff74d800700 0 [alerts INFO  
> > root]
> > > > Starting
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.401+0000 7ff67aa00700 0 [cephadm DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.402+0000 7ff710400700 0 [balancer INFO  
> > root]
> > > > Starting
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.408+0000 7ff710400700 0 [balancer INFO root] Optimize  
> > plan
> > > > auto_2025-04-05_20:33:45
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.408+0000 7ff710400700 0 [balancer INFO root] Mode upmap, max  
> > misplaced
> > > > 0.050000
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.409+0000 7ff710400700 0 [balancer INFO root] Some PGs (1.000000) are unknown; try  
> > again
> > > > later
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.551+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > cephadm
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.552+0000 7ff67aa00700 0 [crash DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.552+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > crash
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.554+0000 7ff67aa00700 0 [dashboard DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.554+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > dashboard
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.555+0000 7ff67aa00700 0 [devicehealth DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.555+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > devicehealth
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.556+0000 7ff676400700 0 [dashboard INFO access_control] Loading user roles  
> > DB
> > > > version=2
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.556+0000 7ff676400700 0 [dashboard INFO sso] Loading SSO  
> > DB
> > > > version=1
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.556+0000 7ff676400700 0 [dashboard INFO root] server: ssl=yes  
> > host=::
> > > > port=8443
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.557+0000 7ff674600700 0 [devicehealth INFO  
> > root]
> > > > Starting
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.558+0000 7ff67aa00700 0 [insights DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.559+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > insights
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.561+0000 7ff674600700 0 [devicehealth INFO root]  
> > Check
> > > > health
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.566+0000 7ff671e00700 0 [insights INFO root] Reset curr health slot key health_history/2025-04-05_20 next flush None checks check count Apr 05 20:33:45 cn02.ceph.xyz.corp ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.567+0000 7ff67aa00700 0 [iostat DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.567+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > iostat
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.572+0000 7ff67aa00700 0 [mirroring DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.579+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > mirroring
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.604+0000 7ff67aa00700 0 [nfs DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.604+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > nfs
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.622+0000 7ff67aa00700 0 [orchestrator DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.622+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > orchestrator
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.623+0000 7ff67aa00700 0 [pg_autoscaler DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.623+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > pg_autoscaler
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.625+0000 7ff662a00700 0 [pg_autoscaler INFO  
> > root]
> > > > _maybe_adjust
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.631+0000 7ff67aa00700 0 [progress DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.631+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > progress
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: Warning: Permanently added '192.168.47.72' (ECDSA) to the list of  
> > known
> > > > hosts.
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: Warning: Permanently added '192.168.47.73' (ECDSA) to the list of  
> > known
> > > > hosts.
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.637+0000 7ff67aa00700 0 [prometheus DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.637+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > prometheus
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.638+0000 7ff661600700 0 [progress INFO  
> > root]
> > > > Loading...
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.638+0000 7ff660200700 0 [prometheus INFO root] server_addr: ::  
> > server_port:
> > > > 9283
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.638+0000 7ff660200700 0 [prometheus INFO root]  
> > Cache
> > > > enabled
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.638+0000 7ff65ee00700 0 [prometheus INFO root] starting metric  
> > collection
> > > > thread
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.640+0000 7ff67aa00700 0 [rbd_support DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.642+0000 7ff661600700 0 [progress INFO root] Loaded [<progress.module.GhostEvent object at 0x7ff7443993c8>, <progress.module.GhostEvent object at 0x7ff744399400>, <progress.module.GhostEvent object at 0x7ff744399438>, <progress.module.GhostEvent object at 0x7ff744399470>, <progress.module.GhostEvent object at 0x7ff7443994a8>, <progress.module.GhostEvent object at 0x7ff7443994e0>, <progress.module.GhostEvent object at 0x7ff744399518>, <progress.module.GhostEvent object at 0x7ff744399550>, <progress.module.GhostEvent object at 0x7ff744399588>, <progress.module.GhostEvent object at 0x7ff7443995c0>, <progress.module.GhostEvent object at 0x7ff7443995f8>, <progress.module.GhostEvent object at 0x7ff744399630>, <progress.module.GhostEvent object at 0x7ff744399668>, <progress.module.GhostEvent object at 0x7ff7443996a0>, <progress.module.GhostEvent object at 0x7ff7443996d8>, <progress.module.GhostEvent object at 0x7ff744399710>, <progress.module.GhostEvent object at 0x7ff744399748>, <progress.module.GhostEvent object at 0x7ff744399780>, <progress.module.GhostEvent object at 0x7ff7443997b8>, <progress.module.GhostEvent object at 0x7ff7443997f0>, <progress.module.GhostEvent object at 0x7ff744399828>, <progress.module.GhostEvent object at 0x7ff744399860>, <progress.module.GhostEvent object at 0x7ff744399898>, <progress.module.GhostEvent object at 0x7ff7443998d0>, <progress.module.GhostEvent object at 0x7ff744399908>, <progress.module.GhostEvent object at 0x7ff744399940>, <progress.module.GhostEvent object at 0x7ff744399978>, <progress.module.GhostEvent object at 0x7ff7443999b0>, <progress.module.GhostEvent object at 0x7ff7443999e8>, <progress.module.GhostEvent object at 0x7ff744399a20>, <progress.module.GhostEvent object at 0x7ff744399a58>, <progress.module.GhostEvent object at 0x7ff744399a90>, <progress.module.GhostEvent object at 0x7ff744399ac8>, <progress.module.GhostEvent object at 0x7ff744399b00>, <progress.modu
le.GhostEvent object at 0x7ff744399b38>, <progress.module.GhostEvent object at 0x7ff744399b70>, <progress.module.GhostEvent object at 0x7ff744399ba8>, <progress.module.GhostEvent object at 0x7ff744399be0>, <progress.module.GhostEvent object at 0x7ff744399c18>, <progress.module.GhostEvent object at 0x7ff744399c50>, <progress.module.GhostEvent object at 0x7ff744399c88>, <progress.module.GhostEvent object at 0x7ff744399cc0>, <progress.module.GhostEvent object at 0x7ff744399cf8>, <progress.module.GhostEvent object at 0x7ff744399d30>, <progress.module.GhostEvent object at 0x7ff744399d68>, <progress.module.GhostEvent object at 0x7ff744399da0>, <progress.module.GhostEvent object at 0x7ff744399dd8>, <progress.module.GhostEvent object at 0x7ff744399e10>, <progress.module.GhostEvent object at 0x7ff744399e48>, <progress.module.GhostEvent object at 0x7ff744399e80>]  
> > historic
> > > > events
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.642+0000 7ff661600700 0 [progress INFO root] Loaded  
> > OSDMap,
> > > > ready.
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.642+0000 7ff661600700 0 [progress INFO root] Processing OSDMap  
> > change
> > > > 27185..27185
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.646+0000 7ff676400700 0 [dashboard INFO root] Configured CherryPy,  
> > starting
> > > > engine...
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.646+0000 7ff660200700 0 [prometheus INFO root]  
> > Starting
> > > > engine...
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE  
> > Bus
> > > > STARTING
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.647+0000 7ff660200700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE  
> > Bus
> > > > STARTING
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.656+0000 7ff654e00700 0 [rbd_support INFO root] recovery  
> > thread
> > > > starting
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.656+0000 7ff654e00700 0 [rbd_support INFO root]  
> > starting
> > > > setup
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.656+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > rbd_support
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: Warning: Permanently added '192.168.47.71' (ECDSA) to the list of  
> > known
> > > > hosts.
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.659+0000 7ff67aa00700 0 [restful DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.659+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > restful
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.660+0000 7ff651c00700 0 [restful INFO root] server_addr: ::  
> > server_port:
> > > > 8003
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.661+0000 7ff654e00700 0 [rbd_support INFO root]  
> > MirrorSnapshotScheduleHandler:
> > > > load_schedules
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.697+0000 7ff651c00700 0 [restful WARNING root] server not running: no  
> > certificate
> > > > configured
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.705+0000 7ff654e00700 0 [rbd_support INFO root] load_schedules:  
> > csxyz,
> > > > start_after=
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.706+0000 7ff67aa00700 0 [snap_schedule DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.707+0000 7ff64f400700 0 [rbd_support INFO root]  
> > MirrorSnapshotScheduleHandler:
> > > > starting
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.708+0000 7ff64ea00700 0 [rbd_support INFO root]  
> > PerfHandler:
> > > > starting
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.709+0000 7ff654e00700 0 [rbd_support INFO root] load_task_task:  
> > csxyz,
> > > > start_after=
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.718+0000 7ff649a00700 0 [rbd_support INFO root]  
> > TaskHandler:
> > > > starting
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.719+0000 7ff654e00700 0 [rbd_support INFO root]  
> > TrashPurgeScheduleHandler:
> > > > load_schedules
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.721+0000 7ff654e00700 0 [rbd_support INFO root] load_schedules:  
> > csxyz,
> > > > start_after=
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.723+0000 7ff649000700 0 [rbd_support INFO root]  
> > TrashPurgeScheduleHandler:
> > > > starting
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.723+0000 7ff654e00700 0 [rbd_support INFO root]  
> > setup
> > > > complete
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.726+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > snap_schedule
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.727+0000 7ff67aa00700 0 [status DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.727+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > status
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.728+0000 7ff67aa00700 0 [telemetry DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.729+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > telemetry
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.729+0000 7ff67aa00700 0 [volumes DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.742+0000 7ff67aa00700 0 [volumes INFO volumes.fs.async_job] queuing job for  
> > volume
> > > > 'cssnfsxyz'
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.742+0000 7ff67aa00700 0 [volumes INFO volumes.fs.async_job] queuing job for  
> > volume
> > > > 'cssnfsxyz'
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.742+0000 7ff67aa00700 1 mgr load Constructed class from  
> > module:
> > > > volumes
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63aa00700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.747+0000 7ff63c800700 -1 client.0 error registering admin socket command: (17)  
> > File
> > > > exists
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:  
> > CherryPy
> > > > Checker:
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: The Application mounted at '' has an  
> > empty
> > > > config.
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE Serving  
> > on
> > > > http://:::9283
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.754+0000 7ff660200700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE Serving  
> > on
> > > > http://:::9283
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:45] ENGINE  
> > Bus
> > > > STARTED
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.754+0000 7ff660200700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:45] ENGINE  
> > Bus
> > > > STARTED
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.754+0000 7ff660200700 0 [prometheus INFO root]  
> > Engine
> > > > started.
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.785+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: CephFS  
> > ->
> > > > /api/cephfs
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.787+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: CephFsUi  
> > ->
> > > > /ui-api/cephfs
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.789+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: IscsiUi  
> > ->
> > > > /ui-api/iscsi
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.789+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Iscsi  
> > ->
> > > > /api/iscsi
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.789+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: IscsiTarget  
> > ->
> > > > /api/iscsi/target
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.791+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: NFSGaneshaCluster  
> > ->
> > > > /api/nfs-ganesha/cluster
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.791+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: NFSGaneshaExports  
> > ->
> > > > /api/nfs-ganesha/export
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.792+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: NFSGaneshaUi  
> > ->
> > > > /ui-api/nfs-ganesha
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.792+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Rbd  
> > ->
> > > > /api/block/image
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.795+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdStatus  
> > ->
> > > > /ui-api/block/rbd
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.795+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdSnapshot  
> > ->
> > > > /api/block/image/{image_spec}/snap
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.796+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdTrash  
> > ->
> > > > /api/block/image/trash
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.797+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdNamespace  
> > ->
> > > > /api/block/pool/{pool_name}/namespace
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.797+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Pool  
> > ->
> > > > /api/pool
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.799+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: PoolUi  
> > ->
> > > > /ui-api/pool
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.800+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RBDPool  
> > ->
> > > > /api/pool
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.801+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Orchestrator  
> > ->
> > > > /ui-api/orchestrator
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.801+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Service  
> > ->
> > > > /api/service
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.802+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroring  
> > ->
> > > > /api/block/mirroring
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.802+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringSummary  
> > ->
> > > > /api/block/mirroring/summary
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.803+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringPoolMode  
> > ->
> > > > /api/block/mirroring/pool
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.803+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringPoolBootstrap  
> > ->
> > > > /api/block/mirroring/pool/{pool_name}/bootstrap
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.803+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringPoolPeer  
> > ->
> > > > /api/block/mirroring/pool/{pool_name}/peer
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.804+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirroringStatus  
> > ->
> > > > /ui-api/block/mirroring
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.805+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Rgw  
> > ->
> > > > /ui-api/rgw
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.805+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwDaemon  
> > ->
> > > > /api/rgw/daemon
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.806+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwSite  
> > ->
> > > > /api/rgw/site
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.806+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwBucket  
> > ->
> > > > /api/rgw/bucket
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.807+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwUser  
> > ->
> > > > /api/rgw/user
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.810+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Auth  
> > ->
> > > > /api/auth
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.811+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Cluster  
> > ->
> > > > /api/cluster
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.811+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: ClusterConfiguration  
> > ->
> > > > /api/cluster_conf
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.813+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: CrushRule  
> > ->
> > > > /api/crush_rule
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.814+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: CrushRuleUi  
> > ->
> > > > /ui-api/crush_rule
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.814+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Daemon  
> > ->
> > > > /api/daemon
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.815+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Docs  
> > ->
> > > > /docs
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.815+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: ErasureCodeProfile  
> > ->
> > > > /api/erasure_code_profile
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.816+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: ErasureCodeProfileUi  
> > ->
> > > > /ui-api/erasure_code_profile
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.817+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: FrontendLogging  
> > ->
> > > > /ui-api/logging
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.817+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Grafana  
> > ->
> > > > /api/grafana
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.817+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Host  
> > ->
> > > > /api/host
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.819+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: HostUi  
> > ->
> > > > /ui-api/host
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.820+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Health  
> > ->
> > > > /api/health
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.820+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: HomeController  
> > ->
> > > > /
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.822+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: LangsController  
> > ->
> > > > /ui-api/langs
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.824+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: LoginController  
> > ->
> > > > /ui-api/login
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.824+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Logs  
> > ->
> > > > /api/logs
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.824+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MgrModules  
> > ->
> > > > /api/mgr/module
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.826+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Monitor  
> > ->
> > > > /api/monitor
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.826+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Osd  
> > ->
> > > > /api/osd
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.828+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: OsdUi  
> > ->
> > > > /ui-api/osd
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.830+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: OsdFlagsController  
> > ->
> > > > /api/osd/flags
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.831+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MdsPerfCounter  
> > ->
> > > > /api/perf_counters/mds
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.831+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MonPerfCounter  
> > ->
> > > > /api/perf_counters/mon
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.832+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: OsdPerfCounter  
> > ->
> > > > /api/perf_counters/osd
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.832+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RgwPerfCounter  
> > ->
> > > > /api/perf_counters/rgw
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.833+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: RbdMirrorPerfCounter  
> > ->
> > > > /api/perf_counters/rbd-mirror
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.833+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MgrPerfCounter  
> > ->
> > > > /api/perf_counters/mgr
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.834+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: TcmuRunnerPerfCounter  
> > ->
> > > > /api/perf_counters/tcmu-runner
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.834+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: PerfCounters  
> > ->
> > > > /api/perf_counters
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.835+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: PrometheusReceiver  
> > ->
> > > > /api/prometheus_receiver
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.835+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Prometheus  
> > ->
> > > > /api/prometheus
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.836+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: PrometheusNotifications  
> > ->
> > > > /api/prometheus/notifications
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.836+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Role  
> > ->
> > > > /api/role
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.838+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Scope  
> > ->
> > > > /ui-api/scope
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.838+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Saml2  
> > ->
> > > > /auth/saml2
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.838+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Settings  
> > ->
> > > > /api/settings
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.840+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: StandardSettings  
> > ->
> > > > /ui-api/standard_settings
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.840+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Summary  
> > ->
> > > > /api/summary
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.841+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Task  
> > ->
> > > > /api/task
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.841+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: Telemetry  
> > ->
> > > > /api/telemetry
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.842+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: User  
> > ->
> > > > /api/user
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.843+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: UserPasswordPolicy  
> > ->
> > > > /api/user
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.843+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: UserChangePassword  
> > ->
> > > > /api/user/{username}
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.843+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: FeatureTogglesEndpoint  
> > ->
> > > > /api/feature_toggles
> > > > Apr 05 20:33:45 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:45.844+0000 7ff676400700 0 [dashboard INFO dashboard.controllers._base_controller] Initializing controller: MessageOfTheDay  
> > ->
> > > > /ui-api/motd
> > > > Apr 05 20:33:46 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:46.050+0000 7ff676400700 0 [dashboard INFO dashboard.module]  
> > Engine
> > > > started.
> > > > Apr 05 20:33:46 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:46.378+0000 7ff67a000700 0 log_channel(cluster) log [DBG] : pgmap v3: 610 pgs: 610 active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109  
> > TiB
> > > > avail
> > > > Apr 05 20:33:47 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:47.379+0000 7ff678c00700 0 log_channel(cluster) log [DBG] : pgmap v4: 610 pgs: 610 active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109  
> > TiB
> > > > avail
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:48] "GET /metrics HTTP/1.1" 200 42104  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.568+0000 7ff65b200700 0 [prometheus INFO cherrypy.access.140699978276536] ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:48] "GET /metrics HTTP/1.1" 200 42104  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff721c00700 0 [cephadm INFO cephadm.serve]  
> > Updating
> > > > cn01.ceph.xyz.corp:/etc/ceph/ceph.conf
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff721c00700 0 log_channel(cephadm) log [INF] :  
> > Updating
> > > > cn01.ceph.xyz.corp:/etc/ceph/ceph.conf
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff719000700 0 [cephadm INFO cephadm.serve]  
> > Updating
> > > > cn02.ceph.xyz.corp:/etc/ceph/ceph.conf
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff6fec00700 0 [cephadm INFO cephadm.serve]  
> > Updating
> > > > cn03.ceph.xyz.corp:/etc/ceph/ceph.conf
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff719000700 0 log_channel(cephadm) log [INF] :  
> > Updating
> > > > cn02.ceph.xyz.corp:/etc/ceph/ceph.conf
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.765+0000 7ff6fec00700 0 log_channel(cephadm) log [INF] :  
> > Updating
> > > > cn03.ceph.xyz.corp:/etc/ceph/ceph.conf
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff721c00700 0 [cephadm INFO cephadm.serve]  
> > Updating
> > > > cn01.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff721c00700 0 log_channel(cephadm) log [INF] :  
> > Updating
> > > > cn01.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff719000700 0 [cephadm INFO cephadm.serve]  
> > Updating
> > > > cn02.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff719000700 0 log_channel(cephadm) log [INF] :  
> > Updating
> > > > cn02.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff6fec00700 0 [cephadm INFO cephadm.serve]  
> > Updating
> > > > cn03.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.768+0000 7ff6fec00700 0 log_channel(cephadm) log [INF] :  
> > Updating
> > > > cn03.ceph.xyz.corp:/etc/ceph/ceph.client.admin.keyring
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.812+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 824b8fde-5a37-4436-af45-a7d8255e2c66 does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.813+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 621cb660-d584-45e4-8033-4a476e804bba does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.813+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 007a4dd2-46d3-4655-9c93-c02f97e555cb does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.814+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 6b0fbff2-881e-4203-9149-f0977bc13eab does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.814+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 8a1a90b8-1f2a-48c8-924f-6e74032b6319 does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.815+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 3c322d26-9edb-4d26-b4bc-97c330f5a1d8 does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.825+0000 7ff6a7400700 0 [progress WARNING root] complete: ev f3e5a517-d643-4fa5-b127-57716264e4e3 does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.825+0000 7ff6a7400700 0 [progress WARNING root] complete: ev 3fb020c7-92e8-4eb1-a884-2d04e5bca889 does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.840+0000 7ff6a7400700 0 [progress WARNING root] complete: ev bf9e0705-075e-4134-b77f-aa64fe92112a does  
> > not
> > > > exist
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.848+0000 7ff6a7400700 0 [cephadm INFO cephadm.upgrade] Upgrade: Need to upgrade  
> > myself
> > > > (mgr.cn02.ceph.xyz.corp.ggixgj)
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.848+0000 7ff6a7400700 0 log_channel(cephadm) log [INF] : Upgrade: Need to upgrade  
> > myself
> > > > (mgr.cn02.ceph.xyz.corp.ggixgj)
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.848+0000 7ff6a7400700 0 [progress INFO root] update: starting ev e7e1a809-558d-43a7-842a-c6229fdc57af (Upgrade  
> > to
> > > > 17.2.0)
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.851+0000 7ff6a7400700 0 [cephadm INFO cephadm.services.cephadmservice] Failing over to  
> > other
> > > > MGR
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.851+0000 7ff6a7400700 0 log_channel(cephadm) log [INF] : Failing over to  
> > other
> > > > MGR
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 -1 mgr handle_mgr_map I was active but no  
> > longer
> > > > am
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > e:
> > > > '/usr/bin/ceph-mgr'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 0:
> > > > '/usr/bin/ceph-mgr'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 1:
> > > > '-n'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 2:
> > > > 'mgr.cn02.ceph.xyz.corp.ggixgj'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 3:
> > > > '-f'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 4:
> > > > '--setuser'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 5:
> > > > 'ceph'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 6:
> > > > '--setgroup'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 7:
> > > > 'ceph'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 8:
> > > > '--default-log-to-file=false'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > 9:
> > > > '--default-log-to-stderr=true'
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn 10:  
> > '--default-log-stderr-prefix=debug
> > > > '
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn respawning with  
> > exe
> > > > /usr/bin/ceph-mgr
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.869+0000 7ff7e7c00700 1 mgr respawn  
> > exe_path
> > > > /proc/self/exe
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ignoring --setuser ceph since I am  
> > not
> > > > root
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ignoring --setgroup ceph since I am  
> > not
> > > > root
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.949+0000 7f5a66492680 0 ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516) pacific (stable), process ceph-mgr,  
> > pid
> > > > 2
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.949+0000 7f5a66492680 0 pidfile_write: ignore  
> > empty
> > > > --pid-file
> > > > Apr 05 20:33:48 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:48.976+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'alerts'
> > > > Apr 05 20:33:49 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.105+0000 7f5a66492680 -1 mgr[py] Module alerts has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:49 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.105+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'balancer'
> > > > Apr 05 20:33:49 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.212+0000 7f5a66492680 -1 mgr[py] Module balancer has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:49 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.212+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'cephadm'
> > > > Apr 05 20:33:49 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.568+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'crash'
> > > > Apr 05 20:33:49 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.677+0000 7f5a66492680 -1 mgr[py] Module crash has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:49 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:49.677+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'dashboard'
> > > > Apr 05 20:33:50 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.276+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'devicehealth'
> > > > Apr 05 20:33:50 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.415+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'diskprediction_local'
> > > > Apr 05 20:33:50 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.640+0000 7f5a66492680 -1 mgr[py] Module diskprediction_local has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:50 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.640+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'influx'
> > > > Apr 05 20:33:50 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.750+0000 7f5a66492680 -1 mgr[py] Module influx has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:50 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.750+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'insights'
> > > > Apr 05 20:33:50 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:50.919+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'iostat'
> > > > Apr 05 20:33:51 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.020+0000 7f5a66492680 -1 mgr[py] Module iostat has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:51 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.020+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'k8sevents'
> > > > Apr 05 20:33:51 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.533+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'localpool'
> > > > Apr 05 20:33:51 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.634+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'mds_autoscaler'
> > > > Apr 05 20:33:51 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:51.894+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'mirroring'
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.019+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'nfs'
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.209+0000 7f5a66492680 -1 mgr[py] Module nfs has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.209+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'orchestrator'
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.386+0000 7f5a66492680 -1 mgr[py] Module orchestrator has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.386+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'osd_support'
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.494+0000 7f5a66492680 -1 mgr[py] Module osd_support has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.494+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'pg_autoscaler'
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.760+0000 7f5a66492680 -1 mgr[py] Module pg_autoscaler has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.760+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'progress'
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.870+0000 7f5a66492680 -1 mgr[py] Module progress has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:52 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:52.870+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'prometheus'
> > > > Apr 05 20:33:53 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.257+0000 7f5a66492680 -1 mgr[py] Module prometheus has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:53 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.257+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'rbd_support'
> > > > Apr 05 20:33:53 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.388+0000 7f5a66492680 -1 mgr[py] Module rbd_support has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:53 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.388+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'restful'
> > > > Apr 05 20:33:53 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:53.799+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'rook'
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.318+0000 7f5a66492680 -1 mgr[py] Module rook has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.318+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'selftest'
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.428+0000 7f5a66492680 -1 mgr[py] Module selftest has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.428+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'snap_schedule'
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.549+0000 7f5a66492680 -1 mgr[py] Module snap_schedule has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.549+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'stats'
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.660+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'status'
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.972+0000 7f5a66492680 -1 mgr[py] Module status has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:54 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:54.972+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'telegraf'
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.089+0000 7f5a66492680 -1 mgr[py] Module telegraf has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.089+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'telemetry'
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.268+0000 7f5a66492680 -1 mgr[py] Module telemetry has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.268+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'test_orchestrator'
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.448+0000 7f5a66492680 -1 mgr[py] Module test_orchestrator has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.448+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'volumes'
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.676+0000 7f5a66492680 -1 mgr[py] Module volumes has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.676+0000 7f5a66492680 1 mgr[py] Loading python  
> > module
> > > > 'zabbix'
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.792+0000 7f5a66492680 -1 mgr[py] Module zabbix has missing  
> > NOTIFY_TYPES
> > > > member
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.793+0000 7f5a52600700 0 ms_deliver_dispatch: unhandled message 0x556eb85f26e0 mon_map magic: 0 v1 from  
> > mon.2
> > > > v2:192.168.47.72:3300/0
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f5a53000700 0 [dashboard DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f5a53000700 1 mgr load Constructed class from  
> > module:
> > > > dashboard
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f5a53000700 0 [prometheus DEBUG root] setting log level based on debug_mgr:  
> > INFO
> > > > (2/5)
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f5a53000700 1 mgr load Constructed class from  
> > module:
> > > > prometheus
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.794+0000 7f58f7800700 0 [dashboard INFO root] server: ssl=yes  
> > host=::
> > > > port=8443
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.795+0000 7f5900400700 0 [prometheus INFO root] server_addr: ::  
> > server_port:
> > > > 9283
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.795+0000 7f5900400700 0 [prometheus INFO root]  
> > Starting
> > > > engine...
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:55] ENGINE  
> > Bus
> > > > STARTING
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.795+0000 7f5900400700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:55] ENGINE  
> > Bus
> > > > STARTING
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.797+0000 7f5a52600700 0 client.0 ms_handle_reset  
> > on
> > > > v2:192.168.47.73:6842/917991990
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.825+0000 7f58f7800700 0 [dashboard INFO root] Configured CherryPy,  
> > starting
> > > > engine...
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.826+0000 7f58f7800700 0 [dashboard INFO root]  
> > Starting
> > > > engine...
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:  
> > CherryPy
> > > > Checker:
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: The Application mounted at '' has an  
> > empty
> > > > config.
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:55] ENGINE Serving  
> > on
> > > > http://:::9283
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.900+0000 7f5900400700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:55] ENGINE Serving  
> > on
> > > > http://:::9283
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: [05/Apr/2025:20:33:55] ENGINE  
> > Bus
> > > > STARTED
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.900+0000 7f5900400700 0 [prometheus INFO cherrypy.error] [05/Apr/2025:20:33:55] ENGINE  
> > Bus
> > > > STARTED
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.900+0000 7f5900400700 0 [prometheus INFO root]  
> > Engine
> > > > started.
> > > > Apr 05 20:33:55 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:55.939+0000 7f58f7800700 0 [dashboard INFO root]  
> > Engine
> > > > started...
> > > > Apr 05 20:33:58 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:58] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:58 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:33:58.569+0000 7f59b8000700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:58] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:00 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:00] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:00 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:00.702+0000 7f59af400700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:00] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:03 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:03] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:03 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:03.286+0000 7f59a6800700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:03] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:08 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:08] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:08 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:08.569+0000 7f599dc00700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:08] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:10 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:10] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:10 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:10.702+0000 7f5995000700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:10] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:13 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:13] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:13 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:13.286+0000 7f598c400700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:13] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:18 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:18] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:18 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:18.569+0000 7f5983800700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:18] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:20 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:20] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:20 cn02.ceph.xyz.corp
> > > >  
> > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]: debug 2025-04-05T20:34:20.702+0000 7f597ac00700 0 [prometheus INFO cherrypy.access.140023317690912] ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:20] "GET /metrics HTTP/1.1" 200 -  
> > ""
> > > > "Prometheus/2.33.4"
> > > >
> > > > cn03
> > > > ——
> > > > Apr 05 20:33:42 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:42] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:42 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:42.753+0000 7f2724a00700 0 [prometheus INFO
> > > > cherrypy.access.139806526902960] ::ffff:192.168.47.72 - -
> > > > [05/Apr/2025:20:33:42] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:43 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:43] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:43 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:43.772+0000 7f271be00700 0 [prometheus INFO
> > > > cherrypy.access.139806526902960] ::ffff:192.168.47.71 - -
> > > > [05/Apr/2025:20:33:43] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:47 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:47] "GET /metrics
> > > > HTTP/1.1" 200 - "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:47 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:47.037+0000 7f2713200700 0 [prometheus INFO
> > > > cherrypy.access.139806526902960] ::ffff:192.168.47.73 - -
> > > > [05/Apr/2025:20:33:47] "GET /metrics HTTP/1.1" 200 - ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.867+0000 7f27d1a00700 1 mgr handle_mgr_map
> > > > Activating!
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.868+0000 7f27d1a00700 1 mgr handle_mgr_map
> > > > I am now activating
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.878+0000 7f2663400700 0 [dashboard INFO
> > > > root] Stopping engine...
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.878+0000 7f2663400700 0 [dashboard INFO
> > > > root] Stopped engine...
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.878+0000 7f2663400700 0 [prometheus INFO
> > > > root] Stopping engine...
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.878+0000 7f2663400700 0 [prometheus INFO
> > > > root] Stopped engine
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > [05/Apr/2025:20:33:48] ENGINE Bus STOPPING
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.879+0000 7f267e600700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:48] ENGINE Bus STOPPING
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.890+0000 7f2675a00700 0 [dashboard INFO
> > > > root] Engine stopped.
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > [05/Apr/2025:20:33:48] ENGINE HTTP Server
> > > > cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.891+0000 7f267e600700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:48] ENGINE HTTP Server
> > > > cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > [05/Apr/2025:20:33:48] ENGINE Bus STOPPED
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.892+0000 7f267e600700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:48] ENGINE Bus STOPPED
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.892+0000 7f267e600700 0 [prometheus INFO
> > > > root] Engine stopped.
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.898+0000 7f2663400700 0 [alerts DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.898+0000 7f2663400700 0 [alerts INFO root]
> > > > Init
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.898+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: alerts
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.899+0000 7f2663400700 0 [balancer DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.899+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: balancer
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.899+0000 7f26f8e00700 0 [alerts INFO root]
> > > > Starting
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.901+0000 7f2663400700 0 [cephadm DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.901+0000 7f26f0200700 0 [balancer INFO
> > > > root] Starting
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.908+0000 7f26f0200700 0 [balancer INFO
> > > > root] Optimize plan auto_2025-04-05_20:33:48
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.908+0000 7f26f0200700 0 [balancer INFO
> > > > root] Mode upmap, max misplaced 0.050000
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.909+0000 7f26f0200700 0 [balancer INFO
> > > > root] Some PGs (1.000000) are unknown; try again later
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.917+0000 7f2663400700 -1 mgr load Failed
> > > > to construct class in 'cephadm'
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.917+0000 7f2663400700 -1 mgr load
> > > > Traceback (most recent call last):
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > File "/usr/share/ceph/mgr/cephadm/module.py", line 470, in __init__
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > self.upgrade = CephadmUpgrade(self)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > File "/usr/share/ceph/mgr/cephadm/upgrade.py", line 112, in __init__
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > self.upgrade_state: Optional[UpgradeState] =
> > > > UpgradeState.from_json(json.loads(t))
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > File "/usr/share/ceph/mgr/cephadm/upgrade.py", line 93, in from_json
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > return cls(**c)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > TypeError: __init__() got an unexpected keyword argument
> > > > 'daemon_types'
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.918+0000 7f2663400700 -1 mgr operator()
> > > > Failed to run module in active mode ('cephadm')
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.919+0000 7f2663400700 0 [crash DEBUG root]
> > > > setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.919+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: crash
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.919+0000 7f2663400700 0 [dashboard DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.919+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: dashboard
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.920+0000 7f2660c00700 0 [dashboard INFO
> > > > access_control] Loading user roles DB version=2
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.921+0000 7f2660c00700 0 [dashboard INFO
> > > > sso] Loading SSO DB version=1
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.921+0000 7f2660c00700 0 [dashboard INFO
> > > > root] server: ssl=yes host=:: port=8443
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.922+0000 7f2663400700 0 [devicehealth
> > > > DEBUG root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.922+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: devicehealth
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.922+0000 7f2663400700 0 [insights DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.922+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: insights
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.923+0000 7f2660200700 0 [devicehealth INFO
> > > > root] Starting
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.925+0000 7f2663400700 0 [iostat DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.925+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: iostat
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.925+0000 7f265f800700 0 [insights INFO
> > > > root] Reset curr health slot key health_history/2025-04-05_20 next
> > > > flush None checks check count 0
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.926+0000 7f2663400700 0 [mirroring DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.931+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: mirroring
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.932+0000 7f2663400700 0 [nfs DEBUG root]
> > > > setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.932+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: nfs
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.932+0000 7f2663400700 0 [orchestrator
> > > > DEBUG root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.932+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: orchestrator
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.933+0000 7f2663400700 0 [pg_autoscaler
> > > > DEBUG root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.933+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: pg_autoscaler
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.934+0000 7f2663400700 0 [progress DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.934+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: progress
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.934+0000 7f2657600700 0 [pg_autoscaler
> > > > INFO root] _maybe_adjust
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.940+0000 7f2656c00700 0 [progress INFO
> > > > root] Loading...
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.940+0000 7f2663400700 0 [prometheus DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.944+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: prometheus
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.944+0000 7f2656200700 0 [prometheus INFO
> > > > root] server_addr: :: server_port: 9283
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.944+0000 7f2656200700 0 [prometheus INFO
> > > > root] Cache enabled
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.945+0000 7f2656c00700 0 [progress INFO
> > > > root] Loaded [<progress.module.GhostEvent object at 0x7f27363ac400>,
> > > > <progress.module.GhostEvent object at 0x7f27363ac438>,
> > > > <progress.module.GhostEvent object at 0x7f27363ac3c8>,
> > > > <progress.module.GhostEvent object at 0x7f27363ac5c0>,
> > > > <progress.module.GhostEvent object at 0x7f27363ac4e0>,
> > > > <progress.module.GhostEvent object at 0x7f27363ac4a8>,
> > > > <progress.module.GhostEvent object at 0x7f278d317e10>,
> > > > <progress.module.GhostEvent object at 0x7f278d4397b8>,
> > > > <progress.module.GhostEvent object at 0x7f278d439908>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed2e8>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed4e0>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed358>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed278>,
> > > > <progress.module.GhostEvent object at 0x7f27363edeb8>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed438>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed400>,
> > > > <progress.module.GhostEvent object at 0x7f27363edb00>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed780>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed240>,
> > > > <progress.module.GhostEvent object at 0x7f27363edba8>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed2b0>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed0b8>,
> > > > <progress.module.GhostEvent object at 0x7f27363edcf8>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed128>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed080>,
> > > > <progress.module.GhostEvent object at 0x7f27363edc18>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed860>,
> > > > <progress.module.GhostEvent object at 0x7f27363eda58>,
> > > > <progress.module.GhostEvent object at 0x7f27363eda90>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed160>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed908>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed940>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed8d0>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed978>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed748>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed898>,
> > > > <progress.module.GhostEvent object at 0x7f27363eda20>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed9b0>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed9e8>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed828>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed7f0>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed7b8>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed668>,
> > > > <progress.module.GhostEvent object at 0x7f27363ed320>,
> > > > <progress.module.GhostEvent object at 0x7f27363ede48>,
> > > > <progress.module.GhostEvent object at 0x7f27363edc50>,
> > > > <progress.module.GhostEvent object at 0x7f27363ede80>,
> > > > <progress.module.GhostEvent object at 0x7f27363edf60>,
> > > > <progress.module.GhostEvent object at 0x7f27363edd68>,
> > > > <progress.module.GhostEvent object at 0x7f27363edda0>] historic events
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.945+0000 7f2656c00700 0 [progress INFO
> > > > root] Loaded OSDMap, ready.
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.946+0000 7f2663400700 0 [rbd_support DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.946+0000 7f2655800700 0 [prometheus INFO
> > > > root] starting metric collection thread
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.947+0000 7f2656200700 0 [prometheus INFO
> > > > root] Starting engine...
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > [05/Apr/2025:20:33:48] ENGINE Bus STARTING
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.948+0000 7f2656200700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:48] ENGINE Bus STARTING
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.948+0000 7f2660c00700 0 [dashboard INFO
> > > > root] Configured CherryPy, starting engine...
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.953+0000 7f2663400700 0 [rbd_support INFO
> > > > root] MirrorSnapshotScheduleHandler: load_schedules
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.957+0000 7f2663400700 0 [rbd_support INFO
> > > > root] load_schedules: csxyz, start_after=
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.989+0000 7f2649a00700 0 [rbd_support INFO
> > > > root] MirrorSnapshotScheduleHandler: starting
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.990+0000 7f2649000700 0 [rbd_support INFO
> > > > root] PerfHandler: starting
> > > > Apr 05 20:33:48 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:48.991+0000 7f2663400700 0 [rbd_support INFO
> > > > root] load_task_task: csxyz, start_after=
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.019+0000 7f2648600700 0 [rbd_support INFO
> > > > root] TaskHandler: starting
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.020+0000 7f2663400700 0 [rbd_support INFO
> > > > root] TrashPurgeScheduleHandler: load_schedules
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.022+0000 7f2663400700 0 [rbd_support INFO
> > > > root] load_schedules: csxyz, start_after=
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.024+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > CephFS -> /api/cephfs
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.026+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > CephFsUi -> /ui-api/cephfs
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.028+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > IscsiUi -> /ui-api/iscsi
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.028+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Iscsi -> /api/iscsi
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.028+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > IscsiTarget -> /api/iscsi/target
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.030+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > NFSGanesha -> /api/nfs-ganesha
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.030+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > NFSGaneshaCluster -> /api/nfs-ganesha/cluster
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.030+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > NFSGaneshaExports -> /api/nfs-ganesha/export
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.031+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > NFSGaneshaUi -> /ui-api/nfs-ganesha
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.031+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller: Rbd
> > > > -> /api/block/image
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.034+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdSnapshot -> /api/block/image/{image_spec}/snap
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.035+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdTrash -> /api/block/image/trash
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.035+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdNamespace -> /api/block/pool/{pool_name}/namespace
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.036+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdMirroring -> /api/block/mirroring
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.036+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdMirroringSummary -> /api/block/mirroring/summary
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.037+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdMirroringPoolMode -> /api/block/mirroring/pool
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.037+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdMirroringPoolBootstrap ->
> > > > /api/block/mirroring/pool/{pool_name}/bootstrap
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.037+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdMirroringPoolPeer -> /api/block/mirroring/pool/{pool_name}/peer
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.038+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller: Rgw
> > > > -> /api/rgw
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.039+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RgwDaemon -> /api/rgw/daemon
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.039+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RgwSite -> /api/rgw/site
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.040+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RgwBucket -> /api/rgw/bucket
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.041+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RgwUser -> /api/rgw/user
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.044+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Auth -> /api/auth
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.044+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Cluster -> /api/cluster
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.045+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > ClusterConfiguration -> /api/cluster_conf
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.046+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > CrushRule -> /api/crush_rule
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.047+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > CrushRuleUi -> /ui-api/crush_rule
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.048+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Orchestrator -> /api/orchestrator
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.048+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Daemon -> /api/daemon
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.049+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Docs -> /docs
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.049+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > ErasureCodeProfile -> /api/erasure_code_profile
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.050+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > ErasureCodeProfileUi -> /ui-api/erasure_code_profile
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.050+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > FeedbackController -> /api/feedback
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.051+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > FeedbackApiController -> /api/feedback/api_key
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.052+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > FeedbackUiController -> /ui-api/feedback/api_key
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.052+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > FrontendLogging -> /ui-api/logging
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.052+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Grafana -> /api/grafana
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.052+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Host -> /api/host
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.054+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > HostUi -> /ui-api/host
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.054+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Health -> /api/health
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.054+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > HomeController -> /
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > [05/Apr/2025:20:33:49] ENGINE Serving on http://:::9283
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.055+0000 7f2656200700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:49] ENGINE Serving on
> > > > http://:::9283
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > [05/Apr/2025:20:33:49] ENGINE Bus STARTED
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.055+0000 7f2656200700 0 [prometheus INFO
> > > > cherrypy.error] [05/Apr/2025:20:33:49] ENGINE Bus STARTED
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.055+0000 7f2656200700 0 [prometheus INFO
> > > > root] Engine started.
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.056+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > LangsController -> /ui-api/langs
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.056+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Logs -> /api/logs
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.056+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > MgrModules -> /api/mgr/module
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.058+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Monitor -> /api/monitor
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.058+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller: Osd
> > > > -> /api/osd
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.060+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > OsdFlagsController -> /api/osd/flags
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.061+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > MdsPerfCounter -> /api/perf_counters/mds
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.061+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > MonPerfCounter -> /api/perf_counters/mon
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.062+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > OsdPerfCounter -> /api/perf_counters/osd
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.062+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RgwPerfCounter -> /api/perf_counters/rgw
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.063+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > RbdMirrorPerfCounter -> /api/perf_counters/rbd-mirror
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.063+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > MgrPerfCounter -> /api/perf_counters/mgr
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.064+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > TcmuRunnerPerfCounter -> /api/perf_counters/tcmu-runner
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.064+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > PerfCounters -> /api/perf_counters
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.065+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Pool -> /api/pool
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.066+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > PoolUi -> /ui-api/pool
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.067+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > PrometheusReceiver -> /api/prometheus_receiver
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.067+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Prometheus -> /api/prometheus
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.068+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > PrometheusNotifications -> /api/prometheus/notifications
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.069+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Role -> /api/role
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.070+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Scope -> /ui-api/scope
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.070+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Saml2 -> /auth/saml2
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.070+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Service -> /api/service
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.071+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Settings -> /api/settings
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.073+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > StandardSettings -> /ui-api/standard_settings
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.073+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Summary -> /api/summary
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.073+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Task -> /api/task
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.074+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > Telemetry -> /api/telemetry
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.074+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > User -> /api/user
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.076+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > UserPasswordPolicy -> /api/user
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.076+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > UserChangePassword -> /api/user/{username}
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.076+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > FeatureTogglesEndpoint -> /api/feature_toggles
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.076+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.controllers._base_controller] Initializing controller:
> > > > MessageOfTheDay -> /ui-api/motd
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.090+0000 7f2647200700 0 [rbd_support INFO
> > > > root] TrashPurgeScheduleHandler: starting
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.096+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: rbd_support
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.102+0000 7f2663400700 0 [restful DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.102+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: restful
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.108+0000 7f2645e00700 0 [restful INFO
> > > > root] server_addr: :: server_port: 8003
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.109+0000 7f2663400700 0 [snap_schedule
> > > > DEBUG root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.116+0000 7f2645e00700 0 [restful WARNING
> > > > root] server not running: no certificate configured
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.257+0000 7f2660c00700 0 [dashboard INFO
> > > > dashboard.module] Engine started.
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.273+0000 7f2663400700 -1 mgr load Failed
> > > > to construct class in 'snap_schedule'
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.273+0000 7f2663400700 -1 mgr load
> > > > Traceback (most recent call last):
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > File "/usr/share/ceph/mgr/snap_schedule/module.py", line 38, in
> > > > __init__
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > self.client = SnapSchedClient(self)
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line
> > > > 158, in __init__
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > with self.get_schedule_db(fs_name) as conn_mgr:
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line
> > > > 192, in get_schedule_db
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > db.executescript(dump)
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > sqlite3.OperationalError: table schedules already exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.274+0000 7f2663400700 -1 mgr operator()
> > > > Failed to run module in active mode ('snap_schedule')
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.274+0000 7f2663400700 0 [status DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.274+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: status
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.275+0000 7f2663400700 0 [telemetry DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.276+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: telemetry
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.276+0000 7f2663400700 0 [volumes DEBUG
> > > > root] setting log level based on debug_mgr: INFO (2/5)
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.284+0000 7f2660200700 0 [devicehealth INFO
> > > > root] Check health
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.290+0000 7f2663400700 0 [volumes INFO
> > > > volumes.fs.async_job] queuing job for volume 'cssnfsxyz'
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.290+0000 7f2663400700 0 [volumes INFO
> > > > volumes.fs.async_job] queuing job for volume 'cssnfsxyz'
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.290+0000 7f2663400700 1 mgr load
> > > > Constructed class from module: volumes
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f2627e00700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.295+0000 7f262b000700 -1 client.0 error
> > > > registering admin socket command: (17) File exists
> > > > Apr 05 20:33:49 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:49.883+0000 7f2662a00700 0
> > > > log_channel(cluster) log [DBG] : pgmap v3: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail
> > > > Apr 05 20:33:50 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:50.875+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v4: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail
> > > > Apr 05 20:33:52 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.72 - - [05/Apr/2025:20:33:52] "GET /metrics
> > > > HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:52 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:52.753+0000 7f264f400700 0 [prometheus INFO
> > > > cherrypy.access.139806389938328] ::ffff:192.168.47.72 - -
> > > > [05/Apr/2025:20:33:52] "GET /metrics HTTP/1.1" 200 33671 ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:52 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:52.876+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v5: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail
> > > > Apr 05 20:33:53 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.71 - - [05/Apr/2025:20:33:53] "GET /metrics
> > > > HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:53 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:53.772+0000 7f264ea00700 0 [prometheus INFO
> > > > cherrypy.access.139806389938328] ::ffff:192.168.47.71 - -
> > > > [05/Apr/2025:20:33:53] "GET /metrics HTTP/1.1" 200 33671 ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:53 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:53.870+0000 7f2663400700 0 [insights INFO
> > > > root] Received health check update 0 pending
> > > > Apr 05 20:33:53 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:53.871+0000 7f265f800700 0 [insights INFO
> > > > root] Applying 1 health updates to slot key
> > > > health_history/2025-04-05_20 next flush None checks check count 0
> > > > Apr 05 20:33:53 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:53.871+0000 7f265f800700 0 [insights INFO
> > > > root] Maybe flushing slot key health_history/2025-04-05_20 next
> > > > flush None checks check count 0 needed False
> > > > Apr 05 20:33:54 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:54.877+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v6: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail; 0
> > > > B/s rd, 12 KiB/s wr, 2 op/s
> > > > Apr 05 20:33:55 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:55.796+0000 7f2662a00700 2 mgr.server
> > > > handle_open ignoring open from mgr.cn02.ceph.xyz.corp.ggixgj
> > > > 192.168.47.72:0/2580067884; not ready for session (expect reconnect)
> > > > Apr 05 20:33:56 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:56.878+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v7: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 439 B/s rd, 27 KiB/s wr, 6 op/s
> > > > Apr 05 20:33:57 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.73 - - [05/Apr/2025:20:33:57] "GET /metrics
> > > > HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> > > > Apr 05 20:33:57 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:57.038+0000 7f264e000700 0 [prometheus INFO
> > > > cherrypy.access.139806389938328] ::ffff:192.168.47.73 - -
> > > > [05/Apr/2025:20:33:57] "GET /metrics HTTP/1.1" 200 33671 ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:33:58 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:58.870+0000 7f2663400700 0 [insights INFO
> > > > root] Received health check update 0 pending
> > > > Apr 05 20:33:58 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:58.878+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v8: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 341 B/s rd, 21 KiB/s wr, 4 op/s
> > > > Apr 05 20:33:58 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:58.897+0000 7f265f800700 0 [insights INFO
> > > > root] Applying 1 health updates to slot key
> > > > health_history/2025-04-05_20 next flush None checks check count 0
> > > > Apr 05 20:33:58 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:33:58.897+0000 7f265f800700 0 [insights INFO
> > > > root] Maybe flushing slot key health_history/2025-04-05_20 next
> > > > flush None checks check count 0 needed False
> > > > Apr 05 20:34:00 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:00.880+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v9: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 279 B/s rd, 24 KiB/s wr, 6 op/s
> > > > Apr 05 20:34:02 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:02] "GET /metrics
> > > > HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:02 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:02.753+0000 7f264d600700 0 [prometheus INFO
> > > > cherrypy.access.139806389938328] ::ffff:192.168.47.72 - -
> > > > [05/Apr/2025:20:34:02] "GET /metrics HTTP/1.1" 200 33671 ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:02 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:02.880+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v10: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 22 KiB/s wr, 5 op/s
> > > > Apr 05 20:34:03 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:03] "GET /metrics
> > > > HTTP/1.1" 200 33671 "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:03 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:03.772+0000 7f264cc00700 0 [prometheus INFO
> > > > cherrypy.access.139806389938328] ::ffff:192.168.47.71 - -
> > > > [05/Apr/2025:20:34:03] "GET /metrics HTTP/1.1" 200 33671 ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:03 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:03.870+0000 7f2663400700 0 [insights INFO
> > > > root] Received health check update 0 pending
> > > > Apr 05 20:34:03 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:03.873+0000 7f265f800700 0 [insights INFO
> > > > root] Applying 1 health updates to slot key
> > > > health_history/2025-04-05_20 next flush None checks check count 0
> > > > Apr 05 20:34:03 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:03.874+0000 7f265f800700 0 [insights INFO
> > > > root] Maybe flushing slot key health_history/2025-04-05_20 next
> > > > flush None checks check count 0 needed False
> > > > Apr 05 20:34:04 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:04.881+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v11: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 59 KiB/s wr, 11 op/s
> > > > Apr 05 20:34:06 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:06.882+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v12: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 511 B/s rd, 66 KiB/s wr, 13 op/s
> > > > Apr 05 20:34:07 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.73 - - [05/Apr/2025:20:34:07] "GET /metrics
> > > > HTTP/1.1" 200 267239 "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:07 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:07.037+0000 7f264c200700 0 [prometheus INFO
> > > > cherrypy.access.139806389938328] ::ffff:192.168.47.73 - -
> > > > [05/Apr/2025:20:34:07] "GET /metrics HTTP/1.1" 200 267239 ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:08 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:08.871+0000 7f2663400700 0 [insights INFO
> > > > root] Received health check update 0 pending
> > > > Apr 05 20:34:08 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:08.883+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v13: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 55 KiB/s wr, 11 op/s
> > > > Apr 05 20:34:08 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:08.907+0000 7f265f800700 0 [insights INFO
> > > > root] Applying 1 health updates to slot key
> > > > health_history/2025-04-05_20 next flush None checks check count 0
> > > > Apr 05 20:34:08 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:08.907+0000 7f265f800700 0 [insights INFO
> > > > root] Maybe flushing slot key health_history/2025-04-05_20 next
> > > > flush None checks check count 0 needed False
> > > > Apr 05 20:34:10 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:10.884+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v14: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 79 KiB/s wr, 16 op/s
> > > > Apr 05 20:34:12 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.72 - - [05/Apr/2025:20:34:12] "GET /metrics
> > > > HTTP/1.1" 200 267239 "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:12 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:12.753+0000 7f264b800700 0 [prometheus INFO
> > > > cherrypy.access.139806389938328] ::ffff:192.168.47.72 - -
> > > > [05/Apr/2025:20:34:12] "GET /metrics HTTP/1.1" 200 267239 ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:12 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:12.884+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v15: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 255 B/s rd, 73 KiB/s wr, 14 op/s
> > > > Apr 05 20:34:13 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > ::ffff:192.168.47.71 - - [05/Apr/2025:20:34:13] "GET /metrics
> > > > HTTP/1.1" 200 267239 "" "Prometheus/2.33.4"
> > > > Apr 05 20:34:13 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:13.772+0000 7f264ae00700 0 [prometheus INFO
> > > > cherrypy.access.139806389938328] ::ffff:192.168.47.71 - -
> > > > [05/Apr/2025:20:34:13] "GET /metrics HTTP/1.1" 200 267239 ""
> > > > "Prometheus/2.33.4"
> > > > Apr 05 20:34:13 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:13.871+0000 7f2663400700 0 [insights INFO
> > > > root] Received health check update 0 pending
> > > > Apr 05 20:34:13 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:13.883+0000 7f265f800700 0 [insights INFO
> > > > root] Applying 1 health updates to slot key
> > > > health_history/2025-04-05_20 next flush None checks check count 0
> > > > Apr 05 20:34:13 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:13.883+0000 7f265f800700 0 [insights INFO
> > > > root] Maybe flushing slot key health_history/2025-04-05_20 next
> > > > flush None checks check count 0 needed False
> > > > Apr 05 20:34:14 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:14.886+0000 7f2661600700 0
> > > > log_channel(cluster) log [DBG] : pgmap v16: 610 pgs: 610
> > > > active+clean; 1.1 TiB data, 3.3 TiB used, 106 TiB / 109 TiB avail;
> > > > 82 KiB/s rd, 103 KiB/s wr, 113 op/s
> > > > Apr 05 20:34:15 cn03.ceph.xyz.corp
> > > > ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn03-negzvb[307291]:
> > > > debug 2025-04-05T20:34:15.754+0000 7f2644000700 0 [dashboard I
> [Text gekürzt]


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx