Hi Eugen,
Thanks so much! Your suggestion did the trick.
[root@ceph-mon1 ~]# ceph health detail
HEALTH_OK
All good now!
Thanks,
Gagan
On Tue, Aug 5, 2025 at 12:14 PM Eugen Block <eblock@xxxxxx> wrote:
> How exactly did you try to get rid of the old daemon?
> Is the previous daemon still listed in 'cephadm ls' output (with a
> different FSID)?
> Is the old daemon still running (since the address is already in use)?
> Do you see it in the output of 'lsof -Pi | grep ":9100"'? Or in
> 'docker ps' or 'podman ps'?
>
> Depending on the information, you might need to remove the daemon with
> 'cephadm rm-daemon --fsid {FSID} --name node-exporter.ceph-mon2
> --force'. But as I said, it would be helpful to see what exactly is
> going on there.
>
>
> Zitat von gagan tiwari <gagan.tiwari@xxxxxxxxxxxxxxxxxx>:
>
> > Hi Guys,
> > Please advice
> >
> > On Mon, 4 Aug, 2025, 9:50 am gagan tiwari, <
> gagan.tiwari@xxxxxxxxxxxxxxxxxx>
> > wrote:
> >
> >> HI Guys,
> >> Good Morning!
> >>
> >> I have re-deployed the ceph cluster using cephadm because of some issues
> >> with osds.
> >> Everything is working fine now with the new cluster except that Cephadm
> is
> >> still trying to deploy node_exporter daemon on one of the monitor hosts
> >> using the old container which no longer exists.
> >>
> >> node_exporter is running fine on that host in the new container.
> >>
> >> But ceph is still trying to deploy it on that host using an old
> container
> >> and since it couldn't find that podman container it throws the warning.
> >>
> >> I did try to remove that node_exporter service on that node and added it
> >> again but no luck. When I reboot that host after its coming ceph status
> >> shows "Health OK" but after sometime it starts showing same waring
> >>
> >> ceph health detail
> >> HEALTH_WARN Failed to place 1 daemon(s)
> >> [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)
> >> Failed while placing node-exporter.ceph-mon2 on ceph-mon2: cephadm
> >> exited with an error code: 1, stderr: Non-zero exit code 125 from
> >> /usr/bin/podman container inspect --format {{.State.Status}}
> >> ceph-1aa23d18-6eb0-11f0-823c-cc28aa448c14-node-exporter-ceph-mon2
> >> /usr/bin/podman: stderr Error: no such container
> >> ceph-1aa23d18-6eb0-11f0-823c-cc28aa448c14-node-exporter-ceph-mon2
> >> Non-zero exit code 125 from /usr/bin/podman container inspect --format
> >> {{.State.Status}}
> >> ceph-1aa23d18-6eb0-11f0-823c-cc28aa448c14-node-exporter.ceph-mon2
> >> /usr/bin/podman: stderr Error: no such container
> >> ceph-1aa23d18-6eb0-11f0-823c-cc28aa448c14-node-exporter.ceph-mon2
> >> Deploy daemon node-exporter.ceph-mon2 ...
> >> Verifying port 0.0.0.0:9100 ...
> >> Cannot bind to IP 0.0.0.0 port 9100: [Errno 98] Address already in use
> >> ERROR: TCP Port(s) '0.0.0.0:9100' required for node-exporter already in
> >> use
> >>
> >> Please let me know how to fix it.
> >>
> >>
> >> Thanks,
> >> Gagan
> >>
> >>
> >>
> >>
> >>
> >>
> > _______________________________________________
> > ceph-users mailing list -- ceph-users@xxxxxxx
> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
>
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
