Hi, I did a fio bench and believe that it destroy one of my OSD, this is the command used : fio --ioengine=libaio --filename=/dev/sda --direct=1 --sync=1 --rw=write --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --name=fio fio --ioengine=libaio --filename=/dev/sda --direct=1 --sync=1 --rw=write --bs=4M --numjobs=1 --iodepth=1 --runtime=60 --time_based --name=fio Log file after the command : 2025-07-23T08:41:44.448+0000 734bf7f6e680 1 bdev(0x59ba86d4ee00 /var/lib/ceph/osd/ceph-2/block) close 2025-07-23T08:41:44.719+0000 734bf7f6e680 1 bdev(0x59ba86d4ee00 /var/lib/ceph/osd/ceph-2/block) open path /var/lib/ceph/osd/ceph-2/block 2025-07-23T08:41:44.719+0000 734bf7f6e680 0 bdev(0x59ba86d4ee00 /var/lib/ceph/osd/ceph-2/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-2/block failed: (22) Invalid argument 2025-07-23T08:41:44.720+0000 734bf7f6e680 1 bdev(0x59ba86d4ee00 /var/lib/ceph/osd/ceph-2/block) open size 1000203091968 (0xe8e0c00000, 932 GiB) block_size 4096 (4 KiB) rotational device, discard supported 2025-07-23T08:41:44.723+0000 734bf7f6e680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-2/block at offset 66: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3] 2025-07-23T08:41:44.724+0000 734bf7f6e680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-2/block at offset 4096: End of buffer [buffer:2] 2025-07-23T08:41:44.724+0000 734bf7f6e680 -1 bluestore(/var/lib/ceph/osd/ceph-2) _check_main_bdev_label not all labels read properly 2025-07-23T08:41:44.724+0000 734bf7f6e680 1 bdev(0x59ba86d4ee00 /var/lib/ceph/osd/ceph-2/block) close 2025-07-23T08:41:44.983+0000 734bf7f6e680 -1 osd.2 0 OSD:init: unable to mount object store 2025-07-23T08:41:44.983+0000 734bf7f6e680 -1 ** ERROR: osd init failed: (5) Input/output error After checking I saw that the block soft link might be wrong On a healthy osd : # ll /var/lib/ceph/cluster-id/osd.5/ total 72 drwx------ 2 167 167 4096 Jul 22 10:36 ./ drwx------ 12 167 167 4096 Jul 22 10:30 ../ lrwxrwxrwx 1 167 167 93 Jul 15 14:39 block -> /dev/ceph-c31f0e16-0460-4bc5-9470-468270b4c68a/osd-block-72aa3074-e2f9-45f8-a468-03c02d36f1de On my breaking osd : /var/lib/ceph/cluster-id/osd.2# ll total 72 drwx------ 2 167 167 4096 Jul 23 10:41 ./ drwx------ 11 167 167 4096 Jul 22 10:35 ../ lrwxrwxrwx 1 167 167 111 Jul 23 10:41 block -> /dev/mapper/ceph--bd0ca671--2b89--4530--be2b--f41443822a91-osd--block--342578c3--4603--4023--a564--8fca6dcc1040 Is there a correct way of changing it or I'm going in the wrong direction ? Thanks Vivien _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |