Hi Mikael, This might be a long shot, but I have to ask: have you checked the average file size on the current CephFS filesystem? Apart from extreme cost efficiency or a bad design, the EC choice on HDD could have had a legitimate reason in the past. It's probably not enough to make it the default data pool, but it might help explain the current design. I don't know the exact amount of data or how many users this filesystem has, but the safest long-term approach in your situation is probably to create a new filesystem and then migrate the data over, as you imagined. Best regards, Frédéric. -- Frédéric Nass Ceph Ambassador France | Senior Ceph Engineer @ CLYSO Try our Ceph Analyzer -- https://analyzer.clyso.com/ https://clyso.com | frederic.nass@xxxxxxxxx Le mer. 10 sept. 2025 à 16:09, Mikael Öhman <micketeer@xxxxxxxxx> a écrit : > The recommendations for cephfs is to make a replicated default data pool, > and adding any EC data pools using layouts: > https://docs.ceph.com/en/latest/cephfs/createfs/ > > > If erasure-coded pools are planned for file system data, it is best to > configure the default as a replicated pool to improve small-object write > and read performance when updating backtraces. > > I have an cephfs that unfortunately wasn't set up like this: they just made > an EC pool on the slow HDDs as the default, which sounds like the worst > case scenario to me. I would like to add a NVME data pool to this ceph fs, > but recommended gives me pause on if i should instead go through the hassle > of creating a new cephfs and migrating all users. > > I've tried to run some mdtest with small 1k files to see if i could measure > this difference, but speed is about the same in my relatively small tests > so far. I'm also not sure what impact I should realistically expect here. I > don't even know if creating files counts as "updating backtraces", so my > testing might just be pointless. > > I guess my core question is; just how important is this suggestion to keep > the default data pool on replicated NVME? > > Setup: > 14 hosts x 42 HDD + 3 NVMEs for db/wal 2*2x25 GbitE bonds > 12 hosts x 10 NVME. 2*2x100 GbitE bonds > > Old CephFS setup: > - metadata: replicated NVME > - data-pools: EC 10+2 on HDD (i plan to add a EC NVME pool here via > layouts) > > New CephFS setup as recommended: > - metadata: replicated NVME > - data-pools: replicated NVME (default), EC 8+2 on HDD via layout, EC 8+2 > on NVME via layout. > > Ceph 18.2.7 > > > Best regards, Mikael > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |