On Wed, Aug 06, 2025 at 04:26:49PM +0800, Mingyu He wrote:
> Hi EXT4 maintainers,
>
> I would like to report a potential bug related to the ext4 allocator
> implementation, specifically in the file `fs/ext4/mballoc.c`.
Yeah, this is a known issue with ext4's RAID support. The problem is
that we're trying too hard to try to find a precise RAID stripe
alignment. There are a couple of things that could be done to solve
the issue, but none of them are easy.
* Cache the number of stripe aligned regions in a particular block
group, so we can skip the block groups where searching is for a
stripe alignment is hopeless. This will reduce the CPU time spent
searching all of the block groups for each alignment, but on a
freshly mounted disk, initial allocations will still be slow since
we would need to read the block allocation bitmaps into memory and
then process them. We would also need to keep the cache of the
number of stripe aligned regions to a minimum.
* Have a hard limit on the amount of time (either wall clock time or
CPU time) spent searching for stripe aligned bitmaps. If none are
available, bail out early.
* Use a more efficient in-memory data structure for storing the free
block information. Today, we use a buddy bitmap, which is great if
we are doing power of two allocations (which for non-RAID file
systems, we try to do whenever possible, up to trying to allocate
more space than what was asked for in case the user tries to append
to the file later). If the RAID stripe size is power-of-two
aligned, the buddy bitmap would be fine, but very often, that isn't
the case. This still requires initially reading the block bitmap
into memory in order to convert to that more efficient in-memory
data structure, but it is simpler than...
* Use a more efficient on-disk data structure, such as a b-tree. This
requires an on-disk format change, which means we would need to
update e2fsprogs, and we would have to worry about backwards
compatibility in case the file system needs to be mounted on an
older kernel.
If someone is interested in working on these options (which I view as
a new feature, not as a bug fix), please contact me and I'm happy to
discuss further.
Alternatively, a workaround is to simply disable the RAID stripe
information in the superblock. You can do this via "tune2fs -E
stripe_width 0 /dev/sdXX". For a file system which is fragmented such
that finding stripe aligned free space is hopeless, this isn't going
to hurt, and it will definitely help. In the most recent version of
e2fsprogs, this is now the default in mke2fs for non-rotational (e.g.,
thin provisioned, or flash based) storage devices:
commit b61f182b2de1ea75cff935037883ba1a8c7db623
Author: Theodore Ts'o <tytso@xxxxxxx>
Date: Sun May 4 14:07:14 2025 -0400
mke2fs: don't set the raid stripe for non-rotational devices by default
The ext4 block allocator is not at all efficient when it is asked to
enforce RAID alignment. It is especially bad for flash-based devices,
or when the file system is highly fragmented. For non-rotational
devices, it's fine to set the stride parameter (which controls
spreading the allocation bitmaps across the RAID component devices,
which always makessense); but for the stripe parameter (which asks the
ext4 block alocator to try _very_ hard to find RAID stripe aligned
devices) it's probably not a good idea.
Add new mke2fs.conf parameters with the defaults:
[defaults]
set_raid_stride = always
set_raid_stripe = disk
Even for RAID arrays based on HDD's, we can still have problems for
highly fragmented file systems. This will need to solved in the
kernel, probably by having some kind of wall clock or CPU time
limitation for each block allocation or adding some kind of
optimization which is faster than using our current buddy bitmap
implementation, especially if the stripe size is not multiple of a
power of two. But for SSD's, it's much less likely to make sense even
if we have an optimized block allocator, because if you've paid $$$
for a flash-based RAID array, the cost/benefit tradeoffs of doing less
optimized stripe RMW cycles versus the block allocator time and CPU
overhead is harder to justify without a lot of optimization effort.
If and when we can improve the ext4 kernel implementation (and it gets
rolled out to users using LTS kernels), we can change the defaults.
And of course, system administrators can always change
/etc/mke2fs.conf settings.
Signed-off-by: Theodore Ts'o <tytso@xxxxxxx>
Cheers,
- Ted
