This patchset adds granular dirty and writeback stats accounting for large
folios.
The dirty page balancing logic uses these stats to determine things like
whether the ratelimit has been exceeded, the frequency with which pages need
to be written back, if dirtying should be throttled, etc. Currently for large
folios, if any byte in the folio is dirtied or written back, all the bytes in
the folio are accounted as such.
In particular, there are four places where dirty and writeback stats get
incremented and decremented as pages get dirtied and written back:
a) folio dirtying (filemap_dirty_folio() -> ... -> folio_account_dirtied())
- increments NR_FILE_DIRTY, NR_ZONE_WRITE_PENDING, WB_RECLAIMABLE,
current->nr_dirtied
b) writing back a mapping (writeback_iter() -> ... ->
folio_clear_dirty_for_io())
- decrements NR_FILE_DIRTY, NR_ZONE_WRITE_PENDING, WB_RECLAIMABLE
c) starting writeback on a folio (folio_start_writeback())
- increments WB_WRITEBACK, NR_WRITEBACK, NR_ZONE_WRITE_PENDING
d) ending writeback on a folio (folio_end_writeback())
- decrements WB_WRITEBACK, NR_WRITEBACK, NR_ZONE_WRITE_PENDING
Patches 1 to 9 adds support for the 4 cases above to take in the number of
pages to be accounted, instead of accounting for the entire folio.
Patch 12 adds the iomap changes that uses these new APIs. This relies on the
iomap folio state bitmap to track which pages are dirty (so that we avoid
any double-counting). As such we can only do granular accounting if the
block size >= PAGE_SIZE.
This patchset was run through xfstests using fuse passthrough hp (with an
out-of-tree kernel patch enabling fuse large folios).
This is on top of commit 4f702205 ("Merge branch 'vfs-6.18.rust' into
vfs.all") in Christian's vfs tree, and on top of the patchset that removes
BDI_CAP_WRITEBACK_ACCT [1].
Local benchmarks were run on xfs by doing the following:
seting up xfs (per the xfstests readme):
# xfs_io -f -c "falloc 0 10g" test.img
# xfs_io -f -c "falloc 0 10g" scratch.img
# mkfs.xfs test.img
# losetup /dev/loop0 ./test.img
# losetup /dev/loop1 ./scratch.img
# mkdir -p /mnt/test && mount /dev/loop0 /mnt/test
# sudo sysctl -w vm.dirty_bytes=$((3276 * 1024 * 1024)) # roughly 20% of 16GB
# sudo sysctl -w vm.dirty_background_bytes=$((1638*1024*1024)) # roughly 10% of 16GB
running this test program (ai-generated) [2] which essentially writes out 2 GB
of data 256 MB at a time and then spins up 15 threads to do 50-byte 50k
writes.
On my VM, I saw the writes take around 3 seconds (with some variability of
taking 0.3 seconds to 5 seconds sometimes) in the base version vs taking
a pretty consistent 0.14 seconds with this patchset. It'd be much appreciated
if someone could also run it on their local system to verify they see similar
numbers.
Thanks,
Joanne
[1] https://lore.kernel.org/linux-fsdevel/20250707234606.2300149-1-joannelkoong@xxxxxxxxx/
[2] https://pastebin.com/CbcwTXjq
Changelog
v1: https://lore.kernel.org/linux-fsdevel/20250801002131.255068-1-joannelkoong@xxxxxxxxx/
v1 -> v2:
* Add documentation specifying caller expectations for the
filemap_dirty_folio_pages() -> __folio_mark_dirty() callpath (Jan)
* Add requested iomap bitmap iteration refactoring (Christoph)
* Fix long lines (Christoph)
Joanne Koong (12):
mm: pass number of pages to __folio_start_writeback()
mm: pass number of pages to __folio_end_writeback()
mm: add folio_end_writeback_pages() helper
mm: pass number of pages dirtied to __folio_mark_dirty()
mm: add filemap_dirty_folio_pages() helper
mm: add __folio_clear_dirty_for_io() helper
mm: add no_stats_accounting bitfield to wbc
mm: refactor clearing dirty stats into helper function
mm: add clear_dirty_for_io_stats() helper
iomap: refactor dirty bitmap iteration
iomap: refactor uptodate bitmap iteration
iomap: add granular dirty and writeback accounting
fs/btrfs/subpage.c | 2 +-
fs/buffer.c | 6 +-
fs/ext4/page-io.c | 2 +-
fs/iomap/buffered-io.c | 281 ++++++++++++++++++++++++++++++-------
include/linux/page-flags.h | 4 +-
include/linux/pagemap.h | 4 +-
include/linux/writeback.h | 10 ++
mm/filemap.c | 12 +-
mm/internal.h | 2 +-
mm/page-writeback.c | 115 +++++++++++----
10 files changed, 346 insertions(+), 92 deletions(-)
--
2.47.3
