##### Dirty Bits syncronization IO fast path will set bits to dirty, and those dirty bits will be cleared by daemon after IO is done. llbitmap_page_ctl is used to synchronize between IO path and daemon; IO path: 1) try to grab a reference, if succeed, set expire time after 5s and return; 2) if failed to grab a reference, wait for daemon to finish clearing dirty bits; Daemon(Daemon will be waken up every daemon_sleep seconds): For each page: 1) check if page expired, if not skip this page; for expired page: 2) suspend the page and wait for inflight write IO to be done; 3) change dirty page to clean; 4) resume the page; Performance Test: Simple fio randwrite test to build array with 20GB ramdisk in my VM: | | none | bitmap | llbitmap | | -------------------- | --------- | --------- | --------- | | raid1 | 13.7MiB/s | 9696KiB/s | 19.5MiB/s | | raid1(assume clean) | 19.5MiB/s | 11.9MiB/s | 19.5MiB/s | | raid10 | 21.9MiB/s | 11.6MiB/s | 27.8MiB/s | | raid10(assume clean) | 27.8MiB/s | 15.4MiB/s | 27.8MiB/s | | raid5 | 14.0MiB/s | 11.6MiB/s | 12.9MiB/s | | raid5(assume clean) | 17.8MiB/s | 13.4MiB/s | 13.9MiB/s | For raid1/raid10 llbitmap can be better than none bitmap with background initial resync, and it's the same as none bitmap without it. Noted that llbitmap performance improvement for raid5 is not obvious, this is due to raid5 has many other performance bottleneck, perf results still shows that bitmap overhead will be much less. following branch for review or test: https://git.kernel.org/pub/scm/linux/kernel/git/yukuai/linux.git/log/?h=yukuai/md-llbitmap Yu Kuai (23): md: add a new parameter 'offset' to md_super_write() md: factor out a helper raid_is_456() md/md-bitmap: cleanup bitmap_ops->startwrite() md/md-bitmap: support discard for bitmap ops md/md-bitmap: remove parameter slot from bitmap_create() md/md-bitmap: add a new sysfs api bitmap_type md/md-bitmap: delay registration of bitmap_ops until creating bitmap md/md-bitmap: add a new method skip_sync_blocks() in bitmap_operations md/md-bitmap: add a new method blocks_synced() in bitmap_operations md: add a new recovery_flag MD_RECOVERY_LAZY_RECOVER md/md-bitmap: make method bitmap_ops->daemon_work optional md/md-bitmap: add macros for lockless bitmap md/md-bitmap: fix dm-raid max_write_behind setting md/dm-raid: remove max_write_behind setting limit md/md-llbitmap: implement llbitmap IO md/md-llbitmap: implement bit state machine md/md-llbitmap: implement APIs for page level dirty bits synchronization md/md-llbitmap: implement APIs to mange bitmap lifetime md/md-llbitmap: implement APIs to dirty bits and clear bits md/md-llbitmap: implement APIs for sync_thread md/md-llbitmap: implement all bitmap operations md/md-llbitmap: implement sysfs APIs md/md-llbitmap: add Kconfig Documentation/admin-guide/md.rst | 80 +- drivers/md/Kconfig | 11 + drivers/md/Makefile | 2 +- drivers/md/dm-raid.c | 6 +- drivers/md/md-bitmap.c | 50 +- drivers/md/md-bitmap.h | 55 +- drivers/md/md-llbitmap.c | 1556 ++++++++++++++++++++++++++++++ drivers/md/md.c | 247 +++-- drivers/md/md.h | 20 +- drivers/md/raid5.c | 6 + 10 files changed, 1901 insertions(+), 132 deletions(-) create mode 100644 drivers/md/md-llbitmap.c -- 2.39.2
