On Wed, Jun 11, 2025 at 03:04:52PM +0530, Ojaswin Mujoo wrote:
> From: "Ritesh Harjani (IBM)" <ritesh.list@xxxxxxxxx>
>
> We brute force all possible blocksize & clustersize combinations on
> a bigalloc filesystem for stressing atomic write using fio data crc
> verifier. We run nproc * $LOAD_FACTOR threads in parallel writing to
> a single $SCRATCH_MNT/test-file. With atomic writes this test ensures
> that we never see the mix of data contents from different threads on
> a given bsrange.
>
> Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@xxxxxxxxx>
> Signed-off-by: Ojaswin Mujoo <ojaswin@xxxxxxxxxxxxx>
> ---
> tests/ext4/061 | 107 +++++++++++++++++++++++++++++++++++++++++++++
> tests/ext4/061.out | 2 +
> 2 files changed, 109 insertions(+)
> create mode 100755 tests/ext4/061
> create mode 100644 tests/ext4/061.out
>
> diff --git a/tests/ext4/061 b/tests/ext4/061
> new file mode 100755
> index 00000000..9d656613
> --- /dev/null
> +++ b/tests/ext4/061
> @@ -0,0 +1,107 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2025 IBM Corporation. All Rights Reserved.
> +#
> +# FS QA Test 061
> +#
> +# Brute force all possible blocksize clustersize combination on a bigalloc
> +# filesystem for stressing atomic write using fio data crc verifier. We run
> +# nproc * 2 * $LOAD_FACTOR threads in parallel writing to a single
> +# $SCRATCH_MNT/test-file. With fio aio-dio atomic write this test ensures that
> +# we should never see the mix of data contents from different threads for any
> +# given fio blocksize.
> +#
> +
> +. ./common/preamble
> +. ./common/atomicwrites
> +
> +_begin_fstest auto rw stress atomicwrites
> +
> +_require_scratch_write_atomic
> +
> +function max()
> +{
> + if (( $1 > $2 )); then
> + echo "$1"
> + else
> + echo "$2"
> + fi
> +}
> +
> +function min()
> +{
> + if (( $1 < $2 )); then
> + echo "$1"
> + else
> + echo "$2"
> + fi
> +}
I've seen these two functions many times, please make them to be common helpers.
> +
> +FS_MAX_CLUSTER_SIZE=$((128*1024))
> +FIO_LOAD=$(($(nproc) * 2 * LOAD_FACTOR))
> +SIZE=$((100*1024*1024))
> +fiobsize=4096
> +
> +# Calculate fsblocksize and FS_MAX_CLUSTER_SIZE as per bdev atomic write units.
> +bdev_awu_min=$(_get_atomic_write_unit_min $SCRATCH_DEV)
> +bdev_awu_max=$(_get_atomic_write_unit_max $SCRATCH_DEV)
> +fsblocksize=$(max 4096 "$bdev_awu_min")
> +FS_MAX_CLUSTER_SIZE=$(min "$FS_MAX_CLUSTER_SIZE" "$bdev_awu_max")
> +
> +function create_fio_config()
> +{
> +cat >$fio_config <<EOF
> + [aio-dio-aw-verify]
> + direct=1
> + ioengine=libaio
_require_aiodio
> + rw=randwrite
> + bs=$fiobsize
> + fallocate=native
> + filename=$SCRATCH_MNT/test-file
> + size=$SIZE
> + iodepth=$FIO_LOAD
> + numjobs=$FIO_LOAD
> + group_reporting=1
> + verify_state_save=0
> + verify=crc32c
> + verify_fatal=1
> + verify_dump=0
> + verify_backlog=1024
> + verify_async=4
> + verify_write_sequence=0
> + atomic=1
> +EOF
> +}
> +
> +# Let's create a sample fio config to check whether fio supports all options.
> +fio_config=$tmp.fio
> +create_fio_config
> +_require_fio $fio_config
> +
> +for ((fsblocksize=$fsblocksize; fsblocksize <= $(_get_page_size); fsblocksize = $fsblocksize << 1)); do
> + for ((fsclustersize=$fsblocksize; fsclustersize <= $FS_MAX_CLUSTER_SIZE; fsclustersize = $fsclustersize << 1)); do
> + for ((fiobsize = $fsblocksize; fiobsize <= $fsclustersize; fiobsize = $fiobsize << 1)); do
Wow, 3 for loops...
> + MKFS_OPTIONS="-O bigalloc -b $fsblocksize -C $fsclustersize"
> + _scratch_mkfs_ext4 "$MKFS_OPTIONS" >> $seqres.full 2>&1 || continue
MKFS_OPTIONS is used in _scratch_mkfs_ext4 by default, you don't need to use it as
an argument.
Or do you want to do:
_scratch_mkfs_ext4 "-O bigalloc -b $fsblocksize -C $fsclustersize" ?
> + if _try_scratch_mount >> $seqres.full 2>&1; then
> + touch $SCRATCH_MNT/f1
> + echo "== FIO test for fsblocksize=$fsblocksize fsclustersize=$fsclustersize fiobsize=$fiobsize ==" >> $seqres.full
> + fio_config=$tmp.fio
Do you change the "$tmp"? If not, you don't need to set fio_config=$tmp.fio
everytime, you've set fio_config above this for loop.
> + fio_out=$tmp.fio.out
If you don't change "$tmp", you can set fio_out once, before the
for loop running.
> + create_fio_config
> + _require_fio $fio_config
I think only $fiobsize will be changed in $fio_config at here, so you're trying
to check "bs=$fiobsize"? If so, that doesn't make sense, due to _require_fio accepts
any "bs" number, except bs <= 0. So you don't need to call _require_fio
at here everytime, especially you've called it before the loop running.
Thanks,
Zorro
> + cat $fio_config >> $seqres.full
> + $FIO_PROG $fio_config --output=$fio_out
> + ret=$?
> + cat $fio_out >> $seqres.full
> + _scratch_unmount
> + [[ $ret -eq 0 ]] || break;
> + fi
> + done
> + done
> +done
> +
> +# success, all done
> +echo Silence is golden
> +status=0
> +exit
> diff --git a/tests/ext4/061.out b/tests/ext4/061.out
> new file mode 100644
> index 00000000..273be9e0
> --- /dev/null
> +++ b/tests/ext4/061.out
> @@ -0,0 +1,2 @@
> +QA output created by 061
> +Silence is golden
> --
> 2.49.0
>
>
