We are currently in the bizarre situation where we are constrained on the number of flags we can set in an mm_struct based on whether this is a 32-bit or 64-bit kernel. This is because mm->flags is an unsigned long field, which is 32-bits on a 32-bit system and 64-bits on a 64-bit system. In order to keep things functional across both architectures, we do not permit mm flag bits to be set above flag 31 (i.e. the 32nd bit). This is a silly situation, especially given how profligate we are in storing metadata in mm_struct, so let's convert mm->flags into a bitmap and allow ourselves as many bits as we like. In order to execute this change, we introduce a new opaque type - mm_flags_t - which wraps a bitmap. We go further and mark the bitmap field __private, which forces users to have to use accessors, which allows us to enforce atomicity rules around mm->flags (except on those occasions they are not required - fork, etc.) and makes it far easier to keep track of how mm flags are being utilised. In order to implement this change sensibly and an an iterative way, we start by introducing the type with the same bitsize as the current mm flags (system word size) and place it in union with mm->flags. We are then able to gradually update users as we go without being forced to do everything in a single patch. In the course of working on this series I noticed the MMF_* flag masks encounter a sign extension bug that, due to the 32-bit limit on mm->flags thus far, has not caused any issues in practice, but required fixing for this series. We must make special dispensation for two cases - coredump and initailisation on fork, but of which use masks extensively. Since coredump flags are set in stone, we can safely assume they will remain in the first 32-bits of the flags. We therefore provide special non-atomic accessors for this case that access the first system word of flags, keeping everything there essentially the same. For mm->flags initialisation on fork, we adjust the logic to ensure all bits are cleared correctly, and then adjust the existing intialisation logic, dubbing the implementation utilising flags as legacy. This means we get the same fast operations as we do now, but in future we can also choose to update the forking logic to additionally propagate flags beyond 32-bits across fork. With this change in place we can, in future, decide to have as many bits as we please. Since the size of the bitmap will scale in system word multiples, there should be no issues with changes in alignment in mm_struct. Additionally, the really sensitive field (mmap_lock) is located prior to the flags field so this should have no impact on that either. Lorenzo Stoakes (10): mm: add bitmap mm->flags field mm: convert core mm to mm_flags_*() accessors mm: convert prctl to mm_flags_*() accessors mm: convert arch-specific code to mm_flags_*() accessors mm: convert uprobes to mm_flags_*() accessors mm: update coredump logic to correctly use bitmap mm flags mm: correct sign-extension issue in MMF_* flag masks mm: update fork mm->flags initialisation to use bitmap mm: convert remaining users to mm_flags_*() accessors mm: replace mm->flags with bitmap entirely and set to 64 bits arch/s390/mm/mmap.c | 4 +- arch/sparc/kernel/sys_sparc_64.c | 4 +- arch/x86/mm/mmap.c | 4 +- fs/coredump.c | 4 +- fs/exec.c | 2 +- fs/pidfs.c | 7 +++- fs/proc/array.c | 2 +- fs/proc/base.c | 12 +++--- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 2 +- include/linux/khugepaged.h | 6 ++- include/linux/ksm.h | 6 +-- include/linux/mm.h | 34 +++++++++++++++- include/linux/mm_types.h | 67 +++++++++++++++++++++++++------- include/linux/mman.h | 2 +- include/linux/oom.h | 2 +- include/linux/sched/coredump.h | 21 +++++++++- kernel/events/uprobes.c | 32 +++++++-------- kernel/fork.c | 9 +++-- kernel/sys.c | 16 ++++---- mm/debug.c | 4 +- mm/gup.c | 10 ++--- mm/huge_memory.c | 8 ++-- mm/khugepaged.c | 10 ++--- mm/ksm.c | 32 +++++++-------- mm/mmap.c | 8 ++-- mm/oom_kill.c | 26 ++++++------- mm/util.c | 6 +-- tools/testing/vma/vma_internal.h | 19 ++++++++- 29 files changed, 239 insertions(+), 122 deletions(-) -- 2.50.1
