Peter Zijlstra wrote:
> On Wed, Jul 09, 2025 at 10:32:16PM -0700, dan.j.williams@xxxxxxxxx wrote:
>
> > Theoretically there could be a threshold at which a CLFLUSHOPT loop is a
> > better option, but I would rather it be the case* that software CXL
> > cache management is stop-gap for early generation CXL platforms.
>
> So isn't the problem that CLFLUSH and friends take a linear address
> rather than a physical address? I suppose we can use our 1:1 mapping in
> this case, is all of CXL in the 1:1 map?
Currently CXL on the unplug path does:
arch_remove_memory() /* drop direct map */
cxl_region_invalidate_memregion() /* wbinvd_on_all_cpus() */
cxl_region_decode_reset() /* physically unmap memory */
...and on the plug path:
cxl_region_decode_commit() /* physically map memory */.
cxl_region_invalidate_memregion() /* wbinvd_on_all_cpus() */
arch_add_memory() /* setup direct map */
Moving this to virtual address based flushing would need some callbacks
from the memory_hotplug code to run flushes for memory spaces that are
being physically reconfigured.
...unplug:
arch_remove_memory()
clwb_on_all_cpus_before_unmap()
cxl_region_decode_reset()
...plug:
cxl_region_decode_commit()
arch_add_memory()
clflushopt_on_all_cpus_before_use()
However, this raises a question in my mind. Should not all memory
hotplug drivers in the kernel be doing cache management when the
physical contents of a memory range may have changed behind a CPUs back?
Unless I am missing something it looks like the ACPI memory hotplug
driver, for example, has never considered that an unplug/replug event
may leave stale data in the CPU cache.
I note drm_clflush_pages() is existing infrastructure and perhaps CXL
should uplevel/unify on that common helper?
