On July 10, 2025 12:11:13 PM PDT, dan.j.williams@xxxxxxxxx wrote: >H. Peter Anvin wrote: >[..] >> >> > In the near term though, current CXL platforms that do not support >> >> > device-initiated-invalidate still need coarse cache management for that >> >> > original infrequent provisioning events. Folks that want to go further >> >> > and attempt frequent DCD events with WBINVD get to keep all the pieces. >> >> >> >> I would strongly prefer those pieces to include WARNs and or worse. >> > >> >That is fair. It is not productive for the CXL subsystem to sit back and >> >hope that people notice the destructive side-effects of wbinvd and hope >> >that leads to device changes. >> > >> >This discussion has me reconsidering that yes, it would indeed be better >> >to clflushopt loop over potentially terabytes on all CPUs. That should >> >only be suffered rarely for the provisioning case, and for the DCD case >> >the potential add/remove events should be more manageable. >> > >> >drm already has drm_clflush_pages() for bulk cache management, CXL >> >should just align on that approach. >> >> Let's not be flippant; looping over terabytes could take *hours*. But those are hours during which the system is alive, and only one CPU needs to be looping. > >Do not all CPUs need to perform the invalidation for L1 copies of the >line? > >Not trying to be flippant, but if wbinvd is only a one-shot per Peter's >proposed policy and the system experiences another CXL reconfiguration >event, then looping is the only option or fail the memory plug event. > >> The other question is: what happens if memory is unplugged and then a >> cache line evicted? I'm guessing that existing memory hotplug >> solutions simply drop the writeback, since the OS knows there is no >> valid memory there, and so any cached data is inherently worthless. > >Right, the expectation is that unplug is always coordinated and that >surprise unplug is unsupported / might lead to system instability. > > CLFLUSH goes through the cache coherency protocol and is therefore system wide, which WBINVD is not.
