Hi Babu, On 5/5/25 12:54 PM, Moger, Babu wrote: > On 5/5/25 11:22, Reinette Chatre wrote: >> On 5/2/25 5:53 PM, Moger, Babu wrote: >>> On 5/2/2025 4:20 PM, Reinette Chatre wrote: >>>> On 4/21/25 3:43 PM, Babu Moger wrote: ... >>>> >>>> Then "bit_usage" will look like: >>>> >>>> # cat /sys/fs/resctrl/info/L3/bit_usage >>>> 0=HHHHXXXXSSSS0000 >>> >>> It is confusing here. To make it clear we may have to print all the CLOSes in each domain. >> >> Could you please elaborate how this is confusing? > > # cat /sys/fs/resctrl/info/L3/bit_usage > 0=HHHHXXXXSSSS0000 > > This may give the impression that the all CLOSes in all domains carries > this property, but in reality, it applies only to one CLOS(15) within each > domain. > > Example below.... > ... > Here is the behaviour after these cahnges. > > === Before io_alloc enabled============================== > > #cd /sys/fs/resctrl/L3/ > # cat io_alloc > disabled > > # cat shareable_bits > 0 (This is always 0 for AMD) > > # cat bit_usage > 0=SSSSSSSSSSSSSSSS;1=SSSSSSSSSSSSSSSS;2=SSSSSSSSSSSSSSSS;3=SSSSSSSSSSSSSSSS Please note that the "S" in above does not have anything to do with "shareable_bits" at this point. The "S" indicates that all L3 instances are currently used by software and that sharing is allowed. "bit_usage" gives insight to user space how all L3 instances are used. If at this point a new resource group is created and it has an "exclusive" allocation then "bit_usage" will change to reflect that. For example, you can try this on the system you are testing on: # echo 'L3:0=fff0;1=fff0;2=fff0;3=fff0' > /sys/fs/resctrl/schemata # mkdir /sys/fs/resctrl/g1 # echo 'L3:0=f;1=f;2=f;3=f' > /sys/fs/resctrl/g1/schemata # echo 'exclusive' > /sys/fs/resctrl/g1/mode The above isolates a portion of all L3 instances for exclusive use by g1. After above changes: # cat /sys/fs/resctrl/info/L3/bit_usage 0=SSSSSSSSSSSSEEEE;1=SSSSSSSSSSSSEEEE;2=SSSSSSSSSSSSEEEE;3=SSSSSSSSSSSSEEEE Note that there is no "closid" or resource group information but instead, "bit_usage" shows to user space how each cache instance is being used across all resource groups and hardware (IO) allocations. > > ==== After io_alloc enabled================================= > > # echo 1 > io_alloc > > # cat io_alloc > enabled > > # cat io_alloc_cbm > L3:0=ffff;1=ffff;2=ffff;3=ffff > > #cat bit_usage > 0=XXXXXXXXXXXXXXXX;1=XXXXXXXXXXXXXXXX;2=XXXXXXXXXXXXXXXX;3=XXXXXXXXXXXXXXXX Looks accurate to me. It shows that both hardware and software can allocate into all portions of all caches. > > ==== After changing io_alloc_cbm ============================ > > #echo "L3:0=ff00;1=ff00;2=ff00;3=ff00 > io_alloc_cbm > > # cat io_alloc_cbm > L3:0=ff00;1=ff00;2=ff00;3=ff00 > > #cat bit_usage > 0=XXXXXXXXSSSSSSSS;1=XXXXXXXXSSSSSSSS;2=XXXXXXXXSSSSSSSS;3=XXXXXXXXSSSSSSSS Looks accurate to me. > ============================================================= > > My concern here is, this may imply that the property is present across all > CLOSes in all the domains, while in fact, it only applies to a single > CLOS(15) within each domain. If a user wants a resource group specific view then the schemata should be used. "bit_usage" presents the view from the cache instance perspective and reflects how each L3 cache instance is being used at that moment in time. It helps system administrator answer the question "how are the caches used at the moment"? "bit_usage" does so by presenting a summary of all allocations across all resource groups and any hardware allocations that may exist. This file helps user space to understand how the cache is being used without needing to correlate the CBMs of all resource groups and IO allocations. For example, "bit_usage" is to be used by system administrator to ensure cache is used optimally (for example, there are no unused portions). Also, a user may be investigating a performance issue in a particular resource group and "bit_usage" will help with that to see if the tasks in that resource group may be competing with IO. Reinette
