Hi Babu/Reinette, On Wed, May 21, 2025 at 1:44 AM Reinette Chatre <reinette.chatre@xxxxxxxxx> wrote: > > Hi Babu, > > On 5/20/25 4:25 PM, Moger, Babu wrote: > > Hi Reinette, > > > > On 5/20/2025 1:23 PM, Reinette Chatre wrote: > >> Hi Babu, > >> > >> On 5/20/25 10:51 AM, Moger, Babu wrote: > >>> Hi Reinette, > >>> > >>> On 5/20/25 11:06, Reinette Chatre wrote: > >>>> Hi Babu, > >>>> > >>>> On 5/20/25 8:28 AM, Moger, Babu wrote: > >>>>> On 5/19/25 10:59, Peter Newman wrote: > >>>>>> On Fri, May 16, 2025 at 12:52 AM Babu Moger <babu.moger@xxxxxxx> wrote: > >>>> > >>>> ... > >>>> > >>>>>>> /sys/fs/resctrl/info/L3_MON/num_mbm_cntrs: Reports the number of monitoring > >>>>>>> counters available for assignment. > >>>>>> > >>>>>> Earlier I discussed with Reinette[1] what num_mbm_cntrs should > >>>>>> represent in a "soft-ABMC" implementation where assignment is > >>>>>> implemented by assigning an RMID, which would result in all events > >>>>>> being assigned at once. > >>>>>> > >>>>>> My main concern is how many "counters" you can assign by assigning > >>>>>> RMIDs. I recall Reinette proposed reporting the number of groups which > >>>>>> can be assigned separately from counters which can be assigned. > >>>>> > >>>>> More context may be needed here. Currently, num_mbm_cntrs indicates the > >>>>> number of counters available per domain, which is 32. > >>>>> > >>>>> At the moment, we can assign 2 counters to each group, meaning each RMID > >>>>> can be associated with 2 hardware counters. In theory, it's possible to > >>>>> assign all 32 hardware counters to a group—allowing one RMID to be linked > >>>>> with up to 32 counters. However, we currently lack the interface to > >>>>> support that level of assignment. > >>>>> > >>>>> For now, the plan is to support basic assignment and expand functionality > >>>>> later once we have the necessary data structure and requirements. > >>>> > >>>> Looks like some requirements did not make it into this implementation. > >>>> Do you recall the discussion that resulted in you writing [2]? Looks like > >>>> there is a question to Peter in there on how to determine how many "counters" > >>>> are available in soft-ABMC. I interpreted [3] at that time to mean that this > >>>> information would be available in a future AMD publication. > >>> > >>> We already have a method to determine the number of counters in soft-ABMC > >>> mode, which Peter has addressed [4]. > >>> > >>> [4] > >>> https://lore.kernel.org/lkml/20250203132642.2746754-1-peternewman@xxxxxxxxxx/ > >>> > >>> This appears to be more of a workaround, and I doubt it will be included > >>> in any official AMD documentation. Additionally, the long-term direction > >>> is moving towards ABMC. > >>> > >>> I don’t believe this workaround needs to be part of the current series. It > >>> can be added later when soft-ABMC is implemented. > >> > >> Agreed. What about the plans described in [2]? (Thanks to Peter for > >> catching this!). > >> > >> It is important to keep track of requirements while working on a feature to > >> ensure that the implementation supports the planned use cases. Re-reading that > >> thread it is not clear to me how soft-ABMC's per-group assignment would look. > >> Could you please share how you see it progress from this implementation? > >> This includes the single event vs. multiple event assignment. I would like to > >> highlight that this is not a request for this to be supported in this implementation > >> but there needs to be a plan for how this can be supported on top of interfaces > >> established by this work. > >> > > > > Here’s my current understanding of soft-ABMC. Peter may have a more in-depth perspective on this. > > > > Soft-ABMC: > > a. num_mbm_cntrs: This is a software-defined limit based on the number of active RMIDs that can be supported. The value can be obtained using the code referenced in [4]. I would call it a hardware-defined limit that can be probed by software. The main question is whether this file returns the exact number of RMIDs hardware can track or double that number (mbm_total_bytes + mbm_local_bytes) so that the value is always measured in events. There's also the mongroup-RMID overcommit use case I described above[1]. On Intel we can safely assume that there are counters to back all RMIDs, so num_mbm_cntrs would be calculated directly from num_rmids. I realized this use case is more difficult to implement on MPAM, because a PARTID is effectively a CLOSID+RMID, so deferring assigning a unique PARTID to a group also results in it being in a different allocation group. It will work if the unmonitored groups could find a way to share PARTIDs, but this has consequences on allocation - but hopefully no worse than sharing CLOSIDs on x86. There's a lot of interest in monitoring ID overcommit in Google, so I think it's worth it for me to investigate the additional structural changes needed in resctrl (i.e., breaking the FS-level association between mongroups and HW monitoring IDs). Such a framework could be a better fit for soft-ABMC. For example, if overcommit is allowed, we would just report the number of simultaneous RMIDs we were able to probe as num_rmids. I would want the same shared assignment scheduler to be able to work with RMIDs and counters, though. Thanks, -Peter [1] https://lore.kernel.org/lkml/CALPaoChSzzU5mzMZsdT6CeyEn0WD1qdT9fKCoNW_ty4tojtrkw@xxxxxxxxxxxxxx/
