Hi Tony, Reinette, On Thu, May 22, 2025 at 2:21 AM Luck, Tony <tony.luck@xxxxxxxxx> wrote: > > > >>> There's also the mongroup-RMID overcommit use case I described > > >>> above[1]. On Intel we can safely assume that there are counters to > > >>> back all RMIDs, so num_mbm_cntrs would be calculated directly from > > >>> num_rmids. > > >> > > >> This is about the: > > >> There's now more interest in Google for allowing explicit control of > > >> where RMIDs are assigned on Intel platforms. Even though the number of > > >> RMIDs implemented by hardware tends to be roughly the number of > > >> containers they want to support, they often still need to create > > >> containers when all RMIDs have already been allocated, which is not > > >> currently allowed. Once the container has been created and starts > > >> running, it's no longer possible to move its threads into a monitoring > > >> group whenever RMIDs should become available again, so it's important > > >> for resctrl to maintain an accurate task list for a container even > > >> when RMIDs are not available. > > >> > > >> I see a monitor group as a collection of tasks that need to be monitored together. > > >> The "task list" is the group of tasks that share a monitoring ID that > > >> is required to be a valid ID since when any of the tasks are scheduled that ID is > > >> written to the hardware. I intentionally tried to not use RMID since I believe > > >> this is required for all archs. > > >> I thus do not understand how a task can start running when it does not have > > >> a valid monitoring ID. The idea of "deferred assignment" is not clear to me, > > >> there can never be "unmonitored tasks", no? I think I am missing something here. You are correct. I did forget to mention something... > > > > > > In the AMD/RMID implemenentation this might be achieved with something > > > extra in the task structure to denote whether a task is in a monitored > > > group or not. E.g. We add "task->rmid_valid" as well as "task->rmid". > > > Tasks in an unmonitored group retain their "task->rmid" (that's what > > > identifies them as a member of a group) but have task->rmid_valid set > > > to false. Context switch code would be updated to load "0" into the > > > IA32_PQR_ASSOC.RMID field for tasks without a valid RMID. So they > > > would still be monitored, but activity would be bundled with all > > > tasks in the default resctrl group. > > > > > > Presumably something analogous could be done for ARM/MPAM. > > > > > > > I do not interpret this as an unmonitored task but instead a task that > > belongs to the default resource group. Specifically, any data accumulated by > > such a task is attributed to the default resource group. Having tasks > > in a separate group but their monitoring data accumulating in/contributed to > > the default resource group (that has its own set of tasks) sounds wrong to me. > > Such an implementation makes any monitoring data of default resource group > > invalid, and by extension impossible to use default resource group to manage > > an allocation for a group of monitor groups if user space needs insight > > in monitoring data across all these monitor groups. User space will need to > > interact with resctrl differently and individually query monitor groups instead > > of CTRL_MON group once. > > Maybe assign one of the limited supply of RMIDs for these "unmonitored" > tasks. Populate a resctrl group named "unmonitored" that lists all the > unmonitored tasks in a (read-only) "tasks" file. And supply all the counts > for these tasks in normal looking "mon_data" directory. I needed to switch to an rdtgroup struct pointer rather than hardware IDs in the task structure to indicate group membership[1], otherwise it's not possible to determine which tasks are in a group when it doesn't have a unique HW ID value. Also this is required for shared assignment so that changing a group's IDs in a domain only requires updating running tasks rather than needing to search the entire task list, which would lead to the same problem we encountered in mongroup rename[2]. -Peter [1] https://lore.kernel.org/lkml/20240325172707.73966-5-peternewman@xxxxxxxxxx/ [2] https://lore.kernel.org/lkml/CALPaoCh0SbG1+VbbgcxjubE7Cc2Pb6QqhG3NH6X=WwsNfqNjtA@xxxxxxxxxxxxxx/
