On 5/23/25 2:06 PM, Rafael J. Wysocki wrote:
> On Fri, May 23, 2025 at 6:25 AM Shashank Balaji
> <shashank.mahadasyam@xxxxxxxx> wrote:
>>
>> Hi Russell,
>>
>> On Thu, May 22, 2025 at 06:15:24AM -0500, Russell Haley wrote:
>>>> The userspace governor requests a frequency between policy->min and
>>>> policy->max on behalf of user space. In intel_pstate this translates
>>>> to setting DESIRED_PERF to the requested value which is also the case
>>>> for the other governors.
>>>
>>> Huh. On this Skylake box with kernel 6.14.6, it seems to be setting
>>> Minimum_Performance, and leaving desired at 0.
>>>
>>>> echo userspace | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
>>> userspace
>>>> echo 1400000 | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_setspeed
>>> 1400000
>>>> sudo x86_energy_perf_policy &| grep REQ
>>> cpu0: HWP_REQ: min 14 max 40 des 0 epp 128 window 0x0 (0*10^0us) use_pkg 0
>>
>> Oh cool, I didn't know about x86_energy_perf_policy.
>>
>> Consider the following on a Raptor Lake machine:
>>
>> 1. HWP_REQUEST MSR set by intel_pstate in active mode:
>>
>> # echo active > intel_pstate/status
>> # x86_energy_perf_policy -c 0 2>&1 | grep REQ
>> cpu0: HWP_REQ: min 11 max 68 des 0 epp 128 window 0x0 (0*10^0us) use_pkg 0
>> pkg0: HWP_REQ_PKG: min 1 max 255 des 0 epp 128 window 0x0 (0*10^0us)
>> # echo 2000000 > cpufreq/policy0/scaling_min_freq
>> # echo 3000000 > cpufreq/policy0/scaling_max_freq
>> # x86_energy_perf_policy -c 0 2>&1 | grep REQ
>> cpu0: HWP_REQ: min 26 max 39 des 0 epp 128 window 0x0 (0*10^0us) use_pkg 0
>> pkg0: HWP_REQ_PKG: min 1 max 255 des 0 epp 128 window 0x0 (0*10^0us)
>>
>> scaling_{min,max}_freq just affect the min and max frequencies
>> set in HWP_REQEST. desired_freq is left at 0.
>>
>> 2. HWP_REQUEST MSR set by intel_pstate in passive mode with userspace
>> governor:
>>
>> # echo passive > intel_pstate/status
>> # echo userspace > cpufreq/policy0/scaling_governor
>> # cat cpufreq/policy0/scaling_setspeed
>> 866151
>> # x86_energy_perf_policy -c 0 2>&1 | grep REQ
>> cpu0: HWP_REQ: min 11 max 68 des 0 epp 128 window 0x0 (0*10^0us) use_pkg 0
>> pkg0: HWP_REQ_PKG: min 1 max 255 des 0 epp 128 window 0x0 (0*10^0us)
>> # echo 2000000 > cpufreq/policy0/scaling_setspeed
>> # x86_energy_perf_policy -c 0 2>&1 | grep REQ
>> cpu0: HWP_REQ: min 26 max 68 des 0 epp 128 window 0x0 (0*10^0us) use_pkg 0
>> pkg0: HWP_REQ_PKG: min 1 max 255 des 0 epp 128 window 0x0 (0*10^0us)
>>
>> scaling_setspeed only changes the min frequency in HWP_REQUEST.
>> Meaning, software is explicitly allowing the hardware to choose
>> higher frequencies.
>>
>> 3. Same as above, except with strictuserspace governor, which is a
>> custom kernel module which is exactly the same as the userspace
>> governor, except it has the CPUFREQ_GOV_STRICT_TARGET flag set:
>>
>> # echo strictuserspace > cpufreq/policy0/scaling_governor
>> # x86_energy_perf_policy -c 0 2>&1 | grep REQ
>> cpu0: HWP_REQ: min 26 max 26 des 0 epp 128 window 0x0 (0*10^0us) use_pkg 0
>> pkg0: HWP_REQ_PKG: min 1 max 255 des 0 epp 128 window 0x0 (0*10^0us)
>> # echo 3000000 > cpufreq/policy0/scaling_setspeed
>> # x86_energy_perf_policy -c 0 2>&1 | grep REQ
>> cpu0: HWP_REQ: min 39 max 39 des 0 epp 128 window 0x0 (0*10^0us) use_pkg 0
>> pkg0: HWP_REQ_PKG: min 1 max 255 des 0 epp 128 window 0x0 (0*10^0us)
>>
>> With the strict flag set, intel_pstate honours this by setting
>> the min and max freq same.
>>
>> desired_perf is always 0 in the above cases. The strict flag check is done in
>> intel_cpufreq_update_pstate, which sets max_pstate to target_pstate if policy
>> has strict target, and cpu->max_perf_ratio otherwise.
>>
>> As Russell and Rafael have noted, CPU frequency is subject to hardware
>> coordination and optimizations. While I get that, shouldn't software try
>> its best with whatever interface it has available? If a user sets the
>> userspace governor, that's because they want to have manual control over
>> CPU frequency, for whatever reason. The kernel should honor this by
>> setting the min and max freq in HWP_REQUEST equal. The current behaviour
>> explicitly lets the hardware choose higher frequencies.
>
> Well, the userspace governor ends up calling the same function,
> intel_cpufreq_target(), as other cpufreq governors except for
> schedutil. This function needs to work for all of them and for some
> of them setting HWP_MIN_PERF to the same value as HWP_MAX_PERF would
> be too strict. HWP_DESIRED_PERF can be set to the same value as
> HWP_MIN_PERF, though (please see the attached patch).
The other governors have been around a lot longer than HWP, though, and
and are used on non-Intel hardware, which may not have a, "this
frequency or higher subject to firmware heuristics," interface.
I tried this on a non-HWP Haswell machine, and there it works like
DESIRED=MIN. Or maybe DESIRED=MAX=MIN; I don't understand when or why
hardware would choose frequencies between DESIRED and MAX (before module
coordination).
IMO, intel_cpufreq_target() being wired up to HWP_MIN_PERF is actually
*more* strange for the other governors than for userspace, because at
least with userspace governor, the userspace program is free to write to
scaling_{min,max}_freq instead of scaling_setspeed if it wants.
The conservative governor on HWP hardware, for example, will cause
strictly higher frequencies (and typically, higher energy consumption)
than HWP powersave. But on non-HWP hardware, conservative is an
efficient, slow-ramping governor.
Changing the behavior of the old-style cpufreq governors is fraught,
because the defaults are schedutil and HWP-powersave, so users of the
other governors likely made an intentional choice, presumably after
tests on a specific platform. A change would invalidate those tests.
But on the other hand, they might *already* be invalid because of an
upgrade from non-HWP hardware. In that case, changing to DES=MIN would
move closer to the tested behavior.
And then there's churn coming from other parts of the stack. For
example, until recently [1] tuned would select conservative for its
"balanced" profile and ondemand for its "powersave" profile, based on
very old data. But that didn't matter until Redhat stopped funding work
on power-profiles-daemon, and the desktop environments' power-profile
selectors got wired up to tuned in Fedora. Hector Martin fixed that,
switching both to schedutil (unless CONFIG_CPU_FREQ_GOV_SCHEDUTIL=n,
which is rare I think). That is at least not terrible on non-HWP
hardware, but given what he was working on at the time, it might not
have been tested on x86.
[1]
https://github.com/redhat-performance/tuned/commit/e24bfef651aa7f4da95727815b2cacbf571b59af
Cheers,
Russell
