Not possible for stats exporter (prometheus node exporter) to provide CPU frequency information about bottlerocket HOT 6 CLOSED

Thanks for the issue @JamesTGrant ! I quickly looked at this and also do not see a cpu_freq directory in sysfs.

bash-5.1# ls /sys/devices/system/cpu/cpu0/
cache  cpuidle  crash_notes  crash_notes_size  driver  firmware_node  hotplug  node0  power  subsystem  topology  uevent

According to our kernel config we do build in the CONFIG_CPU_FREQ kernel module:

bash-5.1# grep CPU_FREQ /boot/config 
CONFIG_ACPI_CPU_FREQ_PSS=y
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_GOV_ATTR_SET=y
CONFIG_CPU_FREQ_GOV_COMMON=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=m
CONFIG_CPU_FREQ_GOV_ONDEMAND=m
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m
CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y

I'll dig into this further and update this issue

from bottlerocket.

Insight into and control over CPU frequency scaling requires access to CPUs' P-states. Bottlerocket does ship with the relevant drivers, but whether Bottlerocket or any other operating system can access P-states depends on the platform (e.g. BIOS/UEFI/firmware settings on bare metal hardware, the hypervisor in virtualized environments). On Amazon EC2, only a subset of instance types pass through control over P-states. Instances that don't allow control over P-states are configured for maximum performance by the hypervisor already.

Please check whether you were running on an EC2 instance type for which the hypervisor exposes P-states to the instance by checking this list in the EC2 documentation. For instances without access to P-states, you will see a message like P-states controlled by the platform in dmesg output.

from bottlerocket.

Thank you. It is not in that list. I don't want to control the c or p state, but I would like access to the frequency metric (for the reasons stated)

from bottlerocket.

Thanks for confirming the used instance type cannot control P-states. On instances without control over P-states the cpufreq directories in sysfs will always be missing as no P-state driver will be claiming them. Please note that this is a limitation put in place by the platform and shared by all distributions.

As an option, the Prometheus Node Exporter may be modified to read the cpu MHz fields of the /proc/cpuinfo file if P-states cannot be managed by the kernel and the cpufreq directories don't exist. Older kernels (before Linux 4.13) would show the value used for TSC calibration here, but newer ones use a couple processor registers incrementing at different speeds (APERF, MPERF) to take a guess at the frequency over the last sampling period.

A couple side notes:

• For cloud environments you can assume the provider is monitoring for thermal throttling on the host level already.
• Another, more common cause for reduced CPU performance may be the depletion of the burst bucket on burstable instance types (e.g. the T instance families on Amazon EC2) which would not show itself as a reduced CPU frequency, but can be measured via metrics specific to the cloud provider (see the docs for Amazon EC2 for an example).

from bottlerocket.

We haven't heard back in a while. As explained above, this is nothing that can be fixed in Bottlerocket. I'm closing the issue now, but please feel free to reopen it in case you have more questions on the topic.

from bottlerocket.

Thank you for the advice. I'm aware of the side notes but well worth pointing out. I wasn't aware that the MHz in /proc/cpuinfo was a dynamic status, I thought it was static so I think that is what I shall experiement with. Thank you again.

from bottlerocket.