TURBOSTAT(8) System Manager's Manual TURBOSTAT(8) (NAME) turbostat - (SYNOPSIS) turbostat [] turbostat [] [--interval ] (DESCRIPTION) turbostat (topology) X86 . turbostat . (command) (fork) . turbostat . --interval . . (OPTIONS) '-' . . --add attributes <<>> (attributes) . 'location' . location: {msrDDD | msr0xXXX | /sys/path... | perf//} msrDDD msr16 msr0xXXX (hex) msr0x10 /sys/path... sysfs perf /sys/bus/event_source/devices/ cstate_core perf "cpu" "cpu_core" "cpu_atom" P E . Turbostat "cpu" "cpu_core" "cpu_atom" . << PERF ( cpu)>> . perf /sys/bus/event_source/devices//events/ c1-residency perf/cstate_core/c1-residency /sys/bus/event_source/devices/cstate_core/events/c1-residency scope: {cpu | core | package} (cpu) (core) (package). : cpu size: {u32 | u64 } MSR u32 . : u64 format: {raw | delta | percent | average} 'raw' MSR . 'delta' . 'percent' . 'average' raw / ( -S) . : delta name: "name_string" . --add pmt,[attr_name=attr_value, ...] PMT ( - Intel Platform Monitoring Technology) --add PMT . XML PMT https://github.com/intel/Intel-PMT . << PMT>> . name="name_string" . type={raw} 'raw' . : raw format={raw | delta} 'raw' . 'delta' . : raw domain={cpu%u | core%u | package%u} 'cpu' /. 'core' . 'package' . '%u' . core4 . offset=%u '%u' MMIO PMT. lsb=%u '%u' 'offset'. 'msb' . msb=%u '%u' 'offset'. 'lsb' . guid=%x '%x' MMIO PMT. --cpu cpu-set (cpu-set) . cpu-set "core" -- HT . cpu-set "package" . . cpu-set ".." "-" : 1,2,8,14..17,21-44 --hide column . . --enable column . "usec" "Time_Of_Day_Seconds" "APIC" "X2APIC". "all" . --show column . . --show CATEGORY --hide CATEGORY show hide (CATEGORIES) : "all" "topology" "idle" "frequency" "power" "cpuidle" "hwidle" "swidle" "cache" "llc" "other". "idle" ( ) "hwidle" "pct_idle" . "cpuidle" ( ) cpuidle . "swidle" "cpuidle" "pct_idle" . "hwidle" . turbostat "sysfs" "swidle" . --Dump . --quiet . --no-msr MSR . --no-perf (API) perf . --force turbostat ( ). --interval seconds . --num_iterations num . --header_iterations num num . --out output_file turbostat output_file . (truncate) . --help . --Joules . --list --show --hide . --Summary . --TCC temperature (Thermal Control Circuit) . (Digital Thermal Sensor) TCC . --version . (command) (fork) . (ROW DESCRIPTIONS) ( --quiet ) . ( ). . Package Core CPU '-' . . ( IRQ) . . MSR 0 . Package/Core/CPU . --cpu turbostat . (COLUMN DESCRIPTIONS) usec (thread migration) -- . "--enable usec" --debug . usec (snapshot) procfs/sysfs . Time_Of_Day_Seconds gettimeofday(2) (. Epoch) . "--enable Time_Of_Day_Seconds" "--debug" . Time_Of_Day_Seconds . Core . (Intel Hyper-Threading Technology HT) . CPU ( ). -- turbostat (topology) HT . Package -- . Avg_MHz . ( ) . Busy% "C0". Bzy_MHz ( "c0"). TSC_MHz (TSC) . IRQ . . turbostat /proc/interrupts . SMI (System Management Interrupts) . SMI . LLCMRPS (Last Level Cache Millions of References Per Second). LLC (L3) L3 ( L2 ) . LLC (L2) L2 ( L1 ) . . . LLC%hit . LLC%hit . LLC%hit . = 100.0 * Hits/References. (100.0 * Sum_Hits/Sum_References). L2MRPS (Level-2 Cache Millions of References Per Second). LLC LLC . . . L2%hit . L2%hit . L2%hit . = 100.0 * Hits/References. (100.0 * (Sum_Hits)/Sum_References). C1, C2, C3... C1 C2 C3 . . C-state /sys/devices/system/cpu/cpu*/cpuidle/state*/name . . MWAIT . "cpuidle" . C1+, C2+, C3+... (idle governor). C1 C2 C3 ( ). /sys/devices/system/cpu/cpu*/cpuidle/state*/below . "cpuidle" . C1-, C2-, C3-... (idle governor). C1 C2 C3 ( ). /sys/devices/system/cpu/cpu*/cpuidle/state*/above . "cpuidle" . C1%, C2%, C3% C1 C2 C3... . . . . "pct_idle" . CPU%c1, CPU%c3, CPU%c6, CPU%c7 . "hwidle" . CoreTmp (DTS) . PkgTmp (Package Thermal Monitor) . CoreThr (Core Thermal Throttling) . /sys/devices/system/cpu/cpu*/thermal_throttle/* . GFX%rc6 (GPU) "render C6" ( rc6) . /sys/class/drm/card0/power/rc6_residency_ms /sys/class/drm/card0/gt/gt0/rc6_residency_ms /sys/class/drm/card0/device/tile0/gtN/gtidle/idle_residency_ms . GFXMHz sysfs . /sys/class/graphics/fb0/device/drm/card0/gt_cur_freq_mhz /sys/class/drm/card0/gt_cur_freq_mhz /sys/class/drm/card0/gt/gt0/rps_cur_freq_mhz /sys/class/drm/card0/device/tile0/gtN/freq0/cur_freq . GFXAMHz sysfs . /sys/class/graphics/fb0/device/drm/card0/gt_act_freq_mhz /sys/class/drm/card0/gt_act_freq_mhz /sys/class/drm/card0/gt/gt0/rps_act_freq_mhz /sys/class/drm/card0/device/tile0/gtN/freq0/act_freq . SAM%mc6 SA Media "module C6" ( mc6) . /sys/class/drm/card0/gt/gt1/rc6_residency_ms /sys/class/drm/card0/device/tile0/gtN/gtidle/idle_residency_ms . SAMMHz sysfs . /sys/class/drm/card0/gt/gt1/rps_cur_freq_mhz /sys/class/drm/card0/device/tile0/gtN/freq0/cur_freq . SAMAMHz sysfs . /sys/class/drm/card0/gt/gt1/rps_act_freq_mhz /sys/class/drm/card0/device/tile0/gtN/freq0/act_freq . Totl%C0 . N N * 100% . Any%C0 . GFX%C0 (GFX compute engine) . CPUGFX% . Pkg%pc2, Pkg%pc3, Pkg%pc6, Pkg%pc7 . . PkgWatt . CorWatt . GFXWatt -- . RAMWatt DRAM DIMM -- . SysWatt (RAPL PSYS). PKG_% (throttling) RAPL . 100% . . RAPL . PkgWatt PkgTmp . RAM_% RAPL DRAM . UncMHz uncore . UMHz1.0 uncore pm_domain=1 fabric_cluster=0 . . Intel Granite Rapids pm_domain IO cluster . "--show" "--hide" "UncMHz" UMHz*.* . (TOO MUCH INFORMATION EXAMPLE) turbostat -- . "--out" . . "--quiet" turbostat . "--hide" "--show" . "--show" turbostat . "--hide" turbostat . --show --hide "--list" . (CATEGORY) . "sysfs" C-state sysfs : sudo ./turbostat --show sysfs --quiet sleep 10 10.003837 sec C1 C1E C3 C6 C7s C1% C1E% C3% C6% C7s% 4 21 2 2 459 0.14 0.82 0.00 0.00 98.93 1 17 2 2 130 0.00 0.02 0.00 0.00 99.80 0 0 0 0 31 0.00 0.00 0.00 0.00 99.95 2 1 0 0 52 1.14 6.49 0.00 0.00 92.21 1 2 0 0 52 0.00 0.08 0.00 0.00 99.86 0 0 0 0 71 0.00 0.00 0.00 0.00 99.89 0 0 0 0 25 0.00 0.00 0.00 0.00 99.96 0 0 0 0 74 0.00 0.00 0.00 0.00 99.94 0 1 0 0 24 0.00 0.00 0.00 0.00 99.84 (ONE SHOT COMMAND EXAMPLE) turbostat (fork) . turbostat (stderr) . --out . "sleep 10" turbostat "ts.out" . "sleep 10" turbostat turbostat . "ts.out" bugzilla . [root@hsw]# ./turbostat -o ts.out sleep 10 [root@hsw]# (PERIODIC INTERVAL EXAMPLE) turbostat . (stdout) --out . "-i sec" . sudo turbostat --quiet --show CPU,frequency Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz CPU%c7 UncMhz - - 524 12.48 4198 3096 74.53 3800 0 0 4 0.09 4081 3096 98.88 3800 0 4 1 0.02 4063 3096 1 1 2 0.06 4063 3096 99.60 1 5 2 0.05 4070 3096 2 2 4178 99.52 4199 3096 0.00 2 6 3 0.08 4159 3096 3 3 1 0.04 4046 3096 99.66 3 7 0 0.01 3989 3096 Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz CPU%c7 UncMhz - - 525 12.52 4198 3096 74.54 3800 0 0 4 0.10 4051 3096 99.49 3800 0 4 2 0.04 3993 3096 1 1 3 0.07 4054 3096 99.56 1 5 4 0.10 4018 3096 2 2 4178 99.51 4199 3096 0.00 2 6 4 0.09 4143 3096 3 3 2 0.06 4026 3096 99.10 3 7 7 0.17 4074 3096 --show . (SYSTEM CONFIGURATION INFORMATION EXAMPLE) turbostat (dump) . "--quiet" . : turbostat version 2022.04.16 - Len Brown Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.18.0-rc6-00001-ge6891250e3b5 ... CPUID(0): GenuineIntel 0x16 CPUID levels CPUID(1): family:model:stepping 0x6:9e:9 (6:158:9) microcode 0xea CPUID(0x80000000): max_extended_levels: 0x80000008 CPUID(1): SSE3 MONITOR - EIST TM2 TSC MSR ACPI-TM HT TM CPUID(6): APERF, TURBO, DTS, PTM, HWP, HWPnotify, HWPwindow, HWPepp, No-HWPpkg, EPB cpu7: MSR_IA32_MISC_ENABLE: 0x00850089 (TCC EIST MWAIT PREFETCH TURBO) CPUID(7): SGX cpu7: MSR_IA32_FEATURE_CONTROL: 0x00000005 (Locked ) CPUID(0x15): eax_crystal: 2 ebx_tsc: 258 ecx_crystal_hz: 0 TSC: 3096 MHz (24000000 Hz * 258 / 2 / 1000000) CPUID(0x16): base_mhz: 3100 max_mhz: 4200 bus_mhz: 100 cpu7: MSR_MISC_PWR_MGMT: 0x00401cc0 (ENable-EIST_Coordination DISable-EPB DISable-OOB) RAPL: 5825 sec. Joule Counter Range, at 45 Watts cpu7: MSR_PLATFORM_INFO: 0x80839f1011f00 8 * 100.0 = 800.0 MHz max efficiency frequency 31 * 100.0 = 3100.0 MHz base frequency cpu7: MSR_IA32_POWER_CTL: 0x002c005d (C1E auto-promotion: DISabled) cpu7: MSR_TURBO_RATIO_LIMIT: 0x2728292a 39 * 100.0 = 3900.0 MHz max turbo 4 active cores 40 * 100.0 = 4000.0 MHz max turbo 3 active cores 41 * 100.0 = 4100.0 MHz max turbo 2 active cores 42 * 100.0 = 4200.0 MHz max turbo 1 active cores cpu7: MSR_CONFIG_TDP_NOMINAL: 0x0000001f (base_ratio=31) cpu7: MSR_CONFIG_TDP_LEVEL_1: 0x00000000 () cpu7: MSR_CONFIG_TDP_LEVEL_2: 0x00000000 () cpu7: MSR_CONFIG_TDP_CONTROL: 0x80000000 ( lock=1) cpu7: MSR_TURBO_ACTIVATION_RATIO: 0x00000000 (MAX_NON_TURBO_RATIO=0 lock=0) cpu7: MSR_PKG_CST_CONFIG_CONTROL: 0x1e008008 (UNdemote-C3, UNdemote-C1, demote-C3, demote-C1, locked, pkg-cstate-limit=8 (unlimited)) Uncore Frequency pkg0 die0: 800 - 3900 MHz (800 - 3900 MHz) /dev/cpu_dma_latency: 2000000000 usec (default) current_driver: intel_idle current_governor: menu current_governor_ro: menu cpu7: POLL: CPUIDLE CORE POLL IDLE cpu7: C1: MWAIT 0x00 cpu7: C1E: MWAIT 0x01 cpu7: C3: MWAIT 0x10 cpu7: C6: MWAIT 0x20 cpu7: C7s: MWAIT 0x33 cpu7: C8: MWAIT 0x40 cpu7: C9: MWAIT 0x50 cpu7: C10: MWAIT 0x60 cpu7: cpufreq driver: intel_pstate cpu7: cpufreq governor: performance cpufreq intel_pstate no_turbo: 0 cpu7: MSR_MISC_FEATURE_CONTROL: 0x00000000 (L2-Prefetch L2-Prefetch-pair L1-Prefetch L1-IP-Prefetch) cpu0: MSR_PM_ENABLE: 0x00000001 (HWP) cpu0: MSR_HWP_CAPABILITIES: 0x01101f53 (high 83 guar 31 eff 16 low 1) cpu0: MSR_HWP_REQUEST: 0x00005353 (min 83 max 83 des 0 epp 0x0 window 0x0 pkg 0x0) cpu0: MSR_HWP_INTERRUPT: 0x00000001 (EN_Guaranteed_Perf_Change, Dis_Excursion_Min) cpu0: MSR_HWP_STATUS: 0x00000004 (No-Guaranteed_Perf_Change, No-Excursion_Min) cpu0: EPB: 6 (balanced) cpu0: MSR_RAPL_POWER_UNIT: 0x000a0e03 (0.125000 Watts, 0.000061 Joules, 0.000977 sec.) cpu0: MSR_PKG_POWER_INFO: 0x00000168 (45 W TDP, RAPL 0 - 0 W, 0.000000 sec.) cpu0: MSR_PKG_POWER_LIMIT: 0x42820800218208 (UNlocked) cpu0: PKG Limit #1: ENabled (65.000 Watts, 64.000000 sec, clamp ENabled) cpu0: PKG Limit #2: ENabled (65.000 Watts, 0.002441* sec, clamp DISabled) cpu0: MSR_VR_CURRENT_CONFIG: 0x00000000 cpu0: PKG Limit #4: 0.000000 Watts (UNlocked) cpu0: MSR_DRAM_POWER_LIMIT: 0x5400de00000000 (UNlocked) cpu0: DRAM Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled) cpu0: MSR_PP0_POLICY: 0 cpu0: MSR_PP0_POWER_LIMIT: 0x00000000 (UNlocked) cpu0: Cores Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled) cpu0: MSR_PP1_POLICY: 0 cpu0: MSR_PP1_POWER_LIMIT: 0x00000000 (UNlocked) cpu0: GFX Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled) cpu0: MSR_IA32_TEMPERATURE_TARGET: 0x00640000 (100 C) (100 default - 0 offset) cpu0: MSR_IA32_PACKAGE_THERM_STATUS: 0x88200800 (68 C) cpu0: MSR_IA32_PACKAGE_THERM_INTERRUPT: 0x00000003 (100 C, 100 C) cpu7: MSR_PKGC3_IRTL: 0x0000884e (valid, 79872 ns) cpu7: MSR_PKGC6_IRTL: 0x00008876 (valid, 120832 ns) cpu7: MSR_PKGC7_IRTL: 0x00008894 (valid, 151552 ns) cpu7: MSR_PKGC8_IRTL: 0x000088fa (valid, 256000 ns) cpu7: MSR_PKGC9_IRTL: 0x0000894c (valid, 339968 ns) cpu7: MSR_PKGC10_IRTL: 0x00008bf2 (valid, 1034240 ns) (max efficiency) (Low Frequency Mode) . TSC -- (brand string) /proc/cpuinfo . . . . (ADD COUNTER EXAMPLE) turbostat CPU cpu0 cpu3 . MSR 0x199 (MSR_IA32_PERF_CTL) "PRF_CTRL" (sleep) . sudo ./turbostat --quiet --cpu 0-3 --show CPU --add msr0x199,u32,raw,PRF_CTRL sleep .1 0.101604 sec CPU PRF_CTRL - 0x00000000 0 0x00000c00 1 0x00000800 2 0x00000a00 3 0x00000800 PERF (ADD PERF COUNTER EXAMPLE) turbostat CPU cpu0 cpu3 . cstate C1 "pCPU%c1" . CPU%c1 . sudo ./turbostat --quiet --cpu 0-3 --show CPU,CPU%c1 --add perf/cstate_core/c1-residency,cpu,delta,percent,pCPU%c1 sleep .1 0.102448 sec CPU pCPU%c1 CPU%c1 - 34.89 34.89 0 45.99 45.99 1 45.94 45.94 2 23.83 23.83 3 23.84 23.84 PERF ( cpu) Meteor Lake . turbostat cpu0 ( - pcore) cpu4 ( - ecore) . L3 "cpu" "VCMISS" . L3 "cpu_core" "PCMISS" . cpu4 . L3 "cpu_atom" "ECMISS" . cpu0 . . sudo ./turbostat --quiet --cpu 0,4 --show CPU --add perf/cpu/cache-misses,cpu,delta,VCMISS --add perf/cpu_core/cache-misses,cpu,delta,PCMISS --add perf/cpu_atom/cache-misses,cpu,delta,ECMISS sleep 5 turbostat: added_perf_counters_init_: perf/cpu_atom/cache-misses: failed to open counter on cpu0 turbostat: added_perf_counters_init_: perf/cpu_core/cache-misses: failed to open counter on cpu4 5.001207 sec CPU ECMISS PCMISS VCMISS - 41586506 46291219 87877749 4 83173012 0 83173040 0 0 92582439 92582458 PMT (ADD PMT COUNTER EXAMPLE) turbostat . DC6 . XML PMT . "XTAL" (type) 'raw' (format) 'delta' (domain) 'package0' (offset) '0' MMIO PMT lsb msb guid '0x1a067102' MMIO PMT 'offset' . DC6 "Die%c6" 'txtal_time' 'delta' 'package0' '0' MMIO PMT lsb msb guid '0x1a067102' MMIO PMT 'offset' . sudo ./turbostat --quiet --cpu 0 --show CPU --add pmt,name=XTAL,type=raw,format=delta,domain=package0,offset=0,lsb=0,msb=63,guid=0x1a067102 --add pmt,name=Die%c6,type=txtal_time,format=delta,domain=package0,offset=120,lsb=0,msb=63,guid=0x1a067102 0.104352 sec CPU XTAL Die%c6 - 0x0000006d4d957ca7 0.00 0 0x0000006d4d957ca7 0.00 0.102448 sec (INPUT) (interval-mode) turbostat . turbostat . Control-C SIGINT turbostat . (SIGNALS) SIGINT . turbostat . SIGUSR1 turbostat . (NOTES) turbostat (root) . turbostat : # setcap cap_sys_admin,cap_sys_rawio,cap_sys_nice=+ep path/to/turbostat # chmod +r /dev/cpu/*/msr # chmod +r /dev/cpu_dma_latency turbostat . . turbostat 2.6.20 2.6.29 acpi-cpufreq MSR APERF MPERF . AVG_MHz = APERF_delta/measurement_interval. -- . TSC (non-stop TSC) . TSC_MHz = TSC_delta/measurement_interval. TSC (invariant TSC) /proc/cpuinfo . TSC TSC_MHz . Busy% = MPERF_delta/TSC_delta Bzy_MHz = TSC_delta*APERF_delta/MPERF_delta/measurement_interval TSC_delta TSC_MHz . Turbostat (atomic) . ( ) turbostat . MSR APERF MPERF (non-halted) . turbostat TSC . (REFERENCES) Volume 3B: System Programming Guide https://www.intel.com/products/processor/manuals (RUN THE LATEST VERSION) turbostat . turbostat . perf(1) . turbostat : "git pull https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git" make tools/power/x86/turbostat/. : "make install" turbostat . /usr/local/bin . turbostat/Makefile "make snapshot" tar . : "git pull https://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux.git turbostat" "turbostat" (Subject) . turbostat "--force" . (FILES) /sys/bus/event_source/devices/ /dev/cpu/*/msr /sys/class/intel_pmt/ /sys/devices/system/cpu/ (SEE ALSO) perf(1), msr(4), vmstat(8) (AUTHORS) Len Brown TURBOSTAT(8)