General System Tweaks

1. CPU mitigations

A public speculative execution attack exploiting return instructions (retbleed) was revealed in July 2022. This has been mitigated in the kernel, but it results in a significant performance regression (14-39%).

The following CPU’s are affected:

AMD: Zen 1, Zen 1+, Zen 2
Intel: 6th to 8th Generation, Skylake, Kaby Lake, Coffee Lake

Check which mitigation affects your CPU by running the following command:

grep . /sys/devices/system/cpu/vulnerabilities/*

Disabling mitigations

While disabling the CPU mitigation’s is going to increase performance, as a downside it introduces security risks.

Add the following to your kernel command-line: retbleed=off or to disable all mitigation’s: mitigations=off

Edit the appropriate file to make the changes persistent:

GRUB: /etc/default/grub
systemd boot: /etc/sdboot-manage.conf
rEFInd: /boot/refind_linux.conf

For more information about the impact on performance:

Downfall

Downfall is characterized as a vulnerability due to a memory optimization feature that unintentionally reveals internal hardware registers to software. With Downfall, untrusted software can access data stored by other programs that typically should be off-limits: the AVX GATHER instruction can leak the contents of the internal vector register file during speculative execution. Downfall was discovered by security researcher Daniel Moghimi of Google. Moghimi has written demo code for Downfall to show 128-bit and 256-bit AES keys being stolen from other users on the local system as well as the ability to steal arbitrary data from the Linux kernel.

This affects the following CPU generations:

Ice Lake
Skylake
Kaby Lake
Coffee Lake
Comet Lake
Rocket Lake
Tiger Lake

Disabling Downfall

Add gather_data_sampling=off to your kernel cmdline parameters list. mitigations=off will also disable downfall.

2. AMD P-State Driver

amd-pstate is the AMD CPU performance scaling driver that introduces a new CPU frequency control mechanism on modern AMD APU and CPU series in the Linux kernel. The new mechanism is based on Collaborative Processor Performance Control (CPPC) which provides finer grain frequency management than legacy ACPI hardware P-States. Current AMD CPU/APU platforms are using the ACPI P-states driver to manage CPU frequency and clocks with switching only in 3 P-states. CPPC replaces the ACPI P-states controls and allows a flexible, low-latency interface for the Linux kernel to directly communicate the performance hints to hardware.

Below are 3 operation modes of the amd-pstate driver and kernel cmdline entries to use them on boot:

AMD P-State (Non-Autonomous Mode): amd-pstate=passive
AMD P-State Guided (Guided Autonomous Mode): amd-pstate=guided
AMD P-State EPP (Autonomous Mode): amd-pstate=active

You can also switch between operation modes at runtime to test the options:

Autonomous mode: platform considers only the values set for Minimum performance, Maximum performance, and Energy Performance Preference.
Terminal window
```
echo active | sudo tee /sys/devices/system/cpu/amd_pstate/status
```
Guided-autonomous mode: platform sets operating performance level according to the current workload and within limits set by the OS through minimum and maximum performance registers.
Terminal window
```
echo guided | sudo tee /sys/devices/system/cpu/amd_pstate/status
```
Non-autonomous mode: platform gets desired performance level from OS directly through Desired Performance Register.
Terminal window
```
echo passive | sudo tee /sys/devices/system/cpu/amd_pstate/status
```

For more information:

3. Using AMD P-State EPP

To use the P-State EPP, there are two CPU frequency scaling governors available: powersave and performance. It is recommended to use the powersave governor and set a preference.

Set powersave governor: sudo cpupower frequency-set -g powersave
Set performance governor: sudo cpupower frequency-set -g performance

To set a preference, run the following command with the desired preference:

echo power | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/energy_performance_preference

Available preferences: performance, power, balance_power, balance_performance

Benchmarks for each preference can be found here: https://lore.kernel.org/lkml/20221219064042.661122-1-perry.yuan@amd.com/

5. AMD 3D V Cache Optimizer

AMD posted a patch to optimize the Cache Scheduling on Dual CCD 3D CPUs, like 7950X3D and 7900X3D. You need to set in the BIOS under the CPPC Option to the “Driver” Option. This will allow to override with the sysfs the used mode.

There are two modes:

Frequency
Cache

If cache is set, the driver will try to put the tasks first on the CCD with the higher Cache, this is mainly profitable on games. The frequency option will try to put the tasks on the second CCD, which has a higher frequency then the 3D Cache CCD.

Frequency (Default):

echo frequency | sudo tee /sys/bus/platform/drivers/amd_x3d_vcache/AMDI0101:00/amd_x3d_mode

Cache:

echo cache | sudo tee /sys/bus/platform/drivers/amd_x3d_vcache/AMDI0101:00/amd_x3d_mode

After you changed the modes, the amd preffered core stats should provide a different ranking. You can read it out with:

grep -v /sys/devices/system/cpu/cpu*/cpufreq/amd_pstate_prefcore_ranking

5. AMD P-State Core Performance Boost

AMD Core Performance Boost aka AMD Turbo Core is a dynamic frequency scaling technology by AMD that allows the processor to dynamically adjust and control the processor operating frequency in certain version of its processors which allows for increased performance when needed while maintaining lower power and thermal parameters during normal operation.

Since linux-cachyos 6.9.6, the kernel is patched with CPB support for AMD’s p-state drivers (includes passive, active and guided). Users can change each CPU’s boost state via the sysfs boost file /sys/devices/system/cpu/cpuX/cpufreq/boost (X refers to the core number e.g. cpu0 is the first core, cpu1 second, etc).

❯ echo 0 | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/boost # Disable boost for all cores
❯ lscpu -ae # This shows that AMD CPB is disabled globally
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE    MAXMHZ   MINMHZ       MHZ
  0    0      0    0 0:0:0:0          yes 3301.0000 400.0000 1212.8250
  1    0      0    0 0:0:0:0          yes 3301.0000 400.0000 1394.2180
  2    0      0    1 1:1:1:0          yes 3301.0000 400.0000 1204.4600

❯ echo 1 | sudo tee /sys/devices/system/cpu/cpu0/cpufreq/boost # Enables boost on cpu0
❯ lscpu -ae
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE    MAXMHZ   MINMHZ       MHZ
  0    0      0    0 0:0:0:0          yes 4564.0000 400.0000 1393.2380
  1    0      0    0 0:0:0:0          yes 3301.0000 400.0000  400.0000
  2    0      0    1 1:1:1:0          yes 3301.0000 400.0000 2157.8469

CachyOS also provides a version of power-profiles-daemon that backports a commit which enables support for AMD CPB. AMD CPB will be disabled if the powersave profile is being used, and will be enabled on balanced or performance.

For more information see:

6. Disabling Split Lock Mitigate

In some cases, split lock mitigate can slow down performance in some applications and games. A patch is available to disable it via sysctl.

Disable split lock mitigate: sudo sysctl kernel.split_lock_mitigate=0
Enable split lock mitigate: sudo sysctl kernel.split_lock_mitigate=1

To make the change persistent, add the following line to /etc/sysctl.d/99-splitlock.conf:

kernel.split_lock_mitigate=0

For more information on split lock, see:

7. Enable RCU Lazy

RCU Lazy helps reducing the power usage at idle or lightly loaded systems. This can be useful for laptops and handheld devices. The improvement is between 5-10% in terms of power savings. However it’s important to note that this power saving feature may come at the cost of slightly reduced performance depending on the scenario. The linux-cachyos-deckify kernel will have this option enabled by default, since power saving is key and necessary for these devices.

To enable RCU Lazy, add the following parameter to your kernel cmdline parameters list:

rcutree.enable_rcu_lazy=1

8. NVIDIA GSP Firmware

The NVIDIA GSP Firmware can “in some cases” lead to decreased performance. While the 555.58.02 NVIDIA Driver has largely addressed this issue, it may persist on certain systems. If you are facing hiccups in KDE or bad performance in some cases, you can disable the GSP Firmware with the following config file: /etc/modprobe.d/nvidia-gsp.conf

options nvidia NVreg_EnableGpuFirmware=0

After creating the file, execute the following command:

sudo mkinitcpio -P

It’s generally recommended to test the GSP firmware after each new NVIDIA driver installation, as it often introduces beneficial features. Moreover, NVIDIA primarily started conducting QA testing using the GSP firmware.

9. Disabling SDDM Wayland Backend

While this a nice step forward, it might introduce some annoyances such as breaking support for overclocking using nvidia-settings or cause incompatibility with older GPUs which struggle under Wayland.

In order to revert this change. Remove cachyos-kde-settings:

sudo pacman -R cachyos-kde-settings

10. Granting Realtime Privileges for the User

# Install the following package:
sudo pacman -S realtime-privileges
# Run the following command:
sudo gpasswd -a $USER realtime
# Reboot your system.