Tweaking kernels for low-RAM use

[fatdoguser]

For those that like to keep up with the latest kernel point releases, on lower memory systems enabling the Memory, Transparent hugepage madvise

CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y

can help with too much memory being unnecessarily allocated/wasted.

I think that is a 6.7 kernel addition, it's certainly in the 6.1.77 point release of the 6.1 kernel that I'm tracking (on a 4GB ram laptop system where a big chunk of that is allocated to vram).

[ozsouth]

@fatdoguser - my latest kernels have that 'not set' & CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y set. I'm guessing those 2 should be reversed in your scenario?

[fatdoguser]

@fatdoguser - my latest kernels have that 'not set' & CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y set. I'm guessing those 2 should be reversed in your scenario?

@ozsouth I changed mine in make menuconfig, zcat /proc/config.gz | grep TRANSPARENT ... indicates

Code: Select all

CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD=y
CONFIG_TRANSPARENT_HUGEPAGE=y
# CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y

... so yes.(reversed).

So wont always allocate a huge page (more ram) by defaullt, but instead only when the kernel deems it appropriate to do so (or not).

Technical detail here : https://www.kernel.org/doc/html/latest/ ... shuge.html

[dimkr]

The dpup kernels are configured like this, makes a big difference.

[jamesbond]

Interesting. I run Fatdog64 on 8GB machines for years (and still do) but never encounter memory pressure problem ... but perhaps I'm just not aware of what missing. On the other hand, from the kernel doc that @fatdoguser referred, enabling THP (transparent huge pages) as "always" (as how Fatdog64 does it) has its performance benefits I guess it's down to the classical memory and speed trade-off again; and different RAM configurations and different workloads requires different tuning.

For others who are reading this thread and wondering about whether they need a special custom compiled kernel to do this, the answer is that you DON'T. Fatdog64 standard kernel can do both "always" and "madvise". The kernel config discussed in this thread only determines which one is used by default.

Standard Fatdog64 kernel "always" uses THP, but it can easily be configured to use "madvise" instead. Per the kernel documentation linked, it can be done in two ways:

1. You can put the boot parameter transparent_hugepage=madvise
2. Or, once Fatdog is up and running, you can run the following command (perhaps put it in rc.local or something):

   Code: Select all

   echo madvise >/sys/kernel/mm/transparent_hugepage/enabled

   And if you don't like it and watch to switch back to "always":

   Code: Select all

   echo always >/sys/kernel/mm/transparent_hugepage/enabled

Note that both (1) or (2) do not survive reboot. Each reboot resets the kernel to use the "always" option, unless overridden by (1) or (2).

---

A little clarification. The "madvise" option doesn't mean that the THP will be enabled by the kernel automatically when it thinks that it is appropriate. What it means is that the kernel leaves it to the application, to determine whether THP will be enabled or not. In other words, the application will have to tell the kernel to enable THP or not, and I would hazard that most applications never do this (except for large server applications, like database server perhaps).

That's why you can see the reduction in RAM-usage when this option is chosen, but on the other hand one could argue that enabling THP will provide performance benefits even for applications that don't explicitly ask for it. The only way to know is to run a benchmark, but most benchmark is artificial and doesn't always represent real-life usage; so in the end it comes back to my above statement: different RAM configurations and different workloads requires different tuning. Test your system with both options, and keep the one that provides you which the best performance.

[ozsouth]

@jamesbond - ok, now I'm confused. I compiled 6.1.74 today with madvise=y & always 'not set' - seems faster (I have 12Gb ram). Can you have both options as '=y' ?

[jamesbond]

No you can't have both enabled as "y", because the configuration is to set the default settings, and you (obviously) can only have one default settings.

But whichever way you configure it, you can always run it one way or the other: boot parameters of either transparent_hugepage=madvise or transparent_hugepage=always, or by tweaking kernel runtime parameters after you boot up (obviously only affecting programs run after you make the tweak): echo madvise >/sys/kernel/mm/transparent_hugepage/enabled or echo always >/sys/kernel/mm/transparent_hugepage/enabled - this is the gist of my previous post. In fact, you can disable THP completely too by passing "never" instead of always/madvise.

As for "faster" - well, if you feel that it's faster, then it probably is. There are reasons why "always" could be faster, and there are also reasons why "madvise" could be faster ... which one ends more dominating depends on your workload.

[ozsouth]

@jamesbond - thanks - I would have been surprised if you could have both, as in some other config situations, one would be '=m', but I see you can toggle the setting regardless of which is builtin. Have to say I don't see much memory-use difference.