Newer kernel update not being used

Support
1

Hello there,

My problem seems to be quite similar as this one : Kernel update not applying but the topic is quite old and I’ve already tried a few things without success.

On one of VMs (only one impacted by this), I’m not seeing the newer Kernel in boot menu and the command grub2-mkconfig -o /boot/grub2/grub.cfg didn’t fix the issue.

Loader entries confirm the missing Kernel:

ll /boot/loader/entries/
total 12
-rw-r--r--. 1 root root 406 Dec 20  2021 690ab83ab4c84c2c9f19f6e7eccaea0f-0-rescue.conf
-rw-r--r--. 1 root root 365 Sep 15 12:42 690ab83ab4c84c2c9f19f6e7eccaea0f-4.18.0-372.26.1.el8_6.x86_64.conf
-rw-r--r--. 1 root root 365 Oct 29 09:15 690ab83ab4c84c2c9f19f6e7eccaea0f-4.18.0-372.32.1.el8_6.x86_64.conf

This is the result of yum check-update:

yum check-update
Last metadata expiration check: 0:08:47 ago on Tue Jan 10 12:53:30 2023.
Security: kernel-core-4.18.0-425.3.1.el8.x86_64 is an installed security update
Security: kernel-core-4.18.0-372.32.1.el8_6.x86_64 is the currently running version

The content of /etc/default/grub:

GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto resume=/dev/mapper/almalinux-swap rd.lvm.lv=almalinux/root rd.lvm.lv=almalinux/swap rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_ENABLE_BLSCFG=true

Command sudo grep -B1 -A1 blscfg /boot/efi/EFI/almalinux/grub.cfg /boot/grub2/grub.cfg

/boot/efi/EFI/almalinux/grub.cfg-#
/boot/efi/EFI/almalinux/grub.cfg:# The blscfg command parses the BootLoaderSpec files stored in /boot/loader/entries and
/boot/efi/EFI/almalinux/grub.cfg-# populates the boot menu. Please refer to the Boot Loader Specification documentation
--
/boot/efi/EFI/almalinux/grub.cfg-
/boot/efi/EFI/almalinux/grub.cfg:insmod blscfg
/boot/efi/EFI/almalinux/grub.cfg:blscfg
/boot/efi/EFI/almalinux/grub.cfg-### END /etc/grub.d/10_linux ###
--
/boot/grub2/grub.cfg-#
/boot/grub2/grub.cfg:# The blscfg command parses the BootLoaderSpec files stored in /boot/loader/entries and
/boot/grub2/grub.cfg-# populates the boot menu. Please refer to the Boot Loader Specification documentation
--
/boot/grub2/grub.cfg-
/boot/grub2/grub.cfg:insmod blscfg
/boot/grub2/grub.cfg:blscfg
/boot/grub2/grub.cfg-### END /etc/grub.d/10_linux ###

Soooooo :slight_smile: what are the next steps to try?
Thanks in advance for you help

PS: big shotout to Almalinux’s team that provided me a reliable and simple replacement solution for my CentOS VMs! :wink:

2

you should be able to fix that using grubby

so here’s my 8.7 vm that is using the 425 kernel ok:

[root@gitlab ~]# grubby --info=ALL 
index=0
kernel="/boot/vmlinuz-4.18.0-425.3.1.el8.x86_64"
args="ro resume=/dev/mapper/almalinux_gitlab-swap rd.lvm.lv=almalinux_gitlab/root rd.lvm.lv=almalinux_gitlab/swap $tuned_params"
root="/dev/mapper/almalinux_gitlab-root"
initrd="/boot/initramfs-4.18.0-425.3.1.el8.x86_64.img $tuned_initrd"
title="AlmaLinux (4.18.0-425.3.1.el8.x86_64) 8.7 (Stone Smilodon)"
id="0a67983483894675bcb6a100250c18c8-4.18.0-425.3.1.el8.x86_64"
index=1
kernel="/boot/vmlinuz-4.18.0-372.19.1.el8_6.x86_64"
args="ro resume=/dev/mapper/almalinux_gitlab-swap rd.lvm.lv=almalinux_gitlab/root rd.lvm.lv=almalinux_gitlab/swap $tuned_params"
root="/dev/mapper/almalinux_gitlab-root"
initrd="/boot/initramfs-4.18.0-372.19.1.el8_6.x86_64.img $tuned_initrd"
title="AlmaLinux (4.18.0-372.19.1.el8_6.x86_64) 8.6 (Sky Tiger)"
id="0a67983483894675bcb6a100250c18c8-4.18.0-372.19.1.el8_6.x86_64"
index=2
kernel="/boot/vmlinuz-4.18.0-372.9.1.el8.x86_64"
args="ro resume=/dev/mapper/almalinux_gitlab-swap rd.lvm.lv=almalinux_gitlab/root rd.lvm.lv=almalinux_gitlab/swap $tuned_params"
root="/dev/mapper/almalinux_gitlab-root"
initrd="/boot/initramfs-4.18.0-372.9.1.el8.x86_64.img $tuned_initrd"
title="AlmaLinux (4.18.0-372.9.1.el8.x86_64) 8.6 (Sky Tiger)"
id="0a67983483894675bcb6a100250c18c8-4.18.0-372.9.1.el8.x86_64"
index=3
kernel="/boot/vmlinuz-0-rescue-0a67983483894675bcb6a100250c18c8"
args="ro resume=/dev/mapper/almalinux_gitlab-swap rd.lvm.lv=almalinux_gitlab/root rd.lvm.lv=almalinux_gitlab/swap"
root="/dev/mapper/almalinux_gitlab-root"
initrd="/boot/initramfs-0-rescue-0a67983483894675bcb6a100250c18c8.img"
title="AlmaLinux (0-rescue-0a67983483894675bcb6a100250c18c8) 8.5 (Arctic Sphynx)"
id="0a67983483894675bcb6a100250c18c8-0-rescue"

most importantly here’s the path to my default kernel (425):

[root@gitlab ~]# grubby --default-kernel
/boot/vmlinuz-4.18.0-425.3.1.el8.x86_64

so you should be able to add it (copying the args from the current default) and set it to become the new default using:

grubby --add-kernel=/boot/vmlinuz-4.18.0-425.3.1.el8.x86_64 --copy-default --make-default

not sure what causes a yum update not to update the kernel - could be as you’re only doing security updates (not really recommended these days) or maybe as you only have kernel-core and not the kernel metapackage installed (was a “feature” of the earlier minimal iso’s).

grubby is the de-facto in el9, but even in 8 its not a great idea to be using grub2-mkconfig, its still a bit bios/mbr -centric.

3

Do you have link to RHEL docs about that?

4

@sej7278 thanks a lot, but no luck for now, I have the following result for grubby --info=ALL

index=0
kernel="/boot/vmlinuz-4.18.0-372.32.1.el8_6.x86_64"
args="ro crashkernel=auto resume=/dev/mapper/almalinux-swap rd.lvm.lv=almalinux/root rd.lvm.lv=almalinux/swap rhgb quiet $tuned_params"
root="/dev/mapper/almalinux-root"
initrd="/boot/initramfs-4.18.0-372.32.1.el8_6.x86_64.img $tuned_initrd"
title="AlmaLinux (4.18.0-372.32.1.el8_6.x86_64) 8.6 (Sky Tiger)"
id="690ab83ab4c84c2c9f19f6e7eccaea0f-4.18.0-372.32.1.el8_6.x86_64"
index=1
kernel="/boot/vmlinuz-4.18.0-372.26.1.el8_6.x86_64"
args="ro crashkernel=auto resume=/dev/mapper/almalinux-swap rd.lvm.lv=almalinux/root rd.lvm.lv=almalinux/swap rhgb quiet $tuned_params"
root="/dev/mapper/almalinux-root"
initrd="/boot/initramfs-4.18.0-372.26.1.el8_6.x86_64.img $tuned_initrd"
title="AlmaLinux (4.18.0-372.26.1.el8_6.x86_64) 8.6 (Sky Tiger)"
id="690ab83ab4c84c2c9f19f6e7eccaea0f-4.18.0-372.26.1.el8_6.x86_64"
index=2
kernel="/boot/vmlinuz-0-rescue-690ab83ab4c84c2c9f19f6e7eccaea0f"
args="ro crashkernel=auto resume=/dev/mapper/almalinux-swap rd.lvm.lv=almalinux/root rd.lvm.lv=almalinux/swap rhgb quiet"
root="/dev/mapper/almalinux-root"
initrd="/boot/initramfs-0-rescue-690ab83ab4c84c2c9f19f6e7eccaea0f.img"
title="AlmaLinux (0-rescue-690ab83ab4c84c2c9f19f6e7eccaea0f) 8.5 (Arctic Sphynx)"
id="690ab83ab4c84c2c9f19f6e7eccaea0f-0-rescue"

And when I try to install it, it doesn’t find it:

grubby --add-kernel=/boot/vmlinuz-4.18.0-425.3.1.el8.x86_64 --copy-default --make-default
The 4.18.0-425.3.1.el8.x86_64 kernel isn't installed in the machine

Which makes sense to me, because the file is not in /boot:

ll
total 239076
-rw-------. 1 root root  4362237 Sep 13 12:16 System.map-4.18.0-372.26.1.el8_6.x86_64
-rw-------. 1 root root  4362925 Oct 25 12:02 System.map-4.18.0-372.32.1.el8_6.x86_64
-rw-r--r--. 1 root root   196018 Sep 13 12:16 config-4.18.0-372.26.1.el8_6.x86_64
-rw-r--r--. 1 root root   196170 Oct 25 12:02 config-4.18.0-372.32.1.el8_6.x86_64
drwx------. 3 root root     4096 Jan  1  1970 efi
drwx------. 2 root root       37 Jan 10 12:04 grub2
-rw-------. 1 root root 89231037 Dec 20  2021 initramfs-0-rescue-690ab83ab4c84c2c9f19f6e7eccaea0f.img
-rw-------. 1 root root 30305759 Sep 15 12:42 initramfs-4.18.0-372.26.1.el8_6.x86_64.img
-rw-------. 1 root root 27203584 Sep 15 12:46 initramfs-4.18.0-372.26.1.el8_6.x86_64kdump.img
-rw-------. 1 root root 30342764 Oct 29 09:15 initramfs-4.18.0-372.32.1.el8_6.x86_64.img
-rw-------. 1 root root 27272192 Nov 12 09:37 initramfs-4.18.0-372.32.1.el8_6.x86_64kdump.img
drwxr-xr-x. 3 root root       21 Dec 20  2021 loader
lrwxrwxrwx. 1 root root       52 Sep 15 12:42 symvers-4.18.0-372.26.1.el8_6.x86_64.gz -> /lib/modules/4.18.0-372.26.1.el8_6.x86_64/symvers.gz
lrwxrwxrwx. 1 root root       52 Oct 29 09:15 symvers-4.18.0-372.32.1.el8_6.x86_64.gz -> /lib/modules/4.18.0-372.32.1.el8_6.x86_64/symvers.gz
-rwxr-xr-x. 1 root root 10211208 Dec 20  2021 vmlinuz-0-rescue-690ab83ab4c84c2c9f19f6e7eccaea0f
-rwxr-xr-x. 1 root root 10473128 Sep 13 12:16 vmlinuz-4.18.0-372.26.1.el8_6.x86_64
-rwxr-xr-x. 1 root root 10641056 Oct 25 12:03 vmlinuz-4.18.0-372.32.1.el8_6.x86_64

But this is not making any sense if we look at this:

yum check-update
Last metadata expiration check: 0:46:23 ago on Tue Jan 10 17:43:21 2023.
Security: kernel-core-4.18.0-425.3.1.el8.x86_64 is an installed security update
Security: kernel-core-4.18.0-372.32.1.el8_6.x86_64 is the currently running version

I’m scratching my head now, any clue? :blush:

5

I don’t recall seeing such output from yum check-update.

What do you get with:

dnf --enablerepo=* clean all
dnf list kernel\*
6

yum check-update was in my very fist post… I agree, it was (very) long… :wink:

Commands results:

dnf --enablerepo=* clean all
53 files removed

dnf list kernel\ 
Installed Packages
kernel.x86_64                                                                                            4.18.0-372.26.1.el8_6                                                                        @baseos  
kernel.x86_64                                                                                            4.18.0-372.32.1.el8_6                                                                        @baseos  
kernel.x86_64                                                                                            4.18.0-425.3.1.el8                                                                           @baseos  
kernel-core.x86_64                                                                                       4.18.0-372.26.1.el8_6                                                                        @baseos  
kernel-core.x86_64                                                                                       4.18.0-372.32.1.el8_6                                                                        @baseos  
kernel-core.x86_64                                                                                       4.18.0-425.3.1.el8                                                                           @baseos  
kernel-modules.x86_64                                                                                    4.18.0-372.26.1.el8_6                                                                        @baseos  
kernel-modules.x86_64                                                                                    4.18.0-372.32.1.el8_6                                                                        @baseos  
kernel-modules.x86_64                                                                                    4.18.0-425.3.1.el8                                                                           @baseos  
kernel-tools.x86_64                                                                                      4.18.0-425.3.1.el8                                                                           @baseos  
kernel-tools-libs.x86_64                                                                                 4.18.0-425.3.1.el8                                                                           @baseos
7

How about:

dnf reinstall kernel*4.18.0-425.3.1.el8
2 Likes
8

Hi @jlehtone

That did the trick! :ok_hand:
At the end, such a simple solution, but as I’m not that confident with Linux, I prefered to ask for such critical component…

I noticed at download that he downloaded more than already installed. Maybe something went wrong on this one at installation (all my VMs have automatic installation of all patches - not only security ones)

Thanks again! :slightly_smiling_face: