HTTPD Segfault issue in Alma Linux Environment

Community
1

Hi Experts,

We are using below HTTPD - RPMs in AlmaLinux 8.5/8.6 and facing HTTPD “Segfault” issue and httpd is crashing. One of the dependent pkg of httpd is mod_proxy. Herewith, attaching the httpd configuration and tools that we are using which is supported by “httpd”.

$ rpm -qa|egrep -i “http|mod_proxy” ------------------------------------------------------ > these are alma linux 8.5

collectd-write_http-5.11.0-1.el8.x86_64

httpd-2.4.37-47.module_el8.6.0+2872+fe0ff7aa.1.alma.x86_64

perl-HTTP-Tiny-0.074-1.el8.noarch

httpd-tools-2.4.37-47.module_el8.6.0+2872+fe0ff7aa.1.alma.x86_64

mod_proxy_html-2.4.37-47.module_el8.6.0+2872+fe0ff7aa.1.alma.x86_64

libnghttp2-1.33.0-3.el8_2.1.x86_64

almalinux-logos-httpd-84.5-1.el8.noarch

httpd-filesystem-2.4.37-47.module_el8.6.0+2872+fe0ff7aa.1.alma.noarch

http-parser-2.8.0-9.el8.x86_64

mod_http2-1.15.7-5.module_el8.6.0+2872+fe0ff7aa.x86_64

$ rpm -qa|egrep -i “http|mod_proxy”------------------------------------------------------ > these are alma linux 8.6

collectd-write_http-5.11.0-1.el8.x86_64

httpd-tools-2.4.37-47.module_el8.6.0+2935+fb177b09.2.x86_64

perl-HTTP-Tiny-0.074-1.el8.noarch

libnghttp2-1.33.0-3.el8_2.1.x86_64

almalinux-logos-httpd-84.5-1.el8.noarch

httpd-filesystem-2.4.37-47.module_el8.6.0+2935+fb177b09.2.noarch

mod_proxy_html-2.4.37-47.module_el8.6.0+2935+fb177b09.2.x86_64

httpd-2.4.37-47.module_el8.6.0+2935+fb177b09.2.x86_64

http-parser-2.8.0-9.el8.x86_64

mod_http2-1.15.7-5.module_el8.6.0+2872+fe0ff7aa.x86_64

We are using a httpd service reload/restart during “log-rotate” in every hour.

“/var/log/messages” error log :

2022-08-01T15:00:01.627582+00:00 kernel: [ 7856.564252] httpd[1488114]: segfault at 0 ip 00007f60f0ca5bc6 sp 00007ffdb9b3f9e0 error 4 in libc-2.28.so[7f60f0c20000+1bc000]

2022-08-01T16:00:01.797314+00:00 kernel: [11456.737690] httpd[2849302]: segfault at 0 ip 00007f60f0ca5bc6 sp 00007ffdb9b3f9e0 error 4 in libc-2.28.so[7f60f0c20000+1bc000]

2022-08-02T01:00:01.517464+00:00 kernel: [43856.485259] httpd[2481909]: segfault at 0 ip 00007f60f0ca5bc6 sp 00007ffdb9b3f9e0 error 4 in libc-2.28.so[7f60f0c20000+1bc000]

2022-08-02T02:00:01.630191+00:00 kernel: [47456.603374] httpd[3831320]: segfault at 0 ip 00007f60f0ca5bc6 sp 00007ffdb9b3f9e0 error 4 in libc-2.28.so[7f60f0c20000+1bc000]

2022-08-02T02:00:01.660156+00:00 kernel: [47456.634199] httpd[3831321]: segfault at 0 ip 00007f60f0ca5bc6 sp 00007ffdb9b3f9e0 error 4 in libc-2.28.so[7f60f0c20000+1bc000]

2022-08-02T03:00:01.498163+00:00 kernel: [51056.474481] httpd[996752]: segfault at 0 ip 00007f60f0ca5bc6 sp 00007ffdb9b3f9e0 error 4 in libc-2.28.so[7f60f0c20000+1bc000]

We have reached out to Redhat on the same query through “Support”, but going over SOS report, they denied support claiming they don’t handle AlmaLinux !!!

We request you kind support on it as our release got halted due to this httpd seg fault issue

Thanks,
Kiran

2

Hi Team,

Could you please help on this query.

Do you think moving to Alma Linux 9.x could address it ? If so, what could be the reason for the same ?

Thanks,
Kiran

3

error 4 is “user-mode read of a page not found”, so i’d suspect something wrong with how you’re rotating the apache logs e.g. you’re compressing huge logs and its taking too long or reloading apache before rotating logs instead of after.

are you using logrotate with the default httpd rotation script? the timing (on the hour) backs that theory up.

does httpd start/restart ok manually, is it just when done via logrotate?

i’d try valgrind logrotate -v /etc/logrotate.d/httpd and see what that says, or post your rotation script.

maybe try disabling some apache modules e.g. mod_proxy to see if that avoids the issue.

if you have support with redhat then build a licensed rhel 8.6 vm with the same setup and see if you can reproduce - and get support that way. they’re never going to support almalinux.