I’m using lsyncd to keep folders up to date between servers. This calls upon rsync and thus I’m doing a lot of rsyncs, a couple 1000 per hour. After upgrading from C8 to Almalinux my ram is filling up on destination servers and that eventually crashes the entire server. When i check /proc/meminfo the “percpu” is the one that fills up.
I tried swtiching kernel to elrepo’s and this did not fix the issue. It has to be some lib that was replaced when i converted. I can reproduce this on all servers i update to alma, the more rsyncs i do the faster it fills up.
UPDATE: Script to reproduce the issue:
#! /bin/bash
# Example test for percpu/rsync memleak. Needs two servers, source can be any linux with rsync. Destination with CD or Alma will leak percpu.
# Can reproduce in Cloudlinux and AlmaLinux
# STEP 1: Generate random files for rsync. CD into any empty DIR
workdir=/opt/test3/
cd $workdir
for n in {1..5000}; do
dd if=/dev/urandom of=file$( printf %03d "$n" ).bin bs=1 count=$(( RANDOM + 1024 ))
done
# STEP 2: Generate SSH keys and make sure you can SSH into remote machine (Alma or Cloud)
ls $workdir | xargs -n1 -P10 -I% rsync -Pa % 192.168.0.212:$workdir
# Open new window, check: cat /proc/meminfo | grep Percpu
# running rm -rf on dir with tranfered files will lower Percpu again. Otherwise no other way i know of so far to reduce mem besides reboot. Dropping caches does not work.