All matches were filtered out by modular filtering for argument: httpd-filesystem

Support
1

Yet another issue with dnf module. I am quite stumped with it.
The module is enabled:

# dnf module list httpd
Last metadata expiration check: 0:01:36 ago on Fri 10 Mar 2023 05:46:27 PM EET.
alma8-appstream
Name             Stream          Profiles                                                Summary
httpd              2.4 [d][e]        common [d], devel, minimal         Apache HTTP Server

It does install when I set module_hotfixes=1, but that should not be solution and it will make updating packages a mess, will have to update just httpd separately with this option enabled and rest of the packages without this option.
After installing it seems to be part of httpd:2.4 so why is it filtering it out?

# rpm -qa --queryformat='%{name} %{evr} %{modularitylabel}\n' | grep httpd-filesystem
httpd-filesystem 2.4.37-51.module_el8.7.0+3281+01e58653 httpd:2.4:8070020221010123511:9edba152

According to above it should be installable from httpd:2.4 module.
I have tried reseting module, tried dnf module enable httpd, dnf module installhttpd, dnf clean all, but nothing seems to work.

# dnf install httpd-filesystem -vvvv
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, kpatch, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, versionlock
DNF version: 4.7.0
cachedir: /var/cache/yum/x86_64/8
...
Versionlock plugin: number of lock rules from file "/etc/dnf/plugins/versionlock.list" applied: 8
Excludes in dnf.conf: leapp, leapp-upgrade-el7toel8, python2-leapp, snactor
All matches were filtered out by modular filtering for argument: httpd-filesystem
Error: Unable to find a match: httpd-filesystem

The versionlock does not contain httpd either.

dnf module install httpd output:

# dnf module install httpd
Last metadata expiration check: 0:06:04 ago on Fri 10 Mar 2023 06:32:59 PM EET.
Unable to resolve argument httpd
No match for package httpd
Unable to resolve argument httpd
No match for package httpd-filesystem
Unable to resolve argument httpd
No match for package httpd-tools
Unable to resolve argument httpd
No match for package mod_ssl
Error: Problems in request:
broken groups or modules: httpd, httpd, httpd, httpd
2

We found the cause of the issue in our environments with modules.

Since we have to clone the repositories, then during regular packages cloning process modules metadata file was not cloned and it kept using initial one. It seems the createrepo is unable to generate modules metadata from packages headers. In modules.yaml file new packages versions were not added and it kept only information about the older version belonging to certain module/stream. Newer packages were “orphaned”.