@qralston thanks for bringing up the issue again here since you're right it still is a concern, and for linking with the old bugzilla issue.

We haven't made progress on the switch from MD5 to SHA256 for scan verdict caching but it is an issue in our Jira that was prioritized for investigation for our next release.

I believe the plan is for MD5 to continue to be supported for signatures for compatibility with existing MD5 signatures. I'll talk with the team about either replacing as many MD5 signatures with SHA256 signatures as possible and dropping the rest, or else consider having the clamav build detect fips-mode and use conditional compilation to remove MD5 support for fips-mode builds.

Having the same issue.
It would appear ClamAV will not run on RHEL9 with FIPS enabled.

When I disable FIPS ClamAV will start.

uname -a
Linux rhel9test 5.14.0-70.26.1.el9_0.x86_64 #1 SMP PREEMPT Fri Sep 2 16:07:40 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux

rpm -qa|grep clam
clamav-filesystem-0.103.7-1.el9.noarch
clamav-data-0.103.7-1.el9.noarch
clamav-lib-0.103.7-1.el9.x86_64
clamav-0.103.7-1.el9.x86_64
clamd-0.103.7-1.el9.x86_64

from /var/log/messages:
Oct 24 12:07:44 rhel9test systemd[1]: Starting clamd scanner daemon...
Oct 24 12:07:45 rhel9test clamd[46661]: LibClamAV Error: Can't load /var/lib/clamav/daily.cvd: Can't allocate memory
Oct 24 12:07:45 rhel9test clamd[46661]: LibClamAV Error: cli_loaddbdir(): error loading database /var/lib/clamav/daily.cvd
Oct 24 12:07:45 rhel9test clamd[46661]: ERROR: Can't allocate memory
Oct 24 12:07:45 rhel9test systemd[1]: [email protected]: Control process exited, code=exited, status=1/FAILURE
Oct 24 12:07:45 rhel9test systemd[1]: [email protected]: Failed with result 'exit-code'.
Oct 24 12:07:45 rhel9test systemd[1]: Failed to start clamd scanner daemon.

@micahsnyder another possibility would be to add one of the available public domain implementations of MD5 (e.g. Solar Designer’s implementation) to ClamAV and call it instead of the OpenSSL implementation if the system is running in FIPS mode.

The problem isn’t using MD5 per se; the problem is asking OpenSSL to perform MD5 when the system is in FIPS mode. ClamAV doesn’t use MD5 in a cryptographic context, but OpenSSL has no way to know that: it must deny all uses of MD5 if the system is in FIPS mode, in order to guarantee that no caller is using MD5 in a cryptographic context.

@micahsnyder: It appears to me that you may be misunderstanding the issue here, possibly due to conflicting uses of the term "signature".

Please correct me if I'm wrong, but it sounds like you are using the word "signature" to refer to checksums that are used to compare scanned files against a list of known safe/malicious files.

FIPS does NOT prohibit the use of MD5 hashes for that purpose. Those "signatures" do NOT need to be replaced in order to resolve this issue.

I'm not confident that I have a full picture of how the scan verdict caching works and is used, however at first glance it looks to me like MD5 should be OK here too. I do NOT think these checksums need to be replaced in order to resolve this issue either.

The real problem is the MD5 checksum and the associated RSA digital signature that are used to verify the integrity and publishing source of the CVD file itself: https://github.com/Cisco-Talos/clamav/blob/main/libclamav/cvd.c#L571
(The term "signature" as it relates to FIPS would refer to something like this RSA digital signature, hence why this "signature" is relevant but the above "signatures" are not.)

Both because of the security context here (depending on the use case, this may be the sole mechanism for verifying that the CVD file has not been tampered with during distribution through insecure distribution channels), and because of the use of an RSA digital signature (which falls under the purview of FIPS), this MD5 checksum does need to be replaced in order to resolve this issue.
It looks like the CVD file format will need to be modified or extended to support a SHA256 checksum and signature in the CVD file header for verification of the CVD file itself.

Are there any plans to address this specific checksum and digital signature in the CVD file header?

@qralston: OpenSSL does have a way to know whether you are using MD5 in a non-cryptographic context: the EVP_MD_CTX_FLAG_NON_FIPS_ALLOW flag. There is no need to replace the MD5 implementation with a different library, just set that flag whenever you are using MD5 for a non-crypto purpose.

clamav is already doing this in some places, like here: https://github.com/Cisco-Talos/clamav/blob/main/libclamav/crypto.c#L191

The problem is that it would NOT be appropriate to set this flag when validating the CVD file itself.

Note that clamav already appears to be setting this flag in the code paths for the file "signature" checks and scan verdict caching, which reinforces my statements above that those uses of MD5 do not need to be replaced. However, since it (appropriately) does not set this flag when validating the CVD file itself, freshclam/clamd/clamav currently all fail when loading/verifying the CVD file.

@PaulSD Thank you for the detailed explanation. I was indeed confused. This helps! I will talk with the team about it.

@PaulSD wrote:

OpenSSL does have a way to know whether you are using MD5 in a non-cryptographic context: the EVP_MD_CTX_FLAG_NON_FIPS_ALLOW flag. There is no need to replace the MD5 implementation with a different library, just set that flag whenever you are using MD5 for a non-crypto purpose.

clamav is already doing this in some places, like here: https://github.com/Cisco-Talos/clamav/blob/main/libclamav/crypto.c#L191

The EVP_MD_CTX_FLAG_NON_FIPS_ALLOW flag was removed in OpenSSL 3.0; see line 218 of OpenSSL’s include/openssl/evp.h. So this work-around will fail with OpenSSL 3.0 and greater.

If the system was not booted in FIPS mode, even if the system’s default OpenSSL 3.0 configuration only loads the fips provider, you should be able to explicitly load the default and/or legacy providers.

But per my experience with attempting to connect to a Wi-Fi network on a RHEL9 / OpenSSL 3.0 laptop booted in FIPS mode, attempts to call cryptographic functions that are not FIPS-validated will fail when the system has been booted in FIPS mode, regardless of what providers you explicitly load. Because that is the entire point of booting a Linux system in FIPS mode: to ensure that all cryptographic providers refuse to provide cryptographic functions that are not FIPS-validated.

The bottom line is that in 2023, you must assume that requesting any of (OpenSSL, GnuTLS, NSS, the Linux kernel) to perform legacy cryptographic functions (MD5, SHA-1, et. al.) will fail, with no possibility of avoiding that failure. Migrating clamav to using SHA-2 or (preferably) SHA-3 hash functions, and removing any/all uses of MD5 or SHA-1, should be a priority for clamav development.

But as per the hostap mailing list thread (linked above), if you cannot immediately rewrite your code so that it no longer needs to call legacy cryptographic functions, the best stopgap measure is to provide internal implementations of those functions, a la hostap.

Finally, I agree with @PaulSD that it is inappropriate to use MD5 to validate the digital signature of a CVD file. MD5 is so hopelessly broken that an attacker with even moderate resources (e.g., an AI/ML system) would likely be able to forge CVD signatures.

The EVP_MD_CTX_FLAG_NON_FIPS_ALLOW flag was removed in OpenSSL 3.0

Good point; A lot has changed in OpenSSL 3, and ClamAV has not yet been updated to work around FIPS with OpenSSL 3. In order to resolve this issue on systems using OpenSSL 3, some code updates will also be needed, although I think those will be relatively easy/minor (some details are below).

However, I have to disagree with a couple of your other statements.

But first, let me start with some additional details on OpenSSL 3:

• A prerequisite to using MD5 in OpenSSL 3 is that the default provider must be loaded.
  • On RHEL9 systems, the default provider is loaded by default regardless of whether FIPS mode is enabled or not. This is configured in /etc/pki/tls/openssl.cnf. (The user can disable the default provider entirely by editing or overriding openssl.cnf, but RHEL9 has it enabled by default.)
  • When FIPS mode is enabled (if either the OPENSSL_FORCE_FIPS_MODE env var or /proc/sys/crypto/fips_enabled is set), RHEL9 systems automatically load the base and fips providers (in addition to any providers configured in /etc/pki/tls/openssl.cnf). This enables FIPS mode to be enabled/disabled globally without making any changes to openssl.cnf files.
• EVP_MD_CTX_FLAG_NON_FIPS_ALLOW technically still exists in OpenSSL 3 (#ifdef will find it). However, the flag is ignored in OpenSSL 3, so setting it does not permit you to use MD5 in FIPS mode.
• OpenSSL 3 supports using "properties" to select/filter algorithms, and this is used to control the FIPS restrictions on RHEL9 (replacing EVP_MD_CTX_FLAG_NON_FIPS_ALLOW among other things).
  • Users can configure default_properties in openssl.cnf. Applications can then override those default properties if needed.
  • A provider=fips property may be used to limit access to only the algorithms provided by the fips provider. Alternatively, fips=yes may be used to limit access to algorithms in any provider that are marked as "FIPS algorithms" (the docs state that there are some non-crypto FIPS algorithms in the default and base providers that match fips=yes but not provider=fips).
  • When FIPS mode is enabled (determined as described above), RHEL9 automatically adds fips=yes to default_properties. Again, this enables FIPS mode to be enabled/disabled globally without making any changes to openssl.cnf files.
  • ClamAV can override this default property (when appropriate) by calling EVP_MD_fetch(NULL, "MD5", "-fips") instead of EVP_get_digestbyname("MD5").

To show this in action:

$ cat /etc/redhat-release 
Red Hat Enterprise Linux release 9.1 (Plow)
$ fips-mode-setup --check
FIPS mode is enabled.
$ echo | openssl dgst -md5
Error setting digest
801B786FE57F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:349:Global default library context, Algorithm (MD5 : 97), Properties ()
801B786FE57F0000:error:03000086:digital envelope routines:evp_md_init_internal:initialization error:crypto/evp/digest.c:237:
$ echo | openssl dgst -md5 -propquery '-fips'
MD5(stdin)= 68b329da9893e34099c7d8ad5cb9c940
$

To dispute some statements:

attempts to call cryptographic functions that are not FIPS-validated will fail when the system has been booted in FIPS mode, regardless of what providers you explicitly load

As described above, this is due to the fips=yes "property", and it can still be overridden (when appropriate) at the application level, just using a different method/flag than was used in OpenSSL <3.

That said, I do agree with this statement: that is the entire point of booting a Linux system in FIPS mode: to ensure that all cryptographic providers refuse to provide cryptographic functions that are not FIPS-validated.
Applications MUST be careful to override FIPS ONLY when performing operations that are not security-sensitive, otherwise they would be defeating the point of FIPS mode, at which point it would be better to just let FIPS mode break things so that users are forced to acknowledge that the software is not FIPS compliant and deal with it accordingly (eg. by disabling FIPS mode, or finding alternative solutions or mitigations, depending on the user's use case and risk profile).

the best stopgap measure is to provide internal implementations of those functions, a la hostap

I would argue that implementing your own crypto (even legacy crypto) is never a good idea, and internalizing or statically linking a random non-FIPS-compliant library/implementation just to work around FIPS issues isn't much better.
My understanding is that hostap provides its own internal implementations primarily to support unit testing, not for use in production systems.
I really think that the better stopgap is to jump through whatever hoops are necessary to allow legacy algorithms in a mainstream library to continue to be used.

Migrating clamav to using SHA-2 or (preferably) SHA-3 hash functions, and removing any/all uses of MD5 or SHA-1, should be a priority for clamav development.

I agree that modern hash functions are always preferable to obsolete ones, and it would be nice if all uses of MD5 in ClamAV could be replaced. However, let's not make perfect the enemy of good.
For example, I think using MD5 to match scanned files against a list of known bad files is perfectly reasonable. This is clearly out of the scope of FIPS, and the worst case is ... what? Maybe someone manages to generate a non-malicious file that matches as a false positive?
Conversely, using MD5 to match scanned files against a list of known good files is less safe. That is still clearly out of the scope of FIPS, but the worst case is someone manages to generate a malicious file that matches as a false negative.
However, even in that case, if I had to choose between the ClamAV team spending a lot of time replacing this use of MD5, or spending the same amount of time significantly improving/extending other aspects of the ClamAV scanning capabilities, I might choose the latter.
For now, I think it's worth focusing on the parts of ClamAV that are clearly both security sensitive and non-FIPS-compliant.

This is preventing me from using ClamAV. What is a workaround? Can I disable signature checking in freshclam?

We are using RHEL 9 and we have the SAME issue. How can we disable the signature checking or get it working with openssl 3.

@breisig to make it just work you have to change libclamav/crypto.c and replace 7 occurrences of

md = EVP_get_digestbyname(alg);

with:

md = EVP_MD_fetch(NULL, alg, "-fips");

Recompile ClamAV and then it works on FIPS enabled system, see also: #959

Thank you @barrydegraaff , Will this fix ever get merged before the next clamav release?

Thank you @barrydegraaff , Will this fix ever get merged before the next clamav release?

The core issue here is that because ClamAV uses MD5 for cryptographic functions, ClamAV is not FIPS-compliant. Since the entire purpose of running a Linux system in FIPS mode is to ensure that only FIPS-validated cryptography is used, ClamAV should fail to work on a system that runs in FIPS mode.

What #959 does is modify ClamAV to represent that all uses of MD5 occur in non-cryptographic contexts (by banning the FIPS provider when fetching an MD5 implementation). This is a chicanery. Metaphorically speaking, this is avoiding being ticketed by a red light camera by spray-painting over the camera, when the correct solution is to stop running red lights.

Thus, #959 must be rejected.

The only correct solution here is for ClamAV to abandon the use of MD5, SHA-1, and all deprecated cryptographic algorithms in favor of modern cryptographic algorithms.

I know that is nontrivial, but as I said in my opening comment, an increasing number of security hardening standards (e.g., DISA STIGs) require FIPS mode. Many of those same standards also require performing malicious code detection. As grateful as we are for ClamAV and to the ClamAV developers, MD5 has been hopelessly broken since 2008; using it in 2023, in any context, is a giant red flag that the software in question does not take security seriously.

@qralston is correct. Sorry @breisig - #959 is not the solution.

What I think we really need to do is switch over the CVD signing and verification logic from MD5 -> SHA256. In the public clamav code that's mostly

We would also probably want to change the hash variable name from md5 -> sha256 in the public API:

And then change our internal signing tool to use sha256 as well.

After that, we'll have to start building and distributing daily.cvd2 or something in addition to the old daily.cvd for older versions, where the .cvd2 version uses the SHA256 algorithm.

We'll have to do the same with a .cdiff2 format. While we're at it, there's a possibility we find a way to make CDIFF patches include a copy of the signature for how the CVD will look after applying a patch. If that's really possible, then we could potentially get rid of the CLD database, which would mean that freshclam could be used by itself to collect all the files required for hosting a private mirror and we wouldn't need cvdupdate anymore. I would want to make this change if we were going to do breaking change to the CVD and CDIFF formats.

In addition to this, we're also making an effort to swap from MD5 to SHA256 for our clean file cache. We would also then replace all MD5 signatures for the newer clamav versions with SHA256 signatures. This last item is already on our roadmap.

Is their any update on this?

No news.

I hate to add to the comments of people asking for an update, but figured I would check in since it's been a couple months since the last post. Assuming no new updates to this issue?

Same issue here. Subscribing to watch for updates and hopefully a resolution.

Any updates on this issue?

No news.

Monthly check-in to see if there might have been some progress here.

true FIPS support should be a priority for the clamav developers. Especially after waiting all this time.

My team is going to have to switch to an ugly alternative soon because of this problem.

Has there been any progress on resolving this issue?

I second this. It needs to be addressed ASAP

It is helpful to see pings on this issue to gauge user interest. I am not going to reply to every monthly ping, however.

This request is on our radar and frequently discussed during roadmap planning. But we have limited resources, particularly now. And this is not the only high priority task that we're being pressed for.

I want to resolve this issue, but I cannot promise a timeline.

Here's my ping for real FIPS support. :-)

Thank you for the response. What resources do you need for this task?

Developer headcount, mostly.

Off the top of my head...

We need research and planning to identify a modern mechanism to sign clamav database archives. Previously I suggested simply switching from MD5 to SHA256. We've talked about it more internally and we'd like to do better. We know we would like to have add support for rotating the signing certificate, which implies support for a root certificate and maybe intermediate certificates. But do we want to use GPG? PKCS? Something else? How would we deploy new public keys? How would we revoke compromised signing certs?

Ideally we would add the new CVD2 signing and extraction code in Rust. We likely want to backport it to previous releases, however, so we can get people off of the old CVD format.

Another thing to consider is where to store the public key for the signing certs. Presumably we'd bundle the latest public key with ClamAV and then add some mechanism to rotate it through a signature update. Does that mean we need to provide a tiny CVD2 in our git repo that contains the latest signing public key? I'm not sure. Is it even a good idea to store public keys for verifying CVD2's in the same directory? If not, why not? And what is the alternative? We'll also need to write up documentation on how this works, how to make and maintain your own signature archive signing certificates so folks like SaneSecurity could deploy their own signed CVD2 databases. Would this be meaningful to third party signature authors? Or would we be adding complexity for no reason?

Something else to consider... ClamAV has a CVD patching feature called CDIFF that extracts a CVD, applies specific line changes to the contained files, and then packages it into a CLD. CDIFF patches are also signed, so we'll certainly have to make a CDIFF2 format.

So relating to CDIFF2 patch files, here's it would be really neat if we could bundle the signature for the next CVD2 with a the CDIFF2 patch such that after applying the patch and re-compressing the signature archive, the CVD2 signature is a match. The CDIFF2 patches would have to be perfect, and rely on compression being 100% reproducible. If we could pull this off, then Freshclam could be used to provide all the files necessary to host a private mirror without downloading a whole CVD2 archive every day. This would reduce traffic against the CDN, and we could deprecate the CVDUpdate program. This could also maybe get rid of the CLD files which would reduce confusion from users who are like "Where my CVD go? 😢" But this isn't a blocker.

After we finish the implementation for the new CVD2 / CDIFF2, which includes both signing and verification, we'll need to make a bunch of changes to our internal processes to build both the old CVD / CDIFF and new CVD2 / CDIFF2 files. Changes will also be required to devops scripts to deploy them, and to Cloudflare CDN configuration to host them.

In short, ... it's a lot. We could use help with recommendations around CVD2 / CDIFF2 design along with a security review of proposed solutions. I'd be open to help with implementation, though I doubt anyone from the community with the required expertise has the time to take on a task of of this size.

Thank you for the response. What resources do you need for this task?

Developer headcount, mostly.

Off the top of my head...

We need research and planning to identify a modern mechanism to sign clamav database archives. Previously I suggested simply switching from MD5 to SHA256. We've talked about it more internally and we'd like to do better. We know we would like to have add support for rotating the signing certificate, which implies support for a root certificate and maybe intermediate certificates. But do we want to use GPG? PKCS? Something else? How would we deploy new public keys? How would we revoke compromised signing certs?

Ideally we would add the new CVD2 signing and extraction code in Rust. We likely want to backport it to previous releases, however, so we can get people off of the old CVD format.

Another thing to consider is where to store the public key for the signing certs. Presumably we'd bundle the latest public key with ClamAV and then add some mechanism to rotate it through a signature update. Does that mean we need to provide a tiny CVD2 in our git repo that contains the latest signing public key? I'm not sure. Is it even a good idea to store public keys for verifying CVD2's in the same directory? If not, why not? And what is the alternative? We'll also need to write up documentation on how this works, how to make and maintain your own signature archive signing certificates so folks like SaneSecurity could deploy their own signed CVD2 databases. Would this be meaningful to third party signature authors? Or would we be adding complexity for no reason?

Just having this TODO list is a good start.
Have other projects made similar transitions? Surely by 2024 this can't be the first time this has happened.

Interesting, I have not experience this. As far as other projects go I have worked with gitlab, ruby bundle audit, and most recently verdaccio.

Reading the comments here and the fixes proposed, it feels possible that we're making perfection the enemy of the good.

While the outline from @micahsnyder gives the long term roadmap, if patch #959 (rejected by @qralston) were instead to be wrapped up inside of a configuration or runtime option, then this would at least allow deliberate execution of Clam inside of a FIPS environment. This option would be disabled by default, but then a user could deliberately allow ClamAV to run in their FIPS environment.

It seems like a logical compromise to have a short win for FIPS users while still allowing a long term path to the most correct FIPS compliant solution.

Ran into same issue testing FIPS on Ubuntu 22.04 which is nearing validation for FIPS 140-3 standard which I can see a lot of orgs moving to from older distributions.

Updated pull request at #1344 that addresses all this issue and other FIPS compliance issues around database signatures. It should be a complete patch for external, extensible signatures going forward.

Just wanted to add my 2 cents that we are also running into this issue on RHEL 9 with FIPS enabled (to show more interest in a fix). Any ETA on a fix?

Our current plan is to fix this with the ClamAV 1.5 release end of this year.

Is 1.5 planned to be the next LTS version?

Yes

Hi @micahsnyder is there a tentative release date for the 1.5 version? (FIPS compliant)

@itsmelodewyk We'd been hoping to release late this month, but we'll be late. I'm still actively working on the implementation for the new FIPS compliant CVD signing mechanism. Then it will need to be reviewed and tested. After merge, we'll create the release candidate. That should last ~3 weeks before stable release. In short, it may be as early as mid-December or as late as mid-January.