A denial-of-service vulnerability in the IKEv2 key derivation if the openssl plugin is used in FIPS mode and HMAC-MD5 is negotiated as PRF was discovered, all strongSwan versions since 5.0.1 may be affected.
One of our users privately reported a denial-of-service vulnerability in strongSwan. All strongSwan versions since 5.0.1 are potentially affected, depending on the configuration. However, it's important to note that many, if not most, installations won't be affected.
The variable storing the SKEYSEED for IKEv2 key derivation is not initialized before using the negotiated PRF. If setting the PRF's key, or using it fails, an attempt to clear the memory to which the uninitialized variable points may cause a crash. Potentially affected are all strongSwan versions since 5.0.1, depending on the configuration.
CVE-2018-10811 has been assigned for this vulnerability.
The keys for IKEv2 are derived from the DH secret and nonces by applying the negotiated PRF to compute SKEYSEED, which is then itself used as key to the PRF+ to derive all the keys required for the IKE and Child SAs.
The code in question looks like this:
/* SKEYSEED = prf(Ni | Nr, g^ir) */
if (this->prf->set_key(this->prf, fixed_nonce) &&
this->prf->allocate_bytes(this->prf, secret, &skeyseed) &&
this->prf->set_key(this->prf, skeyseed))
{
prf_plus = prf_plus_create(this->prf, TRUE, prf_plus_seed);
}
...
chunk_clear(&skeyseed);
If either set_key() or allocate_bytes() fails, skeyseed will not be initialized and chunk_clear() will attempt to write a block of zero bytes to the uninitialized pointer (of a similarly uninitialized length). This will usually result in a crash.
Versions before 5.0.1 are not affected as the two functions had no return value yet - and even now they are generally not expected to fail. Actually, in our own plugins, allocate_bytes() always initializes the passed variable, so the main focus is set_key().
As its name suggests, it takes the given key and copies it so some internal buffer. Depending on the key size and the PRF this might require padding, or the application of the underlying (hash) algorithm to shorten it. But this generally doesn't fail and in most of our own implementations the involved functions simply return TRUE.
However, when using plugins that rely on third-party libraries, such as the openssl plugin, the two functions may fail depending on the return value of called library functions. It's still not something that is expected to occur in general.
But there is one particular scenario where set_key() fails. And that's when OpenSSL is used in FIPS mode and PRF_HMAC_MD5 is negotiated. The PRF implementation can be instantiated fine, as that only retrieves a static reference to a message digest object, which works in regular and FIPS mode. However, set_key() later fails in FIPS mode due to the call to HMAC_Init_ex(), which calls EVP_DigestInit_ex() and that fails for non-FIPS-approved algorithms like MD5.
Remote code execution is not possible due to this issue.
If OpenSSL is not used in FIPS mode it should not be possible to trigger this issue. And because the called HMAC_Init_ex() function only has a
return value since OpenSSL 1.0, setups that link to older versions won't be directly affected either (even in FIPS mode, although the derivation won't result in valid keys).
strongSwan has removed MD5 from the default proposal with 5.6.1 (as PRF and IKE integrity algorithm), so recent releases are not affected in this scenario; unless the algorithm is explicitly configured in an IKE proposal (but such a configuration would not be FIPS-compliant).
For earlier versions, IKE proposals that don't include MD5 may be configured explicitly to avoid the default proposal and this issue.
Another way to mitigate the problem is to load the test-vectors plugin and enable charon.crypto_test.required and charon.crypto_test.on_add (or .on_create). Since the test vectors won't be successful for algorithms unsupported in FIPS mode, the implementations get disabled and won't be added to the default proposals. And if explicitly configured and negotiated, the key derivation would fail earlier when the PRF is instantiated and no implementation is found (or the test vectors fail, if .on_create is enabled).
Finally, the just released strongSwan 5.6.3 fixes this vulnerability. For older releases we provide patches that fix the vulnerability in the respective versions and should apply with appropriate hunk offsets.