A denial-of-service vulnerability in the parser for RSASSA-PSS signatures was discovered in strongSwan 5.6.1.
Our fuzzer on Google's OSS-Fuzz infrastructure revealed a bug in the parser for PKCS#1 RSASSA-PSS signature parameters introduced with 5.6.1 that may lead to a denial-of-service attack.
Incorrectly encoded or crafted RSASSA-PSS signature values may cause a crash while parsing. Potential triggers are signatures in certificates, but also signatures used for IKEv2 signature authentication (RFC 7427). Affected is strongSwan 5.6.1.
CVE-2018-6459 has been assigned for this vulnerability.
ASN.1 encoded algorithm identifier structures for RSASSA-PSS signatures (RFC 8017) contain parameters that specify details, like hash algorithms and the salt length, used to create/verify the signatures. One of the configurable parameters is the mask generation function (MGF). Currently, only MGF1 is specified for this purpose. However, this in turn takes itself a parameter that specifies the underlying hash function. strongSwan's parser did not correctly handle the case of this parameter being absent, causing an undefined data read that results in a potential denial-of-service vulnerability. Whether it actually causes a crash depends on the data on the stack at the time of parsing the MGF1 algorithm identifier.
Remote code execution is not possible due to this issue.
Credit to OSS-Fuzz for finding this vulnerability.
The just released strongSwan 5.6.2 fixes this vulnerability. For 5.6.1 we also provide a patch that fixes it.