Test: use keywords instead of magic numbers for 'rsa_pss_saltlen'

Since commit 137096a7

 it is possible to specify keywords instead
of negative magic numbers for the salt length. This commit replaces
the remaining occurrences of `rsa_pss_saltlen:-3` in the test recipes
by `rsa_pss_saltlen:max`.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8860)