VMS: stricter acquisition of entropy for the pool

Fail harshly (in debug builds) when rand_pool_acquire_entropy isn't
delivering the required amount of entropy.  In release builds, this
produces an error with details.

We also take the opportunity to modernise the types used.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5857)