Some code-cleanup/simplification in apps

Remove some casts on password callback by adding a wrapper function.
Remove level of indent by doing an early-return on failure.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7873)