bn/bn_mont.c: move boundary condition check closer to caller.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: David Benjamin <davidben@google.com>
(Merged from https://github.com/openssl/openssl/pull/6662)

(cherry picked from commit 3c97e412)