fixed public range check in ec_GF2m_simple_oct2point

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/8607)

(cherry picked from commit cad8347b)