Commit 8b37e5c1 authored by Emilia Kasper's avatar Emilia Kasper
Browse files

Fix undefined behaviour in shifts.



Td4 and Te4 are arrays of u8. A u8 << int promotes the u8 to an int first then shifts.
If the mathematical result of a shift (as modelled by lhs * 2^{rhs}) is not representable
in an integer, behaviour is undefined. In other words, you can't shift into the sign bit
of a signed integer. Fix this by casting to u32 whenever we're shifting left by 24.

(For consistency, cast other shifts, too.)

Caught by -fsanitize=shift

Submitted by Nick Lewycky (Google)

Reviewed-by: default avatarAndy Polyakov <appro@openssl.org>
parent 3d6aa6d4
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment