Avoid always generating 0x80 as MSB in primes.

When we ask to generate a prime whose size is a multiple of 8,
the code wrongly computes the mask as 0x00 instead of 0xFF, so the
generated prime always has the MSB set to 0x80 (the highest bit is OR'ed
later in the code).
This commit is contained in:
Giovanni Bajo 2011-09-21 17:27:03 +02:00 committed by Steffen Jaeckel
parent 86ba42797a
commit 08484a1020

View File

@ -35,7 +35,7 @@ int fp_prime_random_ex(fp_int *a, int t, int size, int flags, tfm_prime_callback
}
/* calc the maskAND value for the MSbyte*/
maskAND = 0xFF >> (8 - (size & 7));
maskAND = 0xFF >> ((8 - (size & 7)) & 7);
/* calc the maskOR_msb */
maskOR_msb = 0;