Avoid always generating 0x80 as MSB in primes.

When we ask to generate a prime whose size is a multiple of 8,
the code wrongly computes the mask as 0x00 instead of 0xFF, so the
generated prime always has the MSB set to 0x80 (the highest bit is OR'ed
later in the code).
This commit is contained in:
Giovanni Bajo 2011-09-21 17:27:03 +02:00
parent da5fa59f2c
commit 08b3654ac4

View File

@ -35,7 +35,7 @@ int fp_prime_random_ex(fp_int *a, int t, int size, int flags, tfm_prime_callback
}
/* calc the maskAND value for the MSbyte*/
maskAND = 0xFF >> (8 - (size & 7));
maskAND = 0xFF >> ((8 - (size & 7)) & 7);
/* calc the maskOR_msb */
maskOR_msb = 0;