From 08b3654ac40cf77057fcbba25577d0cc87b91e98 Mon Sep 17 00:00:00 2001 From: Giovanni Bajo Date: Wed, 21 Sep 2011 17:27:03 +0200 Subject: [PATCH] Avoid always generating 0x80 as MSB in primes. When we ask to generate a prime whose size is a multiple of 8, the code wrongly computes the mask as 0x00 instead of 0xFF, so the generated prime always has the MSB set to 0x80 (the highest bit is OR'ed later in the code). --- src/numtheory/fp_prime_random_ex.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/numtheory/fp_prime_random_ex.c b/src/numtheory/fp_prime_random_ex.c index 05fb49a..106d216 100644 --- a/src/numtheory/fp_prime_random_ex.c +++ b/src/numtheory/fp_prime_random_ex.c @@ -35,7 +35,7 @@ int fp_prime_random_ex(fp_int *a, int t, int size, int flags, tfm_prime_callback } /* calc the maskAND value for the MSbyte*/ - maskAND = 0xFF >> (8 - (size & 7)); + maskAND = 0xFF >> ((8 - (size & 7)) & 7); /* calc the maskOR_msb */ maskOR_msb = 0;