[ruby/openssl] Fix modular square root test with LibreSSL >= 3.8

If x is a modular square root of a (mod p) then so is (p - x). Both
answers are valid. In particular, both 2 and 3 are valid square roots
of 4 (mod 5). Do not assume that a particular square root is chosen by
the algorithm. Indeed, the algorithm in OpenSSL and LibreSSL <= 3.7
returns a non-deterministic answer in many cases. LibreSSL 3.8 and
later will always return the smaller of the two possible answers. This
breaks the current test case.

Instead of checking for a particular square root, check that the square
of the claimed square root is the given value. This is always true. Add
the simplest test case where the answer is indeed non-deterministic.

https://github.com/ruby/openssl/commit/93548ae959
This commit is contained in:
Theo Buehler 2023-04-11 19:43:49 +02:00 committed by Kazuki Yamaguchi
parent 1d0a087d45
commit bc0539a9b7

View File

@ -175,7 +175,9 @@ class OpenSSL::TestBN < OpenSSL::TestCase
end
def test_mod_sqrt
assert_equal(3, 4.to_bn.mod_sqrt(5))
assert_equal(4, 4.to_bn.mod_sqrt(5).mod_sqr(5))
# One of 189484 or 326277 is returned as a square root of 2 (mod 515761).
assert_equal(2, 2.to_bn.mod_sqrt(515761).mod_sqr(515761))
assert_equal(0, 5.to_bn.mod_sqrt(5))
assert_raise(OpenSSL::BNError) { 3.to_bn.mod_sqrt(5) }
end