test: Enable specifying flaky tests on fips

Adds a way to mark a specified test as 'flaky' on fips compliant
systems.

Earlier, the ``tools/test.py`` script supported only 'mode',
'system' and 'arch' for test environment specification. This limits the
ability to specify the behavior of tests and setting pre-determined
behavior of the same on other types of systems. As an example, the
feature request below indicates the need to specify certain tests as
'flaky' on fips compliant systems. It hints at future possibility of a
shared library, which in turn may need a specifier for running tests.

This commit introduces a new item in the ``env`` dict, called ``type``
which defaults to ``simple`` type. It also adds an optional command
line argument ``--type``, which inputs strings. Current functionality
extends to setting ``simple`` or ``fips`` for this ``type`` variable.
However, extending it to further uses is rather simple by adding "if"
conditions at appropriate places in the ``tools/test.py`` script.

PR-URL: https://github.com/nodejs/node/pull/16329
Fixes: https://github.com/nodejs/node/issues/14746
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
This commit is contained in:
Nikhil Komawar 2017-10-16 17:39:30 -04:00 committed by Anna Henningsen
parent e8a26e783e
commit f31cf56972
No known key found for this signature in database
GPG Key ID: 9C63F3A6CD2AD8F9

View File

@ -1410,6 +1410,9 @@ def BuildOptions():
result.add_option('--abort-on-timeout',
help='Send SIGABRT instead of SIGTERM to kill processes that time out',
default=False, action="store_true", dest="abort_on_timeout")
result.add_option("--type",
help="Type of build (simple, fips)",
default=None)
return result
@ -1559,6 +1562,21 @@ def ArgsToTestPaths(test_root, args, suites):
return paths
def get_env_type(vm, options_type):
if options_type is not None:
env_type = options_type
else:
if "fips" in subprocess.check_output([vm, "-p",
"process.versions.openssl"]):
env_type = "fips"
# NOTE(nikhil): "simple" is the default value for var 'env_type' and should
# be set last if no if/elif matches. If you plan to add more values, use
# 'elif' above.
else:
env_type = "simple"
return env_type
def Main():
parser = BuildOptions()
(options, args) = parser.parse_args()
@ -1641,6 +1659,7 @@ def Main():
'mode': mode,
'system': utils.GuessOS(),
'arch': vmArch,
'type': get_env_type(vm, options.type),
}
test_list = root.ListTests([], path, context, arch, mode)
unclassified_tests += test_list