policy: add dependencies map for resources

Adds a "dependencies" field to resources in policy manifest files.
In order to ease development and testing while using manifests,
wildcard values for both "dependencies" and "integrity" have been
added using the boolean value "true" in the policy manifest.

PR-URL: https://github.com/nodejs/node/pull/28767
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
This commit is contained in:
Bradley Farias 2019-07-18 09:01:59 -05:00
parent 499533f72a
commit 7e8ad9bad8
19 changed files with 413 additions and 65 deletions

View File

@ -1432,6 +1432,13 @@ An attempt was made to load a resource, but the resource did not match the
integrity defined by the policy manifest. See the documentation for [policy]
manifests for more information.
<a id="ERR_MANIFEST_DEPENDENCY_MISSING"></a>
### ERR_MANIFEST_DEPENDENCY_MISSING
An attempt was made to load a resource, but the resource was not listed as a
dependency from the location that attempted to load it. See the documentation
for [policy] manifests for more information.
<a id="ERR_MANIFEST_INTEGRITY_MISMATCH"></a>
### ERR_MANIFEST_INTEGRITY_MISMATCH
@ -1440,6 +1447,13 @@ entries for a resource which did not match each other. Update the manifest
entries to match in order to resolve this error. See the documentation for
[policy] manifests for more information.
<a id="ERR_MANIFEST_INVALID_RESOURCE_FIELD"></a>
### ERR_MANIFEST_INVALID_RESOURCE_FIELD
A policy manifest resource had an invalid value for one of its fields. Update
the manifest entry to match in order to resolve this error. See the
documentation for [policy] manifests for more information.
<a id="ERR_MANIFEST_PARSE_POLICY"></a>
### ERR_MANIFEST_PARSE_POLICY

View File

@ -38,7 +38,7 @@ node --experimental-policy=policy.json app.js
The policy manifest will be used to enforce constraints on code loaded by
Node.js.
In order to mitigate tampering with policy files on disk, an integrity for
To mitigate tampering with policy files on disk, an integrity for
the policy file itself may be provided via `--policy-integrity`.
This allows running `node` and asserting the policy file contents
even if the file is changed on disk.
@ -105,9 +105,83 @@ When loading resources the entire URL must match including search parameters
and hash fragment. `./a.js?b` will not be used when attempting to load
`./a.js` and vice versa.
In order to generate integrity strings, a script such as
To generate integrity strings, a script such as
`printf "sha384-$(cat checked.js | openssl dgst -sha384 -binary | base64)"`
can be used.
Integrity can be specified as the boolean value `true` to accept any
body for the resource which can be useful for local development. It is not
recommended in production since it would allow unexpected alteration of
resources to be considered valid.
### Dependency Redirection
An application may need to ship patched versions of modules or to prevent
modules from allowing all modules access to all other modules. Redirection
can be used by intercepting attempts to load the modules wishing to be
replaced.
```json
{
"builtins": [],
"resources": {
"./app/checked.js": {
"dependencies": {
"fs": true,
"os": "./app/node_modules/alt-os"
}
}
}
}
```
The dependencies are keyed by the requested string specifier and have values
of either `true` or a string pointing to a module that will be resolved.
The specifier string does not perform any searching and must match exactly
what is provided to the `require()`. Therefore, multiple specifiers may be
needed in the policy if `require()` uses multiple different strings to point
to the same module (such as excluding the extension).
If the value of the redirection is `true` the default searching algorithms will
be used to find the module.
If the value of the redirection is a string, it will be resolved relative to
the manifest and then immediately be used without searching.
Any specifier string that is `require()`ed and not listed in the dependencies
will result in an error according to the policy.
Redirection will not prevent access to APIs through means such as direct access
to `require.cache` and/or through `module.constructor` which allow access to
loading modules. Policy redirection only affect specifiers to `require()`.
Other means such as to prevent undesired access to APIs through variables are
necessary to lock down that path of loading modules.
A boolean value of `true` for the dependencies map can be specified to allow a
module to load any specifier without redirection. This can be useful for local
development and may have some valid usage in production, but should be used
only with care after auditing a module to ensure its behavior is valid.
#### Example: Patched Dependency
Since a dependency can be redirected, you can provide attenuated or modified
forms of dependencies as fits your application. For example, you could log
data about timing of function durations by wrapping the original:
```js
const original = require('fn');
module.exports = function fn(...args) {
console.time();
try {
return new.target ?
Reflect.construct(original, args) :
Reflect.apply(original, this, args);
} finally {
console.timeEnd();
}
};
```
[relative url string]: https://url.spec.whatwg.org/#relative-url-with-fragment-string

View File

@ -1033,9 +1033,15 @@ E('ERR_MANIFEST_ASSERT_INTEGRITY',
}
return msg;
}, Error);
E('ERR_MANIFEST_DEPENDENCY_MISSING',
'Manifest resource %s does not list %s as a dependency specifier',
Error);
E('ERR_MANIFEST_INTEGRITY_MISMATCH',
'Manifest resource %s has multiple entries but integrity lists do not match',
SyntaxError);
E('ERR_MANIFEST_INVALID_RESOURCE_FIELD',
'Manifest resource %s has invalid property value for %s',
TypeError);
E('ERR_MANIFEST_TDZ', 'Manifest initialization has not yet run', Error);
E('ERR_MANIFEST_UNKNOWN_ONERROR',
'Manifest specified unknown error behavior "%s".',

View File

@ -1,19 +1,72 @@
'use strict';
const { Object } = primordials;
const {
ERR_MANIFEST_DEPENDENCY_MISSING,
ERR_UNKNOWN_BUILTIN_MODULE
} = require('internal/errors').codes;
const { NativeModule } = require('internal/bootstrap/loaders');
const { getOptionValue } = require('internal/options');
const experimentalModules = getOptionValue('--experimental-modules');
const { validateString } = require('internal/validators');
const path = require('path');
const { pathToFileURL } = require('internal/url');
const { pathToFileURL, fileURLToPath } = require('internal/url');
const { URL } = require('url');
const debug = require('internal/util/debuglog').debuglog('module');
function loadNativeModule(filename, request, experimentalModules) {
const mod = NativeModule.map.get(filename);
if (mod) {
debug('load native module %s', request);
mod.compileForPublicLoader(experimentalModules);
return mod;
}
}
// Invoke with makeRequireFunction(module) where |module| is the Module object
// to use as the context for the require() function.
function makeRequireFunction(mod) {
// Use redirects to set up a mapping from a policy and restrict dependencies
function makeRequireFunction(mod, redirects) {
const Module = mod.constructor;
function require(path) {
return mod.require(path);
let require;
if (redirects) {
const { map, reaction } = redirects;
const id = mod.filename || mod.id;
require = function require(path) {
let missing = true;
if (map === true) {
missing = false;
} else if (map.has(path)) {
const redirect = map.get(path);
if (redirect === true) {
missing = false;
} else if (typeof redirect === 'string') {
const parsed = new URL(redirect);
if (parsed.protocol === 'node:') {
const specifier = parsed.pathname;
const mod = loadNativeModule(
specifier,
redirect,
experimentalModules);
if (mod && mod.canBeRequiredByUsers) return mod.exports;
throw new ERR_UNKNOWN_BUILTIN_MODULE(specifier);
} else if (parsed.protocol === 'file:') {
return mod.require(fileURLToPath(parsed));
}
}
}
if (missing) {
reaction(new ERR_MANIFEST_DEPENDENCY_MISSING(id, path));
}
return mod.require(path);
};
} else {
require = function require(path) {
return mod.require(path);
};
}
function resolve(request, options) {
@ -114,6 +167,7 @@ function normalizeReferrerURL(referrer) {
module.exports = {
addBuiltinLibsToObject,
builtinLibs,
loadNativeModule,
makeRequireFunction,
normalizeReferrerURL,
stripBOM,

View File

@ -46,6 +46,7 @@ const {
makeRequireFunction,
normalizeReferrerURL,
stripBOM,
loadNativeModule
} = require('internal/modules/cjs/helpers');
const { getOptionValue } = require('internal/options');
const preserveSymlinks = getOptionValue('--preserve-symlinks');
@ -618,11 +619,8 @@ Module._load = function(request, parent, isMain) {
return cachedModule.exports;
}
const mod = NativeModule.map.get(filename);
if (mod && mod.canBeRequiredByUsers) {
debug('load native module %s', request);
return mod.compileForPublicLoader(experimentalModules);
}
const mod = loadNativeModule(filename, request, experimentalModules);
if (mod && mod.canBeRequiredByUsers) return mod.exports;
// Don't call updateChildren(), Module constructor already does.
const module = new Module(filename, parent);
@ -828,8 +826,11 @@ function wrapSafe(filename, content) {
// the file.
// Returns exception, if any.
Module.prototype._compile = function(content, filename) {
let moduleURL;
let redirects;
if (manifest) {
const moduleURL = pathToFileURL(filename);
moduleURL = pathToFileURL(filename);
redirects = manifest.getRedirects(moduleURL);
manifest.assertIntegrity(moduleURL, content);
}
@ -853,7 +854,7 @@ Module.prototype._compile = function(content, filename) {
}
}
const dirname = path.dirname(filename);
const require = makeRequireFunction(this);
const require = makeRequireFunction(this, redirects);
var result;
const exports = this.exports;
const thisValue = exports;
@ -942,7 +943,7 @@ function createRequireFromPath(filename) {
m.filename = proxyPath;
m.paths = Module._nodeModulePaths(m.path);
return makeRequireFunction(m);
return makeRequireFunction(m, null);
}
Module.createRequireFromPath = deprecate(

View File

@ -9,9 +9,9 @@ const {
StringPrototype
} = primordials;
const { NativeModule } = require('internal/bootstrap/loaders');
const {
stripBOM
stripBOM,
loadNativeModule
} = require('internal/modules/cjs/helpers');
const CJSModule = require('internal/modules/cjs/loader').Module;
const internalURLModule = require('internal/url');
@ -93,11 +93,10 @@ translators.set('builtin', async function builtinStrategy(url) {
debug(`Translating BuiltinModule ${url}`);
// Slice 'node:' scheme
const id = url.slice(5);
const module = NativeModule.map.get(id);
const module = loadNativeModule(id, url, true);
if (!module) {
throw new ERR_UNKNOWN_BUILTIN_MODULE(id);
}
module.compileForPublicLoader(true);
return createDynamicModule(
[], [...module.exportKeys, 'default'], url, (reflect) => {
debug(`Loading BuiltinModule ${url}`);

View File

@ -1,15 +1,20 @@
'use strict';
const {
SafeMap,
SafeWeakMap,
Object,
RegExpPrototype,
uncurryThis
} = primordials;
const {
canBeRequiredByUsers
} = require('internal/bootstrap/loaders').NativeModule;
const {
ERR_MANIFEST_ASSERT_INTEGRITY,
ERR_MANIFEST_INTEGRITY_MISMATCH,
ERR_MANIFEST_INVALID_RESOURCE_FIELD,
ERR_MANIFEST_UNKNOWN_ONERROR,
} = require('internal/errors').codes;
const debug = require('internal/util/debuglog').debuglog('policy');
@ -24,6 +29,7 @@ const BufferEquals = uncurryThis(Buffer.prototype.equals);
const BufferToString = uncurryThis(Buffer.prototype.toString);
const { entries } = Object;
const kIntegrities = new SafeWeakMap();
const kDependencies = new SafeWeakMap();
const kReactions = new SafeWeakMap();
const kRelativeURLStringPattern = /^\.{0,2}\//;
const { getOptionValue } = require('internal/options');
@ -52,76 +58,121 @@ class Manifest {
const integrities = {
__proto__: null,
};
const reactions = {
const dependencies = {
__proto__: null,
integrity: REACTION_THROW,
};
let reaction = REACTION_THROW;
if (obj.onerror) {
const behavior = obj.onerror;
if (behavior === 'throw') {
} else if (behavior === 'exit') {
reactions.integrity = REACTION_EXIT;
reaction = REACTION_EXIT;
} else if (behavior === 'log') {
reactions.integrity = REACTION_LOG;
reaction = REACTION_LOG;
} else {
throw new ERR_MANIFEST_UNKNOWN_ONERROR(behavior);
}
}
kReactions.set(this, Object.freeze(reactions));
kReactions.set(this, reaction);
const manifestEntries = entries(obj.resources);
for (var i = 0; i < manifestEntries.length; i++) {
let url = manifestEntries[i][0];
const integrity = manifestEntries[i][1].integrity;
const originalURL = url;
if (RegExpPrototype.test(kRelativeURLStringPattern, url)) {
url = new URL(url, manifestURL).href;
}
let integrity = manifestEntries[i][1].integrity;
if (!integrity) integrity = null;
if (integrity != null) {
debug(`Manifest contains integrity for url ${url}`);
if (RegExpPrototype.test(kRelativeURLStringPattern, url)) {
url = new URL(url, manifestURL).href;
}
debug(`Manifest contains integrity for url ${originalURL}`);
if (typeof integrity === 'string') {
const sri = Object.freeze(SRI.parse(integrity));
if (url in integrities) {
const old = integrities[url];
let mismatch = false;
const sri = Object.freeze(SRI.parse(integrity));
if (url in integrities) {
const old = integrities[url];
let mismatch = false;
if (old.length !== sri.length) {
mismatch = true;
} else {
compare:
for (var sriI = 0; sriI < sri.length; sriI++) {
for (var oldI = 0; oldI < old.length; oldI++) {
if (sri[sriI].algorithm === old[oldI].algorithm &&
BufferEquals(sri[sriI].value, old[oldI].value) &&
sri[sriI].options === old[oldI].options) {
continue compare;
}
}
if (old.length !== sri.length) {
mismatch = true;
break compare;
} else {
compare:
for (var sriI = 0; sriI < sri.length; sriI++) {
for (var oldI = 0; oldI < old.length; oldI++) {
if (sri[sriI].algorithm === old[oldI].algorithm &&
BufferEquals(sri[sriI].value, old[oldI].value) &&
sri[sriI].options === old[oldI].options) {
continue compare;
}
}
mismatch = true;
break compare;
}
}
if (mismatch) {
throw new ERR_MANIFEST_INTEGRITY_MISMATCH(url);
}
}
if (mismatch) {
throw new ERR_MANIFEST_INTEGRITY_MISMATCH(url);
}
integrities[url] = sri;
} else if (integrity === true) {
integrities[url] = true;
} else {
throw new ERR_MANIFEST_INVALID_RESOURCE_FIELD(url, 'integrity');
}
integrities[url] = sri;
}
let dependencyMap = manifestEntries[i][1].dependencies;
if (dependencyMap === null || dependencyMap === undefined) {
dependencyMap = {};
}
if (typeof dependencyMap === 'object' && !Array.isArray(dependencyMap)) {
dependencies[url] = new SafeMap(Object.entries(dependencyMap).map(
([ from, to ]) => {
if (to === true) {
return [from, to];
}
if (canBeRequiredByUsers(to)) {
return [from, `node:${to}`];
} else if (RegExpPrototype.test(kRelativeURLStringPattern, to)) {
return [from, new URL(to, manifestURL).href];
}
return [from, new URL(to).href];
})
);
} else if (dependencyMap === true) {
dependencies[url] = true;
} else {
throw new ERR_MANIFEST_INVALID_RESOURCE_FIELD(url, 'dependencies');
}
}
Object.freeze(integrities);
kIntegrities.set(this, integrities);
Object.freeze(dependencies);
kDependencies.set(this, dependencies);
Object.freeze(this);
}
getRedirects(requester) {
const dependencies = kDependencies.get(this);
if (dependencies && requester in dependencies) {
return {
map: dependencies[requester],
reaction: kReactions.get(this)
};
}
return null;
}
assertIntegrity(url, content) {
debug(`Checking integrity of ${url}`);
const integrities = kIntegrities.get(this);
const realIntegrities = new Map();
const realIntegrities = new SafeMap();
if (integrities && url in integrities) {
const integrityEntries = integrities[url];
if (integrityEntries === true) return true;
// Avoid clobbered Symbol.iterator
for (var i = 0; i < integrityEntries.length; i++) {
const {
@ -139,7 +190,7 @@ class Manifest {
}
}
const error = new ERR_MANIFEST_ASSERT_INTEGRITY(url, realIntegrities);
kReactions.get(this).integrity(error);
kReactions.get(this)(error);
}
}

View File

@ -1,12 +1,7 @@
// Flags: --experimental-modules
/* eslint-disable node-core/require-common-first, node-core/required-modules */
import { createRequireFromPath } from 'module';
import { fileURLToPath as toPath } from 'url';
function createRequire(metaUrl) {
return createRequireFromPath(toPath(metaUrl));
}
import { createRequire } from 'module';
const require = createRequire(import.meta.url);
const common = require('./index.js');

View File

@ -0,0 +1,11 @@
{
"resources": {
"../parent.js": {
"integrity": true,
"dependencies": {}
},
"../dep.js": {
"integrity": true
}
}
}

View File

@ -0,0 +1,10 @@
{
"resources": {
"../parent.js": {
"integrity": true
},
"../dep.js": {
"integrity": true
}
}
}

View File

@ -0,0 +1,10 @@
{
"resources": {
"../parent.js": {
"integrity": true,
"dependencies": {
"./dep.js": "node:util"
}
}
}
}

View File

@ -0,0 +1,13 @@
{
"resources": {
"../parent.js": {
"integrity": true,
"dependencies": {
"./dep.js": "../dep.js"
}
},
"../dep.js": {
"integrity": true
}
}
}

View File

@ -0,0 +1,10 @@
{
"resources": {
"../parent.js": {
"integrity": true,
"dependencies": {
"./dep.js": "node:404"
}
}
}
}

View File

@ -0,0 +1,11 @@
{
"resources": {
"../parent.js": {
"integrity": true,
"dependencies": true
},
"../dep.js": {
"integrity": true
}
}
}

3
test/fixtures/policy/parent.js vendored Normal file
View File

@ -0,0 +1,3 @@
'use strict';
// Included in parent-policy.json
require('./dep.js');

View File

@ -5,4 +5,3 @@ Error: test
at async three (*fixtures*async-error.js:20:3)
at async four (*fixtures*async-error.js:24:3)
at async main (*message*async_error_sync_esm.mjs:7:5)
(node:*) [DEP0130] DeprecationWarning: Module.createRequireFromPath() is deprecated. Use Module.createRequire() instead.

View File

@ -0,0 +1,90 @@
'use strict';
const common = require('../common');
if (!common.hasCrypto)
common.skip('missing crypto');
const fixtures = require('../common/fixtures');
const assert = require('assert');
const { spawnSync } = require('child_process');
const dep = fixtures.path('policy', 'parent.js');
{
const depPolicy = fixtures.path(
'policy',
'dependencies',
'dependencies-redirect-policy.json');
const { status } = spawnSync(
process.execPath,
[
'--experimental-policy', depPolicy, dep,
]
);
assert.strictEqual(status, 0);
}
{
const depPolicy = fixtures.path(
'policy',
'dependencies',
'dependencies-redirect-builtin-policy.json');
const { status } = spawnSync(
process.execPath,
[
'--experimental-policy', depPolicy, dep,
]
);
assert.strictEqual(status, 0);
}
{
const depPolicy = fixtures.path(
'policy',
'dependencies',
'dependencies-redirect-unknown-builtin-policy.json');
const { status } = spawnSync(
process.execPath,
[
'--experimental-policy', depPolicy, dep,
]
);
assert.strictEqual(status, 1);
}
{
const depPolicy = fixtures.path(
'policy',
'dependencies',
'dependencies-wildcard-policy.json');
const { status } = spawnSync(
process.execPath,
[
'--experimental-policy', depPolicy, dep,
]
);
assert.strictEqual(status, 0);
}
{
const depPolicy = fixtures.path(
'policy',
'dependencies',
'dependencies-empty-policy.json');
const { status } = spawnSync(
process.execPath,
[
'--experimental-policy', depPolicy, dep,
]
);
assert.strictEqual(status, 1);
}
{
const depPolicy = fixtures.path(
'policy',
'dependencies',
'dependencies-missing-policy.json');
const { status } = spawnSync(
process.execPath,
[
'--experimental-policy', depPolicy, dep,
]
);
assert.strictEqual(status, 1);
}

View File

@ -28,10 +28,6 @@ const windowsPolicySRI = 'sha512-OeyCPRo4OZMosHyquZXDHpuU1F4KzG9UHFnn12FMaHsvqFU
/* eslint-enable max-len */
const depPolicySRI = `${nixPolicySRI} ${windowsPolicySRI}`;
console.dir({
depPolicySRI,
body: JSON.stringify(fs.readFileSync(depPolicy).toString('utf8'))
});
{
const { status, stderr } = spawnSync(
process.execPath,

View File

@ -71,7 +71,8 @@ function test({
};
for (const [url, { body, match }] of Object.entries(resources)) {
manifest.resources[url] = {
integrity: `sha256-${hash('sha256', match ? body : body + '\n')}`
integrity: `sha256-${hash('sha256', match ? body : body + '\n')}`,
dependencies: true
};
fs.writeFileSync(new URL(url, tmpdirURL.href), body);
}