Modules

edit

Write a module loader

The engine does not read files: the modules that a script or a module imports are given by the module loader of the engine, a class that derives from JsModuleLoader. This loader serves the files of a dictionary, JSON and text files for the type import attribute, and a module implemented in .NET:

/// <summary>
/// Loads the modules of an application: the JavaScript, JSON and text files of a dictionary, and the "app:log" module
/// implemented in .NET.
/// </summary>
public sealed class AppModuleLoader(IReadOnlyDictionary<string, string> files, ICollection<string> logs) : JsModuleLoader
{
    // The same specifier must give the same module each time
    private readonly Dictionary<(string Specifier, string? Type), JsModule> _modules = [];

    protected override void LoadImportedModule(JsModuleLoadRequest request)
    {
        var key = (request.Specifier, request.Type);
        if (!_modules.TryGetValue(key, out var module))
        {
            try
            {
                module = CreateModule(request);
            }
            catch (JsException exception)
            {
                // e.g. a SyntaxError in the module or in the JSON file
                request.Fail(exception);
                return;
            }

            if (module is null)
            {
                request.Fail("Cannot find the module '" + request.Specifier + "'");
                return;
            }

            _modules[key] = module;
        }

        request.Complete(module);
    }

    protected override void InitializeImportMeta(JsSourceTextModule module, JsObject importMeta)
    {
        importMeta.SetProperty("url", module.Url ?? "");
    }

    private JsModule? CreateModule(JsModuleLoadRequest request)
    {
        var realm = request.Realm;
        if (request.Specifier is "app:log")
        {
            // A module whose exports are .NET functions
            return realm.CreateSyntheticModule(["log"], module => module.SetExport("log", realm.CreateFunction("log", 1, (_, arguments) =>
            {
                logs.Add(realm.Engine.ToString(arguments.Length > 0 ? arguments[0] : JsValue.Undefined));
                return JsValue.Undefined;
            })), request.Specifier);
        }

        if (!files.TryGetValue(request.Specifier, out var content))
            return null;

        // The type import attribute: import config from './config.json' with { type: 'json' }
        return request.Type switch
        {
            null => realm.ParseModule(content, request.Specifier),
            "json" => realm.CreateJsonModule(content, request.Specifier),
            "text" => realm.CreateTextModule(content, request.Specifier),
            _ => null,
        };
    }
}
  • LoadImportedModule receives each request: its Specifier ('./format.js'), its Type (the type import attribute, or null), its Realm (where to create the module), and the module or script that imports it (ReferrerModule, ReferrerUrl). Resolve relative specifiers against ReferrerUrl when your modules are in folders.
  • Complete the request with Complete, or fail it with Fail, once: a second call throws an InvalidOperationException. A loader can complete a request later, for instance once a file is downloaded. See Load modules asynchronously.
  • An exception that LoadImportedModule throws before it completes the request fails the request: a JsException with its value, any other exception (a FileNotFoundException...) with a TypeError whose message contains the one of the exception, such as Cannot load the module './format.js': Could not find file 'format.js'. The import() or the loading of the graph is rejected, the exception does not reach your code. A JsTerminationException, and an exception thrown after the request is completed, go to the caller.
  • The same specifier from the same module must give the same module object each time: keep the modules you created.
  • realm.ParseModule creates a JavaScript module; CreateJsonModule, CreateTextModule and CreateBytesModule create the modules of the json, text and bytes types; CreateSyntheticModule creates a module whose exports are set from .NET.
  • InitializeImportMeta fills the import.meta object of each JavaScript module.

Run a module

A module is loaded (with its dependencies), linked, then evaluated. Set the module loader in the options of the engine:

var files = new Dictionary<string, string>(StringComparer.Ordinal)
{
    ["main.js"] = """
        import { log } from 'app:log';
        import { formatPrice } from './format.js';
        import config from './config.json' with { type: 'json' };

        log(`${import.meta.url}: ${formatPrice(config.price, config.currency)}`);
        export const ready = true;
        """,
    ["./format.js"] = "export function formatPrice(amount, currency) { return `${amount.toFixed(2)} ${currency}`; }",
    ["./config.json"] = """{ "price": 12.5, "currency": "EUR" }""",
};

var logs = new List<string>();
using var engine = new JsEngine(new JsEngineOptions { ModuleLoader = new AppModuleLoader(files, logs) });
var realm = engine.CreateRealm();

// Load the module and its dependencies, link them, then evaluate them
var module = realm.ParseModule(files["main.js"], "main.js");
module.LoadRequestedModules();
engine.RunJobs();
module.Link();
var evaluation = module.Evaluate().AsObject();
engine.RunJobs();

Assert.True(evaluation.TryGetPromiseState(out var state, out _, out _));
Assert.Equal(JsPromiseState.Fulfilled, state);
Assert.Equal(["main.js: 12.50 EUR"], logs);

// The exports of the module
using (engine.Enter())
{
    var exports = module.GetNamespace().AsObject();
    Assert.True(exports.GetProperty("ready").AsBoolean());
}
  • LoadRequestedModules asks the loader for every module of the graph. It returns a promise, settled from RunJobs.
  • Link resolves the imports. It throws a JsException (a SyntaxError) when a module does not export a name that another one imports.
  • Evaluate runs the modules and returns a promise. A module that uses top-level await completes from RunJobs.
  • GetNamespace returns the namespace object of a linked module: its exports are its properties, and the default export is the default property.
  • Status tells where the module is (Unlinked, Linked, Evaluated...).

A module without imports needs no loader: parse it and run the same steps.

Handle the errors of a module

The errors of the loading and of the evaluation reject the promises:

var files = new Dictionary<string, string>(StringComparer.Ordinal)
{
    ["main.js"] = "import { missing } from './missing.js';",
};

using var engine = new JsEngine(new JsEngineOptions { ModuleLoader = new AppModuleLoader(files, []) });
var realm = engine.CreateRealm();
var module = realm.ParseModule(files["main.js"], "main.js");

// A module that cannot be loaded rejects the promise of LoadRequestedModules
var loading = module.LoadRequestedModules().AsObject();
engine.RunJobs();
Assert.True(loading.TryGetPromiseState(out var state, out var reason, out _));
Assert.Equal(JsPromiseState.Rejected, state);
Assert.Equal("TypeError: Cannot find the module './missing.js'", engine.ToString(reason));
  • Fail(string) rejects the loading with a TypeError; Fail(JsException) rejects it with the error of the exception, such as the SyntaxError of a module that is not valid.
  • When the evaluation throws, module.HasEvaluationError is true, EvaluationError is the thrown value and EvaluationErrorLocation is where it was thrown.
  • realm.ParseModule itself throws a JsException (a SyntaxError) for a module that is not valid.
  • When a limit stops the evaluation of a module, or a .NET function throws an exception that is not a JavaScript error, the exception goes to the caller of Evaluate (or of RunJobs for a module that uses top-level await). The module and the modules that import it are Evaluated with an error such as Error: The execution was terminated: The script exceeded its time limit: the next evaluation or import() of the module does not run it again, it is rejected with this error.

Load modules asynchronously

A loader does not have to complete a request before it returns. Complete and Fail can be called from any thread:

  • On the thread of the engine, the loading continues immediately: a thread that has entered the engine, such as in LoadImportedModule or in a callback of a script, or the last thread that entered it, such as the event loop of your application between two scripts.
  • On another thread, such as the continuation of a download, the completion is queued: the engine is not touched from that thread, and the loading continues on the thread of the engine, from the next RunJobs. The completions run in the order they were made, even when a script runs while they arrive.

The module given to Complete is created by realm.ParseModule and the other methods of the realm, which must run on the thread of the engine: download the source text on any thread, then create the module from the event loop of your application. Fail(string) creates its TypeError on the thread of the engine, so it is safe to call from a failed download.

Import modules from scripts

A script can import a module with import(). The request goes to the same loader, with IsDynamicImport set, and the module loads from the job queue:

var files = new Dictionary<string, string>(StringComparer.Ordinal)
{
    ["./plugin.js"] = "export default function run(input) { return input.toUpperCase(); }",
};

using var engine = new JsEngine(new JsEngineOptions { ModuleLoader = new AppModuleLoader(files, []) });
var realm = engine.CreateRealm();

// A script can import a module with import(): the loading runs from the job queue
realm.Execute("import('./plugin.js').then(plugin => { globalThis.result = plugin.default('done'); });", "app.js");
engine.RunJobs();

Assert.Equal("DONE", realm.Execute("result").AsString());

Without a module loader, the scripts and the modules cannot import modules.

Deferred re-exports

export defer (deferred re-exports) marks the re-exports of a module, such as the entry point of a library, to load only when another module uses them:

var files = new Dictionary<string, string>(StringComparer.Ordinal)
{
    ["main.js"] = "import { log } from 'app:log'; import { add } from './math.js'; log(`main.js: ${add(1, 2)}`);",

    // The entry point of a library: a deferred re-export is loaded only when one of its bindings is imported
    ["./math.js"] = """
        import { log } from 'app:log';
        export defer { add } from './add.js';
        export defer { subtract } from './subtract.js';
        log('math.js');
        """,
    ["./add.js"] = "import { log } from 'app:log'; log('add.js'); export function add(a, b) { return a + b; }",

    // './subtract.js' is not in the files: nothing imports subtract, so the loader is never asked for it
};

var logs = new List<string>();
using var engine = new JsEngine(new JsEngineOptions { ModuleLoader = new AppModuleLoader(files, logs) });
var realm = engine.CreateRealm();
var module = realm.ParseModule(files["main.js"], "main.js");
module.LoadRequestedModules();
engine.RunJobs();
module.Link();
module.Evaluate();
engine.RunJobs();

// A deferred re-export runs after the module that re-exports it
Assert.Equal(["math.js", "add.js", "main.js: 3"], logs);
  • The module of an export defer { ... } from declaration is loaded, linked and evaluated only when a module imports one of its bindings. It runs after the module that re-exports it, in the order of the declarations.
  • import * as ns loads all the deferred re-exports, and runs each one on the first access to one of its exports. To load fewer modules, name the exports you use: import { add, multiply } as math from './math.js' or import('./math.js', { exports: ['add', 'multiply'] }) give a namespace object with these exports only.
  • export defer * as ns from and export defer { a, b } as ns from re-export a deferred namespace: the module runs on the first access to one of its exports, as with import defer.
  • export * from forwards the deferred re-exports of the module: they are loaded only when they are imported.
  • The module that you load with LoadRequestedModules and Link loads all its own deferred re-exports: its exports are available from GetNamespace.

Source phase imports

import source x from '...' and import.source() give the module source of a module, set by the loader in module.ModuleSource before it completes the request. The object is a JsHostObject that extends realm.AbstractModuleSource and whose ModuleSourceClassName is set (for instance WebAssembly.Module). The source phase import of a module without module source, such as a JavaScript module, throws a SyntaxError.

Modules implemented by the host

CreateHostDefinedModule creates a module implemented in .NET that imports other modules, such as a WebAssembly module. Unlike a synthetic module, it requests modules: they are loaded, linked and evaluated before it, as the imports of a JavaScript module, and it can be part of a cycle of imports. This loader serves a module that reads the step export of ./config.js and exports a live binding:

/// <summary>
/// Loads the JavaScript files of a dictionary, and the "app:counter" module implemented in .NET: it imports the step of
/// './config.js', and its count export is a live binding.
/// </summary>
public sealed class CounterModuleLoader(IReadOnlyDictionary<string, string> files) : JsModuleLoader
{
    // The same specifier must give the same module each time
    private readonly Dictionary<string, JsModule> _modules = [];

    protected override void LoadImportedModule(JsModuleLoadRequest request)
    {
        if (!_modules.TryGetValue(request.Specifier, out var module))
        {
            if (request.Specifier is "app:counter")
            {
                module = CreateCounterModule(request.Realm);
            }
            else if (files.TryGetValue(request.Specifier, out var code))
            {
                module = request.Realm.ParseModule(code, request.Specifier);
            }
            else
            {
                request.Fail("Cannot find the module '" + request.Specifier + "'");
                return;
            }

            _modules[request.Specifier] = module;
        }

        request.Complete(module);
    }

    private static JsHostDefinedModule CreateCounterModule(JsRealm realm)
    {
        var count = 0.0;

        // The modules it requests are evaluated before it: the evaluation steps read their exports. The imported names
        // are checked when the module graph is linked: a missing export is a SyntaxError, and no module runs
        return realm.CreateHostDefinedModule(["./config.js"], ["count", "increment"], module =>
        {
            var step = module.GetImport("./config.js", "step").GetValue().AsNumber();
            module.SetExportGetter("count", () => count); // the getter runs on each access to the binding
            module.SetExport("increment", realm.CreateFunction("increment", 0, (_, _) =>
            {
                count += step;
                return JsValue.Undefined;
            }));
        }, "app:counter", importedNames: [new("./config.js", "step")]);
    }
}
var files = new Dictionary<string, string>(StringComparer.Ordinal)
{
    ["main.js"] = """
        import { count, increment } from 'app:counter';
        const before = count;
        increment();
        increment();
        export const result = `${before} -> ${count}`;
        """,
    ["./config.js"] = "export const step = 5;",
};

using var engine = new JsEngine(new JsEngineOptions { ModuleLoader = new CounterModuleLoader(files) });
var realm = engine.CreateRealm();
var module = realm.ParseModule(files["main.js"], "main.js");
module.LoadRequestedModules();
engine.RunJobs();
module.Link();
module.Evaluate();
engine.RunJobs();

using (engine.Enter())
{
    var exports = module.GetNamespace().AsObject();
    Assert.Equal("0 -> 10", exports.GetProperty("result").AsString());
}
  • The importedNames argument declares the names that the module imports, each with the specifier of its module. They are checked when the module graph is linked, as the imports of a JavaScript module: a name that the requested module does not export throws a SyntaxError from Link, before any module of the graph runs. Without them, a missing export is only found by GetImport, during the evaluation, after the other modules of the graph ran.
  • The evaluation steps run when the module is evaluated, after the modules it requests (except in a cycle: a module that imports this one, directly or not, may be evaluated after it).
  • GetImport resolves an export of a requested module, through its re-exports, and throws a SyntaxError when the module does not export the name. TryGetValue returns false when the binding is not initialized yet (a let or a const of a module of a cycle that is not evaluated yet), and Module is the module that declares the binding.
  • The exports are uninitialized until the evaluation steps set them: reading one throws a ReferenceError. SetExport sets a value, SetExportGetter a live binding whose getter runs on each access.
  • An exception thrown by the evaluation steps is the error of the evaluation of the module.
  • JsModule.FromNamespace returns the module of a namespace object, such as the value of import * as ns.