webmcp

edit

WPT coverage of the tests of webmcp at the revision c8fa86fb13d4. The tests link to their results in the browsers on wpt.fyi. The tentative tests (experimental features), the reftests and the crash tests are listed separately from the stable tests.

GroupPassingTestsDisabledPass rate
Stable42802678%
Tentative000–
Reftests000–
Crash tests550100%
All47852680%

Failure reasons

The tests that do not pass (failing or disabled), by kind of reason. A test with several reasons counts once per kind.

  • TEST BUG: 9
  • NOT IMPLEMENTED: 29

Stable tests

TestResultSubtestsKindDescription
declarative/adopt-form-from-removed-iframe.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/duplicate-tool-name.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-abort.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-detach-toolactivated.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-flexible-types.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-invalid-input.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-navigation.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-no-autosubmit.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-pseudo-classes.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-respondWith-circular-object.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-respondWith-reject.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-respondWith-unserializable.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/executeTool-respondWith.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/execute_tool_change_event.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/execute_tool_submit_from_js.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/form_removal_submit_crash.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/getTools-declarative-schema.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/name-collision-with-imperative-tool.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/no-frame-documents.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/opaque-origin-tools.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/sandboxed-iframe.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/select-multiple-events.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/toolchange-on-attribute-mutation.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/toolchange-on-control-add-remove.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/toolchange-on-name-change.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
declarative/unregister-during-executeTool.https.htmlDisabled–NOT IMPLEMENTEDthe declarative API of WebMCP (the toolname, tooldescription, toolparamdescription and toolautosubmit attributes of the forms, SubmitEvent.agentInvoked and respondWith(), the :tool-form-active and :tool-submit-active pseudo-classes), which only the explainer (https://github.com/webmachinelearning/webmcp/blob/main/declarative-api-explainer.md) and Chromium define: the specification defines the imperative API, which is implemented
idlharness.https.window.jsPass38 / 38
imperative/detached-frame-executeTool.https.htmlPass1 / 1
imperative/detached-frame-getTools.https.htmlPass1 / 1
imperative/detached-frame-modelContext.https.htmlPass1 / 1
imperative/detached-frame-registerTool.https.htmlPass1 / 1
imperative/detached-frame-waitForToolsUntil.https.htmlPass1 / 1
imperative/duplicate_tool_registration.https.htmlPass1 / 1
imperative/executeTool-abort.https.htmlFail5 / 7TEST BUGexecuteTool() resolves with the JSON serialization of the result of the tool, so a tool that returns 'hello' gives '"hello"' (https://webmachinelearning.github.io/webmcp/#invoke-the-execute-callback); the test expects the string itself, as Chromium does, which the editors confirmed is a bug of Chromium and of the tests (https://github.com/webmachinelearning/webmcp/issues/361)
imperative/executeTool-across-trees.https.htmlPass1 / 1
imperative/executeTool-caller-navigate-abort.https.htmlPass4 / 4
imperative/executeTool-detach-toolactivated.https.htmlPass2 / 2
imperative/executeTool-detach-toolcancel.https.htmlPass2 / 2
imperative/executeTool-error-window-onerror.https.htmlPass6 / 6
imperative/executeTool-events.https.htmlFail2 / 3TEST BUGexecuteTool() resolves with the JSON serialization of the result of the tool, so a tool that returns 'hello' gives '"hello"' (https://webmachinelearning.github.io/webmcp/#invoke-the-execute-callback); the test expects the string itself, as Chromium does, which the editors confirmed is a bug of Chromium and of the tests (https://github.com/webmachinelearning/webmcp/issues/361)
imperative/executeTool-invalid-dictionary.https.htmlPass2 / 2
imperative/executeTool-reentrant-cancellation.https.htmlPass3 / 3
imperative/executeTool-signal-cross-origin.https.htmlPass3 / 3
imperative/executeTool-target-detachment.https.htmlPass4 / 4
imperative/executeTool-target-navigation.https.htmlPass1 / 1
imperative/executeTool-unregister-resolution-race.https.htmlPass1 / 1
imperative/exposedTo-cross-origin-child.https.htmlFail1 / 5TEST BUGexecuteTool() resolves with the JSON serialization of the result of the tool, so a tool that returns 'hello' gives '"hello"' (https://webmachinelearning.github.io/webmcp/#invoke-the-execute-callback); the test expects the string itself, as Chromium does, which the editors confirmed is a bug of Chromium and of the tests (https://github.com/webmachinelearning/webmcp/issues/361): the subtests that fail before unregistering their tools leave them registered, so the next subtests of the file see one more tool and fail too
imperative/exposedTo-defaults-cross-origin.https.htmlPass4 / 4
imperative/exposedTo-defaults-same-origin.https.htmlFail0 / 4TEST BUGexecuteTool() resolves with the JSON serialization of the result of the tool, so a tool that returns 'hello' gives '"hello"' (https://webmachinelearning.github.io/webmcp/#invoke-the-execute-callback); the test expects the string itself, as Chromium does, which the editors confirmed is a bug of Chromium and of the tests (https://github.com/webmachinelearning/webmcp/issues/361): the subtests that fail before unregistering their tools leave them registered, so the next subtests of the file see one more tool and fail too
imperative/exposedTo-invalid-origins.https.htmlFail11 / 12TEST BUGexecuteTool() resolves with the JSON serialization of the result of the tool, so a tool that returns 'hello' gives '"hello"' (https://webmachinelearning.github.io/webmcp/#invoke-the-execute-callback); the test expects the string itself, as Chromium does, which the editors confirmed is a bug of Chromium and of the tests (https://github.com/webmachinelearning/webmcp/issues/361)
imperative/exposedTo-multiple-children.https.htmlPass1 / 1
imperative/exposedTo-window-open.https.htmlPass1 / 1
imperative/getTools-filtering.https.htmlPass2 / 2
imperative/getTools-imperative-annotations.https.htmlPass4 / 4
imperative/getTools-imperative-schema.https.htmlPass2 / 2
imperative/getTools.https.htmlPass1 / 1
imperative/initial-about-blank-replacement-register-tool.https.htmlFail0 / 1NOT IMPLEMENTEDthe test sets iframe.src = 'about:blank' on an iframe whose document is its initial about:blank document (its lazy-loaded navigation is deferred) and expects contentDocument to be a new about:blank document at once, as in Chromium; the navigation of the library replaces the document in a later task, so the new document is not there yet
imperative/initial-about-blank-replacement-removes-tools.https.htmlFail0 / 1NOT IMPLEMENTEDthe test sets iframe.src = 'about:blank' on an iframe whose document is its initial about:blank document (its lazy-loaded navigation is deferred) and expects contentDocument to be a new about:blank document at once, as in Chromium; the navigation of the library replaces the document in a later task, so the new document is not there yet
imperative/initial-about-blank-shared-tool.https.htmlPass1 / 1
imperative/javascript-url-document-replacement.https.htmlPass1 / 1
imperative/model_context.https.htmlFail2 / 4TEST BUGexecuteTool() resolves with the JSON serialization of the result of the tool, so a tool that returns 'hello' gives '"hello"' (https://webmachinelearning.github.io/webmcp/#invoke-the-execute-callback); the test expects the string itself, as Chromium does, which the editors confirmed is a bug of Chromium and of the tests (https://github.com/webmachinelearning/webmcp/issues/361)
imperative/non-secure.htmlPass1 / 1
imperative/object-arguments.https.htmlFail0 / 1TEST BUGexecuteTool() resolves with the JSON serialization of the result of the tool, so a tool that returns 'hello' gives '"hello"' (https://webmachinelearning.github.io/webmcp/#invoke-the-execute-callback); the test expects the string itself, as Chromium does, which the editors confirmed is a bug of Chromium and of the tests (https://github.com/webmachinelearning/webmcp/issues/361)
imperative/opaque-origin-tools.https.htmlFail1 / 5TEST BUG4 of 5 subtests: the test expects a document whose origin is opaque to execute its own tools (and the tools of another document of the same opaque origin); the specification rejects executeTool() with a NotSupportedError when the origin of the tool is an opaque origin (https://webmachinelearning.github.io/webmcp/#dom-modelcontext-executetool, https://github.com/webmachinelearning/webmcp/issues/349), as the library does
imperative/permissions-policy.https.htmlFail7 / 8NOT IMPLEMENTEDModelContext.resumeTool() of the continuations proposal (https://github.com/webmachinelearning/webmcp/blob/main/continuations-explainer.md), which the specification does not define: the other subtests pass
imperative/register-tool-input-schema-registers-same-name.https.htmlPass1 / 1
imperative/register-tool-title.https.htmlPass3 / 3
imperative/register_tool_invalid_json_schema.https.htmlPass8 / 8
imperative/register_tool_name_validation.https.htmlPass2 / 2
imperative/register_tool_no_schema.https.htmlPass1 / 1
imperative/register_tool_signal.https.htmlPass4 / 4
imperative/register_tool_toolchange.https.htmlPass1 / 1
imperative/register_tool_with_empty_annotation.https.htmlPass1 / 1
imperative/register_tool_with_schema.https.htmlPass4 / 4
imperative/same-origin-iframe-registerTool-regression.https.htmlPass1 / 1
imperative/tool-re-registration-race.https.htmlPass1 / 1
imperative/unregister-during-executeTool.https.htmlFail1 / 4TEST BUGexecuteTool() resolves with the JSON serialization of the result of the tool, so a tool that returns 'hello' gives '"hello"' (https://webmachinelearning.github.io/webmcp/#invoke-the-execute-callback); the test expects the string itself, as Chromium does, which the editors confirmed is a bug of Chromium and of the tests (https://github.com/webmachinelearning/webmcp/issues/361)
imperative/wait-for-tools-until.https.htmlPass4 / 4
tool-activated-event.https.htmlPass4 / 4
tool-cancel-event.https.htmlPass4 / 4

Crash tests

TestResultVariantsKindDescription
imperative/abort-signal-after-iframe-removal-crash.https.htmlPass1 / 1
imperative/cancel-reentrancy-crash.https.htmlPass1 / 1
imperative/executeTool-same-document-navigation-crash.https.htmlPass1 / 1
imperative/register-tool-input-schema-removes-iframe-crash.https.htmlPass1 / 1
imperative/wait-for-tools-until-detach-crash.https.htmlPass1 / 1