HTML in Canvas API

For years, web developers have had to make a tough architectural choice when building complex, highly-interactive visual applications on the web/ Should you lean on the DOM for its rich semantic features or render directly to the canvas element for low-level graphics performance?

With the new experimental HTML-in-Canvas API—available now in origin trial—you don't have to choose. This API lets you draw DOM content directly into a 2D canvas or a WebGL/WebGPU texture while keeping the UI interactable, accessible, and hooked up to your favorite browser features. By combining HTML with low-level graphics processing, you can create experiences that were previously impossible.

How to use the API

Using the API happens in three phases: Setting up your canvas, rendering into the canvas, and updating the CSS transform so the browser knows where the element physically sits on the screen.

Prerequisites

The HTML-in-Canvas API is in origin trial in Chrome 148 through 150. To test it on your site, use Chrome Canary 149 or later with the chrome://flags/#canvas-draw-element flag enabled. To enable the API for other users, register for the Origin Trial.

Step 1: Basic Canvas setup
First, add the layoutsubtree attribute to your canvas tag. This makes the browser aware of the content nested inside the canvas, preparing it to be displayed inside the canvas, and exposing it to accessibility trees.

<canvas id="canvas" style="width: 200px; height: 200px;" layoutsubtree>
  <div id="form_element">
  <div id="form_element">
  <div id="form_element">
    <label for="name">Name:</label> <input id="name" type="text">
  </div>
</canvas>

Size the canvas grid
To avoid blurriness of the rendered content, make sure to size the canvas grid to match the device scale factor.

const observer = new ResizeObserver(([entry]) => {
  const dpc = entry.devicePixelContentBoxSize;
  canvas.width = dpc ? dpc[0].inlineSize : Math.round(entry.contentRect.width * window.devicePixelRatio);
  canvas.height = dpc ? dpc[0].blockSize : Math.round(entry.contentRect.height * window.devicePixelRatio);
});

const supportsDevicePixelContentBox =
  typeof ResizeObserverEntry !== 'undefined' &&
  'devicePixelContentBoxSize' in ResizeObserverEntry.prototype;
const options = supportsDevicePixelContentBox ? { box: 'device-pixel-content-box' } : {};
observer.observe(canvas, options);

Step 2: Rendering
For a 2D context, use the drawElementImage method. Do this inside the paint event, which triggers whenever the element redraws—for example, during text highlighting or user input. It's crucial to update the element's CSS transform with the return value so interactivity continues to work.

const ctx = document.getElementById('canvas').getContext('2d');
const form_element = document.getElementById('form_element');
const canvas = document.getElementById('canvas');

canvas.onpaint = () => {
  ctx.reset();

  // Draw the form element at x:0, y:0
  let transform = ctx.drawElementImage(form_element, 0, 0);

  // Use the transform returned later on...
};

Render with WebGL
For WebGL, you use texElementImage2D. It functions similar to texImage2D, but takes the DOM element as the source.

canvas.onpaint = () => {
  if (gl.texElementImage2D) {
    gl.texElementImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, form_element);
  }
};

Render with WebGPU
WebGPU uses the copyElementImageToTexture method on the device queue, analogous to copyExternalImageToTexture:

canvas.onpaint = () => {
  root.device.queue.copyElementImageToTexture(
    valueElement,
    { texture: targetTexture }
  );
};

Step 3: Update the CSS transform
Now that you've rendered the element into the canvas, you have to update the browser on where it's located. This ensures spatial synchronization between the canvas and the DOM's layout. This is important so that the browser can correctly map the event zone—such where exactly the user clicks or hovers—with where the element is rendered.

For the 2D context case, apply the transform returned by the rendering call to the .style.transform property:

const ctx = document.getElementById('canvas').getContext('2d');
const form_element = document.getElementById('form_element');
const canvas = document.getElementById('canvas');

canvas.onpaint = () => {
  ctx.reset();
  // Draw the form element at x:0, y:0
  let transform = ctx.drawElementImage(form_element, 0, 0);

  // Sync the DOM location with the drawn location
  form_element.style.transform = transform.toString();
};

With WebGL or WebGPU, the on-screen location of an element depends on how the output texture is used by shader code, and can't be deduced from the canvas rendering context. However, if your shader program uses a typical model view projection to draw the texture, then you can use the new convenience function element.getElementTransform() to compute a transform that can be used in the same way as the return value from drawElementImage(). To facilitate this, you need to do the following:

Convert WebGL MVP Matrix to DOM Matrix.
Normalize the HTML element. HTML elements are sized in pixels (for example, 200px wide). WebGL, however, usually treats objects as "unit squares", for example, ranging from 0 to 1. If you don't normalize, your 200px button will look 200 times larger.
Map to the canvas viewport. This step is the "rescaling" phase. It stretches that unit-space math back out to match the actual pixel dimensions of your canvas element on the screen. It also flips the Y-axis, because in WebGL, up is positive, but in CSS, down is positive.

Calculate the final transform. Multiply the matrices in order: Viewport * MVP * Normalization. Combining them into one final transform produces a "map" that tells the browser exactly where that HTML element layer should sit to align with the 3D drawing.

Apply the transform to the HTML element. This moves the HTML element layer to sit directly on top of its rendered pixels. This ensures that when a user clicks a button or selects text, they're hitting the real HTML element.

if (canvas.getElementTransform) {
  // 1. Convert WebGL MVP Matrix to DOM Matrix
  const mvpDOM = new DOMMatrix(Array.from(htmlElementMVP));

  // 2. Normalize the HTML element (pixels -> 1x1 unit square)
  const width = targetHTMLElement.offsetWidth;
  const height = targetHTMLElement.offsetHeight;

  const cssToUnitSpace = new DOMMatrix()
    .scale(1 / width, -1 / height, 1) // Shrink to unit size and flip Y
    .translate(-width / 2, -height / 2); // Center the element

  // 3. Map to the canvas viewport
  const clipToCanvasViewport = new DOMMatrix()
    .translate(canvas.width / 2, canvas.height / 2) // Move origin to center
    .scale(canvas.width / 2, -canvas.height / 2, 1); // Stretch to canvas dimensions

  // 4. Multiply: (Clip -> Pixels) * (MVP) * (pixels -> unit square)
  const screenSpaceTransform = clipToCanvasViewport
      .multiply(mvpDOM)
      .multiply(cssToUnitSpace);

  // 5. Apply to the transform
  const computedTransform = canvas.getElementTransform(targetHTMLElement, screenSpaceTransform);
  if (computedTransform) {
    targetHTMLElement.style.transform = computedTransform.toString();
  }
}