Canvas Interaction

DevBridge provides 12 canvas-specific sub-actions via the computer tool: one for reading canvas content and 11 for interaction.

Read

Sub-action	Description
canvas_screenshot	Capture the canvas bitmap as an image (downscales large canvases by default)

Interact (11 sub-actions)

Sub-action	Description
canvas_click	Click at coordinates within a canvas
canvas_drag	Drag from one canvas coordinate to another
canvas_draw_path	Draw a multi-point path on the canvas
canvas_gesture	Perform touch-like gestures (pinch, rotate, swipe)
canvas_scroll	Scroll within a canvas element
canvas_hover	Hover over canvas coordinates
canvas_keypress	Send key events while canvas is focused
canvas_pixel	Read the color of a single pixel or pixel region
canvas_pixel_bulk	Read colors of multiple pixels at once (up to 1000 points)
canvas_hit_test	Test which element or region is at given coordinates
canvas_record	Record canvas frame output

Canvas in the accessibility tree

read_page includes canvas elements with metadata: viewportRect, documentRect, bitmapSize, fallback content (if any), and a [large, WxH] label for oversized canvases.

Library hit-test support

DevBridge includes built-in adapters for hit-testing elements drawn by popular canvas libraries:

• Chart.js — identify chart data points and series
• ECharts — identify chart components
• Fabric.js — identify canvas objects
• Konva.js — identify stage nodes