This API is experimental. It is only available to Chrome users on the dev channel.

chrome.automation

Description: The chrome.automation API allows developers to access the automation (accessibility) tree for the browser. The tree resembles the DOM tree, but only exposes the semantic structure of a page. It can be used to programmatically interact with a page by examining names, roles, and states, listening for events, and performing actions on nodes.
Availability: Dev channel only. Learn more.
Manifest: "automation": {...}

Summary

Types
EventType
RoleType
StateType
Rect
FindParams
SetDocumentSelectionParams
AutomationEvent
TreeChange
AutomationNode
Methods
getTree chrome.automation.getTree(integer tabId, function callback)
getDesktop chrome.automation.getDesktop(function callback)
getFocus chrome.automation.getFocus(function callback)
addTreeChangeObserver chrome.automation.addTreeChangeObserver(enum of "noTreeChanges", "liveRegionTreeChanges", "textMarkerChanges", or "allTreeChanges" filter, function observer)
removeTreeChangeObserver chrome.automation.removeTreeChangeObserver(function observer)
setDocumentSelection chrome.automation.setDocumentSelection( SetDocumentSelectionParams params)

Types

EventType

Possible events fired on an automation.AutomationNode.
Enum
"activedescendantchanged", "alert", "ariaAttributeChanged", "autocorrectionOccured", "blur", "checkedStateChanged", "childrenChanged", "clicked", "documentSelectionChanged", "expandedChanged", "focus", "imageFrameUpdated", "hide", "hover", "invalidStatusChanged", "layoutComplete", "liveRegionCreated", "liveRegionChanged", "loadComplete", "locationChanged", "mediaStartedPlaying", "mediaStoppedPlaying", "menuEnd", "menuListItemSelected", "menuListValueChanged", "menuPopupEnd", "menuPopupStart", "menuStart", "mouseCanceled", "mouseDragged", "mouseMoved", "mousePressed", "mouseReleased", "rowCollapsed", "rowCountChanged", "rowExpanded", "scrollPositionChanged", "scrolledToAnchor", "selectedChildrenChanged", "selection", "selectionAdd", "selectionRemove", "show", "textChanged", "textSelectionChanged", "treeChanged", or "valueChanged"

RoleType

Describes the purpose of an automation.AutomationNode.
Enum
"abbr", "alertDialog", "alert", "annotation", "application", "article", "audio", "banner", "blockquote", "busyIndicator", "button", "buttonDropDown", "canvas", "caption", "cell", "checkBox", "client", "colorWell", "columnHeader", "column", "comboBox", "complementary", "contentInfo", "date", "dateTime", "definition", "descriptionListDetail", "descriptionList", "descriptionListTerm", "desktop", "details", "dialog", "directory", "disclosureTriangle", "div", "document", "embeddedObject", "feed", "figcaption", "figure", "footer", "form", "grid", "group", "heading", "iframe", "iframePresentational", "ignored", "imageMapLink", "imageMap", "image", "inlineTextBox", "inputTime", "labelText", "legend", "lineBreak", "link", "listBoxOption", "listBox", "listItem", "listMarker", "list", "locationBar", "log", "main", "mark", "marquee", "math", "menuBar", "menuButton", "menuItem", "menuItemCheckBox", "menuItemRadio", "menuListOption", "menuListPopup", "menu", "meter", "navigation", "note", "outline", "pane", "paragraph", "popUpButton", "pre", "presentational", "progressIndicator", "radioButton", "radioGroup", "region", "rootWebArea", "rowHeader", "row", "ruby", "ruler", "svgRoot", "scrollArea", "scrollBar", "seamlessWebArea", "search", "searchBox", "slider", "sliderThumb", "spinButtonPart", "spinButton", "splitter", "staticText", "status", "switch", "tabGroup", "tabList", "tabPanel", "tab", "tableHeaderContainer", "table", "term", "textField", "time", "timer", "titleBar", "toggleButton", "toolbar", "treeGrid", "treeItem", "tree", "unknown", "tooltip", "video", "webArea", "webView", or "window"

StateType

Describes characteristics of an automation.AutomationNode.
Enum
"busy", "checked", "collapsed", "default", "disabled", "editable", "expanded", "focusable", "focused", "haspopup", "horizontal", "hovered", "invisible", "linked", "multiline", "multiselectable", "offscreen", "pressed", "protected", "readOnly", "required", "richlyEditable", "selectable", "selected", "vertical", or "visited"

Rect

properties
integer left
integer top
integer width
integer height

FindParams

properties
RoleType (optional) role
object (optional) state

A map of automation.StateType to boolean, indicating for each state whether it should be set or not. For example: { StateType.disabled: false } would only match if StateType.disabled was not present in the node's state object.

object (optional) attributes

A map of attribute name to expected value, for example { name: 'Root directory', checkbox_mixed: true }. String attribute values may be specified as a regex, for example { name: /stralia$/ }. Unless specifying a regex, the expected value must be an exact match in type and value for the actual value. Thus, the type of expected value must be one of:

  • string
  • integer
  • float
  • boolean

SetDocumentSelectionParams

properties
AutomationNode anchorObject

The node where the selection begins.

integer anchorOffset

The offset in the anchor node where the selection begins.

AutomationNode focusObject

The node where the selection ends.

integer focusOffset

The offset within the focus node where the selection ends.

AutomationEvent

properties
AutomationNode target

The automation.AutomationNode to which the event was targeted.

EventType type

The type of the event.

string eventFrom

The source of this event.

integer mouseX
integer mouseY
function stopPropagation

Stops this event from further processing except for any remaining listeners on AutomationEvent.target.

TreeChange

properties
AutomationNode target

The automation.AutomationNode that changed.

enum of "nodeCreated", "subtreeCreated", "nodeChanged", "textChanged", or "nodeRemoved" type

The type of change.

nodeCreated
* This node was added to the tree and its parent is new as well, so it's just one node in a new subtree that was added.
subtreeCreated
* This node was added to the tree but its parent was already in the tree, so it's possibly the root of a new subtree - it does not mean that it necessarily has children.
nodeChanged
* This node changed.
textChanged
* This node's text (name) changed.
nodeRemoved
* This node was removed.

AutomationNode

properties
AutomationNode (optional) root

The root node of the tree containing this AutomationNode.

boolean isRootNode

Whether this AutomationNode is a root node.

RoleType (optional) role

The role of this node.

object (optional) state

The automation.StateTypes describing this node. @type {Object}

Rect (optional) location

The rendered location (as a bounding box) of this node in global screen coordinates.

function boundsForRange

Computes the bounding box of a subrange of this node in global screen coordinates. Returns the same as |location| if range information is not available. The start and end indices are zero-based offsets into the node's "name" string attribute.


Returns Rect.
Parameters
integer startIndex
integer endIndex
string (optional) description

The purpose of the node, other than the role, if any.

string (optional) placeholder

The placeholder for this text field, if any.

string (optional) name

The accessible name for this node, via the Accessible Name Calculation process.

enum of "uninitialized", "attribute", "contents", "placeholder", "relatedElement", or "value" (optional) nameFrom

The source of the name.

string (optional) value

The value for this node: for example the value attribute of an <input> element.

string (optional) htmlTag

The HTML tag for this element, if this node is an HTML element.

integer (optional) hierarchicalLevel

The level of a heading or tree item.

array of integer (optional) wordStarts

The start and end index of each word in an inline text box.

array of integer (optional) wordEnds
array of AutomationNode (optional) controls

The nodes, if any, which this node is specified to control via aria-controls.

array of AutomationNode (optional) describedBy

The nodes, if any, which form a description for this node.

array of AutomationNode (optional) flowTo

The nodes, if any, which may optionally be navigated to after this one. See aria-flowto.

array of AutomationNode (optional) labelledBy

The nodes, if any, which form a label for this element. Generally, the text from these elements will also be exposed as the element's accessible name, via the automation.AutomationNode.name attribute.

AutomationNode (optional) activeDescendant

The node referred to by aria-activedescendant, where applicable

string (optional) url

The URL that this link will navigate to.

string (optional) docUrl

The URL of this document.

string (optional) docTitle

The title of this document.

boolean (optional) docLoaded

Whether this document has finished loading.

double (optional) docLoadingProgress

The proportion (out of 1.0) that this doc has completed loading.

integer (optional) scrollX

Scrollable container attributes.

integer (optional) scrollXMin
integer (optional) scrollXMax
integer (optional) scrollY
integer (optional) scrollYMin
integer (optional) scrollYMax
integer (optional) textSelStart

The character index of the start of the selection within this editable text element; -1 if no selection.

integer (optional) textSelEnd

The character index of the end of the selection within this editable text element; -1 if no selection.

string (optional) textInputType

The input type, like email or number.

array of integer lineBreaks

An array of indexes of the break between lines in editable text.

array of integer markerStarts

An array of indexes of the start position of each text marker.

array of integer markerEnds

An array of indexes of the end position of each text marker.

array of integer markerTypes

An array of numerical types indicating the type of each text marker, such as a spelling error.

AutomationNode (optional) anchorObject

The anchor node of the tree selection, if any.

integer (optional) anchorOffset

The anchor offset of the tree selection, if any.

string (optional) anchorAffinity

The affinity of the tree selection anchor, if any.

AutomationNode (optional) focusObject

The focus node of the tree selection, if any.

integer (optional) focusOffset

The focus offset of the tree selection, if any.

string (optional) focusAffinity

The affinity of the tree selection focus, if any.

double (optional) valueForRange

The current value for this range.

double (optional) minValueForRange

The minimum possible value for this range.

double (optional) maxValueForRange

The maximum possible value for this range.

integer (optional) posInSet

The 1-based index of an item in a set.

integer (optional) setSize

The number of items in a set;

integer (optional) tableRowCount

The number of rows in this table.

integer (optional) tableColumnCount

The number of columns in this table.

integer (optional) tableCellColumnIndex

The zero-based index of the column that this cell is in.

integer (optional) tableCellColumnSpan

The number of columns that this cell spans (default is 1).

integer (optional) tableCellRowIndex

The zero-based index of the row that this cell is in.

integer (optional) tableCellRowSpan

The number of rows that this cell spans (default is 1).

AutomationNode (optional) tableColumnHeader

The corresponding column header for this cell.

AutomationNode (optional) tableRowHeader

The corresponding row header for this cell.

string (optional) liveStatus

The type of region if this is the root of a live region. Possible values are 'polite' and 'assertive'.

string (optional) liveRelevant

The value of aria-relevant for a live region.

boolean (optional) liveAtomic

The value of aria-atomic for a live region.

boolean (optional) liveBusy

The value of aria-busy for a live region.

string (optional) containerLiveStatus

The type of live region if this node is inside a live region.

string (optional) containerLiveRelevant

The value of aria-relevant if this node is inside a live region.

boolean (optional) containerLiveAtomic

The value of aria-atomic if this node is inside a live region.

boolean (optional) containerLiveBusy

The value of aria-busy if this node is inside a live region.

object (optional) htmlAttributes

A map containing all HTML attributes and their values @type {Object}

string (optional) inputType

The input type of a text field, such as "text" or "email".

string (optional) accessKey

The key that activates this widget.

string (optional) ariaInvalidValue

The value of the aria-invalid attribute, indicating the error type.

boolean (optional) ariaReadonly

The value of the aria-readonly attribute, if applicable.

string (optional) display

The CSS display attribute for this node, if applicable.

string (optional) imageDataUrl

A data url with the contents of this object's image or thumbnail.

string (optional) language

The language code for this subtree.

boolean (optional) buttonMixed

If a checkbox or toggle button is in the mixed state.

integer (optional) color

The RGBA foreground color of this subtree, as an integer.

integer (optional) backgroundColor

The RGBA background color of this subtree, as an integer.

integer (optional) colorValue

The RGBA color of an input element whose value is a color.

array of AutomationNode children

Walking the tree.

AutomationNode (optional) parent
AutomationNode (optional) firstChild
AutomationNode (optional) lastChild
AutomationNode (optional) previousSibling
AutomationNode (optional) nextSibling
AutomationNode (optional) nextOnLine
AutomationNode (optional) previousOnLine
integer (optional) indexInParent

The index of this node in its parent node's list of children. If this is the root node, this will be undefined.

function doDefault

Does the default action based on this node's role. This is generally the same action that would result from clicking the node such as expanding a treeitem, toggling a checkbox, selecting a radiobutton, or activating a button.

function focus

Places focus on this node.

function getImageData

Request a data url for the contents of an image, optionally resized. Pass zero for maxWidth and/or maxHeight for the original size.

Parameters
integer maxWidth
integer maxHeight
function makeVisible

Scrolls this node to make it visible.

function setSelection

Sets selection within a text field.

Parameters
integer startIndex
integer endIndex
function setSequentialFocusNavigationStartingPoint

Clears focus and sets this node as the starting point for the next time the user presses Tab or Shift+Tab.

function showContextMenu

Show the context menu for this element, as if the user right-clicked.

function resumeMedia

Resume playing any media within this tree.

function startDuckingMedia

Start ducking any media within this tree.

function stopDuckingMedia

Stop ducking any media within this tree.

function suspendMedia

Suspend any media playing within this tree.

function addEventListener

Adds a listener for the given event type and event phase.

Parameters
EventType eventType
boolean capture
function removeEventListener

Removes a listener for the given event type and event phase.

Parameters
EventType eventType
boolean capture
function domQuerySelector

Gets the first node in this node's subtree which matches the given CSS selector and is within the same DOM context.

If this node doesn't correspond directly with an HTML node in the DOM, querySelector will be run on this node's nearest HTML node ancestor. Note that this may result in the query returning a node which is not a descendant of this node.

If the selector matches a node which doesn't directly correspond to an automation node (for example an element within an ARIA widget, where the ARIA widget forms one node of the automation tree, or an element which is hidden from accessibility via hiding it using CSS or using aria-hidden), this will return the nearest ancestor which does correspond to an automation node.

Parameters
string selector
function find

Finds the first AutomationNode in this node's subtree which matches the given search parameters.


Returns AutomationNode.
Parameters
FindParams params
function findAll

Finds all the AutomationNodes in this node's subtree which matches the given search parameters.


Returns array of AutomationNode.
Parameters
FindParams params
function matches

Returns whether this node matches the given automation.FindParams.


Returns boolean.
Parameters
FindParams params

Methods

getTree

chrome.automation.getTree(integer tabId, function callback)

Get the automation tree for the tab with the given tabId, or the current tab if no tabID is given, enabling automation if necessary. Returns a tree with a placeholder root node; listen for the "loadComplete" event to get a notification that the tree has fully loaded (the previous root node reference will stop working at or before this point).

Parameters
integer (optional) tabId
function callback

Called when the AutomationNode for the page is available.

The callback parameter should be a function that looks like this:

function( AutomationNode rootNode) {...};
AutomationNode rootNode

getDesktop

chrome.automation.getDesktop(function callback)

Get the automation tree for the whole desktop which consists of all on screen views. Note this API is currently only supported on Chrome OS.

Parameters
function callback

Called when the AutomationNode for the page is available.

The callback parameter should be a function that looks like this:

function( AutomationNode rootNode) {...};
AutomationNode rootNode

getFocus

chrome.automation.getFocus(function callback)

Get the automation node that currently has focus, globally. Will return null if none of the nodes in any loaded trees have focus.

Parameters
function callback

Called with the AutomationNode that currently has focus.

The callback parameter should be a function that looks like this:

function( AutomationNode focusedNode) {...};
AutomationNode focusedNode

addTreeChangeObserver

chrome.automation.addTreeChangeObserver(enum of "noTreeChanges", "liveRegionTreeChanges", "textMarkerChanges", or "allTreeChanges" filter, function observer)

Add a tree change observer. Tree change observers are static/global, they listen to changes across all trees. Pass a filter to determine what specific tree changes to listen to, and note that listnening to all tree changes can be expensive.

Parameters
enum of "noTreeChanges", "liveRegionTreeChanges", "textMarkerChanges", or "allTreeChanges" filter
function observer

A listener for changes on the AutomationNode tree.

The observer parameter should be a function that looks like this:

function( TreeChange treeChange) {...};
TreeChange treeChange

removeTreeChangeObserver

chrome.automation.removeTreeChangeObserver(function observer)

Remove a tree change observer.

Parameters
function observer

A listener for changes on the AutomationNode tree.

The observer parameter should be a function that looks like this:

function( TreeChange treeChange) {...};
TreeChange treeChange

setDocumentSelection

chrome.automation.setDocumentSelection( SetDocumentSelectionParams params)

Sets the selection in a tree. This creates a selection in a single tree (anchorObject and focusObject must have the same root). Everything in the tree between the two node/offset pairs gets included in the selection. The anchor is where the user started the selection, while the focus is the point at which the selection gets extended e.g. when dragging with a mouse or using the keyboard. For nodes with the role staticText, the offset gives the character offset within the value where the selection starts or ends, respectively.

Parameters
SetDocumentSelectionParams params