Accessibility goes into DOM

PWFG group suggested two new methods for DOM Element interface. These methods reflect role and name accessibility concepts, and corresponding methods were named as computedRole and computedLabel.

I have bunch of issues with the approach I wanted to outline here. Just to keep things in one place.

The purpose

I've been told that primary reason is a testing propose, but having role and name only is not enough to run UAIG tests or any accessibility automation tool since it would require other accessibility properties.

Also they say that it might be used for non accessibility proposes. I realize that semantics, the ARIA adds, can be used by non assistive technologies. In Firefox we have a large number of non AT consumers but we don't have a good idea in most of cases what they are for. So I don't really have the use case, and thus it's hard to say whether accessible role and name only works well for non a11y proposes.

Concerning to assistive technologies I think they also need a much larger API.

Blowing the DOM

Anything useful should require extra accessible properties as I said above. These are accessible description, states, relations, ability to navigate the hierarchy etc. That means sooner or later the Element interface has to be changed to a great extent. Check out AtkObject to get an idea about possible changes.

In beginning of times accessibility interfaces was built on top of DOM and later they were turned into full APIs. Now we are faced to backward process, accessibility APIs are getting back to DOM. I'm not sure if that's a good idea because accessibility tasks are something very specific, and accessibility API might be not suitable for common needs of web apps.

Restrictions

Not every semantically meaningful piece on the screen has a DOM node, for example, list bullets don't necessary have DOM elements associated with them. So Element based accessibility API is too restrictive to fit the requirements of the assistive technologies.

Performance

Last but not least is performance issue. In most of browsers the accessibility engine is kept separately and it gets running on demand. If accessibility is merged with the DOM then nothing tells the user this method may trigger heavy accessibility computations and make his app slower. Surely the browsers will learn how to get smarter but the approach will have a perf hit either way.

What's it going to be then, eh?

The idea is to provide a separate accessibility interface. If you like it then it can be done by parts, for example, introduce role and name only for the first round same as the original proposal says. Later you can think of adding all other properties.

This idea was welcomed initially, then later it was rejected as being too complex and accessibility centric. But - and that's most important thing - it doesn't have disadvantages the Element approach has.

An easy way to understanding the ATK text

As far as I remember myself I've always been in touch with ATK text interface implementation in Mozilla. I started from writing and reviewing some patches in far 2006 year. But I didn't really understand that piece of code so I wasn't sure that a change here don't break things there. At some point we decided that we should get an automated test coverage for the text interface before we do any serious work in this area. At least that allowed us to be sure to a certain extent we don't regress badly from a single bug fix. And then I helped my colleagues in test suite creation. As part of this work we caught a bug in ATK spec (thanks to Evan Yan, a Mozilla community member and those times Sun engineer). So that wasn't easy. It wouldn't be a lie if I said that I've never seen a more complicated API for the things it's designed for.

I should notice that roughly speaking IAccessible2 text interface implementation in Firefox is done via ATK text interface. So having a bad implementation in ATK we deliver all bugs right to IAccessible2 screen readers. It's a hot problem in other words. Recently I've felt myself brave enough (again) to say: we should stop this shame. And I started to look at the code and the spec trying to untwist things. And then I realized I still don't have a good perception of ATK text. First of all I thought it'd be good to add some drawings to stingy ATK spec to let me and everybody else check easily whether the expected results are correct actually.

Preliminaries

ATK provides bunch of methods to get a text:

• atk_text_get_text_before_offset
• atk_text_get_text_at_offset
• atk_text_get_text_after_offset

Of course ATK provides a bunch of other methods but they are trivial and it doesn't make sense to even mention them. Each of methods above take AtkTextBoundary as an argument and its values are:

• char (trivial)
• word start and word end
• line start and line end
• sentence start and sentence end (not implemented in Firefox)

So we have get_text_before/at/after methods and word/line start/end offsets. This is a subject of the talk.

About terms: a chapter for the advanced

If you didn't planned to read the ATK spec or dig into details then you can skip this chapter and move to the pictures part. Otherwise this chapter might be useful since it has some clarifications.

First of all, here are some terms which are used in the spec but aren't defined there:

• word start offset - an offset where the word starts, for example, "hello, all" has two word start offsets: 0 for "hello" and 7 for "all";
• word end offset - 5 for "hello", i.e. an offset after 'o' character, and 10 for "all";
• inside word offset - any offset between word start and end offset (including boundaries), in our case these are 0-5 and 7-9 offsets;
• outside word offset - everything that is not inside a word, in our case this is only 6 offset.

It's pretty much the same for line start and line end offsets.

Also we need to mention edge cases: imaginary offsets. Say we have a paragraph:

  <p>hello</p>

and then we do

  gint startOffset = 0, endOffset = 0;
  atk_text_get_text_at_offset(accessible, 1, 
                              ATK_TEXT_BOUNDARY_WORD_START,
                              &startOffset, &endOffset);
  atk_text_get_text_at_offset(accessible, 1, 
                              ATK_TEXT_BOUNDARY_WORD_END,
                              &startOffset, &endOffset);

In both cases we expect "hello" string with (0, 5) start and end offsets otherwise there's no way traverse this paragraph by words. But actually it goes with spec: "The returned string will contain the word at the offset if the offset is inside a word". But this means that 0 and 5 offsets are both start and end offset the same time because "the returned string is from the word start at or before the offset to the word start after the offset" and "the returned string is from the word end before the offset to the word end at or after the offset".

Summarizing it all a zero offset (0) and a last offset (character count) are special offsets and can be treated as word start, word end, line start and line end offsets.

A Quick-n-Easy Guide

Update. The proposed algorithm must be corrected to handle edge offsets properly, see ATK text pitfalls. I won't spend time to update it since Joanie proposed ATK text simplification and hopefully it will be accepted in foreseeable future.

So we are ready to put the spec verbosity into nice pictures.

atk_text_get_text_at_offset

WORD_START and LINE_START boundaries are illustrated this way (X symbol designates the initial offset):

  or    

Move forward to the boundary and then (if was successful) move backward. The start offset is at or before the initial offset, the end offset is after the initial offset.

WORD_END and LINE_END boundaries:

or

Move backward to the boundary and then (if was successful) move forward. The start offset is before the initial offset, the end offset is at or after the initial offset.

atk_text_get_text_before_offset

WORD_START and LINE_START boundaries:

  or  

If the initial offset is the boundary then move backward to find the start offset. Otherwise move backward twice to pick up the start and end offsets.

WORD_END and LINE_END boundaries:

Move backward twice for start and end offsets.

atk_text_get_after_offset

WORD_START and LINE_START boundaries:

Move forward twice for start and end offsets.

WORD_END and LINE_END boundaries:

 or      

If the initial offset is the boundary then move forward to find the end offset. Otherwise move forward twice to pick up the start and end offsets.

That's all. Hallelujah to Love!

P.S. Well if I'm wrong in sayings above then you'd better say it otherwise this will be implemented in Firefox soon ;)

Debugging the Firefox accessibility

Firefox 18 got a logging system that allows you to analyze accessibility related problems and bugs. It's used to debug the real life examples like web pages where thousands of things happen every second and you don't have any clue what's going wrong there. If you ever debugged a focus issue on the web then I think you agree the logging is something very helpful on this way. On the other hand logging is a great thing to understand the problem you can't reproduce (reliably).

I didn't tried the system in case when the bug reporter is unique person who can see a problem but it works quite well for analysis of automated tests intermittent failures. The difference is the test suite has own logging capabilities and it writes down every step so that you can find out where the test was stumbled. Having these logs enabled both you can see what action was performed and what happened after.

Here's couple examples how to use logging to understand the intermittent failure.

 

Test suite logging

Almost every accessibility related test is based on events watching, i.e. we wait for certain event before starting the next test. It's all caused by async nature of accessibility implementation in Gecko.

You can start logging by setting gA11yEventDumpToConsole variable to true. A log looks like:

registered: event type: document load complete,
  target: tabDocumentAt, arg: 0
registered: event type: document load complete,
  target: tabDocumentAt, arg: 1

Event queue:
  invoke: relations of tabs

Event type: document load complete.
  Target: ['document node', address: 0x1e3b4850, role: document,
           name: 'The Book of Mozilla, 11:9', address: 0x112ddd90].
Event type: document load complete.
  Target: ['document node', address: 0x1e3b4448, role: document,
           name: 'About:', address: 0xe253250].

*****
EQ matched: document load complete
*****

7233 ERROR TEST-UNEXPECTED-FAIL |
  accessible/relations/test_tabbrowser.xul |
  Test timed out.

It is a real world example. We wait for two document load events and as you see we receive them but only one is recognized as expected one. The problem here is we wait events in specific order but sometimes we receive them in reverse order. It happens because documents are loaded asynchronously and thus events order varies. The test was fixed.

This was a case when the test suite log gives you enough information to fix a problem. But often you know there's no expected event and that's all you have. Extra info is wanted. In this case the system logging can be helpful.

 

System logging

The system logging is used to log implementation internals, i.e. it serves to understand what happens under the hood. There's a bunch of modules you can track, for example, 'focus' module logs everything related with accessible focus event processing. Depending on your needs you can enable one or several modules to log. You can do that by setting up the environment variable A11YLOG, for example:

export A11YLOG=focus,tree,DOMEvents;

Alternatively you can manage the logging from java script via nsIAccessibleRetrieval interface:

var accRetrieval =
  Components.classes["@mozilla.org/accessibleRetrieval;1"].
  getService(nsIAccessibleRetrieval);
accRetrieval.setLogging("focus,tree");

Note, you need to have enhanced privileges for that. If you debug a11y automated tests then you can use enableLogging()/disableLogging() helper functions from common.js.

The system logging can be enabled on any debug or nightly build.

 

An example

Bug 782991 is a good example how to debug intermittent failures. The problem was the following: when link is open in a new window then document load event was missed occasionally. That's all you know. And that means you don't even know whether the window was open or not.

Since it's document loading issue then it's reasonable to enable 'docload' module. A log when test fails:

A11Y DOCLOAD: document loaded; 47:41.689
  {
    DOM id: 17D86000, acc id: 1A9545C0
    uri: chrome://browser/content/browser.xul
  }

A11Y DOCLOAD: document loaded; 47:42.014
  {
    DOM id: 0E337000, acc id: 16C60B80
    uri: http://www.example.com/
  }

A11Y DOCLOAD: document loaded *completely*; 47:42.036
  {
    DOM id: 0E337000, acc id: 16C60B80
    uri: http://www.example.com/
    document acc state: completely loaded
    document is load event target: false
  }

A11Y DOCLOAD: document loaded *completely*; 47:42.048
  {
    DOM id: 17D86000, acc id: 1A9545C0
    uri: chrome://browser/content/browser.xul
    document acc state: completely loaded
    document is load event target: false
  }

As you can see 'example.com' document gets loaded (what means the window was open) but document load event wasn't fired. In practice a log when the test doesn't fail is also helpful to find a problem:

A11Y DOCLOAD: document loaded; 02:50.695
  {
    DOM id: 0x91e97a0, acc id: 0xebaf308
    uri: chrome://browser/content/browser.xul
  }

A11Y DOCLOAD: document loaded *completely*; 02:51.043
  {
    DOM id: 0x91e97a0, acc id: 0xebaf308
    uri: chrome://browser/content/browser.xul
    document acc state: completely loaded
    document is load event target: false
  }

A11Y DOCLOAD: document loaded; 02:51.235
  {
    DOM id: 0xd9732b0, acc id: 0xd11ed40
    uri: http://www.example.com/
  }

A11Y DOCLOAD: document loaded *completely*; 02:51.302
  {
    DOM id: 0xd9732b0, acc id: 0xd11ed40
    uri: http://www.example.com/
    document acc state: completely loaded
    document is load event target: true
  }

A11Y DOCEVENT: handled 'load complete' event; 02:51.302
  {
    DOM id: 0xd9732b0, acc id: 0xd11ed40
    uri: http://www.example.com/
  }

The difference between logs is in order of the document load processing. If the 'example.com' document is processed before a document of the main Firefox window then 'example.com' document isn't considered as a target of document loading events and the test fails. It's enough to understand where the bug is and fix it.

If you look for more examples then check out bugs: bug 745788, bug 708927 and bug 691580.

Accessible document handling in Firefox

The accessible document handling is one of tricky parts of accessible Firefox implementation and implicates not evident details. By providing up-to-dated knowledge about this are I hope this information will be interesting for AT developers.

The document accessible is self-contained unit in Firefox and it's mapped into AT APIs by different ways. For example, neither MSAA nor IAccessible2 has similar concept and accessible roles is a primary test used to detect this particular accessible is document accessible. The same time the knowing that this particular accessible is a document accessible having specific document type might be important for AT because it allows to make assumptions about the content and the ways how it should be expose to the user.

Document types

A document accessible may belong to one or more document types. This is mostly terminological split, however documents may be characterized by specific properties depending on type.

Hierarchical and content-dependent types

Tab document
A document accessible created for web page within the browser tab.

On Windows Gecko provides non-standard way to acquire a tab document accessible from any its child accessible object. For that the following GUID:

const GUID SID_IAccessibleContentDocument = {0xa5d8e1f3,0x3571,0x4d8f,0x95,0x21,0x07,0xed,0x28,0xfb,0x07,0x2e}

is used as argument of IServiceProvider::QueryService() method.

Chrome document
A document created for every main Firefox window.

Application document
A document created for any Firefox UI document (also referred as chrome). Note, the application document can be a tab document if it's hosted within a browser tab like add-ons page. In other words, this is either chrome document or tab document.

Dialog document
Different kinds of dialog documents have this type. For example XUL dialog, XUL wizard elements or ARIA dialogs (like role="dialog").

Sub document
A document contained within tab or application document. Sub documents may be nested. For example, documents contained by iframe element accessible objects.

Temporary document
Temporary documents is something we try to avoid but there are few cases when they are created for a short period. The document of this type is replaced by another document shortly.

Native and not native document types

Native document
Native documents are referred to document accessible created for any DOM document. Native document can belong to any of document types listed above.

Not native document
The document accessible can be created when there's no DOM document underneath. It can be done by ARIA like role="dialog" or "document" or "application" placed on HTML element (other than document element or HTML body) or by HTML5 markup like HTML article element. The behavior of such documents may be different from the behavior of native documents since they don't maintain set of features inherent to native documents.

Roles

The document accessible can have one of the following accessible roles:

• DOCUMENT role
• APPLICATION role
• DIALOG role

Exposed role depends on document type. Note that ARIA role can be used to change accessible role.

DOCUMENT role
MSAA / ATK roles: ROLE_SYSTEM_DOCUMENT / ATK_ROLE_DOCUMENT_FRAME.
Primary this role is used for web page documents and for their sub documents.

APPLICATION role
MSAA / ATK roles: ROLE_SYSTEM_APPLICATION / ATK_ROLE_EMBEDDED.
The application role is used for chrome documents when they aren't dialogs. For example, it's used for Firefox UI window, add-ons page or error pages.

DIALOG role
MSAA / ATK roles: ROLE_SYSTEM_DIALOG / ATK_ROLE_DIALOG.
Used for dialog documents.

States

The following states related with document life cycle can be exposed by document accessible.

STALE state
IAccessible2 / ATK states: IA2_STATE_STALE / ATK_STATE_STALE.

The state is exposed when document is not attached to the tree (see Document creation section).

BUSY state

MSAA / ATK states: STATE_SYSTEM_BUSY / ATK_STATE_BUSY.
The state is exposed when document is loading or HTTP request was initiated by user (for example, when clicking on link).

Events

The following events related with document life cycle can be fired.

REORDER event
MSAA event: STATE_SYSTEM_REORDER.
Fired on document container accessible when document accessible is attached to the tree. Target: any document.

LOAD_COMPLETE event
IAcessible2 event / ATK signal: IA2_EVENT_DOCUMENT_LOAD_COMPLETE / "load_complete".
Fired when document is loaded conjointly with its sub documents (the document is referred as completely loaded document). Target: any document, a root of documents loading chain (Firefox 16 and later).

Compatibility.

• [Firefox 15] target: any native document, a root of documents loading chain. 
• [Prior to Firefox 15] target: a tab document.

LOAD_STOPPED event
IAcessible2 event / ATK signal: IA2_EVENT_DOCUMENT_LOAD_STOPPED/ "load_stopped".
Fired when document loading is cancelled. Target: any native document, a root of documents loading chain (Firefox 15 and later).

Compatibility. [Prior to Firefox 15] target: a tab document.

RELOAD event
IAcessible2 event / ATK signal: IA2_EVENT_DOCUMENT_RELOAD / "reload".
Fired when the document is about to reload. 'eventFromUserInput' object attribute on document accessible can be used to detect whether the event was caused by user or programmatically. Target: any native document, a root of documents loading chain (Firefox 15 and later).

Compatibility. [Prior to Firefox 15] target: a tab document. 

STATE_CHANGE event
MSAA event / ATK signal: EVENT_OBJECT_STATECHANGE / "state-change".
Fired when documents is completely loaded or loading was cancelled. Fired immediately after LOAD_COMPLETE or LOAD_STOPPED events. Target: any native document, a root of documents loading chain (Firefox 15 and later).

Compatibility. [Prior to Firefox 15] target: a tab document. 

Document creation

A document accessible is created whenever it gains DOM content or the accessible was requested by AT.

The document has STALE state until its initial content is loaded and until the document is attached to the tree. After that the REORDER event on the document container is emitted. In case of document that is a root in hierarchy the REORDER event is fired on application accessible.

The document keeps BUSY state until it gets completely loaded or loading was cancelled. After that the LOAD_COMPLETE or LOAD_STOPPED events are fired.

Children getting

On Windows the document accessible allows to get any child accessible within it by accessible unique ID (refer to IAccessible::accChild). This is quite useful mechanism to get an event target. Note, this works for ARIA documents as well.

Updated to Firefox 16.