Finally Firefox 4 was released (and finally I finished this post). Marco gave an overview of new accessibility features from user and web developers perspective. I want to tell about technical side, what's new in Firefox 4 for AT developers.
This post doesn't cover all changes introduced in Firefox 4 that may be interesting for AT developers since it's really big area for one blog post. Really, Firefox 4 was developed more than 1.5 year. Near 400 accessibility bugs were fixed. The code is much different than Gecko 1.9.2 (Firefox 3.6) and I don't lie if say more than third of a11y code was changed, redesigned and reworked.
The primary propose of this post is to overview the changes in implementation of ATK and IAccessible2 APIs and highlight important features that come from "under the hood" work.
Performance improvements
Tree traversal.
The accessible tree traversal methods are much faster.
Child by index:
Index of the child in parent:
Relations.
Relation calculation is much faster now. That concerns to relations defined by ARIA and native markups both. There are cases when relations from native markup are more complex than o(1) but no heavy tree traversals anymore.
It's time to start to use IAccessbile2 relation methods over MSAA's accNavigate extension of Firefox if you need multiple relation targets. Of course getting all relations is slower than accNavigate but it shouldn't be a bottleneck.
Group position.
Group position is supplied by:
Text/Hyperlink/Hypertext interfaces.
Implementation of the following methods shares the same cached data. So the first call of any method calculates the data that is associated with the given hypertext accessible. Performance of the fist call is o(n), where n is a child index pointed by given offset. All subsequent calls deals with the cached data until the accessible tree mutates. When tree was altered then cache is cleared and will be constructed again on demand. Firefox 3.6 performance is o(n) and permanent what may be a bottleneck under certain circumstances, e.g. when AT collect information for every hyperlink.
The following methods take o(1) when data is cached.
Character count:
The following methods take o(log n) when data is cached.
Text between offsets:
Redesigned processing of web pages
Lazy tree construction.
Firefox doesn't update accessible tree immediately when the page mutates, it happens asynchronously always. So JS scripts that mutates the DOM heavily can't block UI.
Correct tree.
Firefox 4 got more powerful logic to create accessible tree so that tree creation is performed in unified way. Most discoverable result of the new logic is:
Tree update.
The accessible tree update process is much more robust. Firefox 3.6 had lot of cases when accessible tree wasn't updated properly and therefore Gecko tried to fix it when related information is requested by AT. Correct fix wasn't always possible and led to broken tree (i.e. parent to child and child to parent navigation wasn't consistent) on a case by case basis.
Mutation events.
Another advantage of new tree update logic is mutation events are fired always and don't get lost because of tree update logic gaps. In the other words AT can trust the mutation events to update their cache:
Firefox 4 has much smarter event coalescence algorithm that it was, dupe mutation events is not wide case anymore so AT developers can stop the own coalescence and rely on Firefox. It will make AT faster when page is altered in several unrelated places.
Event dispatching.
Firefox 4 practices async event dispatching. The tree can be changed before event is fired. So hide event is delivered for the accessible that is not in tree and text remove events are delivered when text is removed already.
The new design lead to a problem with text removed events on ATK. There are ongoing changes both on Gecko and ATK side. To all appearances the next Firefox and Gnome will get the fix.
Text interface changes
Except the performance improvements introduced in text interface implementation that were said above there are few things worth to mention.
Text attributes.
The text attributes computation algorithm was modified. The range gets splitted by embedded characters, embedded characters are combined into one range. Opposite to Firefox 4 the Firefox 3.6 looks into embedded characters and break a range iff style is changed.
The following example can illustrate new algorithm:
The text attributes ranges can be presented by notation:
List bullets.
Firefox 4 uses Unicode characters for unordered HTML list bullets depending on bullet type. Firefox 3.6 used the unique '0x2022' character for disc, square and circle list style types.
Polished IAccessibleTable2 implementation
Firefox 3.6 got implementation of IAccessibleTable2 and IAccessibleTableCell interfaces, but not all changes were ready for Firefox 3.6. Firefox 4 gained near complete table support.
Windows hierarchy changes
Shortly the windows hierarchy in Firefox 4 is quite different than in Firefox 3.6. You can read about consequences of this change and approaches how to deal with new hierarchy in David's blog post.
For the reference hierarchies are the following.
The Firefox 3.6 hierarchy:
This post doesn't cover all changes introduced in Firefox 4 that may be interesting for AT developers since it's really big area for one blog post. Really, Firefox 4 was developed more than 1.5 year. Near 400 accessibility bugs were fixed. The code is much different than Gecko 1.9.2 (Firefox 3.6) and I don't lie if say more than third of a11y code was changed, redesigned and reworked.
The primary propose of this post is to overview the changes in implementation of ATK and IAccessible2 APIs and highlight important features that come from "under the hood" work.
Performance improvements
Tree traversal.
The accessible tree traversal methods are much faster.
Child by index:
- ATK: atk_object_ref_accessible_child
- MSAA: get_accChild
Index of the child in parent:
- ATK: atk_object_get_index_in_parent
- IA2: indexInParent
Relations.
Relation calculation is much faster now. That concerns to relations defined by ARIA and native markups both. There are cases when relations from native markup are more complex than o(1) but no heavy tree traversals anymore.
It's time to start to use IAccessbile2 relation methods over MSAA's accNavigate extension of Firefox if you need multiple relation targets. Of course getting all relations is slower than accNavigate but it shouldn't be a bottleneck.
Group position.
Group position is supplied by:
- ATK: atk_object_get_attributes, 'level', 'posinset', 'setsize' object attributes
- IA2: groupPosition
Text/Hyperlink/Hypertext interfaces.
Implementation of the following methods shares the same cached data. So the first call of any method calculates the data that is associated with the given hypertext accessible. Performance of the fist call is o(n), where n is a child index pointed by given offset. All subsequent calls deals with the cached data until the accessible tree mutates. When tree was altered then cache is cleared and will be constructed again on demand. Firefox 3.6 performance is o(n) and permanent what may be a bottleneck under certain circumstances, e.g. when AT collect information for every hyperlink.
The following methods take o(1) when data is cached.
Character count:
- ATK: atk_text_get_character_count
- IA2: nCharacters
- ATK: atk_hyperlink_get_start_index and atk_hyperlink_get_end_index
- IA2: startIndex and endIndex
- ATK: atk_hypertext_get_n_links
- IA2: nHyperlinks
- ATK: atk_hypertext_get_link
- IA2: hyperlink
The following methods take o(log n) when data is cached.
Text between offsets:
- ATK: atk_text_get_text
- IA2: text
- ATK: atk_text_get_text_at_offset, atk_text_get_text_after_offset and atk_text_get_text_before_offset for ATK_TEXT_BOUNDARY_CHAR
- IA2: textAtOffset, textAfterOffset and textBeforeOffset for IA2_TEXT_BOUNDARY_CHAR
- ATK: atk_hypertext_get_link_index
- IA2: hyperlinkIndex
Redesigned processing of web pages
Lazy tree construction.
Firefox doesn't update accessible tree immediately when the page mutates, it happens asynchronously always. So JS scripts that mutates the DOM heavily can't block UI.
Correct tree.
Firefox 4 got more powerful logic to create accessible tree so that tree creation is performed in unified way. Most discoverable result of the new logic is:
- CSS generated content is exposed (CSS :before and :after pseudo-elements)
- no DOM text nodes that don't have rendered text are in tree
- nested CSS visibility props are handled correctly (for example, the case when hidden contains visible children)
Tree update.
The accessible tree update process is much more robust. Firefox 3.6 had lot of cases when accessible tree wasn't updated properly and therefore Gecko tried to fix it when related information is requested by AT. Correct fix wasn't always possible and led to broken tree (i.e. parent to child and child to parent navigation wasn't consistent) on a case by case basis.
Mutation events.
Another advantage of new tree update logic is mutation events are fired always and don't get lost because of tree update logic gaps. In the other words AT can trust the mutation events to update their cache:
- MSAA: EVENT_OBJECT_REORDER, EVENT_OBJECT_SHOW and EVENT_OBJECT_HIDE
- ATK: children_chnaged::add and children_changed::remove events.
Firefox 4 has much smarter event coalescence algorithm that it was, dupe mutation events is not wide case anymore so AT developers can stop the own coalescence and rely on Firefox. It will make AT faster when page is altered in several unrelated places.
Event dispatching.
Firefox 4 practices async event dispatching. The tree can be changed before event is fired. So hide event is delivered for the accessible that is not in tree and text remove events are delivered when text is removed already.
The new design lead to a problem with text removed events on ATK. There are ongoing changes both on Gecko and ATK side. To all appearances the next Firefox and Gnome will get the fix.
Text interface changes
Except the performance improvements introduced in text interface implementation that were said above there are few things worth to mention.
Text attributes.
The text attributes computation algorithm was modified. The range gets splitted by embedded characters, embedded characters are combined into one range. Opposite to Firefox 4 the Firefox 3.6 looks into embedded characters and break a range iff style is changed.
The following example can illustrate new algorithm:
<img>plain<img>plain<img><img><b>bold</b><img><b>bold</b><img>
The text attributes ranges can be presented by notation:
[*)[plain)[*)[plain)[**)[bold)[*)[bold)[*)
* - embedded character, used for image accessible
plain - text without formatting
bold - text with altered text weight
List bullets.
Firefox 4 uses Unicode characters for unordered HTML list bullets depending on bullet type. Firefox 3.6 used the unique '0x2022' character for disc, square and circle list style types.
Polished IAccessibleTable2 implementation
Firefox 3.6 got implementation of IAccessibleTable2 and IAccessibleTableCell interfaces, but not all changes were ready for Firefox 3.6. Firefox 4 gained near complete table support.
Windows hierarchy changes
Shortly the windows hierarchy in Firefox 4 is quite different than in Firefox 3.6. You can read about consequences of this change and approaches how to deal with new hierarchy in David's blog post.
For the reference hierarchies are the following.
The Firefox 3.6 hierarchy:
- MozillaUIWindowClass - main window, contains titlebar
- MozillaWindowClass - Firefox window
- MozillaContentWindowClass - window for tab document
- MozillaWindowClass
