Alex's blog

Windows 8 Narrator on the web

2012-03-01T09:21:00.000-08:00

I should notice I'm not a screen reader user and therefore my front-end observations might be not complete or not accurate. Please correct me if I say something wrong.

Windows 8 beta is out. I saw a bunch of posts saying that Microsoft improved accessibility support, in particular that Narrator was greatly refined over previous versions including outstanding performance in IE. So I wanted to play with Narrator and see how it works in Firefox.

The background of the interest is sort of pragmatic. I'm curious whether the market share of screen readers can be shifted after Windows 8 release. The rationally is simple. If some OS provides a set of built-in applications then the user gets all-in-one solution. And if those applications are good enough then many users don't need anything else. So I wasn't going to do any truth worth research but I wanted to see myself how the things look.

On Windows Mozilla bets on IAccessible2, an open standard developed by number of companies including both AT client vendors (screen readers, screen magnifiers etc) and AT server vendors. I don't think this is going to change but if Firefox wasn't nice on Narrator and if Narrator takes a certain percent of the market share then we at Mozilla can't leave this unaddressed.

This time I didn't pay an attention to touch-based accessibility features despite this is definitely a crucial part for mobile web. But this is certainly big area deserving own topic. I focused on how Narrator is doing on the web in classical desktop case.

Narrator

The first noticeable thing is Narrator highlights traversed objects similar to what Voice Over does. This should be a neat feature for teaching and training. I liked the help dialog to see command shortcuts which is good for newbies. Another interesting feature the Narrator allows to bring a dialog containing the list of available actions on the current object and the user can pick up the one to invoke.

Narrator is fast, at least I didn't noticed any lags when I navigated the web. Moreover I've got an impression that sometimes it's faster than NVDA (both on IE and Firefox).

It provides hierarchical virtual cursor: arrow keys allows you to move into container, navigate through siblings within a container and move outside a container. It seems that the implementation is quite plain, it just moves the virtual cursor through accessible tree. You may consider this as a benefit since often this allows you to skip easily a large portions of the page. However taking into account that containers aren't provided by any good description then this is rather useless feature, at least until the user learns the hierarchy for each web site he visits. To Narrator credit I must say it announces the amount of items within certain containers.

On the other hand the web pages and especially web applications have quite complicated trees and that makes navigation not trivial. I'm not experienced user but I feel like NVDA approach is nicer. Narrator approach sounded for me like it's more suitable for tree inspection for debugging proposes rather than for document navigation in the wild.

I looked how Narrator goes with some applications like GMail. I was able to navigate the whole GMail, however sometimes the virtual cursor reached a certain point and didn't move further. I was forced to use tab to skip this place. Also I wasn't able to locate some menus of menu buttons. I think this is because these menus weren't in hierarchy of the button. So you might want to blame Gmail.

Narrator allows to navigate the text, provides commands to traverse the page by paragraphs and supports table navigation but I didn't make it to do more complicated structural navigation like navigation by headings, links etc. It doesn't sound like Narrator has it.

MSAA / UIA / IAccessible2

Windows 8 makes some work to map MSAA to UIA and that allows Narrator to work with MSAA applications. That makes Firefox accessible in principle. For example, you can navigate the web pages and do basic interaction with control elements like open a link, click a button etc. But MSAA accessibility is quite restricted.

Narrator allows the user to navigate by paragraphs, lines, words and characters. It provides some support of text attributes. None of these features work in Firefox. When the document is loaded then Narrator starts reading it. Again not in Firefox.

In summary Firefox doesn't look cool in Narrator. It's sort of expected because Firefox doesn't implement UIA.

I didn't really believe that Microsoft will bridge IAccessible2 to UIA but I hoped. That would allow a number of applications like Eclipse, Open Office or Firefox to work great for Narrator users. Sometimes dreams may not come true.

How I booked the flight or the curse of cancel button

2012-02-29T08:40:00.000-08:00

Recently I booked the flight for me and my family. After some search I stopped at Rossiya airlines (Saint-Petersburg's airlines). It was new airlines for me but they offered the best option.

So I started booking and somewhere in the middle I decided to use another bank card. I pressed cancel button (instead the pay button) and it says the payment failed. No option to retry.

I tried to dial the call-center of Rossiya airlines, several times, but no any luck to ring till the bell is answered. So I wrote them email and in few minutes I got an answer:

Unfortunately you can't pay by bank card online if you rejected to do that once. But you can pay in our office, choose another payment type or book another flight.

Sure I tried to book the flight again but the price surprised me. It wasn't the best option anymore. So I decided to pay existing one and since another payment type usually assumes a noticeable fee then I said:

Ok. What is address of your office in my city.

They said:

Unfortunately we don't have offices in your city. You can try to pay by VTB-24 ATM or in VTB-24 bank office.

I asked about the fee. They said no fee. Ok, nice.

I located VTB-24 ATM near of my house and tried to pay. When I typed the payment info then the ATM said that it can't process this request. I thought maybe VTB-24 ATM can vary, thus I checked Rossiya airlines web site which ATM can be used to make this payment. Next day I tried another VTB-24 ATM. This time when I specified all payment info it said "the feature is not implemented". The farther into the forest the thicker the trees I thought.

So I came to the office of VTB-24 bank to make a payment. They said I can't pay by my bank card. Of course if that was a bank card of their bank then I could do that but otherwise they can't help me and they need a cash. So I moved to ATM of my bank and got that cache, got back to that office and oh happiness I did that.

Say me what a heck I pressed that cancel button.

Firefox 12 for AT developers

2012-02-24T02:15:00.001-08:00

Firefox 12 is still on aurora channel and expected to enter into beta stage at 13 March. It doesn't have much "front-end" features but there's couple ones that might be interesting for AT developers. We did some under the hood work and crash fixes. Also we improved Mac accessibility support but it's still not a part of the default build.

ARIA support gained some sugar for multi-selectable widgets like ARIA list and tree. Now the selected state is exposed on list options and tree items by default, i.e. until aria-selected="false" is specified. This prevents screen readers like JAWS and NVDA to say "unselected" for many ARIA widgets on the web when the user traverse through their items. So a bit of sugar to keep users less annoyed.

We fixed NAVRELATION_EMBEDS MSAA relation on Firefox application accessible (ROLE_SYSTEM_APPLICATION/IA2_ROLE_FRAME). It failed to return a selected tab document accessible under certain circumstances, for example, when Firebug is installed.

Invisible and offscreen states

The most important change is we altered a logic how we expose MSAA INVISIBLE and OFFSCREEN states and ATK VISIBLE and SHOWING states. Let's do some background on this.

ATK VISIBLE means visible or the user can make it visible. ATK SHOWING means visible, i.e. on the screen. See the post for the reference.

MSAA expectations are different. INVISIBLE means not visible on the screen (in either case), OFFSCREEN means it's not visible on the screenbut the user can make it visible. See the post for the reference.

So based on that we'd need to do the following:

on the screen

MSAA: no INVISIBLE & no OFFSCREEN
ATK: VISIBLE & SHOWING

not on the screen

MSAA: INVISIBLE & OFFSCREEN
ATK: VISIBLE & no SHOWING

hidden accessible

MSAA: INVISIBLE & no OFFSCREEN
ATK: no VISIBLE & no SHOWING

However Firefox didn't expose hidden accessible this way on MSAA so we decided to not fix this and keep the logic closer to ATK spec:

on the screen

MSAA: no INVISIBLE & no OFFSCREEN
ATK: VISIBLE & SHOWING

not on the screen

MSAA: no INVISIBLE & OFFSCREEN
ATK: VISIBLE & no SHOWING

hidden accessible

MSAA: INVISIBLE & no OFFSCREEN
ATK: no VISIBLE & no SHOWING

Also we started to treat accessibles from the background as offscreen (previously they were exposed as invisible). The reason of doing this is usually ATs ignore invisible accessibles but they are interested in changes happening in background tab.

If you need more information on this then please refer to the bug to read discussion.

aria-hidden and role="presentation"

2012-02-20T23:51:00.000-08:00

John Foliot pinged me about his blog post devoted to ARIA techniques used to hide the content from assistive technologies. Since I don't have straight answer then I decided to put my thoughts here.

presentation role

ARIA role="presentation" technique is intended to hide an element from AT users. A classical example is presentational images. If you place role="presentation" on the image then it gets removed from accessible tree. Also this technique can be used to remove HTML table semantics, i.e. if role="presentation" is specified on HTML table then table structure is not exposed.

ARIA role="presentation" is completely ignored if it's used on focusable element. That means the element isn't removed from accessible tree and its native semantic is exposed. For example, if you put this role on HTML button then it's revealed as normal HTML button to AT. This is reasonable because if the user focuses the button then focus doesn't get lost and screen reader announces something meaningful.

aria-hidden
The most noticeable difference from role="presentation" is aria-hidden affects on whole subtree. John gave a good use case: hide excess links from screen reader users. A common pattern is a clickable image inside HTML:a element. ARIA spec confirms this saying "Authors MAY, with caution, use aria-hidden to hide visibly rendered content from assistive technologies only if the act of hiding this content is intended to improve the experience for users of assistive technologies by removing redundant or extraneous content."

I should notice that ARIA is not designed to change visual presentation and affect on behavior. Thus if this is an ordinal link then no ARIA technique can be used to hide it from screen reader user since this contradicts to ARIA design. But if the author puts @tabindex="-1" on the link to make it unfocusable then aria-hidden looks like a proper way to achieve desired result.

Technical side

ARIA implementation guide allows to remove elements with aria-hidden from accessible tree but it doesn't require that. Also it states "If the object is in the accessibility tree, map all attributes as normal. In addition, expose object attribute hidden:true". Additionally it points that if the aria-hidden attribute is changed then the browser should emit attribute_changed event (in case of IAccessible2). That's exactly what Firefox does.

So Firefox does a minimal implementation of aria-hidden allowing a screen reader to do what it thinks it should. That means each screen readers should invent a wheel and actually that what happens (or doesn't) - check this table to see how Firefox is different depending on screen reader running.

Implementation of aria-hidden both on browser side and screen reader side is not straightforward. For example if some element having aria-hidden contains a focusable element then its *whole* subtree can't be ignored. At the first glance reasonable solution would be if the browser doesn't create an accessible for each element in subtree until the element is focusable, in other words it treats aria-hidden as role="presentation" was specified on each node in subtree. Otherwise AT needs to crawl the accessible tree to check if there's an element with aria-hidden in ancestor chain. The current Firefox implementation forces AT to do that.

Things to think about.

If the browser doesn't create an accessible for aria-hidden subtree then there is a black box having certain dimensions on the screen. Thus if the user investigates the page layout by mouse then screen reader says nothing when the mouse pointer is above that black box but if the user clicks at this area then he gets unexpected behavior.

On the other hand some screen magnifiers use dimensions of accessible objects for page zooming. No accessible then zoom is likely broken.

So at the second glance I think aria-hidden shouldn't change the tree at all. Instead the browser should expose hidden:true object attribute on every accessible from the subtree of aria-hidden element. That allows AT to decide whether they want to ignore the accessible or not and prevents AT to crawl the accessible hierarchy.

I'd say this rule should be applicable to role="presentation" as well. So if the user explorers the page by mouse then screen reader can say this is presentational image but exclude it from keyboard navigation.

DOM Inspector as accessibility tool

2012-02-10T02:56:00.000-08:00

As you probably know DOM Inspector (aka DOMi) is Firefox add-on designed to inspect web pages and Firefox UI. In some sense it's ancestor of Firebug but perhaps not so fancy and miss some useful features like integrated JavaScript console. Nevertheless DOMi allows you to inspect Firefox UI what makes it quite different from Firebug. You may say there's ChromeBug but it has less amount of features than DOMi, for example, it doesn't allow to inspect XBL elements.

Anyway, DOMi is really nice accessibility tool also. It's not feature complete and far from ideal but it's quite powerful. And that's probably something new for you. Each platform provides set of accessibility tools like accprobe on Windows, accerciser on Linux or built-in tools on Mac. These tools have own benefits and disadvantages but main thing I dislike they don't allow me to inspect DOM. When you need to understand why your web page or your ARIA widget has accessibility issues then it's quite handy to inspect both DOM and accessible trees and what's most important to see *relation* between elements of these trees. That's what DOMi gives you.

DOM Tree

If you check 'Show Accessible Nodes' menu option then 'DOM Nodes' view of the left panel marks accessible DOM nodes by bold font. This view is good to understand where the specific accessible is located in DOM tree.

Accessible Tree

If you don't want to see inaccessible DOM nodes then you can switch to 'Accessible Tree' view of the left panel. The tree displays accessible role, name and related DOM node information for each accessible. Also you can inspect different properties and relations of interesting accessible by selecting it and choosing corresponding view in the right panel.

Also you should choose this view if you want to work with complete accessible tree. 'DOM Nodes' view doesn't always allow you to do this because accessible tree is not a strict subset of DOM tree. For example, XUL trees often don't have underlying DOM however they have rows and cells in accessible tree.

JavaScript Console

JavaScript console is available in 'Accessible Tree' view from context menu on selected accessible. It gets open in separate window (what is not super comfortable) and it doesn't have set of handy functions to help you with tree inspection but you can evaluate arbitrary JS code and call any method on selected accessible object.

Accessible Properties

'Accessible Properties' view of the right panel allows you to inspect accessible properties of the accessible selected in the left panel. You can inspect accessible role, name, states, object attributes i.e. standard set of accessible properties. Also it allows you to invoke accessible actions. Switch to 'Actions' tab to find it out.

Accessible Relations

'Accessible Relations' view of the right panel allows you to see accessible relations and inspect relation targets in separate window.

Accessible Events

'Accessible Events' is a super feature to inspect web pages and ARIA widgets in dynamics. It allows you to

watch accessible events and choose events you want to watch;
add custom event handlers and see output.

If you switch to 'Watched Events' tab then you can choose a number of events you'd like to listen and add custom event handlers. Custom event handlers feature provides you a set of helper functions to inspect accessible tree and accessible properties at time of event handling. For example, you can use outputTree() function to make a snapshot of accessible tree. Small help button marked by '?' is available to get a list of helper functions.

You can see custom event handler output by selecting handled event in the left panel and choosing 'Accessible Event' view in the right panel.

Accessible Event

'Accessible Event' view of the right panel is used to inspect properties of handled accessible event and contains output of custom event handlers. Here's a snapshot of accessible tree when accessible focus event was handled. Highlighted accessible in the tree is focus event target.

Firefox 11 for AT developers

2012-02-08T01:38:00.000-08:00

Here's an update what's new in Firefox 11 (beta, release on March 13) for assistive technology developers.

HTML

CSS generated tables (CSS display:table style) are exposed as layout tables. Originally this piece of work was targeted to Firefox 10 but we weren't in time to make this happen.

HTML table cells (HTML th and td elements) gained new not standard axis and abbr object attributes.

The axis object attribute is direct mapping of HTML @axis attribute. This object attribute is supposed to help AT to extract semantic of rich HTML tables in the web. Granted, this HTML attribute is not wide used on the web but we wanted to break the chicken-egg problem: browsers/AT don't support it iff web authors don't use it.

The abbr object attribute is less academic than previous one and useful to pick up short accessible name for header cells, for example, it makes sense when the user traverses through table cells and screen reader announces related heading information for each cell. The user doesn't want to hear long header cell names on and on: that's what abbr object attribute is supposed to help to. This object attribute is exposed in two cases:

HTML abbr element is inside the table cell
@abbr attribute is used on table cell

Examples:

  <th id="th1"><abbr title="Social Security Number">SS#</abbr></th>
  <th id="th2" abbr="SS#">Social Security Number</th>

HTML acronym and abbr elements allow @title attribute for accessible name computation. This can be illustrated by following example:

  <input id="input">
  <label for="input"><acronym title="O A T F">OATF</acronym></label>

Accessible name of input accessible is "O A T F" now.

HMTL5

HTML5 figure and figcaption elements are now accessible. The figure element is exposed with generic MSAA ROLE_SYSTEM_GROUPING and ATK ATK_ROLE_PANEL roles because neither IAccessible2 nor ATK provide more suitable roles. AT can rely on xml-roles:figure object attribute to detect figure element. HTML figure element picks up accessible name from figcaption element which is exposed with IA2_ROLE_CAPTION / ATK_ROLE_CAPTION role. The figure and figcaption accessible objects are linked by LABELLED_BY / LABEL_FOR relations.

The content of HTML5 canvas element is not accessible still but Firefox 11 started to expose an accessible object having IA2_ROLE_CANVAS/ATK_ROLE_CANVAS role for canvas element itself. Not big deal but it's a good first step on canvas accessibility.

ARIA

ARIA attributes used on HTML file element (input@type="file") are propagated to underlying text field and "Browse" button, i.e. accessible states defined by these attributes are inherited.

ARIA combobox (@role="combobox") fires MSAA EVENT_OBJECT_VALUECHANGE event and ATK accessible-value signal when option is changed. Here's an example of ARIA combobox widget.

Correctness

IA2_STATE_ACTIVE/ATK_STATE_ACTIVE state is exposed on active item for standard composite widgets like HTML select elements. The state can be used for example to detect the current item of the widget when the widget isn't focused. We make our implementation closer to ARIA widgets where aria-activedescendant technique is used. Another side of this code unification is IAccessible::accSelect called with SELFLAG_TAKEFOCUS flag can be used on widget items now.

Small fix for IAccessible::get_accName that returns S_FALSE when the accessible object doesn't have accessible name. Not big deal. Done for consistence and meet MSAA spec.

Frefox merry-go-round

2011-11-24T06:49:00.000-08:00

As you know Mozilla switched to fast release cycle this year and no doubts this is a great decision for Firefox users because they don't need to wait couple years anymore before wanted features are added or disturbing bugs are fixed. But everything would be ok if it weren't so bad. There's a downside of this approach. 3d party vendors of products that rely on Firefox this or that way like screen readers aren't always able to pick up new things so fast as they appear. That can happen due to various reasons for example because of different release cycles. That's why users can suffer and blame screen readers. Screen readers blame Firefox and new ways of collaboration must be figured out. One way is to provide up to dated information what's new for assistive technologies (AT) developers in Firefox. I continue Marco's blog tradition where he gives general overview of accessibility improvements and I will highlight the changes that might be interesting for AT developers.

Since many Firefoxes were released after my last post about Firefox 4 for assistive technologies then I list accessibility related changes for both released and upcoming Firefox versions. Note, the post doesn't cover UI changes in Firefox which can be also important for AT developers.

Firefox 5

Firefox 5 appeared shortly after Firefox 4 and actually it was a release we wanted to see as Firefox 4. Firefox 4 was really great, so innovative and so different from Firefox 3.6 but due to tight schedule we were unable to make sure all changes work perfectly. So Firefox 5 is Firefox 4 that contains several crash fixes and few fixed regressions.

Firefox 6

The most interesting thing happened on Linux is plugins accessibility. Thanks to Brad Taylor who contributed this feature the plugins can be made accessible on Linux now. The solution is based on socket mechanism, refer to bug for technical discussion and example. This work might be found remarkable because it could happen that this was a last thing we did for plugin accessibility. It sounds like era of plugins is ending.

ARIA implementation gained support of MSAA accChild method on ARIA documents. So you can get a child belonging to ARIA document by passing unique ID of the child into accChild method called on document accessible. This is quite neat feature for AT developers since they don't need to have special support for ARIA documents.

We've got some ARIA improvements and HTML5 progress element accessibility support. Some crashes and regressions are fixed. Read Marco's blogpost for details.

Firefox 7

Firefox 7 is mostly under the hood work, less memory usage, more performance, more stability.

ARIA role="presentation" if specified on focusable elements doesn't override accessible role coming from native markup. So if the author puts it by accident on focusable HTML table then table accessible is still exposed to AT.

HTML tables got more correct mapping of summary attribute and caption element to accessibility APIs. HTML5 aside element is exposed as complimentary xml-roles object attribute what makes it an equivalent to ARIA role="complimentary" technique.

Thunderbird 7 is a little more accessible now. Orca announces the number of unread messages when inbox is refreshing. Read more in Marco's blog.

Firefox 8

Access keys and keyboard shortcuts are exposed in ATK for any element (previous releases were restricted to menus). I hope Orca users are little happier now.

IAccessibleRelation::get_targets method was regressed. ATs don't rely on this method currently so the regression was noticed too late after Firefox 8 was shipped. The problem is fixed in Firefox 9.

See Marco's post for more details.

Firefox 9

Currently Firefox 9 is in beta stage. Release is expected in December. It doesn't have much new for AT developers and web authors. However it's worth to notice ARIA autocomplete attribute is now respected when used on HTML input element.

Firefox 10

Firefox 10 is in alpha stage now.

Firefox 10 has a great improvement of focus management. If you still stay on Firefox 3.6 due to some reason then it's time to seriously think if it's worth to continue ;)

New focus management makes different widgets of same type to work consistently.

Listboxes (like HTML select@size>1) claims to be focusable but they never get the focus as long as they have navigable items. Items are focusable and receive focus events.
Comboboxes (like HTML select) and its options are focusable. Combobox gets the focus until it's expanded, otherwise selected option keeps the focus. If the user navigates items in collapsed combobox then no focus event is fired, items receive selection change events and combobox itself receives value change events.
Autocompletes like Firefox awesomebar work similar to comboboxes. Except they don't provide a way to navigate options when autocomplete popup is closed.

Also Firefox 10 has changed the support of selection events. Now it fires the following MSAA events for selectable controls:

EVENT_OBJECT_SELECTION when single item was selected, event is targeted to item.
EVENT_OBJECT_SELECTIONADD and EVENT_OBJECT_SELECTIONREMOVE when item(s) was unselected and/or items were selected, events are targeted to changed items.
EVENT_OBJECT_SELECTIONWITHIN when selection within control was changed much, for example, when all items were selected. In this case screen readers can announce that selection was changed rather than announce every changed item. Event target is control.

ATK emits "selection_changed" signal. It doesn't provide information about how selection was changed and it's supposed to be used as generic notification.

caretOffset method of text interface can be used now to change the caret position within a document. That makes this release a little bit closer to virtual cursor implementations based on Gecko's caret navigation mode.

HTML table 'layout-guess' object attribute now takes into account @datatable="0" attribute placed on HTML table. We did that to make the web is a little more accessible. Web authors should use ARIA role="presentation" instead.

HTML map element used as navigation links container is always exposed to AT even if imagemap techniques are used on it. See an example:

  <map id="mapgroup" title="Navigation Bar" name="mapgroup">
    [<a href="#how">Bypass navigation bar</a>]
    [<a href="home.html">Home</a>]
  </map>

Default state on submit button is exposed correctly. So if the form has two submit buttons by accident then only one which is really default exposes this state. HTML options and XUL tree items don't pick up relations from their controls anymore. These are good fixes allowing the screen readers to avoid the confusion.

HTML5 gained complete support of @datalist attribute on HTML input controls which is exposed as SUPPORTS_AUTOCOMPLETION state and supported by IAccessible2 and ATK APIs.

Random thoughts

2011-11-21T20:11:00.001-08:00

They replaced children's playground by parking lots near of my house. Children don't play soccer, volleyball or tennis anymore. They've got some free time and they can spend it for something else. But don't worry, sooner or later children will grow up and they will buy cars too.

Рядом с моим домом убрали детскую площадку, заменили на автомобильную парковку. Детям негде играть в футбол, волейбол или тенис. У них появилось свободное время, чтобы потратить его на что-нибудь еще. Но не стоит переживать, рано или поздно дети вырастут и у них тоже будут машины.

ARIA autocomples implementation insight

2011-11-17T06:32:00.001-08:00

Autocomple widget is a text field having associated list of options so the user is allowed to type value into text field or choose it from available options. Basically autocomplete is variety of combobox control and the user expects it behaving similarly to combobox, for example, keyboard shortcuts should work mostly the same way.

ARIA allows the author to create autocomplete widgets by putting aria-autocomplete attribute on element having role="textbox" or role="combobox" attributes. There are examples on the web, for instance, here or here.

Usually authors prefer to use HTML input control as a base of autocomplete widget because they get typing implementation for free. All they need is to implement autocomplete list, navigation through list options and support standard combobox shortcuts.

When the user navigates autocomplete list then he's able to start typing to adjust list of available options. From implementation point of view the author tends to keep DOM focus on text field. That makes sense because if he keeps it on option element or autocomplete list then he'd need to implement typing on his own. But what is perceived focus in this case? I'd say the focus is on currently traversed option but when the user starts typing then focus goes into text field and visa versa. It's quite similar to comboboxes: when the user navigates options then focus is on option, if the user dismiss popup then focus goes into combobox itself.

How does it look from AT perspective? When the user navigates through options then AT should announce where the user is. For that the author can manage the DOM focus by tabindex technique, use ARIA live regions or try something else. Tabindex technique is really great because it's mapped into accessibility focus concept. The focus concept exists in all AT APIs what makes it universal and all ATs support it very well. Whenever focus is changed it gets announced to the user. ARIA live regions technique is good too because it's supported well by modern ATs.

Reality is the author doesn't want to manage DOM focus because it means special support for typing and he doesn't want to use ARIA live regions because it's hacky, complicated and sort of weird. So we have something else in the end. ARIA provides one more technique called active descendant which is supposed to be mapped to accessible focus but not restricted to it. So this one looks like authors could rely on.

The author wants to keep DOM focus on text field so he sets aria-activedescendant attribute on autocomplete list element which points to currently traversed option. ARIA implementation guide states that aria-activedescendant attribute change results in accessible focus event on pointed descendant iff the container has DOM focus. Since DOM focus is on text field then there's no focus event. Author can't manage aria-activedescendant on text field since autocomplete popup can't be a child of text field due to markup restrictions and strictly speaking it shouldn't be required to be a logical child (aria-owns). Actually the relation between autocomplete and autocomplete popup should be described by aria-controls attribute but ARIA spec doesn't allow to map aria-activedescendant changes into focus event for elements of this relation.

So what do we have? Some AT APIs (like IAccessible2) has a concept of active descendant and the author could hope that ATs are smart enough so they can pick up active descendant changes and announce them to the user. Reality is active descendant concept is well supported neither by browsers nor by ATs. Some AT APIs (like MSAA) doesn't have it. I'm not aware of other use cases of this concept so that makes me think that active-descendant is going to require special support from ATs. But ATs like focus, they don't want to support new techniques when there's working old one. Autocomple widgets are not special and user interaction can be described in focus terms. That's how Firefox awesome bar works.

What can we do? I think ARIA spec should be changed to extend rules of active-descendant mapping into accessible focus. The following proposition sounds reasonable with me. Allow ARIA menu, listbox and tree widgets controlled (aria-controls) by the widget having DOM focus to manage accessible focus by aria-activedescendant technique. For example:

<input aria-autocomplete="list" aria-controls="autocomplelist">
<ul role="listbox" id="autocompletelist" aria-activedescendant="option1">
<li role="listitem" id="option1">first option</li>
<li role="listitem" id="option2">second option</li>
</ul>

In this case AT should report accessible focus on the first option when the text field has DOM focus.

When HTML table is an accesible table

2011-10-26T22:31:00.000-07:00

In addition to previous topic Data vs layout tables another aspect of the issue is when layout-guess computation algorithm is applied for HTML table elements.

Shortly the algorithm starts working if HTML table element has associated table accessible. In particular that means it doesn't run if HTML table is not accessible, i.e. it doesn't have any associated accessible object in the accessible objects hierarchy. This happens if HTML table is

not visible (CSS display:none or visibility:hidden styles are applied)
not focusable and ARIA role="presentation" is used on it.

Another case is when table element is accessible but it's not a table accessible. From AT perspective the table accessible is an accessible object that exposes a number of table specific interfaces (like IAccessibleTable2 in case of IAccessible2) and has proper accessible roles (like MSAA table or outline roles). So this case is effective (no table accessible is created) when ARIA role used to override native semantics is set on the element, for example role="button". Note, the same time weak ARIA roles like role="banner" don't affect on table semantics and the element gets treated as a table by AT.

In either case HTML table accessibility state affects on its underlying accessible hierarchy. So that if HTML table is not accessible or is not a table accessible then no table structure is exposed, i.e. no rows and no cells.

Also table accessibles may be created for elements different than HTML tables. There are two options to do that:

CSS display:table
table specific ARIA roles like role="grid" or role="treegrid".

The layout-guess algorithm returns true in former case and false in second case.

Data vs layout table

2011-10-19T08:06:00.000-07:00

Tables in accessible world may be treated as data or layout table. The data tables are supposed to contain data and user should be able to navigate through its structure and get information like headings. Typical example of data table is grid. Layout tables are supposed for elements arrangement on the screen and structure of these tables is likely ignored by AT.

Firefox provides semi-standard layout-guess object attribute on table accessible to say whether this table is for data or layout. Shortly, this attribute is exposed for layout tables only, for data table it's omitted. This object attribute wasn't adopted by IAccessible2 or ATK specifications yet but there is discussion on IAccessible2 list.

In Gecko only HTML tables can be treated as layout tables. So the following algorithm is considered in context of HTML tables. This algorithm is pretty static between Firefox releases but Firefox 10 introduces slight changes to it.

Algorithm

Table inside editable area is data table always since the table structure is crucial for table editing
Table having table related ARIA role is data table (like ARIA grid or treegrid)
Table having landmark role is data table
Table having datatable="0" attribute is layout table (Firefox 10)
Table made by CSS display style is layout table (Firefox 10)
Table having summary attribute or legitimate data table structures is data table, these structures are

caption element
col or colgroup elements
tfoot or thead elements
th elements

Table having nested table is layout table
Table having only one row or column is layout table
Table having many columns (>= 5) is data table
Table having borders around cells is data table
Table having differently colored rows is data table
Table having many rows (>= 20) is data table
Wide table (more than 95% of the document width) is layout table
Table having small amount of cells (<= 10) is layout table
Table containing embed, object, applet of iframe elements (typical advertisements elements) is layout table
Otherwise it's data table

Accessible document handling in Firefox

2011-10-19T03:26:00.000-07:00

The accessible document handling is one of tricky parts of accessible Firefox implementation and implicates not evident details. By providing up-to-dated knowledge about this are I hope this information will be interesting for AT developers.

The document accessible is self-contained unit in Firefox and it's mapped into AT APIs by different ways. For example, neither MSAA nor IAccessible2 has similar concept and accessible roles is a primary test used to detect this particular accessible is document accessible. The same time the knowing that this particular accessible is a document accessible having specific document type might be important for AT because it allows to make assumptions about the content and the ways how it should be expose to the user.

Document types
A document accessible may belong to one or more document types. This is mostly terminological split, however documents may be characterized by specific properties depending on type.

Tab documents
The tab document is a document accessible created for web page within the browser tab. Tab document is unique document type what document load events are fired for (see Events section below).

On Windows Gecko provides not standard way to acquire the tab document accessible from any its child accessible object. For that the following GUID:

const GUID SID_IAccessibleContentDocument = {0xa5d8e1f3,0x3571,0x4d8f,0x95,0x21,0x07,0xed,0x28,0xfb,0x07,0x2e}

is used as argument of IServiceProvider::QueryService() method.

Application documents
Application document is a document created for piece of Firefox UI (also referred as chrome). Note, the application document can be a tab document if it's hosted within browser tab like add-ons page.

Dialog documents
Different kinds of dialogs falls into this category (like XUL dialog, XUL wizard elements or ARIA dialogs).

Sub documents
Sub documents are documents contained within tab or application documents. Sub documents may be nested. For example, documents contained by iframe element accessible objects.

Temporary documents
Temporary documents is something we try to avoid but there are few cases when they are created for a short period. This sort of documents is replaced by another type document shortly.

Native documents
Native documents are document accessibles created for DOM documents.

ARIA documents
ARIA can be used to create a document accessible by placing ARIA role attribute on DOM element, for example, "dialog", "document" or "application" roles are used for that. In contrast to native documents ARIA documents can have behavior distinct from native documents. This happens when ARIA role is used on DOM element other than document element or HTML body element, in other words when there's no native document accessible is in base of ARIA document.

Roles
The document accessible can have one of the following accessible roles:

DOCUMENT role
APPLICATION role
DIALOG role

Exposed role depends on document type. Note that ARIA role can be used to change accessible role.

DOCUMENT role
MSAA / ATK roles: ROLE_SYSTEM_DOCUMENT / ATK_ROLE_DOCUMENT_FRAME.
Primary this role is used for web page documents and for their sub documents.

APPLICATION role
MSAA / ATK roles: ROLE_SYSTEM_APPLICATION / ATK_ROLE_EMBEDDED.
The application role is used for chrome documents when they aren't dialogs. For example, it's used for Firefox UI window, add-ons page or error pages.

DIALOG role
MSAA / ATK roles: ROLE_SYSTEM_DIALOG / ATK_ROLE_DIALOG.
Used for dialog documents.

States
The following states related with document life cycle can be exposed by document accessible.

STALE state
IAccessible2 / ATK states: IA2_STATE_STALE / ATK_STATE_STALE.

The state is exposed when document is not attached to the tree (see Document creation section).

BUSY state

MSAA / ATK states: STATE_SYSTEM_BUSY / ATK_STATE_BUSY.
The state is exposed when document is loading or HTTP request was initiated by user (for example, when clicking on link).

Events
The following events related with document life cycle can be fired.

REORDER event
MSAA event: STATE_SYSTEM_REORDER.
Fired on document container accessible when document accessible is attached to the tree.

LOAD_COMPLETE event
IAcessible2 event / ATK signal: IA2_EVENT_DOCUMENT_LOAD_COMPLETE / "load_complete".
Fired on tab document accessible when document is loaded together with its sub documents (referred as completely loaded document).

LOAD_STOPPED event
IAcessible2 event / ATK signal: IA2_EVENT_DOCUMENT_LOAD_STOPPED/ "load_stopped".
Fired on tab document accessible when document loading is cancelled.

RELOAD event
IAcessible2 event / ATK signal: IA2_EVENT_DOCUMENT_RELOAD / "reload".
Fired on tab document when the document is about to reload. 'eventFromUserInput' object attribute on document accessible can be used to detect whether the event was caused by user or programmatically.

STATE_CHANGE event
MSAA event / ATK signal: EVENT_OBJECT_STATECHANGE / "state-change".
Fired on tab document when documents is completely loaded or loading was cancelled. Fired immediately after LOAD_COMPLETE or LOAD_STOPPED events.

Document creation
A document accessible is created whenever it gains DOM content or the accessible was requested by AT. It has STALE state until it gets initial content and gets attached to the tree. After that the REORDER event on the document container accessible is emitted. In case of document that is root in hierarchy the REORDER event is fired on application accesible.
The document keeps BUSY state until it gets completely loaded or loading was cancelled. After that the LOAD_COMPLETE or LOAD_STOPPED events are fired in case of tab document.

Children getting
On Windows the document accessible allows to get any child accessible within it by accessible unique ID (refer to IAccessible::accChild). This is quite useful mechanism to get an event target. Note, this works for ARIA documents as well.

Travel passport online in Russia!

2011-05-03T05:57:00.000-07:00

Finally they saved us from standing in long lines and millions of forms fillings when you need to get social services from government. Now you can fill up millions of forms online. Yep, that's great. My travel passport gets expired soon so I decided to obtain a new one online.

I went to gosuslugi.ru where I were asked to sign up. I entered my ID number (well, it's not ID in terms of USA but close at some precision) and they said they will send me activation number by post mail. "Ok" - I thought and after couple weeks I've got a letter with activation number.

Then I activated my account. It was tabbed UI. First tab "enter ID and activation numbers". Second tab "pick up a password". Third tab: "you see a message that account is activated". So due to some reason they skipped a second tab. They said my account was activated and now I can use it. "Ok" - I thought and tried to use it with empty password. That didn't work. Fortunately, they allowed me to pick up new passport after usual procedure with confirmation by email. "Ok, great" - I thought - "let's move further".

I was ready to fill up the form to request a new travel passport. That was a big form, yep, really big form. I should point who I am, my birthplace, my existing passports, all my jobs and etc, shortly speaking all my biography.

That was a bit unusual UI. I need to choose where I want to pick up the passport when it's ready. You can imagine how Russia is big and how many places to pick up the passport it could have. These options weren't sorted by cities. I listed them one by one looking for a proper one. My wife suggested to start from the end (she had experience to get the travel passport online, well, she didn't get yet) and - oh happiness - my option was in first dozen of options from the end.

I'm not sure why but my browser shows enabled and disabled controls on this web page the same way (I assume that's page styling). The required control elements are marked by red asterisk, this applied to enabled and disabled controls both. Usually when you choose a value in combobox then some controls gets enabled or disabled. So what I did? I clicked any form control to make sure I don't miss anything important.

What UI control would I use if I need to ask a question with answers "yes" or "no". First of all I think of checkboxes. These guys decided to use comboboxes with three options: yes, no and empty (default). I guess they did this because they wanted to make sure the user chooses something and don't skip the question by mistake. Well, this point makes sense but obviously it's not user friendly.

Every region in Russia has a number. So, Irkutsk where I'm located is 38. How many people knows this number and how many people think about the number where they are asked about their region? I'm pretty sure if you have a car then you know the number (because the car number contains a region number). But I'm not sure your answer would be 38 if you were asked about the region you are located. What do I say this for? They have a combobox with options like "38 - Irkutsk region". What do I do when I want to choose my region from list of options? I focus it and start typing "Irk" and etc and expect that I choose the Irkutsk region. Yep, that doesn't happen because they used numbers as prefixes. I run through all options and find a needed one and only then I understand what all these number mean and why I wasn't able to find Irkutsk by typing.

Ok, let's move next. I was asked to point all jobs I had (period, company name, job title, address). They provided a button "add item", by clicking it the new row appears and you can describe next job. They provide neither button "remove item" nor UI to change items order. But I was lucky and typed everything correct.

Finally I reached the form end. I didn't attached my picture yet but clicked on checkbox "Sign the form", I guess that should be similar to the statement "all information I pointed above is true". But to all appearances it wasn't right guess because I've got an alert message saying that I can sign the form in Internet Explorer browser 6.0 or higher only. I'm Firefox user. I'm Mac user. I don't have IE. I hung for a couple minutes thinking in pretty warm way of site developers. And then I saw the last-hope button "save the form on your computer", I clicked it. The form was cleared and I didn't noticed anything else. Firefox downloads dialog doesn't have anything downloaded. Downloads folder is empty.

I think it's too much for me to repeat these steps today if I find IE 6.0 or higher. I'll fill up the form next time.

Firefox 4 for AT developers

2011-03-29T04:53:00.000-07:00

Finally Firefox 4 was released (and finally I finished this post). Marco gave an overview of new accessibility features from user and web developers perspective. I want to tell about technical side, what's new in Firefox 4 for AT developers.

This post doesn't cover all changes introduced in Firefox 4 that may be interesting for AT developers since it's really big area for one blog post. Really, Firefox 4 was developed more than 1.5 year. Near 400 accessibility bugs were fixed. The code is much different than Gecko 1.9.2 (Firefox 3.6) and I don't lie if say more than third of a11y code was changed, redesigned and reworked.

The primary propose of this post is to overview the changes in implementation of ATK and IAccessible2 APIs and highlight important features that come from "under the hood" work.

Performance improvements

Tree traversal.

The accessible tree traversal methods are much faster.

Child by index:

ATK: atk_object_ref_accessible_child
MSAA: get_accChild

Children count:

ATK: atk_object_get_n_accessible_children
MSAA: get_accChildCount

Index of the child in parent:

ATK: atk_object_get_index_in_parent
IA2: indexInParent

Methods implementation depends on platform and thus the speed of the first call varies. On Windows all necessary data is calculated when accessible tree is created, thus method evaluation is always o(1). On Linux the data is cached on demand in some cases, so when method is called for the first time then it can take o(n). The second time it'll be o(1) too. The difference lies in discrepancy between accessible trees on Windows and Linux.

Relations.

Relation calculation is much faster now. That concerns to relations defined by ARIA and native markups both. There are cases when relations from native markup are more complex than o(1) but no heavy tree traversals anymore.

It's time to start to use IAccessbile2 relation methods over MSAA's accNavigate extension of Firefox if you need multiple relation targets. Of course getting all relations is slower than accNavigate but it shouldn't be a bottleneck.

Group position.

Group position is supplied by:

ATK: atk_object_get_attributes, 'level', 'posinset', 'setsize' object attributes
IA2: groupPosition

Group position calculation is smart for most cases. If you request group position for every item in the set then it takes o(n) now, while it took o(n^2) in Firefox 3.6. However there are a few cases where Firefox 4 is not optimal.

Text/Hyperlink/Hypertext interfaces.

Implementation of the following methods shares the same cached data. So the first call of any method calculates the data that is associated with the given hypertext accessible. Performance of the fist call is o(n), where n is a child index pointed by given offset. All subsequent calls deals with the cached data until the accessible tree mutates. When tree was altered then cache is cleared and will be constructed again on demand. Firefox 3.6 performance is o(n) and permanent what may be a bottleneck under certain circumstances, e.g. when AT collect information for every hyperlink.

The following methods take o(1) when data is cached.

Character count:

ATK: atk_text_get_character_count
IA2: nCharacters

Hyperlink start and end offsets:

ATK: atk_hyperlink_get_start_index and atk_hyperlink_get_end_index
IA2: startIndex and endIndex

Hyperlink count:

ATK: atk_hypertext_get_n_links
IA2: nHyperlinks

Hyperlink by index:

ATK: atk_hypertext_get_link
IA2: hyperlink

The following methods take o(log n) when data is cached.

Text between offsets:

ATK: atk_text_get_text
IA2: text

Character at/before/after offset:

ATK: atk_text_get_text_at_offset, atk_text_get_text_after_offset and atk_text_get_text_before_offset for ATK_TEXT_BOUNDARY_CHAR
IA2: textAtOffset, textAfterOffset and textBeforeOffset for IA2_TEXT_BOUNDARY_CHAR

Hyperlink at offset:

ATK: atk_hypertext_get_link_index
IA2: hyperlinkIndex

Redesigned processing of web pages

Lazy tree construction.

Firefox doesn't update accessible tree immediately when the page mutates, it happens asynchronously always. So JS scripts that mutates the DOM heavily can't block UI.

Correct tree.

Firefox 4 got more powerful logic to create accessible tree so that tree creation is performed in unified way. Most discoverable result of the new logic is:

CSS generated content is exposed (CSS :before and :after pseudo-elements)
no DOM text nodes that don't have rendered text are in tree
nested CSS visibility props are handled correctly (for example, the case when hidden contains visible children)

Tree update.

The accessible tree update process is much more robust. Firefox 3.6 had lot of cases when accessible tree wasn't updated properly and therefore Gecko tried to fix it when related information is requested by AT. Correct fix wasn't always possible and led to broken tree (i.e. parent to child and child to parent navigation wasn't consistent) on a case by case basis.

Mutation events.

Another advantage of new tree update logic is mutation events are fired always and don't get lost because of tree update logic gaps. In the other words AT can trust the mutation events to update their cache:

MSAA: EVENT_OBJECT_REORDER, EVENT_OBJECT_SHOW and EVENT_OBJECT_HIDE
ATK: children_chnaged::add and children_changed::remove events.

A note for AT developers on Windows: it's time to consider seriously whether to use show/hide events instead the reorder event. It may be more performant to update accessibles that were added or removed rather than update all child accessibles of the parent the reorder event is delivered for.

Firefox 4 has much smarter event coalescence algorithm that it was, dupe mutation events is not wide case anymore so AT developers can stop the own coalescence and rely on Firefox. It will make AT faster when page is altered in several unrelated places.

Event dispatching.

Firefox 4 practices async event dispatching. The tree can be changed before event is fired. So hide event is delivered for the accessible that is not in tree and text remove events are delivered when text is removed already.

The new design lead to a problem with text removed events on ATK. There are ongoing changes both on Gecko and ATK side. To all appearances the next Firefox and Gnome will get the fix.

Text interface changes

Except the performance improvements introduced in text interface implementation that were said above there are few things worth to mention.

Text attributes.

The text attributes computation algorithm was modified. The range gets splitted by embedded characters, embedded characters are combined into one range. Opposite to Firefox 4 the Firefox 3.6 looks into embedded characters and break a range iff style is changed.

The following example can illustrate new algorithm:

<img>plain<img>plain<img><img><b>bold</b><img><b>bold</b><img>

The text attributes ranges can be presented by notation:

[*)[plain)[*)[plain)[**)[bold)[*)[bold)[*)

* - embedded character, used for image accessible

plain - text without formatting

bold - text with altered text weight

List bullets.

Firefox 4 uses Unicode characters for unordered HTML list bullets depending on bullet type. Firefox 3.6 used the unique '0x2022' character for disc, square and circle list style types.

Polished IAccessibleTable2 implementation

Firefox 3.6 got implementation of IAccessibleTable2 and IAccessibleTableCell interfaces, but not all changes were ready for Firefox 3.6. Firefox 4 gained near complete table support.

Windows hierarchy changes

Shortly the windows hierarchy in Firefox 4 is quite different than in Firefox 3.6. You can read about consequences of this change and approaches how to deal with new hierarchy in David's blog post.

For the reference hierarchies are the following.

The Firefox 3.6 hierarchy:

MozillaUIWindowClass - main window, contains titlebar

MozillaWindowClass - Firefox window

MozillaContentWindowClass - window for tab document

Firefox 4 hierarchy:

MozillaWindowClass