ATK text pitfalls

As soon as I ensured myself I've got a good understanding of ATK text they put me back into reality. One more time I must admit myself that ATK text is unknowable like the universe. Seriously, shortly after I started the work on fixing ATK text bugs in Firefox then Orca, a Linux screen reader, suddenly felt bad. I've been suggested to compare Firefox and GEdit to see if there's a difference in implementations. So did I and then I realized that results depend on whence you start the ATK spec reading (btw, GEdit implementation doesn't always follow the spec). If you read the spec from beginning (a first sentence) then you get one result. If you read it from the end (a second sentence) then you might conclude that a different result is expected. I filed a bug against ATK. But let's read it again together, I might be missing something.

Let's consider an example: "a funny word".

* atk_text_get_text_at_offset for BOUNDARY_WORD_END

The returned string is from the word end before the offset to the word end at or after the offset.

I think you will agree that there's no word end *before* 0 offset so it can be treated as an author error. ATK doesn't say how error values should be handled so I guess any reasonable return value is allowed. Firefox returned a ('', 0, 0) triplet and that confused Orca.

Read the spec next:

The returned string will contain the word at the offset if the offset is inside a word.

This means we should return a ('a', 0, 1) triple because 0 offset is inside 'a' word (btw, that's what GEdit did).


* atk_text_get_text_at_offset for BOUNDARY_WORD_START

It is a dual problem to the issue above for the offset equal to a text length. Spec says:

The returned string is from the word start at or before the offset to the word start after the offset.

and

The returned string will contain the word at the offset if the offset is inside a word.

There's no word start after the offset but the same time the offset is inside 'word' word. Reading next.


* atk_text_get_text_after_offset for BOUNDARY_WORD_END

The returned string is from the word end at or after the offset to the next work end.

It might be not evident but 0 offset is a word end offset. A proof by contradiction. If 0 offset is not the end offset then

get_text_at_offset(0, BOUNDARY_WORD_END)

in case of single word (like 'word') should return an empty text. But this contradicts to get_text_at_offset method name semantic and Orca expectations (see the case above). Therefore 0 offset is a word end.

Then it means that the method at 0 offset should return the first word ('a' in our example). But the second sentence says that it must be a second word ('funny' in our case).

The returned string will contain the word after the offset if the offset is inside a word.

* atk_text_get_text_before_offset for BOUNDARY_WORD_START.

This is a dual problem to get_text_after_offset (word end boundary) case. Let's take an offset equal to text length.

The returned string is from the word start before the word start before or at the offset to the word start before or at the offset.

Text length offset is a word start offset. A proof is by analogy (see above). That means that 3d word is expected ('text' in our case). However the second sentence says that it should be a 2nd word ('funny' in our case):

The returned string will contain the word before the offset if the offset is inside a word.

IAccessible2 text vs ATK text

We started our accessible text rework in Firefox. It's time to revisit our IAccessible2 text implementation and compare it to ATK text.

Both IAccessible2 and ATK allows you to navigate the text by characters, words and lines. To do so they both provide these three methods:

• get text *at* offset
• get text *after* offset
• get text *before* offset

It makes clear that these methods are used to get the text at, after or before offsets. It's true in IAccessible2 world but it's no so univocal in case of ATK. ATK provides bunch of tricky boundary types that may change things. Below I will example the difference.

A difference #1. IAccessible2 getTextAtOffset may return nothing when you asked for a word. This happens when the given offset is between words (see the spec).

If the index is valid, but no suitable word (or other text type) is found, a NULL pointer is returned.

ATK always returns a word, no matter where you are because it requires to return the text between two word start or word end offsets (check out the spec).

A difference #2. IAccessible2 getTextAfter/BeforeOffset always return the requested lexem after/before the given offset.

Returns the substring of the specified text type that is located after/before the given character and does not include it. The result of this method should be same as a result for IAccessibleText::textAtOffset with a suitably increased/decreased index value.

ATK get_text_after/before_offset methods may return a lexem at the offset under certain circumstances. For example here's the case of word end boundary in get_text_after_offset method:

If the boundary_type is ATK_TEXT_BOUNDARY_WORD_END the returned string is from the word end at or after the offset to the next work end.

In short ATK and IAccessible2 supply two resembling but different concepts of text traversal. ATK allows you to move through the whole text by any method. For example, text_get_text_at_offset works nice if you pass the end offset you obtained at previous call. In IAccessible2 word you need to use a couple consisting of getTextAtOffset and getTextBefore/AfterOffset to move backward/forward. I'd say IAccessible2 text is simplified (a human friendly) version of ATK text. I won't object if somebody says that ATK looks more powerful. But the same time I'm not sure screen readers need this power. 

Anyway IAccessible2 text  looked so close to ATK and thus originally we mapped IAccessible2 boundary constants into ATK constants and we ended up with shared logic between these APIs. It was done several years ago and we never revisited this code.

Now we are going to change. IAccessible2 consumers please keep the track of our progress to catch regressions early.

Accessible text support in Firefox

So getting back shortly after ATK text insight to share our plans about text support in Firefox. In fact we started the work already and couple patches were landed into Firefox 22 (currently in Aurora stage). The primary work is targeted to Firefox 23 and Firefox 24.

We need to know if the changes we introduce deliver any problems for screen readers. Recently we figured out that we broke Orca. We cherished our bugs through years and probably screen readers learned how to live with them. Putting a screen reader into fresh water might make it feel bad. I guess that's what happens with Orca.

So it's worth to give the assistive technology and the users a notice and ask them to keep eyes open. And of course be ready to report bugs against us.

 

An overview

The text problem is wide and consists of several parts (or has several layers like ogres or onions).

Caret navigation

Caret navigation in Firefox is not good in general. The behavior is inconsistent and

accidental. From time to time I observe weirdness myself writing a next blog post. The caret may move somewhere I didn't want to or it may hang somewhere in the text and I need to use mouse to move it. I don't exclude a possibility that Google Blogger product "helps" the Firefox but I don't recall any web app for rich text editing running really smooth in Firefox. If you deal with lists, tables or control elements then as a rule you should be ready to surprises.

We can't bypass a caret navigation problems, for example, recently I fixed one bug but it's likely we won't pay much attention until UIA text implementation.

ATK and IAccessible2 text

These interfaces are very similar. IA2 interface is simplified (I'd say a human friendly) version of ATK. Actually this is a reason that these interfaces have shared implementation in Firefox. We have a *lot* of bugs related to this implementation.

UIA text

UIA has a different concept of accessible text support and likely requires a separate implementation. We don't have immediate plans on it.

 

Technical part

In short our plans is ATK/IAccessible2 text bugs. On technical layer the work is scoped by following items:

* Fix ATK/IAccessible2 getTextAt/Before/After method logic. Check out our understanding of ATK text to make sure our understanding is the same with yours one.
* Fix soft line breaks problem. Currently start and end of lines produced by soft line break share the same offset. So we never know which line we should return when that offset is given.
* Do not use embedded characters for inline elements.

Please keep your eyes open and let us know if you observe a problem when reading the text in your favorite screen reader.

An easy way to understanding the ATK text

As far as I remember myself I've always been in touch with ATK text interface implementation in Mozilla. I started from writing and reviewing some patches in far 2006 year. But I didn't really understand that piece of code so I wasn't sure that a change here don't break things there. At some point we decided that we should get an automated test coverage for the text interface before we do any serious work in this area. At least that allowed us to be sure to a certain extent we don't regress badly from a single bug fix. And then I helped my colleagues in test suite creation. As part of this work we caught a bug in ATK spec (thanks to Evan Yan, a Mozilla community member and those times Sun engineer). So that wasn't easy. It wouldn't be a lie if I said that I've never seen a more complicated API for the things it's designed for.

I should notice that roughly speaking IAccessible2 text interface implementation in Firefox is done via ATK text interface. So having a bad implementation in ATK we deliver all bugs right to IAccessible2 screen readers. It's a hot problem in other words. Recently I've felt myself brave enough (again) to say: we should stop this shame. And I started to look at the code and the spec trying to untwist things. And then I realized I still don't have a good perception of ATK text. First of all I thought it'd be good to add some drawings to stingy ATK spec to let me and everybody else check easily whether the expected results are correct actually.

Preliminaries

ATK provides bunch of methods to get a text:

• atk_text_get_text_before_offset
• atk_text_get_text_at_offset
• atk_text_get_text_after_offset

Of course ATK provides a bunch of other methods but they are trivial and it doesn't make sense to even mention them. Each of methods above take AtkTextBoundary as an argument and its values are:

• char (trivial)
• word start and word end
• line start and line end
• sentence start and sentence end (not implemented in Firefox)

So we have get_text_before/at/after methods and word/line start/end offsets. This is a subject of the talk.

About terms: a chapter for the advanced

If you didn't planned to read the ATK spec or dig into details then you can skip this chapter and move to the pictures part. Otherwise this chapter might be useful since it has some clarifications.

First of all, here are some terms which are used in the spec but aren't defined there:

• word start offset - an offset where the word starts, for example, "hello, all" has two word start offsets: 0 for "hello" and 7 for "all";
• word end offset - 5 for "hello", i.e. an offset after 'o' character, and 10 for "all";
• inside word offset - any offset between word start and end offset (including boundaries), in our case these are 0-5 and 7-9 offsets;
• outside word offset - everything that is not inside a word, in our case this is only 6 offset.

It's pretty much the same for line start and line end offsets.

Also we need to mention edge cases: imaginary offsets. Say we have a paragraph:

  <p>hello</p>

and then we do

  gint startOffset = 0, endOffset = 0;
  atk_text_get_text_at_offset(accessible, 1, 
                              ATK_TEXT_BOUNDARY_WORD_START,
                              &startOffset, &endOffset);
  atk_text_get_text_at_offset(accessible, 1, 
                              ATK_TEXT_BOUNDARY_WORD_END,
                              &startOffset, &endOffset);

In both cases we expect "hello" string with (0, 5) start and end offsets otherwise there's no way traverse this paragraph by words. But actually it goes with spec: "The returned string will contain the word at the offset if the offset is inside a word". But this means that 0 and 5 offsets are both start and end offset the same time because "the returned string is from the word start at or before the offset to the word start after the offset" and "the returned string is from the word end before the offset to the word end at or after the offset".

Summarizing it all a zero offset (0) and a last offset (character count) are special offsets and can be treated as word start, word end, line start and line end offsets.

A Quick-n-Easy Guide

Update. The proposed algorithm must be corrected to handle edge offsets properly, see ATK text pitfalls. I won't spend time to update it since Joanie proposed ATK text simplification and hopefully it will be accepted in foreseeable future.

So we are ready to put the spec verbosity into nice pictures.

atk_text_get_text_at_offset

WORD_START and LINE_START boundaries are illustrated this way (X symbol designates the initial offset):

  or    

Move forward to the boundary and then (if was successful) move backward. The start offset is at or before the initial offset, the end offset is after the initial offset.

WORD_END and LINE_END boundaries:

or

Move backward to the boundary and then (if was successful) move forward. The start offset is before the initial offset, the end offset is at or after the initial offset.

atk_text_get_text_before_offset

WORD_START and LINE_START boundaries:

  or  

If the initial offset is the boundary then move backward to find the start offset. Otherwise move backward twice to pick up the start and end offsets.

WORD_END and LINE_END boundaries:

Move backward twice for start and end offsets.

atk_text_get_after_offset

WORD_START and LINE_START boundaries:

Move forward twice for start and end offsets.

WORD_END and LINE_END boundaries:

 or      

If the initial offset is the boundary then move forward to find the end offset. Otherwise move forward twice to pick up the start and end offsets.

That's all. Hallelujah to Love!

P.S. Well if I'm wrong in sayings above then you'd better say it otherwise this will be implemented in Firefox soon ;)

Accessible Mozilla: Tech overview of Firefox 21

Next week Firefox 21 reaches a beta status so it's time to list accessibility related changes we introduced for assistive technology. This release we were focused on under-the-hood improvements and ARIA bugs. Let's start.

ARIA

* We pick up accessible value of ARIA combobox in name computation. An example:

  <label for="test">Flash the screen 
    <div role="combobox">
      <div role="textbox"></div>
      <ul role="listbox" style="list-style-type:none;">
        <li role="option">1</li>
        <li role="option" aria-selected="true">2</li>
        <li role="option">3</li>
      </ul>
    </div>
    times.

  </label> 

In this case accessible name of the label is "Flash the screen 2 times" (refer to bug).

* ARIA grid has editable state by default. ARIA grid editable or readonly states (a case of aria-readonly attribute usage) are inherited by grid cells until aria-readonly attribute on a gridcell overrides it (see bug).

* We no longer expose hidden:false object attribute for aria-hidden="false" because ARIA group members concluded that aria-hidden mirrors CSS display:none (check out the bug for details). In particular this means

  <div aria-hidden="true">
    <input aria-hidden="false">
  </div>

HTML input element is still ARIA hidden.

Sure the author doesn't have any single reason to use aria-hidden="false" in his web app but you know people do crazy things every time. If we exposed it then it could be confusing for screen readers (if they wouldn't know that "false" value is an author error of course). However you may ask why ARIA spec doesn't like to treat "false" value as "true" value like it does for any other error value. It's less work for browsers, no burden for AT. You see that wouldn't bad. I do not know. Go ask your dad.

* We supported ARIA based text attributes. Now you can use aria-invalid="grammar" to mark that your text has a grammatical error. From AT perspective it means that we expose invalid:grammar text attribute.

Unfortunately ARIA spec doesn't look perfect about defining the aria-invalid. For example, it doesn't allow a list of values

  <p aria-invalid="spelling,grammar">my wrong text</p>

and it says nothing about aria-invalid inheritance which would allow you to do a trick

  <p aria-invalid="spelling">
    <span aria-invalid="grammar">my wrong text</span>
  </p>

to say that your text is misspelled and has grammatical error. Also it doesn't say how the browser should resolve collisions between aria-invalid and native spelling support which is also mapped into invalid text attribute.

HTML

* HTML5 main element was implemented. It's exposed as xml-roles:main object attribute on accessibility layer.

* We sorted out name computation for HTML input buttons:

• HTML input@type="button", @type="submit", type="reset" gets name from:
• @value attribute
• @title attribute 
• HTML input@type="image" gets name from:
• @alt attribute
• @value attribute
• @title attribute
• HTML input@type="image" having no valid @src attribute gets name from:
• @alt attribute
• @value attribute
• "Submit Query" - a visible label of the button

Everything else

* We did one more fix in our name computation algorithm and now we don't jam together a plain text and name coming from a control element. For example:

  <label>foo<input type="text" value="bar">baz</label>

The accessible name of the label is "foo bar baz". Visually "foo" and "baz" are separated from HTML input, thus we wrap the control's name by spaces when we compute the label name.

* We learned how to coalesce state change events so you should get only one event when the object changes its state fast enough, for example, it may happen during document loading when state busy of a document accessible is switched quickly.

* You can use magic offsets now to get text attributes.

DOM Inspector for accessibility. New features.

A couple new features were introduced after I wrote the previous post about DOM Inspector (aka DOMi). Note, some features aren't available in release version yet. So if you want to get an access to them then you need to build DOMi yourself.

Text attributes

You can inspect text attributes of the accessible object (also refer to IAccessible2 text attributes). All you need is to choose 'Accessible Properties' view in the right panel (make sure 'Show Accessible Nodes' menu option of 'View' menu is on) and then switch to 'Text attributes' tab as it shown below.

You can see default text attributes for the selected accessible and inspect text attributes and offsets for each text range where text attributes stay permanent. For example if you have

  <p><b>hello</b><i>italic</i></p>

then HTML p element is divided into two ranges, the first text range corresponding to 'hello' text has 'font-weight:700' text attribute, the second text range corresponding to 'italic' text exposes 'font-style:italic' text attribute.

Note, if the selected accessible doesn't support nsIAccessibleText interface then 'Accessible Properties' view doesn't have a 'Text attributes' tab.

Note, this feature is not available in current 2.0.13 version.

Search the tree

'Accessible Tree' view of the left panel gained a JS style search. Now when you open 'JS console' (right click on the selected item and choose 'Evaluate JavaScript' menu item) you can operate on JS object named tree having following methods:

search
  Search though the accessible tree and highlights accessibles complying with
    the given criteria.
  Arguments
    accessible of type nsIAccessible
      The root of the subtree the search is performed in.

    criteriaFunc of type Function
      Function invoked for each traversed accessible to determine whether it complies
        with the criteria.
      Arguments
      accessible of type nsIAccessible
        The traversed accessible.
      Return value
        true if the traversed accessible complies with the criteria, otherwise false.

clearSearch
  Clears search results (unhihglights highlighted accessibles)

So say you want to find all text leaf accessibles in the document:

To do that you write the criteria function and invoke the tree.search() on the document accessible.

  function searchTextLeafs(aAcc)
  {
    // Ci is shortcut for Components.interfaces
    // nsIAccessibleRole is an interface defining accessible roles 
    return aAcc.role == Ci.nsIAccessibleRole.ROLE_TEXT_LEAF;
  }

  // 'accessible' is the selected accessible in the left panel, in our case
  // it's the document accessible.
  tree.search(accessible, searchTextLeafs);

When you press 'Evaluate' button then all text leaf accessibles in the document are highlighted.

I should add that all diversity of Gecko accessibility API is available for you from JS console and that allows you to create any criteria you need.

In short, this search tool is rather for JS hackers but it's very flexible and easy enough.

Build DOM Inspector

From time to time I'm getting asked how to build DOM Inspector (aka DOMi) from sources. It's helpful when you need to access to the latest features which aren't available in release yet or if you want to hack on it.

So if you build Firefox on your machine then building the DOMi is easy.

1. You need to download DOM Inspector sources: hg clone http://hg.mozilla.org/dom-inspector/ inspector 
2. Copy the inspector folder under extension folder of Mozilla sources you build Firefox from
3. Add ac_add_options --enable-extensions=default,inspector into your .mozconfig file
4. Build Firefox

That worked fine for a long time. But recently I noticed it doesn't make a trick and DOMi isn't shown in installed add-ons anymore.

In this case cd firefox-objdir/extension; make; helps to appear the DOMi. When you open Firefox for the first time then you will see an extra tab with request to enable the DOMi. You just click the checkbox and restart Firefox.

After that you can summon DOMi from Tools -> Web Developer -> DOM Inspector menu.