Firefox preferences for screen readers

I realized that nobody yet shared that new user preferences in Firefox were introduced for screen readers so I do it here.

There are couple preferences to make screen readers support smoother. And all of them - you might be not surprised - are Adobe Flash related. Flash is headache for both assistive technology vendors and for Firefox developers because flash is tightly related with hangs, crashes and security problems. Unfortunately we need to balance between usability and stability when dealing with Flash so we provide a bunch of preferences to let the user make a choice.

No plugin sandboxing for JAWS

You probably heard or even encountered yourself that JAWS started to hang occasionally in Firefox 14. It was a regression from bug where OOPP (out of process plugins) option was forced on by default on Windows Vista and later. Since it was done on security proposes then we had to choose between making Firefox unstable for JAWS users and keeping JAWS working. So we let users to decide and introduced a preference

  dom.ipc.plugins.enabled.a11y

The preference is not public and you should add it manually via about:config. For that you need to type about:config into address bar and then choose 'New' -> 'Boolean' items from the context menu. As far as I know you don't need any actions on Flash side because Adobe turns sandboxing off as long as Firefox's one is off.

AFAIK JAWS fixed the issue on their side: they started to deal with flash content as separate document. It is definitely different user experience but it must be better than security hazard. So we have an open bug to remove this pref. If you have a reason why we shouldn't do this then please comment into the bug.

Note, the preference was introduced in Firefox 18 and it has an affect only if JAWS is running.

Delayed plugin creation

We had another report about hanging screen readers (Windows Eyes and JAWS) on web pages with Flash. To workaround the problem we introduced a new boolean preference:

  accessibility.delay_plugins

This pref makes the browser to delay a plugin accessible creation. As a rule it saves from hangs. There's accompanying numeric preference used to specify a value of the delay in seconds:

  accessibility.delay_plugin_time

By default the value is 10 seconds. This means you can't interact with Flash content during 10 secs after the plugin was loaded on the web page. It's definitely bad user experience but it's still an option to keep screen readers working.

You should be aware that user entered value of accessibility.delay_plugins preference is ignored as long as JAWS or Windows Eyes screen reader is detected. In this case a plugin accessible creation is delayed automatically.

This feature was introduced in Firefox 22.

UIA text vs ATK / IAccessible2

After IAccessible2 comparison to ATK text it'd be good to get a quick look at ATK / IAccessible2 APIs and UIA text. Firefox doesn't have UIA implementation yet and there's no nearest plans about it but it's worth to compare these API because one day UIA text might become a good alternative to IAccessible2 I think.

UIA text is closer to user actions since UIA has concept of range which can be moved like a cursor through the web page. You can move it (and extend it) by characters, words, lines the same way as the user would do. And then you can get a text the range is spanned to. This means you won't ever run into restrictions of accessible tree and embedded characters as you probably do in case of IAccesisble or ATK APIs. A couple examples might be good to explain what I mean.

Example #1

If accessible tree is DOM based i.e. it's close to DOM hierarchy then ATK / IA2 text interface implementation might be tricky. For example, the following HTML paragraph

<p>hello
  <aside style="position:absolute;top:0px;left:0px;">meine freund</aside>
my friend</p>

can have the accessible tree

paragraph (html:p)
  text leaf ('hello' text)
  section (html:aside)
    text leaf ('meine freund' text)
  text leaf ('my friend' text)

If the browser is not smart enough then it doesn't remove embedded character designating html:aside element from the text of parent html:p element. In other words if the paragraph text is "hello*my friend" (where * is an embedded character for html:aside) then a screen reader have to to deal with it and it should ignore out of flow content somehow. If the screen reader is not smart enough to ignore that then it will move the user through "meine freund" text when the user moves through the paragraph text.

Example #2

In case of Firefox implementation which tends to use embedded characters for everything you can observe another kind of weird behavior. Screen reader must be smart to move by words, etc because embedded characters are used for inline objects like anchors. For instance if you have

<p>hel<a>lo</a></p>

then screen reader must juggle offsets to detect that this paragraph technically consists of one word. You can get into similar troubles when an anchor is spanned through multiple lines. So if you move by lines then the end offset pointing after embedded character never says to you whether a line end is in the middle or after the embedded object. You need to look into embedded object to detect that. It makes a screen reader logic not performant and not trivial.

A summary 

In short UIA lets you to move through the page in correspondence to web page layout while ATK and IAccessible2 allow you to move in correspondence to the accessible tree. Sometimes it makes a difference. 

So accessible tree dependent approach makes the text implementation not trivial on certain platforms (granted, on Gecko). I'm sure that everything (or mostly everything) can be implemented right on the browser side or can be workarounded by screen readers but implementation in either case must be not seamless. Note, somebody told me that Webkit has a nice ATK text implementation (or nicer than Gecko's one? I don't recall). So it's an attestation it's doable, just can be not easy.

I should notice we didn't prototyped anything yet. Of course before making any judgements (did I?) we need to implement it and screen readers should adopt it. And only after that I have a right to say whether it was so good as it looks. On the other hand ATK appeared years ago, IAccessible2 just adopted and simplified ATK ideas so that Microsoft had enough time after MSAA to invent something nice. So I'm ready to believe they did it.

ATK text pitfalls

As soon as I ensured myself I've got a good understanding of ATK text they put me back into reality. One more time I must admit myself that ATK text is unknowable like the universe. Seriously, shortly after I started the work on fixing ATK text bugs in Firefox then Orca, a Linux screen reader, suddenly felt bad. I've been suggested to compare Firefox and GEdit to see if there's a difference in implementations. So did I and then I realized that results depend on whence you start the ATK spec reading (btw, GEdit implementation doesn't always follow the spec). If you read the spec from beginning (a first sentence) then you get one result. If you read it from the end (a second sentence) then you might conclude that a different result is expected. I filed a bug against ATK. But let's read it again together, I might be missing something.

Let's consider an example: "a funny word".

* atk_text_get_text_at_offset for BOUNDARY_WORD_END

The returned string is from the word end before the offset to the word end at or after the offset.

I think you will agree that there's no word end *before* 0 offset so it can be treated as an author error. ATK doesn't say how error values should be handled so I guess any reasonable return value is allowed. Firefox returned a ('', 0, 0) triplet and that confused Orca.

Read the spec next:

The returned string will contain the word at the offset if the offset is inside a word.

This means we should return a ('a', 0, 1) triple because 0 offset is inside 'a' word (btw, that's what GEdit did).


* atk_text_get_text_at_offset for BOUNDARY_WORD_START

It is a dual problem to the issue above for the offset equal to a text length. Spec says:

The returned string is from the word start at or before the offset to the word start after the offset.

and

The returned string will contain the word at the offset if the offset is inside a word.

There's no word start after the offset but the same time the offset is inside 'word' word. Reading next.


* atk_text_get_text_after_offset for BOUNDARY_WORD_END

The returned string is from the word end at or after the offset to the next work end.

It might be not evident but 0 offset is a word end offset. A proof by contradiction. If 0 offset is not the end offset then

get_text_at_offset(0, BOUNDARY_WORD_END)

in case of single word (like 'word') should return an empty text. But this contradicts to get_text_at_offset method name semantic and Orca expectations (see the case above). Therefore 0 offset is a word end.

Then it means that the method at 0 offset should return the first word ('a' in our example). But the second sentence says that it must be a second word ('funny' in our case).

The returned string will contain the word after the offset if the offset is inside a word.

* atk_text_get_text_before_offset for BOUNDARY_WORD_START.

This is a dual problem to get_text_after_offset (word end boundary) case. Let's take an offset equal to text length.

The returned string is from the word start before the word start before or at the offset to the word start before or at the offset.

Text length offset is a word start offset. A proof is by analogy (see above). That means that 3d word is expected ('text' in our case). However the second sentence says that it should be a 2nd word ('funny' in our case):

The returned string will contain the word before the offset if the offset is inside a word.

IAccessible2 text vs ATK text

We started our accessible text rework in Firefox. It's time to revisit our IAccessible2 text implementation and compare it to ATK text.

Both IAccessible2 and ATK allows you to navigate the text by characters, words and lines. To do so they both provide these three methods:

• get text *at* offset
• get text *after* offset
• get text *before* offset

It makes clear that these methods are used to get the text at, after or before offsets. It's true in IAccessible2 world but it's no so univocal in case of ATK. ATK provides bunch of tricky boundary types that may change things. Below I will example the difference.

A difference #1. IAccessible2 getTextAtOffset may return nothing when you asked for a word. This happens when the given offset is between words (see the spec).

If the index is valid, but no suitable word (or other text type) is found, a NULL pointer is returned.

ATK always returns a word, no matter where you are because it requires to return the text between two word start or word end offsets (check out the spec).

A difference #2. IAccessible2 getTextAfter/BeforeOffset always return the requested lexem after/before the given offset.

Returns the substring of the specified text type that is located after/before the given character and does not include it. The result of this method should be same as a result for IAccessibleText::textAtOffset with a suitably increased/decreased index value.

ATK get_text_after/before_offset methods may return a lexem at the offset under certain circumstances. For example here's the case of word end boundary in get_text_after_offset method:

If the boundary_type is ATK_TEXT_BOUNDARY_WORD_END the returned string is from the word end at or after the offset to the next work end.

In short ATK and IAccessible2 supply two resembling but different concepts of text traversal. ATK allows you to move through the whole text by any method. For example, text_get_text_at_offset works nice if you pass the end offset you obtained at previous call. In IAccessible2 word you need to use a couple consisting of getTextAtOffset and getTextBefore/AfterOffset to move backward/forward. I'd say IAccessible2 text is simplified (a human friendly) version of ATK text. I won't object if somebody says that ATK looks more powerful. But the same time I'm not sure screen readers need this power. 

Anyway IAccessible2 text  looked so close to ATK and thus originally we mapped IAccessible2 boundary constants into ATK constants and we ended up with shared logic between these APIs. It was done several years ago and we never revisited this code.

Now we are going to change. IAccessible2 consumers please keep the track of our progress to catch regressions early.

Accessible text support in Firefox

So getting back shortly after ATK text insight to share our plans about text support in Firefox. In fact we started the work already and couple patches were landed into Firefox 22 (currently in Aurora stage). The primary work is targeted to Firefox 23 and Firefox 24.

We need to know if the changes we introduce deliver any problems for screen readers. Recently we figured out that we broke Orca. We cherished our bugs through years and probably screen readers learned how to live with them. Putting a screen reader into fresh water might make it feel bad. I guess that's what happens with Orca.

So it's worth to give the assistive technology and the users a notice and ask them to keep eyes open. And of course be ready to report bugs against us.

 

An overview

The text problem is wide and consists of several parts (or has several layers like ogres or onions).

Caret navigation

Caret navigation in Firefox is not good in general. The behavior is inconsistent and

accidental. From time to time I observe weirdness myself writing a next blog post. The caret may move somewhere I didn't want to or it may hang somewhere in the text and I need to use mouse to move it. I don't exclude a possibility that Google Blogger product "helps" the Firefox but I don't recall any web app for rich text editing running really smooth in Firefox. If you deal with lists, tables or control elements then as a rule you should be ready to surprises.

We can't bypass a caret navigation problems, for example, recently I fixed one bug but it's likely we won't pay much attention until UIA text implementation.

ATK and IAccessible2 text

These interfaces are very similar. IA2 interface is simplified (I'd say a human friendly) version of ATK. Actually this is a reason that these interfaces have shared implementation in Firefox. We have a *lot* of bugs related to this implementation.

UIA text

UIA has a different concept of accessible text support and likely requires a separate implementation. We don't have immediate plans on it.

 

Technical part

In short our plans is ATK/IAccessible2 text bugs. On technical layer the work is scoped by following items:

* Fix ATK/IAccessible2 getTextAt/Before/After method logic. Check out our understanding of ATK text to make sure our understanding is the same with yours one.
* Fix soft line breaks problem. Currently start and end of lines produced by soft line break share the same offset. So we never know which line we should return when that offset is given.
* Do not use embedded characters for inline elements.

Please keep your eyes open and let us know if you observe a problem when reading the text in your favorite screen reader.