Browsers traditionally overlook MathML accessibility besides MathML implementation itself by some of them. Because of that there is a bunch of 3d parties software to display MathML and expose math content to assistive technology. They do a good job but what relates to screen readers support it's far from being perfect.
So MathPlayer works with IE only. MathJax, a cross browser math renderer, claims they do accessibility by means of MathPlayer. Last time I checked MathPlayer did a trick by adding accessible properties like accessible name to MathML nodes in accessible tree. There's a bunch of problems with this approach like the user can't control the speech output. However I must admit it makes screen readers to read math.
Main problem here there's no right API and they have to use existing APIs which is Procrustean bed for math. The browser and assistive technology have to invent something that describes math well. Until that there's no right tool to succeed. I should notice that above-mentioned primarily applies to Windows and Linux worlds. Much to my surprise WebKit has got a math accessibility API on OS X last year. I didn't hear VoiceOver picked it up yet but as soon as it does the MathML content should become accessible on Mac.
In Firefox we have a meta bug to track MathML accessibility work. As the first step on this way Firefox 27 started to create generic accessible objects for MathML elements. It's not so valuable by itself because a generic accessible don't expose any math semantic but it allows the assistive technology to navigate the math and watch for tree mutations. Next we could follow WebKit effort by extending ATK and IAccessible2 APIs to make math accessible on Windows and Linux but I have been told libraries that screen readers rely on to process MathML speak also MathML language. It might be unwise to split MathML into atoms of high level API to make the assistive technology to reconstruct MathML on its side.
Gecko exposes ISimpleDOMNode interface providing a direct access to DOM for assistive technology. I never was a devotee of ISimpleDOM because I believe that high-level APIs like IAccessible2 are more efficient. But in case of MathML it's apparently not true. Having said that I think it'd be good to have something more sophisticated than plain DOM to implement, for example, an extended math navigation. Otherwise I think AT have to learn some MathML.
So we stopped at this point for now. Assistive technology can navigate the MathML tree, get MathML markup and feed it to utility libraries processing the math. It looks good for a start, at least after years of keeping silence. Ostap Bender would say the ice has broken, ladies and gentlemen of the jury!
If you have ideas, thoughts to share you're welcome to comment our meta bug.
So MathPlayer works with IE only. MathJax, a cross browser math renderer, claims they do accessibility by means of MathPlayer. Last time I checked MathPlayer did a trick by adding accessible properties like accessible name to MathML nodes in accessible tree. There's a bunch of problems with this approach like the user can't control the speech output. However I must admit it makes screen readers to read math.
Main problem here there's no right API and they have to use existing APIs which is Procrustean bed for math. The browser and assistive technology have to invent something that describes math well. Until that there's no right tool to succeed. I should notice that above-mentioned primarily applies to Windows and Linux worlds. Much to my surprise WebKit has got a math accessibility API on OS X last year. I didn't hear VoiceOver picked it up yet but as soon as it does the MathML content should become accessible on Mac.
In Firefox we have a meta bug to track MathML accessibility work. As the first step on this way Firefox 27 started to create generic accessible objects for MathML elements. It's not so valuable by itself because a generic accessible don't expose any math semantic but it allows the assistive technology to navigate the math and watch for tree mutations. Next we could follow WebKit effort by extending ATK and IAccessible2 APIs to make math accessible on Windows and Linux but I have been told libraries that screen readers rely on to process MathML speak also MathML language. It might be unwise to split MathML into atoms of high level API to make the assistive technology to reconstruct MathML on its side.
Gecko exposes ISimpleDOMNode interface providing a direct access to DOM for assistive technology. I never was a devotee of ISimpleDOM because I believe that high-level APIs like IAccessible2 are more efficient. But in case of MathML it's apparently not true. Having said that I think it'd be good to have something more sophisticated than plain DOM to implement, for example, an extended math navigation. Otherwise I think AT have to learn some MathML.
So we stopped at this point for now. Assistive technology can navigate the MathML tree, get MathML markup and feed it to utility libraries processing the math. It looks good for a start, at least after years of keeping silence. Ostap Bender would say the ice has broken, ladies and gentlemen of the jury!
If you have ideas, thoughts to share you're welcome to comment our meta bug.
Speaking for MathJax, we are very interested in discussing accessibility APIs, see e.g. this discussion on our dev list https://groups.google.com/forum/#!topic/mathjax-dev/UvFSEx8GOh8.
ReplyDeleteAs you point out, we originally worked with the MathPlayer team to ensure compatibility (that was 5 years ago when MathPlayer was the only existing math accessibility solution). This year, we worked with the ChromeVox team to ensure MathJax output is as accessible as MathML to them (and in fact they use MathJax to make TeX and asciimath notation in image alt-tags acccessible).
Of course, once ShadowDOM becomes available we can start leaving MathML in the DOM no matter what. Then again, I personally hope Firefox will improve enough for us to go back to Native MathML output anyway. In any case, thanks for the post!
If you start using ARIA (like aria-label technique) then you should get half backed solution similar to what MathPlayer has but it will be cross browser and cross screen readers approach. Everything else would require changes on each side. Since you are JS-based software then ideal solution would be to have JS accessibility API. But either way we'd need to have some math a11y API I think. I'm curious to know details about your progress with ChromeVox, how it works.
DeleteWe at Design Science, makers of MathPlayer and other math apps, are very interested in accessibility APIs. The main reason MathPlayer only works in IE is that there is not a way to plug its functionality into other browsers easily. It would be nice if our software could implement some sort of IMathToSpeechServiceProvider API that all browsers accepted.
ReplyDeleteThat's right, having a platform API (like MS COM API on Windows) and JS API is what we'd need I think. Taking everybody on board usually takes years though. What concerns to Mozilla I think a11y team is always ok to collaborate and experiment. But probably it's out of box for you until MathPlayer is IE-based.
DeleteI don't follow your last "out of box" sentence. Can you explain please?
DeleteI meant it may be hard for you to find resources to work on theoretical aspects with Mozilla until you are expanding to Firefox
DeleteIf I understand what you are proposing, and what I am also interested in, is developing APIs that work in many environments, not just Firefox. Assuming that is what your team wants to work on too, I don't have any resource issues at my end. In fact, lack of such APIs is the only thing stopping us from providing MathPlayer's functionality in Firefox and other environments.
DeleteAwesome. Since presumably you know MathML and rules of math expr reading well then would you sketch up an interface that works for you? It would be a good matter to start from.
DeleteI don't really know what kind of interface you have in mind exactly. Who else are you working with?
DeleteYou can keep IAccessible2 in mind designing a new interface. NVDA has shown interest in it.
Delete