xability / maidr Goto Github PK
View Code? Open in Web Editor NEWMultimodal Access and Interactive Data Representation
License: GNU General Public License v3.0
Multimodal Access and Interactive Data Representation
License: GNU General Public License v3.0
Currently, pressing arrow keys or any other keys trigger help menu.
For lack of better word, boxplot empty cell sound is a little bit annoying. Could we explore other smoother sound going forward?
If you turn on verbose mode and navigate "suv" lower and upper outliers, you would hear/see the following:
Lower Outliers are 126.18
5 Upper Outlier is 302.48, 318.51, 334.53, 350.56, 366.59
First, the plural and singular verbs are unmatched with the the number of outliers.
Second, no information on the number of "lower outlier". Currently, it displays as:
Lower Outliers are 126.18
However, the expected outcome would be:
1 Lower Outlier is 126.18
In verbose mode, go to upper outlier area in "suv"
Arrow down to focus on the upper outlier of "subcompact": you would hear and see the following:
class is subcompact, Upper Outlier2 is 590.97, 639.05
However, the expected outcome is as follows:
class is subcompact, 2 Upper Outlier are 590.97, 639.05
heatmap.html
.b
.In boxplot.html, application role is not activated in the svg area.
Open scatterplot.html.
Focus in the graph area.
Hit page-up key to go to the second layer (the smoothed line).
Press Ctrl+Shift+Left-Arrow to autoplay.
While the autoplay is working, press Ctrl+Shift+Right-Arrow to play the other direction: two autoplays get played at the same time.
The last autoplay needs to kill the previous autoplay job.
While addresing #61, please also fix the following issue.
Open boxplot.html
Press arrow keys to go to upper outlier area of suv.
Hit down-arrow key: You will see:
2 subcompact Upper Outliers 590.97, 639.05
This needs to be fixed as:
subcompact, 2 Upper Outliers 590.97, 639.05
5 suv Upper Outliers 302.48, 318.51, 334.53, 350.56, 366.59
This needs to be fixed as:
suv, 5 Upper Outliers 302.48, 318.51, 334.53, 350.56, 366.59
As we discussed today, please implement an event-driven log tracker.
Log tracker needs to get started as a page is loaded.
We may want to bind log tracker to each SVG object.
Default log file format: JSON
Use F10
to bring up "Save As" dialog box for exporting log file.
Use sessions and cookies to address multi user access.
In our user studies, we will embed each plot in separate pages, which means we may need to append multiple log data from the same user agent until they explicitly press F10
.
Structure (please feel free to add more if any):
graph type
timestamp
keystroke
current modes (BTS)
misc settings (like custom frequency range, volume, braille display size, etc)
current position on chart
user OS platform info
user browser and version
boxplot.html
.b
to activate braille mode and do the same thing (pressing left or right arrows): You would hear redundant group label even though it's still in terse mode.BBeeeeIt seems like the braille cursor position is one column behind the visual highlight.
For Mac / PC switcher: use navigator.platform
or scrape from navigator.userAgent
, because navigator.userAgentData.platform
does not exist on all browsers (only exists for Chrome and Edge).
Boxplot empty cell sound is not well pronounced because it uses low pitch and may be confusing. Let's keep using the same empty sound as the one used in heatmap.
I know there is a tradeoff, but please bind all the keydown events to svg units instead of the whole document. Because we will likely embed more than one graph on the same page and the document-level key bindings can cause some issues. We may want to isolate them within an SVG object.
When help menu is displayed, "document" role must work as tagged to activate screen readers' virtual cursor (AKA browse mode). However, currently it does not take any effects. Usually, assigning tabindex 0 with "document" does the trick, but it does not work this time. Probably, it inherits from "dialog" ARIA role, but it was not the issue b/c removing the "dialog" did not make it work.
I have pushed a heatmap svg sample file at accessible-graphs/static/heatmap.svg
.
With this sample file, please make a heatmap.html
file just like accessible-graphs/barplot.html
:
Visual highlight moves along with left/right/up/down arrows.
Play audio tone that addresses the following
left and right stereo panning for x-axis.
Similar to left and right panning mode, make sound moves along up and down (not the pitches, just the spatial audio) along with y-axis.
different pitches for values (higher pitch for higher value; lower pitch for lower value).
@yujuny2 -- I have just uploaded the following scatterplot SVG:
Please work on these files instead of the old one because these did not use jitter noise. The only difference between 1. and 2. is whether there is an added best line or not.
FYI, the backend raw data that I previously provided in json format as well as HTML are still valid because I created them without jitter noise at that time. I guess this no jittered SVGs would be more similar to the raw data.
In barplot and other graphs, autoplay (ctrl+shift+arrows) does not work with braille mode being turned on.
The braille display works nicely. Great job! The only issue I have found is that the braille cursor locations are not precisely coupled with its corresponding sound pitches. When I press left and right arrows, cursor location moves along with braille columns. However, this cursor location is one column after its corresponding sound. For example, The highest tone/y_value in our sample svg file is at the far right. This pattern is also represented in braille unicode, but I here the highest tone when my braille cursor is one column after the target location.
It would make more sense that we turn off braille mode when it loses its focus. Could you please add event handler to the braille div for "focusout" event?
Currently, the first landing point is the very bottom gorup instead of the top one. Given that the screen reader's reading order is top-to-bottom and left-to-right, it would be more natural to set the first focus to the first top group and at the lower outlier.
At some point in the near future, we need to integrate our interface with Chart2Music. To do this, we need:
Create a JSON-looking general API like C2M that does not necessarily depend on visual formats.
Allow users to fall back to C2M for any graph types we don't support, such as candlestick plot, band plot, etc.
In any graphs and in any position, press CMD+Shift+LeftArrow or CMD+Shift+RightArrow to start autoplay.
When autoplay is on, text mode needs to be muted so that our users only hear the sonification sound. However, autoplay triggers the current position in text output. Please turn off the text mode as soon as the autoplay key is pressed and turn it on again right after the autoplay is finished.
For the log tracker feature, we need to host a server. I will take care of this.
1 Lower Outlier 126.18
0 Out Lier
Toggle focus between braille output div and user's previous location (on the chart) with B key.
Up and down arrow keys move between different boxplot groups.
Left and right arrow keys move section by section within selected boxplot.
I will add thorough description on the system design soon.
Also CC'ing @ellvix
We need Ctrl+Home key for going to the very first element and Ctrl+End key for going to the very last element in graphs. This is useful for two-dimensional graphs, such as heatmap and boxplot. However, please make these keys work for other types of graphs including barplot and scatterplot as a general solution. I know, in barplot and scatterplot, these keys' behaviors would be exactly the same as Ctrl+LeftArrow and Ctrl+RightArrow, but we need to make this generalizable so that we can be prepared for any types of graphs going forward.
Investigate smooth audio for scatterplot curve. Try AudioParam.setValueCurveAtTime()
Add Command + key for mac, mirror current Ctrl + key for PC controls.
Try using navigator.userAgent
or navigator.platform
to detect OS and switch both controls for all plots, and key control info on help menu.
When landed in either lower or upper outlier region, it would be great to announce the total number of outliers like below:
{total_number} upper outliers: . . .
Currently, the autoplay speed controls (faster and slower) only work for scatterplot; however, please make this work for all types of graphs we support.
Also, please replace the current shortcut keys with the following:
Faster: period key (.)
Slower: comma key (,)
Alt+Shift+Left-Arrow: From the beginning to the current position.
Alt+Shift+Right-Arrow: From the end to the current position.
In boxplot, changing group refreshes the cursor location, defaulting to min area instead of preserving the current stat area. For example, if the current cursor is at 25% of group A, users may expect to land in 25% of group B when they press down arrow key.
I think this is an inadvertent regression after we patch aria role bound to SVG element.
Open any graphs, such as boxplot.html or barplot.html
Tab to focus in SVG object.
Press arrow left and right to move around.
Press B
to turn on braille mode.
Turn off braille mode by pressing B
key.
See if your focus goes back to your original position: it does not.
boxplot.html
.b
to activate braille mode and do the same thing (pressing left or right arrows): You would hear redundant group label even though it's still in terse mode.BBIt would be nicer if ESC key event can close the help menu dialog.
We currently use min frequency, and sometimes custom tweak the oscillator type, but we should have a unique repeatable null sound used everywhere that we can tweak to improve UX.
Currently, we obtain boxplot stat values from svg object. However, we may want to use exact values exported from R back end: boxplot_data.json
While hosting our project on GitHub, we may want to publish our solution to NPM as an open-source library at some point so that people can download and import our library.
After multiple experiment on my end, I have found that double lines are more pronounced. The following includes my suggested logics. I will share more ideas on some thresholds.
left_boxside = median - Q1
right_boxside = Q3 - median
if (left_boxside == right_boxside) {
box_brl = "⠿⠸⠇⠿"
} else if (left_boxside < right_boxside) {
box_brl = "⠿⠸⠇⠿⠿"
} else {
box_brl = "⠿⠿⠸⠇⠿"
}
lower_whisker_length = Q1 - min
upper_whisker_length = max - Q3
if (lower_whisker_length == upper_whisker_length) {
"⠒" + box_brl + "⠒"
} else if (lower_whisker_length < upper_whisker_length) {
"⠒" + box_brl + "⠒⠒"
} else {
"⠒⠒" + box_brl + "⠒"
}
[JooYoung will take care of this issue]
It would be great if we could bypass some screen reader's virtual cursor conflicting keys. See yihui/xaringan#262
When navigating a stat vertically across groups, sometimes focus goes to another stat category.
Open boxplot.html
Focus on low (AKA minimum) of "suv"
Keep pressing down-arrow keys (autoplay does not have this issue so please test with manual arrow keys).
If you reached the low of "2seater" without any issues, keep pressing up-arrow multiple times to go back to the suv low.
Repeat this up and down round trip multiple times: you may notice that your focus sometimes moves to lower outlier group out of the blue.
Currently, all the keydown events and their corresponding event handlerss are bound to window
object. This is a great start. However, what if there are more than one svg objects contained in the current web page? addEventListener()
will be triggered against when the whole browser window is loaded. Probably, a better approach would be binding these events and handlers to svg
object unit instead of the window
object. This way would allow users to interact with any number of svg objects in the current window.
I have finally been able to do reverse-engineering jobs to obtain the real data for both point and smoothed line layers within scatter plot. Please use the following data for scatter plot.
Point layer (I know you already used the raw data for point values, but this is the exact one. Please replace): scatterplot/layers/point_layer.json
Smoothed line layer: scatterplot/layers/smooth_layer.json
Please note: if there are multiple values on the same spot, use the larger volume to indicate the weight. For example, let's say there are x: 2, y: 4 and x: 2, y: 4 (the same values on the same spot). Sighted people would see them thicker, but blind people cannot see them. Therefore, please use larger volume parameter as a way of weighted value.
CC'ing @ellvix as well.
This is a known issue, but we need to track the progress. Currently, autoplay doesn't work in heatmap while braille mode is turned on.
I suggest we reword the following terms in boxplot:
Minimum
-> Low
Maximum
-> High
This is because these terms are misleading. The ending points of left and right wiskers actually are not matched with data min and max values.
Here is a formula how low and high (i.e., the ending point of both wiskers) values are calculated:
High:
Low:
Outliers are any values outside either low or high values.
Autoplay response speed is a little bit slow. I think there is about one second lag to respond after pressing autoplay keys (Ctrl+Shift+LeftArrow or Ctrl+Shift+RightArrow). We may want to make the response rate faster.
Currently, oscillator
play is triggered via spacebar keydown event. This would cause some issues when there are multiple SVG plots included on the same page. After addressing #2, please modify this in the following ways:
Add tabindex="0" attribute to each svg objects via setAttribute()
method.
Make oscillator
event handler work against focus
event (when the current SVG object gets focused) instead of spacebar keydown event.
In the long run, we need to switch the current project into Typescript for better scalability, compatibility, and maintenance.
Autoplay is not currently supported in boxplot. Go to the first and last location (Ctrl+arrows) does not work, either.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.