Group Abstract Group Abstract

Message Boards Message Boards

9
|
20K Views
|
32 Replies
|
64 Total Likes
View groups...
Share
Share this post:

Can Mathematica support a quadriplegic teenager in his learning journey?

Posted 5 years ago

My 14-year-old son is medically complex, including being a quadriplegic. He currently uses software (Grid 3) to communicate ideas and an eye tracker (Tobii) with software that allows him to "type" using his eyes.

I am looking for tools that will allow him to work out a math problem through its various steps (since he cannot hold a pencil and write on paper to show his work), and I thought Mathematica might be a tool that could help.

Has anybody successfully used Mathematica using mouse movements alone and not typing anything on the keyboard?

Thanks.

POSTED BY: Myles Dear
32 Replies
Posted 5 years ago

Hi, everybody. A few more details of my son's computer. The resolution is 1024/768 on a Tobii I12+ Windows PC. The eye tracking click menu takes between 10% and 20% of the screen width and all of its height (as per the enclosed snapshots). The eye tracking keyboard takes the bottom half of the screen.

I'll actually be using the desktop version of Mathematica rather than the cloud version, since my son needs palettes, but since I'm having some license problems I thought I'd post these to give everybody an idea of the playing field.

Attachment

Attachment

Attachment

POSTED BY: Myles Dear
POSTED BY: Kyle Keane
Posted 4 years ago
POSTED BY: Myles Dear
Posted 5 years ago
POSTED BY: Myles Dear
Posted 5 years ago

Hello Myles Dear,

For item #3, if you only want to remove some things from the Classroom Assistant Palette for your son's use, here is something you might try: 1. Open the Classroom Assistant Palette. 2. Right-click somewhere inside. 3. Click "Generate Notebook." 4. Manually select and delete parts of the palette in the new notebook graphically. 4. Select the cell in the notebook. 5. In the Palettes menu, click "Generate Palette from Selection."

This process may be tedious. I hope it helps though. Maybe someone has a less tedious solution.

POSTED BY: Jibiana Jakpor
Posted 5 years ago

Myles:

I'm still new to Mathematica and the Wolfram Language. But, as I understand things, Mathematica is a general purpose tool that for most functions require entering text in some fashion.

Given your question, I am assuming that your son would not be able to use a speech-to-text interface. But for those that can use speech-to-text, I think Mathematica's natural language interface would work well.

That said, I do believe it would be possible for someone to develop applications in Mathematica that would allow your son to work through math problems utilizing the power of Mathematica and the Wolfram language.

This is a great community that I'm sure would like to help. But first, it would help to understand what sort of math problems your son is ready to tackle.

POSTED BY: Mike Besso

Ah I think you want him to write things that look like typeset math not code but here is the code-ier version which we could use for some of the subtopics even if some subtopics are using templates. Having simple code commands might be very useful for things like Simplify[] and Solve[]

enter image description here

POSTED BY: Kyle Keane
POSTED BY: Kyle Keane

This is good info Myles.

What is the size (in pixels) of a single on-screen keyboard button? If we stick a tiny template indicator inside of a button of the same size, will your son be able to see it? Currently the keyboard buttons have just a single easy to read letter in each button, will a template with thin lines inside of their work? Can you verify the smallest template that will be comfortable for him to read?

If we follow the keyboard button sizes, then we can fit a list of topics at the top, subtopics below that, a row of templates from the subtopic, and finally a row of commonly used buttons below that.

If the screen is 1024x768 and the tobii controls on the left are 15% of the width and we need the top half of the screen to display the math he is inputting, then we have about 870x380 pixels for the control area. I guess that means the buttons are around 65 pixels wide with 10 pixel padding on each side?

Attached is a notebook that you can modify to test various size buttons and layouts that will generate an image with controllable pixel size so you can open it on his computer to see if it look as you want and your son can read it.

enter image description here

Attachments:
POSTED BY: Kyle Keane
Attachments:
POSTED BY: Kyle Keane

It is good to see that Kyle Keane has reached out to you. I am also curious if I can help a bit. Both the Basic Math Assistant and Classroom Assistant palettes were created with the idea that a keyboard was not available and they could be made large by changing the magnification level. However, as you have noticed, they are packed with functionality that is not useful to your son is this year. How would you suggest modifying the palette(s) so they could better serve your son? You can email me directly: ericschulz at mac.com.

POSTED BY: Eric Schulz

I just realized there is no private messaging on the community. you can email me at my mit.edu email kkeane

POSTED BY: Kyle Keane

Send me a PM and we can have a zoom chat. I teach assistive tech at MIT and am an avid Wolfram user.

POSTED BY: Kyle Keane
Posted 5 years ago
POSTED BY: Jibiana Jakpor
POSTED BY: Stephen Wilson
Posted 5 years ago

Thanks for the reply, Stephen. My son does not have the head motor control to consistently operate head-based switches or to use a head mouse, but he does not have spasticity (in fact, his muscles are limp like wet noodles except for a very little amount of slow left-to-right neck movement). In contrast, his eyes are very fast and he has trained on how to use his gaze-based eye tracking system for several years and has become quite good at it. I think a raw command-line interface would not be feasible for him, but rather a series of graphical elements that allow for higher-level control.

POSTED BY: Myles Dear
POSTED BY: Stephen Wilson
Posted 5 years ago
POSTED BY: Mike Besso
Posted 5 years ago
POSTED BY: Myles Dear

Hi Myles
I don't have much experience with Tobii, but I know that it supports an on-screen keyboard with English layout. If your son can successfully click on the buttons of this keyboard using Tobii, then he probably will be able to use it to type in Wolfram notebook. If you are using Windows 10, this page can help

https://support.microsoft.com/en-us/windows/get-started-with-eye-control-in-windows-10-1a170a20-1083-2452-8f42-17a7d4fe89a9

POSTED BY: Ahmed Elbanna
Posted 4 years ago

Hi, Kyle. For your first point, please look at the enclosed video that will hopefully explain exactly what a Grid 3 zoom mouse control looks like (which I call zoom-to-select in my earlier comment). First, the function (left click, right click, double click, drag) is selected, and then the user moves their eyes and fixes their attention on a point. The zoom proceeds, with the user making course corrections during the progress of the zoom until the exact point is identified. The zoom ends, the original screen is restored, and the mouse action then occurs.

If my son deselects the zoom function he would then be free to read the contents of the screen without "interference" from the zoom process, and my request is that all text in the non-zoomed screen be 25 point or larger. He would not be reading anything while the zoom mouse is active. I understand that this means he might have to traverse several screens in order to see a whole sentence of help text.

For your second point, I am not looking for just a simple calculator workflow, I want to open up the full power of the tool to my son, whether it be typesetting, entering functions, or even writing programs. I understand that in order to get a cell ready for evaluation that it may take multiple palette functions to enter the data appropriately before evaluating the entire cell's contents, and I see now why the play button is necessary in order to trigger that evaluation. Sorry for my misunderstanding.

Attachments:
POSTED BY: Myles Dear
Posted 5 years ago

Thanks for the reply, Jiabiana. I was able to get my workbook evaluating correctly, after reading the documentation fine print, I see Rectangle accepts two points, while Triangle accepts a list of points. Both are working as inputs to Graphics[] now.

POSTED BY: Myles Dear
Posted 5 years ago

Regarding item #1, I believe I've found a link that may help, I'll investigate further : https://reference.wolfram.com/language/howto/ConnectToARemoteKernel.html

POSTED BY: Myles Dear
Posted 5 years ago

Regarding item 4, Andy D responded saying he didn't know why Graphics[] did not accept Rectangle[] but said that the developers must have had their reasons for doing it that way. Sometimes trial and error is needed to make things work !

POSTED BY: Myles Dear
Posted 5 years ago

Thanks for the replies, everybody. I really appreciate your kind interest in my son's development.

  1. Could you please give me some pointers on how I can run Mathematica on my resource-constrained PC (using the palettes feature), while running the kernel on my more capable home-based server, with both located on my home network? Would doing this require more than one license?

  2. I had a meeting with Andy Giddens (Sales) and Andy Dorsett (Sales Engineer) and they explained I could set magnification settings on a palette to ensure the targets were bigger for my son. They also recommended use of the CreatePalette command. This looks like a fairly complex command, does anybody have any examples on its use (other than the very few examples in the product documentation)? I am a software developer and am a quick study.

  3. I looked at the built-in Classroom Assistant palette and although it had some good things my son would benefit from (basic algebraic entry, expressions and equations, navigation, evaluation to name a few), only around 40% of it would be immediately useful. Rather than starting from scratch, I'd love to be able to take a copy of the code that generated this template and cut it down to include only the basics for an early learner. Any idea how to best approach this ?

  4. I'm attaching a sample 2d exercise I created to demonstrate to my son's tutor that Mathematica could be used to help my son with his daily work. Any idea why the Graphics[] function did not accept a Rectangle[] object as input ?

Attachments:
POSTED BY: Myles Dear

To Myles Dear,

By the way, have you been here: https://www.wolfram.com/legal/accessibility/wolfram-mathematica/ ?

In listening to National Public Radio, I learned much from Edward Reynolds Price (February 1, 1933 – January 20, 2011). His cancer treatment with surgery and radiation led to paralysis below the waist. Chuck Close,[Born: July 5, 1940 (age 80 years)], the artist came up with this phrase: ‘temporarily abled.’ Mr. Price embraced it and ran with it[so to speak]. https://adogcalledpain.wordpress.com/2016/01/04/reynolds-price-a-whole-new-life-an-illness-and-a-healing/ https://www.washingtonpost.com/archive/entertainment/books/1994/06/12/though-much-is-taken-much-abides/e06d08f7-e315-42d2-abd7-4b9667ec5daa/ http://chuckclose.com

POSTED BY: Stephen Wilson
Posted 5 years ago

I have downloaded a desktop copy of Mathematica onto my home server, this is a big software package, I'm not completely sure it would run on my son's small resource-constrained Tobii I12+ eye tracking computer. I'm quite interested in looking into palettes - I agree, the basic palettes that ship with Mathematica are much too small (typically eye tracking targets can be no smaller than the size of a nickel, and I haven't yet experimented with zoom-dwell-to-click control models with my son that might allow him access to smaller targets). Currently, he is using dwell-to-click against fairly large targets.

POSTED BY: Myles Dear
Posted 5 years ago

My son is nonverbal, so would rely on eye based access to interface with his computer. He is doing math at a Grade 8 level and we are bridging him with various online tools manually. However, he will reach a point very soon in which doing things in his head becomes next to impossible and he requires some kind of tool to allow him to write things such as algebraic equations so he can reduce/solve them. Rather than just having him use a "dumb" sketchpad app, I am looking for tools that could provide a higher level of value and help him engage with his math more independently.

POSTED BY: Myles Dear
Posted 2 years ago

A sad piece of news : my wonderful son passed away in my arms on July 31st of this year.

I would be honored if you could join us in the celebration of his life by subscribing and viewing the five videos we have produced : https://forever.link/arbormemorial-kelly-orleans/kyram-dear

This topic remains incredibly pertinent, however, as the lack of an onramp for those with physical disabilities to the Wolfram ecosystem remains a barrier that must still be surmounted.

I strongly believe, more so as I continue to listen to Stephen Wolfram's videos, that having this kind of onramp for those with physical disabilities who rely on assistive technology to perceive the world and interact with it would expose many fine new minds to the wonderful ecosystem Stephen has created. I truly believe we need EVERY voice to move forward to develop this New Kind of Science.

Who is with me?

POSTED BY: Myles Dear
Posted 4 years ago

Thanks very much to Eric Schulz for posting this sample / possible implementation.

Attachments:
POSTED BY: Myles Dear
Posted 5 years ago

Thanks, Kyle, I'll take a look at this second draft skeleton. As noted below in my Dec 30 post, any buttons cannot be smaller than 21 pixels by 21 pixels (which will appear five times larger when completely zoomed into with his zoom mouse).

POSTED BY: Myles Dear

Awesome, thanks Eric. I am thinking I will work to refine the design and code up a prototype and maybe you could take it from there to mature my implementation?

POSTED BY: Kyle Keane
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard