Smart Headset for Nystagmus Correction
(Future Solutions, Proposal 1)
Could you help me develop technology that would benefit someone who has nystagmus? Would you work as hard as I do, or possibly harder?
About Nystagmus
If you’re sighted, then the animated GIF above from Doc James is a start to understanding the condition.
To quote the American Optometric Association:
Nystagmus is a vision condition in which the eyes make repetitive, uncontrolled movements.
Nystagmus can prevent someone from forming a stable image.
I don’t have nystagmus, but I’ve known people who do. Someone can have nystagmus without having other vision conditions. Nystagmus can occur alongside other conditions affecting vision. Nystagmus can also be the cause of blindness.
Proposal
The purpose here is to be reasonably quick about a writeup. If there’s sufficient interest, contact me, but be sure to first read the Introduction to this series about when it’s appropriate to contact me, what sort of inquiries I’d ignore, and what help would be needed. Also review the other info below.
Since I don’t have nystagmus, I’ll make assumptions about the condition that may be wrong. A partial solution would need to be user tested to determine feasibility.
Problem
The eyes of someone with nystagmus move repeatedly and uncontrollably, preventing the formation of a stable image.
Solution
An initial solution could be implemented for a smart vision headset such as Apple Vision Pro that includes eye tracking, that can block the user’s view entirely, and that can generate images spanning the user’s vision in real time using on-board processing.
As has been proposed previously, scene stabilization could be provided by generating images that move as the user’s eyes move, thus
…stabilizing the image on the retina (digital Retinal Image Stabilization) by moving the digital image synchronal with the unintended eye movement using gaze contingent display technology.
A virtual reality headset such as Meta Quest 3 might have sufficient processing power to provide image stabilization, but the Quest 3 lacks eye tracking.
A solution might be implemented using an eye tracking camera to provide stabilization on a large desktop screen or wall screen.
Difficulties
As of May 2024, the Apple Vision Pro may be the only commercially available device that has the eye tracking sensors, image generation capability, and app distribution channel (the App Store) to support wide distribution of an image stabilization app for people with nystagmus. But the headset is expensive at $3500 (base price).
Developing an app for the Apple Vision Pro is time-consuming, and with little to no assurance that R&D costs can be recouped. Even if a 3rd party developer creates such an app, Apple may develop its own image stabilization app, using internal resources, effectively undercutting the original developer’s business.
Workaround
Prototyping on a smart tablet with front- and back-facing cameras would make it possible to test image stabilization for a portion of the tester’s field of view.
Proof of Concept
A no-code demo isn’t practical. However, a proof of concept could be developed using Protopie, an excellent prototyping tool that can access a smart phone’s sensors.
The proof of concept would show an animated pair of eyes moving at the top of the screen, and a moving image representing image stabilization occupying most of the rest of the screen.
A no-code or low-code prototype would be better than writing a bunch of code for an AR/VR headset. If the experience of image stabilization could be demonstrated to a handful of people with nystagmus, and if those people showed sufficient enthusiasm, then development could proceed with caution.
An interactive prototype for one eye would require writing an app for an iPad and/or Android tablet with front-facing (selfie) and back-facing cameras.
- The table would be mounted in a fixed position.
- A soft “offset guide” such as a piece of foam would indicate where the user should place her head.
- The selfie camera would be used to track the user’s eye motion.
- The tablet’s screen would display live video from the back-facing camera, shifted to move synchronously with the user’s eye. This would provide a somewhat stabilized view of a small portion of the user’s field of view.
- In a follow-up version, motion prediction of the user’s eye movements would present a smoother experience.
My suspicion is that a standard Kalman filter wouldn’t be quite optimal for motion prediction, but it’d be a reasonable start.
Existing Products that Could Provide a Stable Image
The sensors and processing power necessary to solve this problem suggest that the only technology that could provide a solution would be augmented reality smart headsets or virtual reality headsets.
A solution should block as much of the user’s view as possible — preferably the entire field of view — and present real-world images that move fast enough to keep up with the user’s saccades (eye motions). Academic work may already provide the specifications to be met.
Some tech platforms that might be suitable:
- Apple Vision Pro (U.S. $3500 and higher). They’re new, big, have a relatively short battery life, and could be fatiguing to wear. But an app running on Apple Vision Pro may work well enough to provide an “existence proof” that a future solution is feasible.
- Meta Quest 3 (U.S. $500 and higher) plus eye tracking, which is missing. Maybe the processing power and ability to generate images is sufficient. I haven’t used Quest 3, and I don’t know what the experience is like.
- Snap Spectacles (demo only) would be useful to demonstrate a proof of concept, but likely at reduced speed. They’re not currently for sale; a group has to apply for funding to use them.
Other smart glasses on the market may use the same Snapdragon XR1 chip (or later version) as the Snap Spectacles, but may not offer the same Open AI as Snap, or a reliable, robust means of distributing apps and updates.
Meta / Ray-Ban smart glasses don’t have waveguides or other in-lens displays.
Academic / Professional References
In the United States, the National Institute of Health (NIH) is a source of information about government-funded research.
Using an ARX model… (2017)
“Assisting people with Nystagmus through image stabilization: Using an ARX model to overcome processing delays” by Stephan Polzer, Klaus Miesenberger
Abstract:
Pathological Nystagmus is characterized by an unintended and involuntary eye-movement, which tends to impact on visual acuity. Today only view therapies (for instance medication or surgeries) to treat nystagmus are at hand and the existing therapies only show partial improvement. Only general Assistive Technology (AT) solutions like glasses, screen magnifiers, speech output, display adaptation and concepts holders are at hand to support daily living. More specific ATs to reduce the impact of nystagmus are missing. This paper presents conceptual research and feasibility studies with the attempt to reduce the impact of nystagmus by stabilizing the image on the retina (digital Retinal Image Stabilization) by moving the digital image synchronal with the unintended eye movement using gaze contingent display technology. Further an identification and validation process using recorded eye-movements is presented since the synchronization approach relies on predicting eye-movements to reduce the impact of delays due to processing time.
(Bold emphasis added to “…the synchronization approach … due to processing time)
Treatment of Nystagmus and Syccadic Oscillations (2013)
This web page also provides a prevalence (relative frequency of occurrence) for nystagmus: 24 per 10,000, or 0.24%.
That’s a small percentage, but a lot of people.
Nystagmus is often encountered in ophthalmology practice, having a prevalence of about 24 per 10,000 in the general population.
There are two references. Good enough! Moving on.
Prevalence of Nystagmus
Sources present a range of estimates for the prevalence of nystagmus. Some of the sources below may quote statistics from the same source — I’m trying to be quick about this.
24 per 10,000 is the number provided by the uiowa.edu page linked above.
24.0 per 10,000 is also provided by ArvoJournals.org (2009): https://iovs.arvojournals.org/article.aspx?articleid=2164820
6.1 per 10,000 is the prevalence for a study in Denmark (from only 103 patients): https://pubmed.ncbi.nlm.nih.gov/32067411/
6.72 per 100,00 is reported for pediatric (child) nystagmus for patients younger than 19 years (with diagnoses for just 71 children): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5610648/
At the low end that’s 6.72 per 100,000, and at the high end that‘s 24 per 10,000. Quite a range, especially if there are no typos in the sources.
The range 6 to 24 per 10,000 seems reasonable enough. For now, let’s ignore the much lower number of 6.72 per 100,000.
Pick a number in the middle, why don’t we: 15 per 10,000.
The prevalence is very roughly 0.15%, or one in 667 people.
That would be about 12 million people worldwide with nystagmus. Probably more.
The Problems of Low Prevalence
This first proposal is a good place to describe a problem when a small percentage of people are affected by a disability.
Even though a small percentage of people may have a condition or disability, the global number of people with the condition may number in the millions, or tens of millions. There may be two million or more people in the world with nystagmus. However…
- Low prevalence means there are relatively few people in any single geographic area who have a condition such as nystagmus. Finding and reaching these people can be difficult. People with nystagmus may visit the same ophthalmologists and hospitals in the area, but only infrequently.
- Selling tech through ophthalmologists, hospitals, and other specialists can be tough. You’re selling new technology unfamiliar to many, not wedge prisms, tissues, or eye drops to dilate pupils.
- Low prevalence means people will be scattered geographically. How could you reach those people cost effectively?
- When prevalence is low, relatively few people worldwide will have the financial ability to pay for a high tech solution. People with disabilities are chronically underemployed.
- If only a few million people have some condition, it could be very hard for a profit-seeking business to justify development. Thus some solutions that could be developed, won’t be.
Other Problems of Assistive Tech
In some countries, certain types of assistive technology may require government approval, especially if the tech can be considered as “medical device.” Don’t expect a fast turnaround time on getting approval.
In some countries, the government may have a monopoly on purchasing and selling / providing assistive technology for its citizens. This can be good: if “the market” doesn’t provide an assistive tech solution, a government may fund this solution to respect the rights of its citizens. That makes sense to me, but it could mean that you may have to have fully developed technology ready to sell — which is hard to fund — and only then will you be able to pitch the technology to the state agency that buys and provides assistive technology. If they say no, that may be it.
This is one reason why, unfortunately, assistive tech might first be sold in countries with looser restrictions, and with a sufficient population of well-to-do people who happen to have a condition such as nystagmus, and who are willing to take a risk trying new technology.
Liability insurance is a must for assistive tech developers. Even if someone isn’t harmed by your technology, if someone happens to be wearing your technology when an accident occurs, or if that person misuses the technology, that person or their family may track down an ambitious lawyer who is all too happy to file a lawsuit. (The United States is a litigious country. Sigh.) Liability insurance may cover such conditions. But that raises another concern.
Getting liability insurance in the first place could be hard. No one has to underwrite an insurance policy if the risk is too high.
Articles on Nystagmus
You can google as well as I can, perhaps even better, but here you go.
Links are provided separately from text, rather than inline, so that people using screen readers finding it less annoying.
American Optometric Association: Nystagmus
https://www.aao.org/eye-health/diseases/what-is-nystagmus
WebMD
https://www.webmd.com/eye-health/nystagmus
Wikipedia (includes GIF animation of nystagmus)
How to Contact Me
Before considering whether you should contact me, please read both the Preamble and the Introduction to this series.
You might also check out the index to proposals I’ve written so far.
Too Long?
It’s easier and quicker to write a lot than it is to write concisely. So be it!
Having written the Preface, Introduction, List, and this proposal on one Sunday afternoon, I think it’s time for a break.
