Virtual Sensors: Transforming the Way We Think About Accommodation Stevens Institute of Technology-Hoboken, New Jersey Katherine Grace August, Avi

Size: px

Start display at page:

Download "Virtual Sensors: Transforming the Way We Think About Accommodation Stevens Institute of Technology-Hoboken, New Jersey Katherine Grace August, Avi"

Silvester Floyd
5 years ago
Views:

Virtual Sensors: Transforming the Way We Think About Accommodation Stevens Institute of Technology-Hoboken, New Jersey Katherine Grace August, Avi Hauser, Dave Nall

1 Virtual Sensors: Transforming the Way We Think About Accommodation Stevens Institute of Technology-Hoboken, New Jersey Katherine Grace August, Avi Hauser, Dave Nall Fatimah Shehadeh-Grant, Jennifer Chen, Hong Man, Victor Lawrence 22 November 2014 Demo: opt 8; participant:

2 Virtual Sensors: Transforming the Way We Think About Accommodation SensoSmart Virtual Cloud Sensors Stevens Institute of Technology sensosmartvirtualsensors/home

3 Outline SensoSmart Virtual Sensors: Transforming the Way We Think About Accommodation 1. Background Robots, Hearing Aids, & Assistive Devices are Available but Expensive & Difficult to Update Medical Model, Wireless/ HAC Architecture Segregate People & Technology into Disabled & Not Disabled Wireless Example of Hearing Assistance & Hearing Aids Accommodation becomes Personalization 2. Mainstream Wireless Cloud Connectivity for Personalization New Model for Accommodation Machine & Human Sensing & Learning Inferences, Analytics, Recommendations, & Social Sharing Reduce Barriers for All People 3. About Hearing Assistance & Wireless Cloud Connectivity Display how people hear, statistics, audiograms, examples Predict issues for person & context Wireless Applications solve problems in context 4. Conclusion Goal is a One Click Solution SensoSmart Virtual Sensors Website Stevens Institute of Technology

4 1. Background Many excellent assistive technologies are available today including robots, prostheses, hearing aids, & mobility devices. However, they are often expensive, employ proprietary approaches, are frequently not covered by insurance, & they can be complex to use, update, or personalize. The Medical Model segregates people into Disabled and Not Disabled.

1. Background Wireless and Hearing Assistance Highly Regulated HAC, Hearing Aid Compatibility Separate infrastructures Telecoil (T-coil), Acoustic Coupling, Relay Services Require Doctor s

with this architecture - Bluetooth limited to 7 users & 3 meters - Only ios uses Bluetooth to stream media a proprietary link - Consumers become the testers of the System - Single hearing aids limit

5 1. Background Wireless and Hearing Assistance Highly Regulated HAC, Hearing Aid Compatibility Separate infrastructures Telecoil (T-coil), Acoustic Coupling, Relay Services Require Doctor s Prescription & State or Federal Accommodation for Devices and Services - Inductive Loop for PA System or School - Expensive Multiple Devices - High battery use - Public spaces are required to comply with this architecture - Bluetooth limited to 7 users & 3 meters - Only ios uses Bluetooth to stream media a proprietary link - Consumers become the testers of the System - Single hearing aids limit signal processing for example reduce noise in the environment - Purchase lots of adapters for streaming media or to hear over acoustic coupling or T-Coil - Shifting frequencies smears Speech, creates maskers, doesn t help with Speaker ID - Caption phones require Rx & State & Federal funding - Waiting lists - Phones default to non-caption mode

2. Mainstream Wireless Cloud Connectivity for Personalization Virtual Sensors for Smarter Robots, Hearing Aids, and Devices use Mainstream approach of Wireless Cloud Connectivity to provide a social

6 2. Mainstream Wireless Cloud Connectivity for Personalization Virtual Sensors for Smarter Robots, Hearing Aids, and Devices use Mainstream approach of Wireless Cloud Connectivity to provide a social network for people and their assistive devices to create, learn and share new functions & features bringing Personalized easy to use affordable Accessibility solutions to anyone, anywhere. People are living longer; graying of the planet creates challenges in the workplace, healthcare, education, social settings, and in family situations. Personalization using available Wireless Cloud Connectivity can reach more people and bridge the gap to Accessibility.

7 2. Mainstream Wireless Cloud Connectivity for Personalization SensoSmart aims to create a transformative new culture of global Accessibility that will improve economic well-being for those with disabling conditions. Reducing barriers. Through information theory, crowd sourcing, network learning, and viral sharing, SensoSmart will reach many more people with affordable accessibility solutions on popular platforms including smart phones, tablets, and other or any devices.

2. Mainstream Wireless Cloud Connectivity for Personalization Social Networks for Users and Developers, cloud data, peer to peer solutions Media training system, human

8 2. Mainstream Wireless Cloud Connectivity for Personalization Social Networks for Users and Developers, cloud data, peer to peer solutions Media training system, human and machine learning, analytics, inferences Simplify control of robots and devices Improve context awareness and device autonomy Increase and share new features and knowledge

- 1:5 in the US - 1:40 in developing countries have

Sensors - Mainstream wireless access to the Cloud for

Medical Model alone - Network features, databases, GPS,

9 2. Mainstream Wireless Cloud Connectivity for Personalization Example Hearing Aids To Have and Have Not - 1:5 in the US - 1:40 in developing countries have hearing aids who need hearing aids Solution: Virtual Sensors - Mainstream wireless access to the Cloud for Personalization Instead of relying on regulation & Medical Model alone - Network features, databases, GPS, machine learning, social networks, recommenders, human learning Next: Human Hearing Examples

10 3. About Hearing Assistance & Wireless Cloud Connectivity Insight into Hearing Loss Audio display Phonemically balanced phrases. Recordings based on individual audiograms. Hearing aids are important but do not solve all problems. We see opportunities in Personal Hearing Assistants. While 17% of American adults has hearing loss, only 20% of those has a hearing aid. The entire world s production of hearing aids only equals 10% of the people who need them. NIDCD Quick Statistics Fiona Barker - A Cochrane Systematic Review of Interventions

11 How a Person With Hearing Loss Hears the Phrases Diminished hearing, even one ear, creates inequities in people s lives. Statistics reveal loss affects social, educational, healthcare, business, employment, income, and other aspects of life. People are living longer. 1/3 over age 65 have hearing loss. 5% of the World population or 360 million people have disabling hearing loss, a majority living in developing countries. WHO World Health Organization -- Deafness and Hearing Loss

Challenges for hearing in developing countries include: Cost

Cost of hearing aids and batteries Loss to follow-up 1:40

Sensors Hearing aids have many features and can cost $2000 to

Yet these devices do not resolve all issues faced by people

There is opportunity to reach more people & solve issues

14 Challenges for hearing in developing countries include: Cost and availability of medical care, prevention, and diagnosis Cost of hearing aids and batteries Loss to follow-up 1:40 have hearing aids who would benefit SensoSmart Virtual Sensors Hearing aids have many features and can cost $2000 to $6000. Yet these devices do not resolve all issues faced by people with hearing loss. There is opportunity to reach more people & solve issues using mainstream wireless cloud connected Virtual Sensors. Social & Machine Networks & Learning. Viral Sharing. Low cost personalization for anyone, to any device, anywhere.

15 SensoSmart Virtual Cloud Sensors

SensoSmart Acoustic Scene Analysis and Acoustic Geo Tagging will use human

16 Acoustic Scene Analysis and Acoustic Geo Tagging What information is missing from this scene? SensoSmart Acoustic Scene Analysis and Acoustic Geo Tagging will use human labeling, machine learning, and history and analytics overlaid on a map to create an important source of information for those who cannot hear every sound.

17 SensoSmart Virtual Cloud Sensors

Speaker uses Smart Phone Anyone can listen over

issues: - Direction of speaker - Identity of

18 Speaker uses Smart Phone Anyone can listen over their own Smart Phone Use for Audio and or Video Low Talkers? Measure them Hearing aids do not resolve all issues: - Direction of speaker - Identity of speaker, gender of speaker - Speakers further than 1 meter - Background noise, fans, chairs, pen tapping, etc.

19 HearHere example noisy rooms Speaker uses Smart Phone - recognizes context Existing Wireless Architecture - GPS, Network Features Anyone can listen over their own Smart Phone Use for Audio and or Video

22 Dr. Robert Wilson uses the bull horn to speak to a group gathered to mark the 50 th Anniversary of the Big Bang.

23 HearHere! even when you re far, far away.

4. Conclusion a One Click Solution SensoSmart Virtual Sensors Wireless Cloud Connectivity for Personalization and Accessibility NAO watch a way cool robot learn a new song and dance Step 1

24 4. Conclusion a One Click Solution SensoSmart Virtual Sensors Wireless Cloud Connectivity for Personalization and Accessibility NAO watch a way cool robot learn a new song and dance Step 1 YouTube or Video encoded with machine readable message Step 2 Mobile phone grabs machine readable message, like QR or product code Step 3 One click to the cloud to retrieve data or new control program

26 SensoSmart Virtual Sensors

Roger TM at work. Focus on work rather than on hearing

Roger TM at work. Focus on work rather than on hearing Roger TM at work Focus on work rather than on hearing Communicate, participate and contribute The modern workplace can be a challenging listening environment. It is also a place where successful communication