REPORT PROJECT: TRANS2WORK - School-to-Work Transition for Higher education students with disabilities in Serbia, Bosnia & Herzegovina and Montenegro Document title: Report of dissemination at the World Federation of the Deaf conference in Budapest, 8.11.2017 10.11.2017 Partner: Type of DEV: Authors: University of Maribor Report dr. Matjaž Debevc The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the information contained therein.
1 About the conference The WFD conference, which held at the Congress center in Budapest, Hungary from 8.11.2017 till 10.11.2017 elaborated different themes on plenary, section and board meetings: bilingual education, sign language in the families, deaf employees at employment market, communication without barriers, new information and technology revolution, and participation in the decision-making process. The World Federation of the Deaf (WFD), which was organiser, want to ensure also with such events that members of Deaf Communities in every country have the right to use sign language as their primary language in all walks of life with the result being, the preservation of and development of deaf culture. WFD President Mr. Colin Allen, and the SINOSZ President Dr. Ádám Kósa, Member of the European Parliament, emphasized that the WFD Conference, which enjoyed significant support by the Hungarian government, highlights recognition by the World Federation of the Deaf and of the Hungarian National Association of the Deaf and Hard of Hearing, the oldest organization representing the interests of persons with disabilities in Hungary. The WFD conference addressed deaf people, especially researchers (academics and representatives), parents of deaf children, teachers, teachers trainers, administrators and politicians involved in deaf education and any interested parties. At the WFD conference were about 600 participants from 83 countries. The URL link to the conference: http://wfdbudapest2017.com The Facebook link to the conference: https://www.facebook.com/wfdbudapest2017 2 Presentation of the project Trans2Work Dr. Matjaž Debevc from University of Maribor, Slovenia, who is also a member of the Expert Group on the Accessibility of WFD had a 20-minutes PowerPoint presentation on the second day of the conference. He presented the importance of the technology used for deaf and hard of hearing, which are also results from the project Trans2Work. In his presentation he emphasized the importance of the use of technology for video relay services and subtitling also in higher education and on the transition way to work for student with disabilities.
He presented some interesting best practice examples and among other he presented solution for web based video sign language interpreter, which can be easily adopted to the web page. Figure 1 shows dr. Matjaž Debevc during his presentation at the WFD conference. It can be seen from the figure the use of two sign language interpreters (one for English language and one for Hungary language) together with live subtitles for both languages. Figure 1: Presentation of the project Trans2Work by dr. Matjaž Debevc at WFD conference
ANNEX 1: Schedule
ANNEX 2: Powerpoint slides Slide 1 SUPPORTING HIGHER EDUCATION DEAF AND HARD OF HEARING STUDENTS WITH TECHNOLOGY Assoc. Prof. Dr. Matjaž Debevc Member of the Expert Group on Accessibility and Technology of WFD Dr. Ines Kožuh University of Maribor Faculty of Electrical Engineering and Computer Science Budapest, 10.11.2017 Copyright by Debevc, Kozuh, 2017 1/20 Dear Ladies and Gentlemen, It is my pleasure to present you our work on supporting Deaf and Hard of Hearing (DHH) students with technology. We, at the University of Maribor, already have introduced some possibilities and solutions at using ICT successfully in more European projects. Currently, we are working in this field in the European project Trans2Work. The project is intended to help students to go to work after their study. Today, I would like to present you briefly firstly, the right of DHH students, secondly current available technical solutions and, at the end, some future trends in the use of technology for DHH. Slide 2 RIGHTS OF DEAF AND HARD OF HEARING SIGN LANGUAGE STUDENTS Copyright by Debevc, Kozuh, 2017 2/20
Slide 3 Before deciding which technology we should use in the education we should be aware which are the rights of DHH people. After knowing these rights, we can link them more easily with the corresponding technology solutions and products. Copyright by Debevc, Kozuh, 2017 Deaf People and Human Rights, by Ms Hilde Haualand and Mr Colin Allen for the World Federation of the Deaf and the Swedish National Association of the Deaf, 2009 3/20 The first and the most important right is the use of sign language. Sign languages are the natural languages of deaf people. Recognition of the use of sign languages in daily and educational live is critical in the movement for deaf rights. The next key factor to accessibility is the right to sign language interpreting, and this can be accomplished with sign language interpreter network and centres. Deaf students are best instructed in subjects through national sign language, along with the written form of the national language. Thus a bilingual education for deaf children is crucial. For deaf students, accessibility often rests upon the availability of sign language interpreters and accessible e-learning material. Slide 4 Now I would like to show you how we can use ICT for DHH. USE OF INFORMATION AND COMMUNICATION TECHNOLOGIES Copyright by Debevc, Kozuh, 2017 4/20
Slide 5 WFD Statement on the Adoption and Adaptation of Technologies and Accessibilities Vision 2020: Barrier-free access to communications Accessible information Equal interaction with society Accessible products and services Detailed WFD Working document: Adoption and adaptation of Technologies and Accessibilities and how it applies to the UN CRPD Copyright by Debevc, Kozuh, 2017 5/20 The expert group on the accessibility work on the Statement on the Adoption and Adaptation of Technologies and Accessibilities, where we emphasise explicitly the right for barrier-free communication, like using real-time text; the right for accessible information from the environment and in school with the help of a sign language interpreter video, for example; the right for equal interaction with society, like through social networks, and, finally, the right for accessible products and services, like watching movies with subtitles. If you are interested in the work of the expert group you could contact WFD. Slide 6 UN Convention for the Rights of Persons with Disabilities (CRPD) "...all persons with all types of disabilities must enjoy all human rights and fundamental freedoms..." EXAMPLES Article 9: Accessibility obligations... Article 21: Sign language, captions Video Relay Video chat 6 Accessible Websites Captions With the UN CRPD, many articles make reference to where countries and governments who have ratified the Convention, are committed to create service and products that meet these obligations across all facets in community life: Education, community, work, home, media, communication, etc. Here, we can see almost the minimum requirements, which services are most important for deaf users, according to the UN Convention. These are Video Relay Services or video relay interpreting. Then, there is a need to have accessible websites with the glossary of hard to understand terms, and with sign language interpreter videos. Furthermore, video chat conference with one-to-one or up to many-to-many conversation type is needed. Finally, there is a strong need to use captions or subtitles in all video materials. According to the reports from the UK, the BBC has provided 100% captioning of its programme content already since 2008. Unfortunately, this is not the case in other European countries.
Slide 7 INFORMATION AND COMMUNICATION TECHNOLOGIES IN EDUCATION Copyright by Debevc, Kozuh, 2017 7/20 Slide 8 Common types of ICT use in education TOOLS CONTENT COMMUNICATION Teaching and learning Real-time text E-Learning material Administrative (with SL translation Videoconferencing video) Collaborative Social network A medium of creative Cooperative Speech recognition expression Speech synthesis Copyright by Debevc, Kozuh, 2017 8/20 Let we see where we can use ICT in education. DHH can use ICT as a tool for teaching, for example, a teacher can use an app for providing e-learning material, together with a video and for learning, for example, students look at the video and use subtitles for easier understanding of the content of the video. Next, we can use ICT for administrative purposes, like contacting the tutor or school attendant for providing more information about the school process. Finally, we can use ICT for cooperative and collaboration activities, like working in groups on the same document, like Google Doc. Further, we can use ICT for providing the content, like for e-learning material, made together with a sign language video interpreter. And, finally, DHH students can use ICT very efficiently for communication purposes. Here, we are still waiting for reliable and effective speech recognition and synthesis for many world languages.
Slide 9 Proposal for basic ICT for deaf students Laptop with WiFI and Video camera Bluetooth Smartphone with voice recognition Touch screen tablet Copyright by Debevc, Kozuh, 2017 9/20 Here, we suggest some basic ICT technologies for DHH students. The first and most important is the laptop with a video camera. Further, a good and realiable smartphone, with a high quality video camera with the possibility to recognise and sythesise voice and other sounds. Finally, touch screen tablets are becoming popular, since they are light weight and can be used easily everywhere as can the electronic book. Slide 10 Proposal for basic ICT for higher education institutions Multipoint videoconferencing system Video remote interpreting Accessible e-learning system (Source: www.ediv.be) Copyright by Debevc, Kozuh, 2017 10/20 Here, we suggest some basic ICT technologies for higher education institutions. In our opinion, one of the most important ICT products is multipoint video conferencing for grouping deaf students and interpreters together. Also, an accessible and easily portable video display is needed for enabling video remote interpreting, so students could also follow a lecture in school. Next, an important solution is providing a video interpreter for the most important and difficult to understand e-learning material and, especially, for off-line quizzes. Also a video glossary of important terms would be welcome.
Slide 11 Accessible e-learning material (suggestion by University of Maribor) Accessible E-Learning Portal Transparent and movable sign language translator video with captions Communication: Video conference, Video forum, Text chat, Collaboration: Individual assignments, Quizzes, Questionnaires. Copyright by Debevc, Kozuh, 2017 11/20 Regarding the e-learning material adopted to the needs of DHH students, we are proud to show you our accessible e-learning portal, made on the Moodle Learning Management System. For this work, we received the Comenius EduMedia Award in the year 2011. We redesigned the Moodle completely in such a way that is easy to understand and intuitive to use. We also invented a transparent and movable sign language translator video together with captions. We, for example, made a study together with experts from Switzerland, where we proved the need for subtitles for better comprehension of the content, derived by the video interpreter. Next, we suggest to use, in the same learning environment, the use of communication tools, so the user does not need to go outside of the system. These communications tools are video conferencing, video forum and text chat. And, finally, we provided collaboration tools for doing exercises, quizzes and questionnaires. This environment has proven to be successful in terms of usability evaluation and effectiveness of results after the course. Slide 12 Events: Real-time captioning - CART https://youtu.be/nxmdeuso_5s Mobile based:https://captionfirst.com/demo/ Speech recognition: http://www.livecaptionapp.com/video/ Another technology which can be used in a school is real-time captioning. We already know this technology from other events, where they use this technology for making events accessible to hard of hearing people. Copyright by Debevc, Kozuh, 2017 12/20 If you like, you can try the demo from caption first yourself. Just type the link in your web browser and enjoy the demo.
Slide 13 Personal Amplifiers FM System with T-Coil Bluetooth streamer Assistive Listening Devices for school Bring sound closer to the ear Separate the good noise from unwanted background noise Infrared System In progress: Live audio and caption streaming via app Induction Loop Systems Many hard-of-hearing students, including those with cochlear implants, may find Assistive Listening Devices (ALDs) beneficial. ALDs are portable electronic devices that help hard of hearing individuals hear better by bringing the sounds they want to hear directly to the ear. Here, we can see (show the images in the slide). Copyright by Debevc, Kozuh, 2017 13/20 The tutor speaks into a microphone and the student picks up the sound through a receiver with a volume control and listens through a head set or some other device to get the sound to the ear, depending on whether or not the student uses hearing aids or has a cochlear implant. Depending on the student, you may still need to maintain visual contact. Nonetheless, ALDs are very beneficial. They help reduce background noise, which can be very intrusive for hearing aid users (hearing aids amplify everything, not just speech sounds). They also amplify only what is coming in through the microphone, so they help the student to focus in on the teacher`s voice. This allows the student to attend to the content, and reduces the strain of simply decoding the message. Relying on lipreading requires a great deal of energy.
Slide 14 WFD 2020 Vision Accessibility - Education E-book sign translation Children s book - sign translation School - remote captioning Deafblind education Live subtitling 14 Deaf Teachers and education access Here, we can see as an abstract some of the most important factors for education according to the mission of the WFD, more exactly of the Group on the Accesibility. We all know that education is a right for all, not a privilege. Many deaf people are not fluent in their native language, which includes signing, written and spoken language - that is because of the poor education opportunities available in many countries. This needs to change - there are models across the world, some of them we can see on the slide, which show a good education for deaf people is when they have access to all the resources required and, most importantly, to a language.
Slide 15 Trends: Augmented Reality (AR) Captioning: http://www.gretaundstarks.de/ Mixed reality: Facebook AR Instagram AR ARKit Hololens Copyright by Debevc, Kozuh, 2017 15/20 At the end of my presentation I would like to present some trends in providing information to the sign language students. One company, Greta & Starks, is on the verge of launching an augmented reality headset called Starts AR that displays film subtitles for the deaf and hearing impaired, as well as for those who need dialogue translated into their native language. A micro display, similar to those seen on Google Glass, streams subtitles in any language to the user's field of view, as we can see from the image on the slide. Augmented reality trend also helps DHH at acquiring information from the real environment if they are available to see it on the display of their mobile phones. Facebook, Instagram, Microsoft with their Hololens already provide some interesting solutions which can help DHH users to bring more accessible and understandable information to them, even as a Sign Language Interpreter video inside of the real video camera field. Slide 16 Future trends: Holograms delivering Sign Language https://youtu.be/ulfvmpghvc4 Copyright by Debevc, Kozuh, 2017 16/20 Here, we can see a quite interesting solution, called hologram, which was also designed together with Microsoft. Currently, PRSONAS units can deliver sign language and spoken word communications (in any language). The company have built, and are currently testing, voice interactions. With these solutions, sign language recognition is still not possible. Maybe this will be changed with Machine Learning, more exactly with Artificial Intelligence technology, as we can see in another slide.
Slide 17 Future trends: Sign Language recognition using Machine Learning By Microsoft: Advancing Research in Sign Language Recognition Copyright by Debevc, Kozuh, 2017 17/20 The problem of sign language recognition, which is still in research, consists of three tasks to be done in real-time. Firstly, obtaining video of the user signing (input); secondly, classifying each frame in the video to a letter and thirdly, reconstructing and displaying the most likely word from classification scores (output). Moreover, the problems are environmental concerns, like lighting and background visual noise; problems with movements outside of the camera view; problems when signs starts and ends, and with co-articulation of signs. Some results show improvements in this way, but are still too slow and not 100% reliable. On the other hand, we encourage and support this research field, since we believe there is a lot of free room for solutions for DHH students. Slide 18 Assoc. Prof. Dr. Matjaž Debevc e-mail: matjaz.debevc@um.si Contact Here, on my last slide, you have my contact data in case you want to know more in this field. Thank you! University Maribor Faculty of Electrical Engineering And Computer Science Smetanova 17, 2000 Maribor www.feri.um.si Copyright by Debevc, Kozuh, 2017 18/20
Project Coordinator Erasmus+ KA2 Capacity Building in Higher Education Project contact person: Lefkothea Kartasidou, Associate Professor lefka@uom.edu.gr Project email 561847.trans2work@gmail.com Project website Project duration: 15 th October 2015-14 th October 2018 The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the