Progress Has Been Made But We Are Not Yet Where We Need to Be

Transcription

1 Progress Has Been Made But We Are Not Yet Where We Need to Be Re-published with permission from Family Center on Technology and Disability Technology Voices December 2009

2 Progress Has Been Made But We Are Not Yet Where We Need to Be An Interview with C-Print research team members Mike Stinson, founder and principal investigator, Pam Francis, development coordinator and Lisa Elliot, senior research scientist Tremendous technological progress has been made in the past three decades as we ve evolved from classroom interpreters to tablet PC-based speech-to-text captioning, declares C-Print founder Mike Stinson. Today, he adds, although language and communication skills of deaf and hard of hearing students are aided by the widespread use of cochlear implants, early identification of hearing loss and early intervention, many of our students continue to lag behind hearing students and we are not yet where we need to be. The research path from the early days has been a hard but steady climb. While novel for its day and beloved by 1980s students, the stenographic system based on court reporting does not does not take advantage of certain new technologies that can improve the communication access and learning of deaf students. For example, steno-based systems do not display graphical information, or enable students to easily adapt the display to meet individual needs or add their own input to the information being displayed. In contrast, the tablet PC-based speech-to-text system being refined by the C-Print team in its Steppingstones grant has all these features, C-Print researcher Lisa Elliot says. Nothing Is Left Out Research and development for C-Print commenced in 1989, Dr. Elliot recalls. In the beginning Mike Stinson and collaborator Ross Stuckless (now retired) began with realtime captioning, i.e. stenography. They discovered quickly that captioning for the deaf requires the generation of a lot of text. The court reporter stenography provides every single word and was expensive. Nothing is left out.

3 Mastery of the craft of stenographic court-reporter-style transcribing, which employs abbreviations, requires 3-4 years of study, she explains. The equipment was expensive. From an educational perspective the court reporting form of captioning was far too costly for most schools to provide. Mike wanted to find a different way. I still have samples of stenographic transcripts from the 1980s, Dr. Stinson says. They are very long and very hard to study. We heard a story from a student who used C- Print in secondary school here in Rochester and then went to a college where only a stenographic system was available. The student was so frustrated with the notes that she gave up using a stenographic system and went to using an interpreter. The speech-to-text display produced with C-Print software represents the spoken dialogue of the classroom using a software application called C-Print Pro. The text can be displayed simultaneously to one or more students in different ways, including additional computers (laptops) or display monitors. The captionist includes as much information as possible, generally providing a meaning-for-meaning (not verbatim) translation of the spoken English content. Changes in delivery technology aside, C-Print has always been based on a dictionary comprised of complete words accompanied by corresponding abbreviations. Captionists learn a rule system in order to type abbreviations instead of complete words. Says Dr. Stinson: They can t memorize every word so they use the rules. With Pam overseeing the evolution, we have continued to revise and improve the rules and the abbreviation system. Even in retrospect, Ms. Francis says, the early 1990s version of C-Print was far from cumbersome. The key was phonetics. The process is auditory. Using a phonetics-based abbreviation system for many is a natural process because they are typing what they hear. It can certainly help those who are not strong spellers. Conceptually it doesn t seem cumbersome even now. As technology changes there are different things that can be done to improve the application that we developed. I still think the original concept is very effective. Dr. Stinson and his team began the transition from stenography with basic computers, using basic word processing programs, he recalls. At first, Dr. Elliot 2

4 remembers, we used off-the-shelf computer programs - word processing programs, abbreviation programs and a program that allowed the computers to communicate with each other. When several programs were used, it was difficult for captionists to operate all of the applications simultaneously, and it was difficult for the project team to maintain continuity with changing software versions. We eventually decided to write our own application that would combine aspects of all three programs to create a single, smooth program. The C-Print Pro software ( has been developed with the aid of RIT students, who work with us through the RIT co-op program or as part-time employees while they re in school. Keeping Up with Technology: the Tablet PC C-Print employs the latest in popular technology. Keeping pace with that technology, says Pam Francis, presents the team with a continuing challenge because we re an educational institution. We aren t a company that produces commercial tools. We try to keep up with technology within the limits of our grant funding. Things change so fast and we are trying to do the research and also the development. There are commercially available options out there. Ironically, many of the products offered by commercial entities are based on the research we ve conducted at C-Print. She adds: We are not only doing product development or providing a service. We are conducting the research that shows the effectiveness of existing applications, ours and others. We ve been doing this for a long time and have developed the technology and conducted the research that has allowed others to move forward technologically. The current Steppingstones grant calls for the use of C-Print software with a tablet PC, extending the capabilities of the software. Says Dr. Elliot: With the tablet things have changed. We can still do the traditional (speech-totext) C-Print but we can also use the tablet in other ways. Those ways, she explains, include real-time speech-to-text services with graphics and real-time notetaking. The speech-to-text with graphics option includes 3

5 teacher-student classroom exchanges; that is, the student views a real-time display of the text of what is being said, just as with traditional C-Print, and in addition, can view graphics produced by the provider. With the real-time notetaking option, the student is able to view in class what is written as it is written by a notetaker. This new form of notetaking is quite different from traditional notetaking. Students using traditional notetaking services often received their notes from another person paid to take notes in their classes. That person took notes on four-part carbon as she sat in the rear of the room. Students received those notes at day s end. The notetaker did highlighting and filled in worksheets. Student and notetaker fulfilled their respective tasks independent of each other. This was a very labor intensive system and very hard on notetakers hands. It can be frustrating too, especially if the carbon doesn t work properly. With the incorporation of C-Print software, however, notetaker and student have tablet PCs that communicate wirelessly. The student can see what the provider who is notetaking is doing in real-time. The student can also add to whatever the notetaker is doing and the notetaker can see what the student is doing, which can be very beneficial in that it provides an indication of areas in which a student is weak and needs improvement. The Study The current Steppingstones study encompasses students in grades 6-12 in mainstream settings, Dr. Elliot says. College students are the centerpiece of a companion study that dovetails with the Steppingstones grant. Our projects blend with each other; they evolve. We ve received U.S. Department of Education funding since The current Steppingstones grant is the fourth such grant we ve received. The Steppingstones study is comprised of three groups of students: the realtime notetaking group; students who receive speech-to-text with graphics; and a control group. When the C-Print team applied for Steppingstones funding from OSEP, team members identified the following research questions: What is the effectiveness of tablet technology? What is the fidelity of implementation? 4

6 What are the factors that contribute to the effectiveness of the tablet technology? In regard to the last question, Dr. Elliot notes, what we re looking at are student characteristics such as level of hearing loss, reading ability and the ways students employed the technology. There are questions about student motivation. We also look at teacher practices in the classroom and at student familiarity with computers; some students are more comfortable with computers than others. The study segment of the current grant encompasses 90 students in several metropolitan areas nationwide. Some of the students may have multiple disabilities, but whether or not they have multiple disabilities is not a contingency for participating in the research. All the student participants have been diagnosed as deaf. Some are hard of hearing. Some are profoundly deaf. Some use hearing aids. Some have cochlear implants. Some use FM systems or interpreters. The support services they receive or the kinds of assistance they have don t matter. Every student who is in a mainstream setting can participate. According to Dr. Elliot, the 10-week period of study is modeled after the traditional 10-week school marking period. Students in the technology trial groups use the tablet PC technology in just one science, technology, engineering or math (STEM) class for five weeks. Classroom teachers evaluate student progress made using the new technology. Classroom teachers also evaluate student progress for five weeks before or after the trial period. For 10 weeks the control group receives no technology and employs only the supports used previously. Control group students are also evaluated by their teachers. After that period the control group gets a one-week trial, but no data is collected from the group. Students in the study participate in an experiment in which they all view the same simulated lecture. According to Dr. Elliot, they take a pre-test, view a video, then they do a post-test. The way they view the video depends on which group they ve been in, because we ve tried to simulate a class experience for them with the support technology that they actually used. If they were in the real-time notetaking group during the regular trial, then there is a screen that shows the teacher and the lecture and a second screen that replicates the notetaking that is taking place. It s the same arrangement with the speech-to-text with graphics group. The third group the control group -- has an interpreter. The objective, she notes, is to recreate the 5

7 students classroom experience and to compare student performance on the minilesson assessment in each of the three groups. Phases of C-Print Research According to Dr. Stinson, phase one of C-Print research consists of device development. We have to make something that will work, in this case, two options: a system of real-time notetaking created for tablet PCs; and a combination of traditional C-Print captioning with the addition of graphics. Focus groups are conducted with students, parents, service providers, and teachers. We bring in consultants and others with technology backgrounds to provide us with their input. Trials are then conducted. We test [alternative] options and constantly get feedback in order to improve them. We conduct 15 trials for the typical project, inputting continuous improvements as the trials progress. All phase one activities are conducted in the Rochester, NY area. In phase two scaling up occurs as the trials are conducted at schools located throughout the country. In the current Steppingstones grant, trials are being conducted in San Diego, Bucks County, PA and New Bedford, MA, as well in the Rochester, NY area. This phase, Dr. Stinson says, involves a more comprehensive evaluation to determine the educational effectiveness of the technology. In this case we use a randomized design. Some students are trying the notetaking option, some are using the C-Print text-and-graphics combination and some remain with their standard IEP service. We include teacher ratings and student opinions as well as objective measures to ascertain how much they are learning with the option they are using. These strategies, he explains, enable us to develop a more quantitative assessment of how much the students benefit from the technology. We also take measures to address fidelity of implementation, i.e., how do the programs implement the services with their service providers. 6

8 Fidelity of Implementation Research, Dr. Elliot emphasizes, is about being systematic. But I can only be in one place at one time. It s always a challenge when we work on studies in multiple locations nationwide to make certain that procedures are followed uniformly everywhere, which is a requirement when conducting research. Coordination and timing are paramount. Because members of the small C-Print research team are unable to deliver services everywhere at once, fidelity of implementation, according to Dr. Elliot, must provide answers to the following questions: How are students, classroom teachers, and school administrators approached about participating in the study? How are students trained to use the equipment? How is the service provider trained to use the equipment? We ve created brochures and thought hard about these approaches. We conduct orientation meetings with site staff to determine what should be said in a meeting to introduce a trial with the technology and who should attend. Details in regard to implementation of a technology in a particular classroom are settled before we enter a classroom. Randomized Design Challenges The randomized design requirement of the current Steppingstones grant the first time ever in C-Print s relationship with Steppingstones that random design has been required presented an execution issue for the C-Print team. Randomized design is difficult to execute with a low-incidence population, Dr. Elliot declares. Finding enough students is difficult and a randomized design requires many students. We need to have a certain number of student participants in order to be able to demonstrate statistical significance. Fortunately, she adds, we ve been successful in gathering students to produce statistically significant results because our reputation is a good one. We ve worked with some programs for years. Sometimes it can take longer to assemble the student participants than we originally planned. 7

9 When a randomized design is required, the volunteer participants are not told which group they ll be in it could be the control group. We devised a way to reward the students who had to be in the control group; they received a free week to use our technology. After the data is collected we return to the school and provide the participating student an opportunity to sample the new technology. Unfortunately, a week is not a lot of time, Dr. Elliot comments. It s a reality that when the study ends, the new equipment departs with the researchers. All of our research and development is based on feedback from students, teachers and service providers. I observe how the technology is used and then talk to participants about their experiences. All our changes are based on that feedback. It s an iterative and customized process. The problem is that when we test the software in a live classroom and the students like it and need it, removing it can be a wrenching experience for all. Often the administrators, parents and students volunteer to participate because they think the tool will be beneficial, and it often is. And then what? That s the downside. The upside is that we are careful to develop something that will work and will eventually get to the schools. Sometimes, she adds, administrators may be reluctant to allow us to come into the school because if our equipment is effective families will want it and the school will have to provide it. We re not in this to make money. We re not in this to make a profit. We ve created the technology so it can be provided to schools and individuals at low cost. Some administrators are reluctant to get involved because they don t want to make an implied commitment to the future. The C-Print Team: Ongoing Enthusiasm and Varied Skills Are Hallmarks According to Pam Francis, ongoing enthusiasm for its mission and varied but complementary skills are the hallmarks of the C-Print team. Our team s enthusiasm flows from grant to grant. We have a variety of backgrounds. Lisa conducts qualitative and quantitative research. My background is development and training. Our two research captionists have educational notetaking and business backgrounds, and we have a talented software engineer who decided to stay with the project after graduation. We ve had excellent technical personnel throughout, 8

10 although they tend not to remain with the team for long because of their student status. Everyone is a very important part of our overall mission and the variety of backgrounds generates a more complete vision. The team s development effort, she continues, has been tasked to software engineering and computer science students from RIT. The RIT connection is a strong positive. This is training and preparation for these students to move on to other software development activities, to graduate and find other work. Often, though, she adds, their work with us has encouraged them to seek out meaningful, non-commercial work. Because our project encompasses aspects of software research, development, and implementation the students who emerge from this project are usually well-rounded. This is consistent with the RIT tradition of preparing students for valuable technology careers and is well-received by federal funding agencies who like the idea that we are creating a learning environment for the students who are going to move on. During the 16 years since their first Steppingstones grant, the C-Print team has trained more than 1,500 captionists at RIT and elsewhere. According to Ms. Francis, about 25% of the practicing captionists provide services in middle school and high school settings. Evolving with the Technology C-Print research has kept pace with technological advances, Ms. Francis says. For example, in an earlier era when laptops needed to be connected to one another to communicate, they were joined by cables. Now we re in a WiFi era. We can route the information through the Internet. Not only do the computers no longer need to be connected by a cable, they no longer need to be in the same room. The centerpiece of C-Print s current Steppingstones grant, the tablet PC, is especially effective for students taking courses like math, technology, science and engineering in which information is presented by alternative means that include graphs, illustrations and formulas. Typing, which suffices for courses like English and social studies, does not adequately capture this information, Dr. Elliot declares. Students using an interpreter must keep their eyes on the interpreter while trying to absorb other visual information. It s very difficult for them to look at the interpreter 9

11 and the overhead projector or the board. If they move their gaze away from the interpreter they lose the information. We ve been able to bring some of that information into the same visual space by using the tablet PC. Notetaking Is Active Improved notetaking and use of notetaker s notes by the students who view the C- Print display remains high on the C-Print team s agenda, Dr. Elliot says. Hearing students take notetaking for granted. There is much research to suggest that notetaking in class aids students in tying the immediate class material with previous knowledge, in making relationships and in jotting down notes to remember specific items. Notetaking is active. Previously, she explains, deaf students had a notetaker who sat in the rear of the room and delivered the notes to students later in the day. Our software enables students to interact with the notes, she states. The C-Print software, she explains, enables students to highlight the information transcribed by the captionist as it is being transcribed. Students create a notepad on which they can copy and paste. They can write questions for themselves. Because they are working on a tablet, students don t even have to be able to type; they can draw with a stylus. They can create symbols of emphasis like stars or asterisks. We have learned that students who are able to manipulate the notes, not just passively read them, tend to do better on quizzes and tests. Another C-Print software advantage, she points out, is that it empowers two-way communication. We ve included ways to allow students to communicate with the service provider via an instant messaging (IM) system. If the student is unable to voice, she can type/write a note and the provider can voice the student s question or comment to the class. It enables the student to have a more private conversation with the provider, perhaps about a word or phrase. There s communication support as well as access to information. It s often very difficult for deaf students to participate the same way as hearing students. The software enables them to do that. According to Pam Francis, the basic C-Print software is straightforward. There are two applications that communicate with each other. The information that is 10

12 produced by the service provider is sent to the client. The two computers communicate via the software. The basic connection protocol is not fancy. Most computers have this protocol today. The computers do not require a broader network in order to be connected. They can be connected via an ad hoc connection. Introducing Complementary Technology One of the C-Print team s major objectives, Ms. Francis says, is to broaden the capability to introduce complementing technology, like the tablet PCs or mobile phones, for example. A mobile version of the software can be used via a cell phone, she notes. There might be a need to have text in an environment that does not necessarily have a traditional network access. Our companion National Science Foundation project calls for us to develop a mobile version that will enable deaf and hard of hearing students to use text in an environment that does not have local network access, like a field trip, where students can use a cellular Internet connection. A recent trial of the tool, she says, involved a visit to a local museum by a client with low vision. The results, she notes, could have benefits for deaf and hard of hearing students as well. Explains Ms. Francis: The student used some capabilities built into our client application that would allow her to have a larger font. She could view the captioning on a cellphone, adjusting it to a setting appropriate to her visual needs. This assisted her in a number of ways. For example, often museums are dimly lit to preserve the artifacts, so they lack the lighting that would enable an interpreter to work effectively. This device allowed the student to read the captioning despite the museum lighting. That was a situation in which we were able to use the cellular network. We re still using the Internet but we re using it over the cellular network. A campus network is equally effective and can be used instead of a cellular network. We use IP addresses, which broadens our capabilities. Another example involved the use of the remote during a field trip to a local phone company. We found we could use the device in an environment that was very noisy. We have a Bluetooth headset on a speaker that has noise canceling 11

13 capability. The speaker delivers the information over a regular cell line via a voice connection to the captionist. The information is transmitted to the student on the student s phone via a cellular connection. What was fascinating, she remarks, was that hearing students joined the deaf student in looking at the phone display because the noise level was so high that the students were unable to hear what the speaker was saying. The text was effective for all the students. What the C-Print team aims to accomplish, Ms. Francis explains, is to provide a display in environments that are non-traditional, that require a little more creativity in getting the information to the captionist, who can in turn send it to student. This is achieved in real time. There s a slight lag, the same lag that occurs in any remote situation. But it s a version of remote that is not dependent on a local area network. Adds Dr. Stinson: This kind of a mobile phone works much better in this situation than having the student and captionist carry two heavy laptops on a tour. Ensuring Access to the Curriculum A longtime deaf rights advocate, Dr. Stinson strives to ensure access to the curriculum for students using assistive technology. I ve chosen to deal with that question as a researcher and developer, Dr. Stinson declares. I ve put my energy into leading a team to try to create something that is very useful to deaf students. That s my number one motivation, not publishing articles, but instead creating something of value. That s not to say that I don t want to publish articles in order to disseminate the results of what we re achieving or not achieving. My main objective, though, is to produce tools that can be used by deaf and hard of hearing children. His team, he says, has much experience in advising families who want C-Print. We often provide information to families who would like their child s school system to at least try out the technology in order to get a sense of it. We are reluctant to tell parents that they should tell their school system, You must make a full-fledged commitment to this. Implementing C-Print represents a big commitment. A person has to be trained and paid a salary to provide the service. This is a costly step for a school system, although it s not more expensive than an interpreter. We understand why schools may hesitate. The grant offers an opportunity to conduct a trial, and if 12

14 it s clear during the trial that a child is benefiting from the service, it becomes easier for a school system to support that child. Adapting to the Curriculum C-Print, Dr. Elliot insists, adapts smoothly to the curriculum because whatever is being said in the classroom can be provided to students. In addition, there is a human who interfaces with the material. One difficulty, she notes, is in not interfering or being interfered by the other technology in use in the classroom. For example, she says, we re finding that schools have Internet access in their classrooms. Our computers are communicating with each other. Sometimes there is just too much traffic. If there s an interactive whiteboard in the classroom occasionally there will be interference. The service provider has to be flexible and adapt to whatever technology crisis surfaces. A better question, she says, is Can the technology keep pace with the ideas we generate? For example, the first time we considered writing on a computer was in In 1998 there weren t many tablet PCs anywhere. We had to wait until the technology caught up with us. Can the school keep up? We try to make sure that the software is usable on a range of computers that the technology is not a memory hog, for instance. Culturally, she adds, schools approach computers differently. For instance, will the school allow our computers in the building? Does the school have a printer for us to print notes on? Is it a color printer? Most schools do not use a color printer. Therefore, printing is a challenge. What Teachers Say Teachers have mostly been very receptive to the C-Print research effort, Dr. Elliot insists. Over the years we ve learned that there may be a benefit for the student but there may be secondary benefits for the classroom teachers. There are benefits also for itinerant teachers of the deaf who are not in the classroom. Many itinerant teachers work with their students outside of the classroom, she explains. The students receive notetaking support in the classroom but because it s 13

15 an electronic file we also have the capability of printing out notes, ing the notes and posting the notes on the web. There are different ways of distributing them, but teachers of the deaf can have access to those notes. That gives them a very good picture of what s going on in class by providing context regarding ways to help students [it provides] information teachers can use to work more productively with their students. There are other benefits for classroom teachers, she adds. Sometimes they retain the notes because the notes help them plan for the following year. The notes may also be useful in preparing a substitute teacher. If the regular teacher is absent she can obtain the notes and learn what has occurred in her class in her absence. The notes also help teachers prepare exams, Dr. Elliot says. This year, with so many students out with the flu, the notes can be distributed to other students in the class. The notes can also provide teachers with some insight into their own classroom performance. Some teachers worry that the speech-to-text version will record their every word, including instances in which discipline is meted out. Most teachers, though, once they become accustomed to it, cease to be worried. Captionists can work with teachers to decide what will be in the final copy of the notes, she explains. In some schools providers edit the notes. Teachers, she adds, appreciate the ability of the tablet PC notetaking option to capture students writing, which enables teachers to acquire insight into the information a student is likely to retain. This opportunity didn t exist previously. Occasionally, she says, the team encounters a random teacher who is uncomfortable with the presence of another adult in the class. Some teachers are distracted by a provider of real-time captioning who is tapping keys while the teacher is speaking. Fortunately, the tapping of keys for most of us has become the background noise of everyone s life. While there are certainly criticisms there are concerns about intellectual property being captured teachers for the most part have appreciated having the information notetaking provides. The Intervention: What s been Learned? Although Dr. Elliot confirms that the C-Print intervention has been beneficial, the 14

16 question of observable patterns elicits a nuanced response from the veteran researcher. From studies associated with our earlier Steppingstones grants we ve learned that C-Print is definitely a viable support service for students. Based on past studies we ve done we know that providing C-Print to a student doesn t hurt the student. In some circumstances use of C-Print may provide more information than an interpreter. In other circumstances an interpreter is better. C-Print, she continues, provides access to students. Not all students know sign language. C-Print can provide that access, sometimes the only access students have. This access was previously unavailable. The team also learned that the notetaking capability is useful. Students are responding positively to the graphics. We can use various colored highlighters and pens. Students find the ability to use color is helpful. Take a math class, for example. Students know that working a complicated math problem requires many steps. In some middle school and high school classrooms teachers will use colored chalk or colored markers on a whiteboard to indicate steps. We can mimic that with our multi-colored pen so a student can more clearly follow the teacher s presentation. Students also found various ways to code the information, she says. For instance, if three topics are being discussed during a class a student might choose a certaincolored pen to delineate subtopics. These colors aid students in organizing the information. The team learned that educators expect students to know how to take their own notes, but teachers don t always explain the mechanics of notetaking to students. We can use this as a learning opportunity for students and teachers. We can use notetaking as a way to teach students how to study. We ve surveyed the ways that students study. We can feed that back to them and show them how to make vocabulary lists, for example. We, of course, expect that students will create study systems on their own but not all students have that capability. In terms of the current study, Dr. Elliot adds, we re only about one-third of the way through the data collection process so data analysis has not yet begun. 15

17 C-Print Staying Power and the Future The C-Print team is hopeful that C-Print in the future will be as resilient, adaptable and relevant as in the past. My involvement with text-based systems goes all the way back to 1982, Dr. Stinson comments. The provision of speech-to-text services didn t take hold until around We ve been around for a long time. And we have persisted. Ms. Francis calls herself a closet geek because she loves technology. I love working with people in training. It s been a great challenge to move things from research to the next step. Implementing that progression requires us to bridge the gap between research and practice a sometimes treacherous transition -- and aid others in making that transition. Even at NTID that transition is often difficult. There are more than 50 C-Print captionists here at NTID and the Institute may require more to fill the need. The need for interpreters is growing as well. The point is that the text is not going away. In fact, the need for it is growing. Our goal is to meet that need and evolve along with the technology. For his part, Dr. Stinson is hoping for yet another Steppingstones grant after I ve written some preliminary drafts in the anticipation that there will be funding. We want to continue this work. We think there are many new kinds of situations that are not currently being addressed, such as group work and communication. Deaf students often are not participating in group activities. We d like to do something that helps them participate. 16