White Paper: micon Directivity and Directional Speech Enhancement

Transcription

1 White Paper: micon Directivity and Directional Speech Enhancement Eghart Fischer, Henning Puder, Ph. D., Jens Hain Abstract: This paper describes a new, optimized directional processing with integrated spatial, transient and stationary noise reduction, introduced in the Siemens hearing instruments powered by micon, the new platform behind BestSound Technology.

2 Micon Directivity and Directional Speech Enhancement 1 Improved Directivity and the Directional Speech Enhancement Feature of micon Noisy environments and conversations in large groups, e.g., party or restaurant situations, continually have been reported to be the most challenging listening situations for hearing instrument users [1]. This paper describes a new, optimized directional processing with integrated spatial, transient and stationary noise reduction, introduced in the Siemens hearing instruments powered by micon, the new platform behind BestSound Technology. I. Introduction Different types of interfering noise can impede the ability to follow conversations for the hearing instrument user in various hearing situations. In order to define the terminology of the signal processing schemes introduced in the later part of this paper, it is useful to classify interferers as depicted in Figure 1 into Speech interferers: Noise created by interfering single or multiple speakers Stationary noises: Noises with very low modulation, i.e. the signal varies much slower than speech in its temporal and spectral properties. Typical examples are the noise in a crowded street or from a passing car. Transient noises: Impulse-type noises, which change in temporal and spectral properties much faster than speech. Typical examples are the sound of a clinking glass or a hammer. Figure 1: Different type of noises (transient, stationary, speech interferers) which limit the hearing impaired individual s ability to follow a conversation

3 Micon Directivity and Directional Speech Enhancement 2 Beamforming and Noise Reduction: Merging two worlds of hearing instrument signal processing As a typical solution to enhance communication abilities in noise, directional microphone systems, also called beamforming systems, process and combine two or more microphone signals per frequency channel into one single directional output signal. They are able to improve the signal to noise ratio (SNR) and thus increase speech intelligibility, especially in situations mentioned above. On the other hand, noise reduction systems process one single input signal per frequency channel, mostly by applying a gain or a filter. In hearing instrument applications, the output of a beamformer is usually fed to the noise reduction system which provides further ease of listening. Previously, beam forming and noise reduction have been considered as separate units in hearing instrument signal processing. By introducing a new type of spatial noise reduction called Directional Speech Enhancement, this scheme has now changed into a beneficial integration of both techniques and complete the comprehensive environmental noise reduction system consisting of transient (SoundSmoothing), stationary (Speech and Noise Management), and the new spatial noise reduction in Siemens hearing instruments. Figure 2: Block diagram of a fully integrated beamforming and environmental noise reduction system Figure 2 depicts a block diagram of the fully integrated speech enhancement system. The noise reduction system consists of three functional blocks: Transient noise reduction reduces signals of which the temporal spectrum changes much faster compared to speech independent of the spatial direction of the interferer. Steeply-rising slopes in the signal envelope and faster than speech onsets are detected and instantaneously attenuated to a still audible but comfortable level. (SoundSmoothing feature in Connexx). Stationary noise reduction reduces noise whose of which the temporal spectrum changes more slowly compared to speech independently of the spatial direction of the interferer. Typically, noise spectrum changes in the range of a few 100ms can be tracked and reduced via Wiener filtering. In case of very stationary noise environments without speech activity, the overall noise is additionally reduced. (Speech and Noise Management feature in Connexx) Spatial noise reduction reduces noise from undesired directions independently of its temporal spectrum. Directional side information from the differential beam-

4 Micon Directivity and Directional Speech Enhancement 3 former is utilized to further attenuate signals in the same directions as the beamformer. (Directional Speech Enhancement feature in Connexx) In attempting to maximize their effectiveness and still preserve excellent sound quality in any given situation, both beamforming and noise reduction schemes pose specific algorithmic challenges. Some of these challenges and how they are solved are described in this paper. II. Solutions i.) Improved Directivity Differential Beamforming and its challenges To realize beneficial directivity in hearing instruments with very small dimensions, typically differential beamforming is applied because this method is able to achieve high directional effect using two closely spaced (3-12mm) omnidirectional microphones (Figure 3). Front microphone Rear microphone Figure 3. Siemens Ace micon hearing instrument The principle of a differential beamformer is simple: The signal of the rear microphone is first delayed and then subtracted from the front microphone. Using different delays allows the adjustment of the angular direction of the maximum interference cancellation, called notch direction as shown in Figure 4. Choosing the delay to zero (no delay), will result in a directional pattern called figure eight, which attenuates interferers from ±90. A d elay that is equal to the propagation time of sound from the back to the front microphone will cancel out interferers from the back. This directivity pattern with notch direction to 180 is called cardioid. In state-of-the-art hearing instruments, the notch direction is chosen automatically by an adaptive procedure which tracks the position of the interfering sources in the rear hemisphere and places the notch to an optimal position for maximum interference cancellation (adaptive differential beamformer)[2].

5 Micon Directivity and Directional Speech Enhancement 4 Figure 4. Beampatterns for different delays. Blue tracing: no delay, beampattern Figure Eight, Green tracing: delay = propagation time back-to-front microphone, beampattern cardioid, Red tracing: beampattern with delay between eight and cardioid These ideal directivity patterns can be achieved in free field conditions. Placing the hearing instrument on the ear (such as with a BTE instrument) or in the ear (for an ITE) compromises the ideal conditions. This is due to the diffraction and shadowing effects caused by the head which modifies the propagation times of different frequencies by different amounts. As a consequence, it is not sufficient to just use one frequency-independent delay for all frequencies for a hearing instrument in wearing position. A second contributing real-world restriction when realizing a differential set-up is due to the fact that it is physically impossible to build microphones without any self-induced noise. The largest disadvantage of basic differential beamformers is their strong amplification of low frequency microphone noise components. This occurs because a low pass filter has to be applied to compensate for the inherent high pass characteristic of the differential processing. Both real world issues are addressed by micon high-resolution directivity. High frequency resolution processing to improve directivity Former realizations of differential directional processing often used broadband signals or just a few frequency channels because of limited computational resources. The latest generation chip technology, micon, makes it possible to make a significant advancement in hearing instrument directivity. The directivity achieved in micon with high-frequency resolution in 48 channels shows clear advantages with respect to the ability of cancelling out interfering noise from the back hemisphere ( interference cancellation ). Optimal interference cancellation Head shadowing provokes a certain frequency dependency of the notch direction for a frequency independent (broadband) implementation of the beamformer, as shown in Figure 5 on the right. Notches are spread around the intended angle which prevents a constantly high attenuation of the interfering source (red arrow) over the complete frequency range. In case of a frequency dependent implementation in multiple channels, the adaptation procedure is able to choose, in each channel, the optimum placement of the notch for the best interference cancellation, as shown in Figure 5, left.

6 Micon Directivity and Directional Speech Enhancement dB dB dB 80dB dB 80dB 60 70dB 70dB 60dB 60dB Figure5. Beampattern measured at the left ear of an artificial head (KEMAR) for interfering signals from 240. Optimized beam pattern with freq uency dependent adaptation (left) compared to frequency independent adaptation (right). The colors indicate different frequencies. Precise, head-related optimization The two static directional characteristics cardioid (see Figure 4, green) and anti-cardioid (reversed cardioid with notch direction to 0 ) are the underlying basic building blocks of the adaptive differential beamformer. By optimizing these important building blocks separately for each frequency channel for a typical head, it is possible to compensate for frequency-dependent head shadow effects more precisely and thereby achieve a transparent transfer of signals from the desired direction (very close to 0dB). In summary, the micon 48 channel directional microphone can provide very precise, deep and fast adapting notches which are the preconditions for good interference cancellation and an important component for improved speech intelligibility. Test results showing the clear benefit of the 48-channel high-resolution directivity can be found in section III of this paper. Impact of microphone noise How the tradeoff between directivity and microphone noise can be solved The differential directional approach provides desired directivity but as a side effect reduces the amplitude of the resulting output signal for low frequencies. This high pass effect has to be compensated by applying an amplifying low pass filter. Thus a flat frequency response of the target signal can be restored, but only at the cost of amplifying low frequency microphone noise. This problem is not unique to micon, but true for all directional products. This well-known property concerning the inherent noise of differential directional processing is not a problem when there is loud background noise, as the resulting amplified external noise signal exceeds the internal noise. In soft or moderately loud situations, the undesirable microphone noise can be audible for the hearing instrument wearer.

7 Micon Directivity and Directional Speech Enhancement 6 Until now, the way to minimize this problem was to generally reduce and limit the directional performance for low frequencies, typically below khz. By applying this limitation, virtually no microphone noise can be perceived in quiet environments, but, on the other hand, beamforming is effective only above this frequency. This solution is seen in many commercially available state-of-the-art systems. However, in the presence of a target speaker or interfering sources, this extreme limitation is actually not necessary since the target speaker or the residual noise sources will mask major components of the microphone noise anyway. And more importantly, a fixed limitation as described above would prevent the system from realizing the full potential benefit of the beamformer, and thus would not lead to the theoretically achievable improved speech understanding, which is essential for these challenging listening situations. The solution is to use an adaptive directional beamformer with a constrained adaptation. This patented method [3] is able to cope with the inherent trade-off between more directivity and less microphone noise. The adaptation range of the adaptive directional beamformer spans from an omnidirectional setting (no directivity, lowest possible microphone noise) to cardioid and hypercardioid (Figure 6) up to a possible directivity pattern which can have notches to ±90 (most extreme directivity, highest microphone noise, not shown in Figure 6) dB 30-5dB Amplification of the Microphone Noise Figure 6. Amplification factor of the microphone noise (left) and corresponding directivity patterns (right). The directivity is optimized to attenuate unwanted signals as much as possible but under the constraint that the microphone noise is constantly masked by natural environmental signal components and that the desired signal from the frontal direction is left completely untouched. Using this approach, maximum directivity and speech intelligibility with high sound quality can be reached for higher-level noisy situations separately for each frequency channel. At the same time, directivity and the resulting audible microphone noise is automatically limited for more quiet situations and frequencies with low level signals. The internal noise is masked so that the user has the maximum benefit with respect to superb sound quality and the best directionality possible.

8 Micon Directivity and Directional Speech Enhancement 7 ii.) Integrated Noise Reduction Why different types of noise need different types of algorithms As we have discussed, directional microphone processing (beamforming) has proven to be an effective method for improving speech intelligibility in noise, and is usually complemented with noise reduction algorithms, which are able to provide additional benefit regarding listening comfort and perceived signal enhancement. Two important types of noise reduction algorithms, implemented in hearing instruments, are based on modulation analysis, meaning that the temporal spectrum of the signal envelope is used for deciding which parts of the signal shall be regarded as noise (and attenuated) and which parts belong to the target signal and shall be kept or even enhanced. These two types of modulation based noise reduction are: transient and stationary noise reduction. high modulation low Speech signal from DirMic interfered by stationary and transient noise Transient noise Speech Stationary noise Sound Smoothing Speech Enhancement Stationary Noise Reduction 0-30dB 0-12dB Clear speech with balanced and comfortable environmental sound Figure 7. Noise reduction scheme with stationary and transient noise reduction but still without spatial noise reduction. Real-world environments not only consist of extremely fast or slow modulated interfering signals. What was missing until now was a method to reduce interfering signals with speech-like modulation properties. Here, the obvious problem arises that modulation analysis cannot be used to distinguish between speech the user wants to listen to, and on the other hand, speech that is regarded as disturbing. The solution for this dilemma is to change the criterion which separates a signal into the target and noise part. Therefore, instead of using modulation properties, the criterion which is used in the new Directional Speech Enhancement is changed to DOA (direction of arrival) based one:

9 Micon Directivity and Directional Speech Enhancement 8 Signals from frontal directions are considered as desired target signals, whereas signals not arriving from front are per definition regarded as interfering signals. The required information about the direction of incidence of the signals can be obtained directly from the beamformer. That is, using a cardioid and an anti-cardioid characteristic ( anticardioid is a reversed cardioid response which ideally completely cancels out signals from the frontal direction). Thus, the beamforming stage has become an integrated part of the noise reduction system. high modulation low Speech signal after directional processing, still interfered by stationary, m odulated and transient noise Transient noise Sound Smoothing Interfering Speech Speech from desired direction Modulated noise Stationary Noise Stationary Reduction noise 0-30dB 0-12dB Direction dep.noise reduction Direction dep.noise reduction Clear speech from front direction w ith balanced and com fortable environm ental sound Eghart Fischer, RDSP, 04/2008 Figure 8. Noise reduction scheme with stationary, transient, and spatial (direction dependent) noise reduction. Figure 8 shows the complete noise reduction system. Note that this processing occurs following the directional microphone algorithm. A wide array of disturbing noises can now be addressed while keeping the frontal target speech signal clear and understandable. This unique approach provides two interesting aspects: Simultaneous attenuation of the complete back hemisphere The effect of beamforming at the very front of the signal path is enhanced: Whereas differential beamformers can provide only one notch (direction of minimal sensitivity, e.g. 180 for cardioid characteristic, 120 for supercar dioid), the directional noise reduction can simultaneously attenuate signals from any direction, as long as it is not frontal. The effects of beamforming and spatial noise reduction therefore ideally complement each other. Individual optimization opportunity With this scheme, a question can arise for one class of signals: What about stationary signals from frontal directions? For the modulation-based criterion, stationary noise from a frontal direction is considered noise because it is relatively stationary; for the direction based criterion, it is desired signal because it is from the front.

10 Micon Directivity and Directional Speech Enhancement 9 Here micon offers the possibility to select between presets that were optimized for different rationales: o Keeping a certain amount of stationary frontal noise as wanted signal and attenuating it less will provide a clear and natural sound impression for frontal signals. o Including frontal stationary noise fully into the class of unwanted signals will provide a stronger overall attenuation of the complete environmental noise including the stationary part, and thus result in a smoother, more comfort oriented sound impression. Based on these two options, the Connexx med and max setting of the Directional Speech Enhancement feature were optimized. The med setting preferably preserves part of the stationary frontal sources, whereas max is more comfort-oriented and reduces frontal stationary noise to a larger extent. Target speech from the frontal direction is kept bright and clear for each of the settings. Combining Spatial and Stationary Noise Reduction: How to keep the advantages of both Most applications of noise reduction, including the described spatial and stationary approaches use a filter rule based on the so-called Wiener filter principle, which applies a frequency dependent attenuation to the input signal, according to an estimated, frequency dependent signal-to-noise ratio (SNR). When applying this formula, the typical ''Musical Tones'' problem can occur, i.e., the residual noise after applying the noise reduction can have a fluctuating characteristic. This is due to the fact that the estimation of the SNR obviously can never be perfect. The problem becomes the more serious the more the intended attenuation for interferers is increased. Especially for the suppression of non-stationary noises by the spatial noise reduction, high attenuation has to be allowed to achieve a clearly perceivable benefit. Stationary noise reduction normally allows up to 6-12dB attenuation, whereas for spatial noise reduction, it can be necessary to temporarily apply more than 20dB attenuation in the respective frequency channels e.g. for effective suppression of loud interfering speech from the back. The problem of integrating stationary and directional noise reduction is now to keep the 6-12 db limitation for stationary noise in order to achieve a pleasant and stable sound impression for stationary parts of the signal and simultaneously extend the allowed range of attenuation for non-stationary noises to 20dB and more. The goal is to achieve a clearly beneficial overall noise reduction effect, and still preserve an excellent target signal quality. There are several integration possibilities, e.g., a combination of the noise estimates, or a combination of separately calculated noise reduction gains. Based on intensive investigations and user preferences, we found the best performance with the combination of the Wiener filter noise reduction gains shown in Figure 9. This approach is patented [4].

11 Micon Directivity and Directional Speech Enhancement 10 Stationary noise reduction Limiter g1 Stationarity detection g3 Switch / fader g4 Spatial noise reduction g2 Figure 9. Integration of stationary and spatial noise reduction The basic idea of the procedure is to detect the degree of stationarity of the input signal independently in each frequency channel and fade between the appropriate noise reduction procedures. The core component for this is the block ''stationarity detection'' as seen in Figure 9. Based on the result of this analysis, either the stationary or the spatial noise reduction is applied. To obtain a smooth sound impression, the fading occurs in just a few milliseconds. The combination allows for the selection of the appropriate noise attenuation. When stationary signals are detected, the stationary noise reduction is applied which ensures a pleasant residual noise without any ''Musical Tones''. During the activity of non-stationary signal components, however, the spatial noise reduction gain is used which guarantees strong attenuation of spatial interferences. The control block, i.e., the stationarity detection itself can be realized using the stationary noise reduction gain. Whenever this value is close to the noise floor, this is a strong indication that only a non-stationary signal is present. This evaluation is performed independently in each frequency channel.

12 Micon Directivity and Directional Speech Enhancement 11 III. Study Results Research using these new algorithms included objective results based on Speech Reception Threshold' (SRT) measurements with nine normal hearing people. Roughly described, the SRT is the ratio of the target and interference signal levels for which 50 % speech intelligibility is obtained. Using this method, the lower the SRT, the better is the system performance. The measurement set-up consisted of one target speech signal at 0 and three speech mixture interferers at 120, 180, and 240. In Figure 10, the SRT is compared for a state-of-the-art differential adaptive beamformer in four frequency channels, and the micon Multi-channel Directional Microphone System in 48 frequency channels using Directional Speech Enhancement. Omnidirectional settings SRT [db] better speech understanding Beamformer settings BF in 4 bands BF in 48 bands with SpNR micon Figure 10. SRT measurements for a differential beamformer in 4 channels (left) compared to the micon Multi-channel Directional Microphone System with Directional Speech Enhancement (right). The boxplots indicate the mean (red), 25 / 75 % percentiles (blue box), and min/max ratings (black). The upper part of Figure 10 shows the boxplots for an omni-directional setting without beamforming (BF) or noise reduction (NR), whereas the lower plots show the results with the respective enhancement procedures. The micon 48-channel solution shows an improvement of about 2 db SRT in comparison to a realization with 4 channels.

13 Micon Directivity and Directional Speech Enhancement 12 Figure 11. Subjective noise reduction (left side) and overall preference rating (right side) with respect to Directional Speech Enhancement in min /med / max seeting and with stationary noise reduction (no DSE) as reference. Boxplots: max/min 75%/25% quartiles median. Figure 11 shows the subjective preference obtained by MUSHRA (Multiple Stimuli with Hidden Reference and Anchor) [5] listening tests with ten hearing impaired persons. The setup consisted of a target speaker from 0 and one speech i nterferer from 180. The subjective noise reduction (left side) and the subjective overall preference (right side) have been analyzed for a stationary noise reduction approach (no DSE) and the spatial noise reduction (Directional Speech Enhancement) for three different settings (DSE min, med, and max) using a beamformer system without any noise reduction as MUSHRA ''anchor'' (corresponding to the 0 line in the figure). One can observe the benefit of Directional Speech Enhancement compared to stationary noise reduction alone for both criteria. As intended, the perceived noise reduction increases for stronger settings of Directional Speech Enhancement, while the subjective preference values remain high compared to the stationary noise reduction (no DSE) in each of the three settings. References [1] Kochkin, S. MarkeTrak VIII: Customer satisfaction with hearing instruments is slowly increasing. The Hearing Journal, Vol. 63 (1), January 2010, pp [2] G.W. Elko and A.N. Pong: A simple adaptive first-order differential microphone, Proc IEEE Workshop on Applications of Signal Processing to Audio an10 Acoustics, pp , 1995 [3] Patent by H. Puder & E.Fischer (e.g. US ): Method for reducing interference power in a directional microphone and corresponding acoustical system [4] Patent by E. Fischer: DE Hörvorrichtung und Verfahren zum Reduzieren eines Störgeräuschs für eine Hörvorrichtung [5] ITU-R BS Standard