Acoustic Echo Cancellation (AEC) Instructions¶
AEC is the abbreviation of Acoustic Echo Cancellation. With reference, this algorithm adaptively tracks the transformation of echo path and suppresses the echo signal of speakers arriving at the microphone terminal in real time to improve the recognition effect of target speech, and evaluates whether to turn on AEC according to the actual application scenario. When this function is enabled, voice recognition can also be interrupted when there is a long playing content or when playing audio media resources (such as MP3 songs played by speakers). This document mainly describes how to use this function.
1. AEC algorithm¶
The block diagram of AEC’s application principle is as follows: the reference signal generates broadcast signal B through the loudspeaker, and the voice is the target voice signal A. When playing the voice, signal B and signal A enter the chip after complex mixing in the application environment. The AEC algorithm suppresses signal B according to the reference signal and the mixed signal, improves the signal-to-noise ratio of signal A, and then enters the speech recognition engine to improve the recognition effect.
2. Software configuration method of AEC¶
You can open the ci in the SDK package_ ssp_ The echo cancellation algorithm in the config. c file has the following parameters for debugging:
aec_config_t aec_config =
{
.mic_channel_num = 1, //Number of microphone signal channels
.ref_channel_num = 1, //Number of test signal channels
.aec_control_mode = ENABLE_PLAYING_STATE_MODE, //ENABLE_PLAYING_STATE_MODE:Conduct aec control according to broadcast status; COMPUTE_ REF_ AMPL_ MODE: perform aec control according to the reference amplitude
.aec_gain = 4.0f, //Gain value
.aec_enable_threshold = 3000.0f,//Threshold value of reference signal judgment
.nlp_flag = 2, //The non-linear processing module selects the mode. By default, mode 2 is used. 0: Not used. 1: Mode 1 has greater distortion. 2: Mode 2 has less distortion. 3: Mode 2 is used first, then Mode 1
.aggr_mode = 1, //Default 1
.fft_size = 256, //Frequency number of frequency domain processing
};
Note
Pay attention to dual channel stereo application: modify parameters ref_channel_num 、mic_channel_num
3. Application configuration method of AEC¶
The AEC reference signal (and the signal played by the horn) has the following two sources:
-
1: Refers to the loudspeaker playing of the voice module itself, which is called internal AEC for short**
-
2: Refers to the horn sound playing from other external players. This application is called external AEC for short**
3.1 Precautions for use of internal AEC¶
3.1.1 Hardware circuit description¶
In the single microphone scheme, another unused codec channel (micpr) of the CI110X chip is used as the AEC reference signal input channel.
In the case of low volume, AB power amplifier is recommended, such as SMG4890; Class D power amplifier is recommended for high volume.
AEC feedback signal is standard I2S, 16bits 16K sampling rate, full amplitude voltage 3.3V, then the voltage amplitude range after the reference voltage divider circuit is 100-150mv at the maximum volume.
3.1.2 matters needing attention¶
If the hardware is designed by your company, we can assist you to check the schematic diagram.
In case of stereo dual channel, dual channel codec shall be used to process the stereo after recovery as AEC reference signal
If Class D power amplifier is adopted, filtering circuit shall be added (refer to Class D power amplifier filtering circuit in subsequent external AEC ).
3.2 Precautions for use of external AEC¶
In the external AEC application, if it is a single mic application, the external reference signal can be input from MIC R in the form of analog signal; If it is a dual mic application, the external codec is required to pick up the reference signal. At the same time, the different signal output types (digital and analog) of the power amplifier will lead to inconsistent peripheral hardware lines. The following describes how to apply it according to the classification of the power amplifier.
3.2.1 Power amplifier output analog signal¶
(1) Hardware principle
AEC reference signal is standard I2S, 16bits 16K sampling rate.
The output signal of the power amplifier is analog signal (such as Class AB power amplifier), and the reference signal path uses adc (Shunxin es7243E, single ended input) as the sampling of the reference signal (as shown in the following figure). The full amplitude voltage is 3.3V, and the voltage amplitude range after the reference voltage divider circuit is 100-150mv at the maximum volume.
The voltage divider circuit can refer to the following figure:
Synchronize the player’s playing status to the AEC module:
In sample internal sample_ XXX\src\system_ msg_ deal. Check in c_ current_ Playing is a function for the aec module to obtain the playing state. The external player needs to inform the aec module by GPIO (or changing the playing state variable through the serial port).
The following is the routine of informing the aec module in GPIO mode: (When playing music, the external player pulls up the pin of the corresponding GPIO, and when stopping playing, pulls down the pin of the GPIO):
bool check_current_playing(void)
{
// if(AUDIO_PLAY_STATE_IDLE != get_audio_play_state()) //Inform aec module status during internal playback
if(gpio_get_input_level_single(GPIO2,gpio_pin_7)==1)//When it is high, the external player is playing
{
return true;
}
else
{
return false;
}
}
3.2.2 Power amplifier output digital signal¶
(1) Hardware principle
AEC feedback signal is standard I2S, 16bits 16K sampling rate.
The output signal of the power amplifier is digital signal (such as Class D power amplifier), and the reference signal path uses adc (such as AD51050, Shunxin es7243E, differential input) as the sampling of the reference signal (see the figure below). The full amplitude voltage is 3.3V, and the voltage amplitude range after the reference voltage divider circuit is 100-150mv at the maximum volume.
The voltage divider circuit can refer to the following figure:
If the sound source played by the product (or voice module) is dual channel stereo, it needs to go through dual channel codec before being connected to the power amplifier.
Synchronize the player’s playing status to the AEC module:
In sample internal sample_ XXX\src\system_ msg_ deal. Check in c_ current_ Playing is a function for the aec module to obtain the playing state. The external player needs to inform the aec module by GPIO (or changing the playing state variable through the serial port).
The following is the routine of informing the aec module in GPIO mode: (When playing music, the external player pulls up the pin of the corresponding GPIO, and when stopping playing, pulls down the pin of the GPIO):
bool check_current_playing(void)
{
// if(AUDIO_PLAY_STATE_IDLE != get_audio_play_state()) //Inform aec module status during internal playback
if(gpio_get_input_level_single(GPIO2,gpio_pin_7)==1)//When it is high, the external player is playing
{
return true;
}
else
{
return false;
}
}
4. Precautions and influencing factors of AEC effect¶
4.1 Precautions for reference signal¶
Avoid reference analog signal saturation
Debugging suggestion：The signal in front of the voltage divider circuit is distorted, and the volume of, or should be adjusted or replaced with a low impedance horn.
The reference analog signal of AEC is a small analog signal, which is easy to be interfered. Special attention should be paid to the cleanliness of the reference signal during layout. When designing, the reference analog signal should be as far away from the high-frequency signal as possible. If there is a problem in the circuit design, bad signals, such as pulses, clutter, and aliasing, are usually introduced into the collected signal or microphone signal, which will lead to data loss and affect the effect.
Debugging suggestion：In this case, the hardware circuit design should be modified first to ensure that the circuit signal is clean before the acoustic test. The interference of the reference signal has a significant impact on the AEC effect.
The reference signal collected from the back end of the power amplifier is the closest to the real sound of the horn. When the power amplifier has no delay, the front-end signal of the power amplifier can be used as a reference. When the power amplifier has delay (especially some power amplifiers with eq), it is recommended to take the reference signal from the back-end of the power amplifier. If the signal is introduced from the back end of the PA, pay attention that the output range of the power amplifier should not exceed the ADC sampling range.
4.2 Precautions for mic signal¶
Avoid intercepting the collected mic voice signal
Debugging suggestion： play the full sweep signal at the maximum volume of the loudspeaker, the loudspeaker shall not have broken sound, resonance, etc., and the mic acquisition signal and reference signal shall not have clipping. Excessive loudspeaker will lead to more serious distortion. In practical application, it is necessary to select an appropriate volume range (loudspeaker is not distorted).
4.3 Precautions for loudspeaker¶
The reduction of harmonic distortion is due to the fact that the system is not completely linear. In the whole audio channel, the influencing factors of signal harmonic distortion include: distortion curve of single loudspeaker, structural design of loudspeaker rear cavity, loudspeaker front mesh, microphone receiver hole structure, etc; All of these will increase the distortion of the microphone signal. The greater the distortion of microphone signal, the worse the similarity between microphone signal and recovery signal, and the worse the performance of echo cancellation. The frequency response of the horn has poor consistency at 200~4khz, and the structure has resonance, which will affect the AEC effect.
Generally, speakers have a certain degree of distortion. When the volume is close to the speaker’s playback limit, the distortion will be increased. Try to make the speakers work in a small range of distortion when they are used. Different speakers will have different fixed distortion interference.
The speakers with low distortion should also be selected. Refer to Figure 4-3-1. The following figure shows the sound played by the four speakers. As can be seen in the figure, some speakers have large distortion in different frequency bands, such as the last three in the figure below. The speech distortion in these different frequency bands will seriously affect the AEC effect, so try not to use such speakers. The first horn in Figure 5-5 has good frequency response.
When using the loudspeaker, avoid the top clipping of the microphone signal caused by excessive loudspeaker sound.
Debugging suggestion： Try not to use micro speakers, ultra-thin speakers and other speakers with high resonance frequency, because such single units have large low-frequency distortion; If the rear cavity structure of the loudspeaker allows, the design of passive radiator or guide tube can increase the low frequency and reduce the nonlinear vibration of the loudspeaker.
4.4 Environmental precautions¶
Reduce mechanical noise In household appliances, robots and other products and equipment, the variable introduced by fan noise and motor noise will cause the bottom noise of microphone signal to be particularly large, and the recognition effect will be greatly affected.
Debugging suggestion： In this kind of equipment, special attention should be paid to the distance between the internal fan, motor and microphone. If the structure space allows, the microphone can be sealed separately to prevent internal sound transmission.
4.5 Position relationship between microphone and loudspeaker¶
- Debugging suggestions: the position relationship between the microphone and the speaker. In principle, if the structure space allows, the microphone should be as far away from the speaker as possible. The microphone should not exceed the horizontal plane where the speaker faces. The closer the microphone is to the speaker, the lower the signal-to-noise ratio, the greater the possibility of microphone distortion. Considering the structure size, it is recommended that the microphone be about 10cm away from the speaker, The position relationship between microphone and loudspeaker has little influence on AEC processing effect.
- 注意扬声器播放口背离麦克风,避免朝向麦克风.
5.Method of debugging effect¶
Figure 5-1 shows the time domain effect of sound acquisition of the recording board. It can be seen from the figure that the original left channel data (the upper half figure) is a mixture of the target voice and the speaker playing voice. After the aec algorithm processing, the speaker playing voice is suppressed and the signal-to-noise ratio of the target voice is improved, as shown in the right channel data (the lower half figure) in Figure 5-1.
The following figure 5-2 shows the frequency domain display of the same audio, and the result is normal.
Set the IIS output parameter to audio_ pre_ rslt_ write_ When data ((int16_t *) ref, (int16_t *) micl) is used, the reference signal data and microphone left channel voice acquisition data are output, as shown in Figure 5-3. It can be seen in the figure that the reference signal has data when there is play, and the reference signal is very small and close to 0 when there is no play.
Note
Try to ensure that the reference signal and microphone signal are not truncated or distorted.
6.Abnormal debugging steps¶
Note
In the process of debugging exceptions, record and save the audio, which is conducive to rapid analysis by other colleagues.
When the application of aec does not meet the expectation, it is recommended to take standard sdk and hardware as reference, and analyze the specific reasons through reference sub module of standard accessories.
-
Step 1: Determine whether the sdk configuration is abnormal;
-
step2 2: Determine the cause of abnormality by collecting the reference signal (ref), output signal (dst) and microphone signal (mic) from the acoustic panel; At present, many abnormalities occur in the reference signal. Examples of existing problems are: The problems with abnormal reference signals include:
1: The reference channel has no data, so it is necessary to check the hardware channel and whether ref is a left channel input or a right channel input. The default is the left channel
2: The reference signal interference is relatively large, so check the hardware path
3: The amplitude of the reference signal deviates from the standard value greatly (too large or too small), so it is necessary to check the voltage divider circuit and alc gain value, taking the standard board as a reference
4: In case of saturation during voltage division of reference signal, it is necessary to check the voltage division circuit and alc gain value, and AEC abnormal debugging cases
Abnormal microphone signal:
1: In the process of aec processing, alc is not closed, so check the playback status and alc switching status
2: If the microphone signal is truncated, check the playback status, alc switching status and alc gain value
3: The microphone signal sensitivity is different, and the microphone needs to be replaced to confirm whether the microphone is abnormal
4: The amplitude of the microphone is too small, and the aec does not work. The threshold value in the aec processing needs to be adjusted and changed
-
Step 3: Replace different modules to analyze whether it is the difference of peripheral hardware devices that causes the abnormal peripheral devices: microphone, speaker, voltage divider, power amplifier, codec, etc;
-
Step 4: Check whether the aec effect is reduced due to environmental factors. If the reverberation is high, the aec effect will be reduced;
-
Step 5: If the above methods cannot effectively solve the problem, the algorithm colleagues will support it.
Note that the playback port of the speaker faces away from the microphone and avoids facing the microphone.
5. The method of debugging effect¶¶
Figure 5-1 is the time-domain effect of the recording board sound collection. In this figure, you can see that the original left channel data (the upper half of the figure) is a mixture of the target voice and the sound played by the speaker. After being processed by the AEC algorithm, the sound played by the speaker is suppressed. Sound, improve the signal-to-noise ratio of the target speech, as shown in the right channel data (bottom half of the figure) in Figure 5-1.
Figure 5-2 below shows the frequency domain display of the same audio, and the result is a normal result.
Figure 5-2 Frequency domain display of audio
When the IIS output parameter is set to audio_pre_rslt_write_data((int16_t*)ref, (int16_t*)micl ), the reference signal data and the microphone left channel sound sampling data will be output, as shown in Figure 5-3. In the figure, it can be seen that when there is playback The reference signal has data, and when there is no playback, the reference signal is very small and close to 0.
Figure 5-3 Reference signal and microphone signal waveforms
Figure 5-4 Reference signal and microphone signal frequency domain
Notice
Try to ensure that the reference signal and the microphone signal are not clipped and distorted, and cannot be too small.
6. Abnormal debugging steps¶¶
Notice
In the process of debugging abnormalities, record and save the audio, which is helpful for other colleagues to quickly analyze.
When the aec application does not meet expectations, it is recommended to use the standard sdk and hardware as a reference, and analyze the specific reasons through the standard accessories reference sub-module.
- Step1: Determine whether there is an exception in the sdk configuration;
- Step2: Collect the reference signal (ref), output signal (dst) and microphone signal (mic) to determine the cause of the abnormality through the sound board; currently, most abnormalities occur in the reference signal. Examples of existing problems include:
- The problems of abnormal reference signal are:
- 1: There is no data in the reference channel, you need to check the hardware path, check whether the ref is the left channel input or the right channel input, the default is the left channel;
- 2: The interference of the reference signal is relatively large, and the hardware path needs to be checked;
- 3: The amplitude of the reference signal deviates greatly from the standard value (too large or too small), you need to check the voltage divider circuit and ALC gain value, and use the standard board as a reference;
- 4: Saturation occurs during the voltage division process of the reference signal. It is necessary to check the voltage division circuit and the ALC gain value, AEC abnormal debugging case
- Abnormal microphone signal problems include:
- 1: In the process of aec processing, alc is not closed, you need to check the playback status and alc switching status;
- 2: The microphone signal is clipped, you need to check the playback status, ALC switching status, and ALC gain value;
- 3: The sensitivity of the microphone signal is different, and the microphone needs to be replaced to confirm whether the microphone is abnormal;
- 4: The amplitude of the microphone is too small, the aec does not work, the threshold value in the aec processing needs to be adjusted, and the alc gain needs to be changed.
- Step3: Replace different modules to analyze whether the difference in peripheral hardware equipment causes abnormal peripheral equipment: microphone, speaker, voltage divider circuit, power amplifier, codec, etc.;
- Step4: Check whether the environmental factors lead to the decline of the aec effect, and the aec effect will decline in the environment with large reverberation;
- step5: If the above methods cannot effectively solve the problem, please support the algorithm colleagues.