Automatic identification test¶
Under normal circumstances, the speech recognition test takes a long time, and there are often multiple tests during product development. For example, the traditional manual test method takes a long time and is inefficient, which will slow down the product development to mass production. Therefore, our company has developed a set of automatic identification testing methods, which can effectively reduce the use of personnel, improve the testing efficiency and accelerate the product testing process by replacing manual testing with computers. The details are as follows.
Equipment and instruments required for testing¶
To conduct automatic identification test, the following equipment needs to be prepared first (some equipment models are optional, and users can replace equipment with the same specifications and functions):
| S/N | Category | Equipment | Equipment Model | Equipment Brand | Function | Picture |
|---|---|---|---|---|---|---|
| 01 | Computer | Desktop/laptop | Unlimited | Unlimited | Monitor whether the voice module feedback accurately outputs the test results |  |
| 02 | Sound source | Artificial mouth | 4227-A | Br ü el&Kj ｜ r | Play audio signal |  |
| 03 | Noise monitoring | Precision noise meter (sound level meter) | 1357 | TES | Test the sound pressure reaching the microphone |  |
| 04 | Noise source | Speaker/TV | Recommended model of monitor speaker: FX8 | Fluid Audio | Play noise and simulate external interference |  |
| 05 | Audio Collection | High Fidelity Recording Equipment | R44 | Loran/Roland | Audio Recording |  |
Table 1 Speech Recognition Test Equipment
Layout of test equipment and instruments¶
Artificial mouth
Model: 4227-A
Performance index:
-
Rated output sound pressure SPL:
-
200Hz - 2kHz ----- 110dB
- 100Hz - 8kHz ----- 100dB *Distortion (@ 94dB):
- 200Hz - 250Hz ----- <2%
- ’>‘250Hz ----- <1%
- Impedance ------ 4 Ω
- Maximum bearing ----- 10W
- Instantaneous withstand power ----- 50W
- Nozzle diameter ----- 20mm
The artificial mouth is used to play the sound source, which is located in front of the microphone of the voice module, 4-5m horizontally and straightly. The sound source is 1-1.5m from the ground (normal height of simulated human voice), as shown in Figure 1.
Noise source (monitor speaker)
Model: Fluid Audio FX8
Performance index:
- Frequency response: 35Hz - 22kHz (± 3dB);
- Cross frequency: 2.4kHz;
- Low frequency amplifier power: 80 watts;
- High frequency amplifier power: 50 watts;
- Signal noise:>100dB (typical A-weighted);
- Polarity: positive signal+input generates an outward low-frequency displacement;
- Input impedance: 20 kOhm (balanced type), 10 kOhm (unbalanced type);
- Input sensitivity: when the volume control is set to the maximum value (102dB of maximum sound pressure), the input of 85 mV pink noise will produce an output sound pressure of 95dBA;
- Power supply: 115V~50/60 Hz or 230V~50/60 Hz (user can switch);
- Protection device: RF interference, output current limitation, over temperature protection, transient on/off;
- Protection, subwoofer filter, external power fuse;
- Box: medium density fiberboard with ethylene base;
- Size (single monitor speaker): 340mm (height) x254mm (width) x270mm (length);
- Weight (single monitor speaker): 9.8kg.
If there is no monitor speaker, TV can also be used instead.
The voice module and the precision noise meter are located at the same plane (1m from the ground). The distance from the microphone of the voice module is more than or equal to 1.5m, but not more than 2m, as shown in Figure 2.
Precision noise meter
Model: TES 1357
Performance index:
- 0.1dB resolution;
- The measuring range is 30 to 130dB;
- 1/1,⅓,⅙,1/12,1/24 octave spectrum analysis software (optional);
- Accuracy± 1.5dB (ref 94dB @ 1KHz);
- Weighted measuring range: 30dB to 130dB;
- C-weighted measuring range: 35dB~130dB;
- Measuring gear 30-80dB, 50-100dB, 60-110dB, 80-130dB;
- Frequency response 31.5 Hz to 8KHz;
- Digital display 4-digit LCD, 0.1dB resolution, updated every 0.5s;
- AC/DC signal output 2Vrms/full scale of each gear, 10mV/dB.
The precision noise meter shall be as close as possible to the microphone of the voice module (the distance between them shall be ≤ 5cm), but shall not contact the microphone of the voice module, as shown in Figure 2.
Computer and module to be tested
The computer and the module to be tested are connected by USB to serial port tool, and the computer can receive the serial port information from the module. The location relationship between computer, module, precision noise meter (decibel meter) and noise source is shown in Figure 2.
After arranging various instruments and equipment and modules to be tested, we can use the automated testing tools provided by our company to test.
Description of automated testing tools¶
Our company has independently developed computer-based automated testing tools, which users can ☞contact us to obtain and use. The explanation for this tool is as follows
Principles of automated testing tools¶
The principle of this tool is shown in the figure above. Two computers that can be connected to the network in the same LAN are used. One is a noise playing computer, and the other runs an automated test tool. The noise player computer stores the required noise set and can be connected with the automated test computer through the local area network. The automated test computer needs to be configured with a USB serial port connection tool, and the USB serial port is used to connect the voice module to be tested.
When using the tool, you need to configure the audio file, firmware and test parameters. After running the automated test tool, the tool reads the configuration file, and gradually confirms whether the serial port information and test audio are normal according to the order of the configured items. The tool will automatically upgrade the firmware according to the test parameters. At the same time, it will send instructions to the noise player computer to play the noise according to the configured test parameters, and start the test process. After the automatic test, the identification results will be automatically counted to generate documents, and finally the e-mail will automatically send the identification results to the specified mailbox.
Directory structure of tools¶
The tool is a Windows based file package. The main directory contains 7 folders and an executable file. The folders are as follows:
- /audio_ Files: test broadcast audio storage directory
- /param_ Save: parameter save directory */playnoise: broadcast noise tool
- /preproccess_ Audio: directory of audio preprocessing tools */result: storage directory of intermediate and final results
- /result_ merge_ Tool: Other summary tools
- /test_ Fw: Test firmware storage directory
The executable file is the tool execution program. Double click it to start running.
Tool interface¶
The interface of this tool is shown in the figure above. It has the following functions, which are described as follows:
- Command word play interval text box: used to fill in the test audio play interval, which can be manually set according to the situation;
- Wake up words play interval text box: available after checking off the online check box, used to fill in the wake-up words audio play interval, which can be manually set according to the situation;
- Email check box: used to determine whether to enable the function of automatically sending result emails after test completion. If checked, it is enabled;
- Off line check box: used to enable one wake-up one recognition mode;
- Automatic upgrade check box: used to determine whether the automatic upgrade test firmware function is enabled. If checked, it is enabled;
- Result saving path text box: fill in the storage directory of detailed test results after the completion of automated testing, and users can choose an effective saving path for the results;
- Audio preprocessing button: used to filter the audio used for testing during automatic testing in the prepared test audio library and put it into the test broadcast audio storage directory;
- Parameter configuration button: used to manually set test parameters, including test serial port, baud rate and other information;
- Result processing tool button: used to open the statistical tool of test results and other functions;
- Audio file path button: used to set the storage directory of test broadcast audio;
- Only broadcast button: used to select only broadcast audio_ Test audio in files directory;
- Start Test Button: used to start automatic test.
Test broadcast audio selection and storage¶
Test audio selection
The test audio recorded by our company is placed under the specified path of the test computer. When the test is required, prepare the command word file to be tested, and filter the audio through the preprocessing audio tool. The specific operations are as follows.
Open the tool and click “Audio Preprocessing” on the tool interface. The following interface pops up
The interface of this tool is shown in the figure above. It has the following functions, which are described as follows:
- Audio source path: used to fill in the specified path for recording test audio to be placed on the test computer, which can be manually set according to the situation;
- Audio target path: it is used to fill in the path of audio storage filtered out after the preprocessing audio tool is completed. It can be manually set according to the situation;
- Command word file path: used to specify the path of the command word file to be tested, which can be manually set according to the situation;
- Single command words radio box: used to select the command words;
- Select among people: used to confirm the screening criteria and screen the audio according to the tester;
- Selected words: used to confirm the filtering criteria and filter the audio according to the test command words.
- Get audio files
Select the corresponding file path and the number of audio required by a single command word. According to the actual needs, select and click “Select among people” or “Select among words”, wait for a few minutes, and then you can obtain the corresponding test audio under the audio target path D: test< br/> Command word file format and encoding format Command word file is in text format, as follows:
Note: The file coding format is utf-8 without BOM format.
- Audio preprocessing
In most cases of audio standardization, the volume of each recorded test audio is inconsistent, so the audio must be standardized before the test. The audio before and after standardization is as follows< br/>
To speed up the test and reduce unnecessary time waste, mute before and after the audio is removed. Before the test broadcast, mute before and after the single audio. The audio before and after processing is shown below.
Test audio storage requirements
The user can store the prepared audio in the test broadcast audio storage directory audio_ Under files, there are two ways to store test broadcast audio in this directory:
- Put the audio of different speakers with the same audio content in one audio directory;
- Put different audio content of the same speaker in one audio directory.
Test audio format requirements
The audio file format is *. wav. The audio file is named as follows:
[0001] Xiaowei Xiaowei. wav
[0002] Heat for two minutes. wav
[0187]ALL-OPEN.wav
[0188]HELLO-JENNY.wav
Note: [0001] represents that the serial number can be sorted by itself, and “Xiaowei Xiaowei” is the command word wake-up word (it must be the same as the expected output result, because the internal identification is correct through string matching).
Operation steps of automatic identification test¶
After the above instruments and equipment are ready, and the two computer networks are confirmed to be unblocked and the automated test tools are available in the automated test computer, the test can be started. The steps are as follows:
Firmware production¶
When the firmware is created, the user_ Configure the following macro in config. h:
#define DEBUG_ PRINTF_ FUNC_ EN (0)
The sending result of module board serial port 2 must be in the following form: send result: Xiaowei Xiaowei 0.343994.
Note: To avoid test inconsistency caused by hardware differences, it is recommended to use two module boards for each firmware during testing.
Hardware connection and power supply¶
Automatic test computer
It is used to connect test modules, run automatic test tools, and display test results.
Noise playing computer
It is used to connect the automatic test computer and play the test noise environment.
Serial port for receiving identification results
It is used to test the identification result of the computer receiving test module, and test the TX2 of the computer receiving serial port RX connection test module.
Firmware upgrade serial port
It is used to test the module upgrade firmware, test the computer firmware upgrade serial port RX to connect the TX1 of the test module, and TX to connect the RX1 of the test module.
Sending broadcast noise command mode
It is used to automatically play the test noise environment. During the automatic noise environment test, the test computer can send instructions to the noise broadcast computer to broadcast the specified noise. There are serial port and socket (network) modes for sending broadcast instructions, which can be distinguished by different configurations in the configuration file.
Power supply
It is used to test the 5V power supply of the module board. If HUB is used to supply power to multiple test boards, HUB needs to use stable DC power supply, otherwise the identification result will be poor due to insufficient or unstable power supply.
Fill in configuration file¶
Parameter configuration file and save directory
The automatic test tool can use the parameter configuration file schema Xls to set parameters, saving manual configuration time. The file is stored in the parameter save directory param_ Under save, the specific configuration mode is shown in Figure 6:
- Test round: used for multiple tests (one round must be left blank after the last round, as shown in the fourth round of Figure 13);
- Audio file directory name: used to fill in the audio file name and audio_ Corresponding audio file names in the files directory;
- Upgrade/test serial port: used to fill in the serial port number of the test module connection;
- Baud Rate: used to fill in the serial baud rate;
- Test Object Name: used to fill in the name of the test object, which can be manually set according to the situation;
- Upgrade enable serial port: used to fill in the upgrade enable serial port number of the test module;
- Firmware Name: used to fill in the firmware name (corresponding to the firmware name in the atest_fw directory);
- Noise: used to fill in the name of noise audio (corresponding to the name of noise audio in the folder of noise tool playnoise by_socket noise in the computer);
- Email sender: used to fill in the email number. The email address filled in will receive the test results after the test is completed, which can be manually set or not filled according to the situation.
Note: The maximum number of firmware to be tested is determined by the line number of “Round 1” and “Round 2” in the first column of the configuration file, so the two merge areas in this column cannot be empty.
Broadcast noise IP configuration of computer
- If the serial port is used to send the broadcast noise command, the directory param_ Play in the save directory_ noise_ Type. txt is configured as shown in Figure 14 below.
- If the socket is used to send the broadcast noise command, the two computers must be in the same LAN. Then add the directory param_ Play in the save directory_ noise_ Type. txt is configured as shown in Figure 15 below.
Prepare audio and test firmware¶
Store the prepared audio in audio_ The files directory should correspond to the configuration file, as shown in Figure 16 below.
The test firmware is stored in the directory test_ The fw directory should correspond to the configuration file, as shown in Figure 16 below.
Test software setup and related preparations¶
Double click to open the executable file under the working directory, and select the saving path according to your needs. The path cannot be. . The audio broadcast interval is 2S by default, which can be manually set according to the situation. Copy the noise playing tool (directory playnoise) to the noise playing computer, and start it before the normal operation of the test software.
Start testing¶
When ready, click the “Start Test” button. If the email check box is checked, the system will automatically send the results to the mailbox of the relevant personnel through email after each item is tested.
Test results¶
The test results after the test are completed will be stored in the result directory of the main directory, as shown in Figure 17.
Taking the test of six items and broadcast by five people as an example, the test result summary.xls file is the average recognition rate of each item after the completion of five test voice broadcasts, as shown in Figure 18.
The identification details of each project are shown in CI1102_ 1 as an example, as shown in Figure 19.
- . Txt is the detailed identification of each command word for each person, as shown in Figure 20. When the test tool automatically or forcibly exits during testing, the file remains.
- . Xls refers to the detailed recognition and final average recognition rate of each command word for each person. It can be generated only after a person’s test voice broadcast is completed, as shown in Figure 21.
