Instructions for Offline Command Word Self-learning¶¶

1 Content overview¶¶

This document mainly explains the offline command word self-learning project in the single-mic offline SDK of the third-generation CI130X chip (CI130X_SDK_Offline_V1.5.7 and above). The content includes the concepts, scenarios, usage instructions and precautions of offline command word self-learning.

2 Basic concepts of offline command word self-learning¶¶

In the offline state, the user changes the default command words through voice dialogue; this change method meets the personalized customization needs of end users and can bring better user experience to customers.

3 Applicable scenarios for offline command word self-learning¶¶

This solution is applicable to various voice control-related smart home appliances, home furnishing and other products, enabling users to customize command words offline.

4 Offline command word self-learning project description¶¶

In CI130X_SDK_Offline, a demo of command word self-learning is added, and the project name is projects\cwsl_sample. After compiling and downloading, it can be used.

5 Instructions for Offline Command Word Self-learning¶¶

The underlying code of offline command word self-learning is under CI130X_SDK\components\ci_cwsl, and the application layer code is under CI130X_SDK\projects\cwsl_sample\src. See cwsl_app_sample1.c and cwsl_app_sample1.h. The relevant functions have detailed notes, and this article lists some important ones. function.

/* 
 * @brief Command word self-learning message processing function
 * @param asr_msg ASR identification result message
 * @param cmd_handle command handle
 * @param cmd_id command ID
 * @retval 1 The data is valid, and the message has been processed
 * @retval 0 Invalid data, message not processed
 */
uint32_t cwsl_app_process_asr_msg(sys_msg_asr_data_t *asr_msg, cmd_handle_t *cmd_handle, uint16_t cmd_id)
/* 
 * @brief Module initialization event response, command word self-learning parameter initial function, which will be called initially
 * @param cwsl_init_parameter Parameter pointer
 * @retval 1 The number of templates that can be learned must be returned
 */
int on_cwsl_init(cwsl_init_parameter_t *cwsl_init_parameter)
/* 
 * @brief The learning starts the event response, and the function is called at the beginning of each learning
 * @param cmd_id command word ID
 * @param group_id tearm ID
 * @param word_type Types of learning,CMD_WORD command word ；WAKEUP_WORD wakeup word
 * @retval 0
 */
int on_cwsl_reg_start(uint32_t cmd_id, uint16_t group_id, cwsl_word_type_t word_type)
/* 
 * @brief Learn to stop event response
 * @param no
 * @retval 0
 */
int on_cwsl_reg_abort()
/* 
 * @brief 录制开始事件响应
 * @param 无
 * @retval 0
 */
int on_cwsl_record_start()
/* 
 * @brief 录制结束事件响应
 * @param times 当前学习的次数
 * @param result 当前学习结果的状态：学习成功,学习失败,...
 * @retval 0
 */
int on_cwsl_record_end(int times, cwsl_reg_result_t result)
/* 
 * @brief 删除模板成功事件响应
 * @retval 0
 */
int on_cwsl_delete_successed()
/* 
 * @brief 识别成功事件响应
 * @param cmd_id 命令词ID 
 * @param distance 参数未使用,当前无效
 * @retval 0
 */
int on_cwsl_rgz_successed(uint16_t cmd_id, uint32_t distance)
/* 
 * @brief 开始启动学习
 * @param word_type 学习的命令词类型
 * @retval 0
 */
void cwsl_app_reg_word(cwsl_word_type_t word_type)
/* 
 * @brief 删除指定类型模板
 * @param cmd_id 指定要删除的命令词ID, 传入-1为通配符,忽略此项
 * @param group_id 指定要删除的命令词分组号, 传入-1为通配符,忽略此项
 * @param word_type 指定要删除的命令词类型,传入传入-1为通配符,忽略此项
 * @retval 无
 */
void cwsl_app_delete_word(uint32_t cmd_id, uint16_t group_id, cwsl_word_type_t word_type)

Under CI130X_SDK\projects\cwsl_sample\src, open the user_config.h file.

#define USE_CWSL                   1 //1打开离线命令词自学习,0：关闭离线命令词自学习

illustrate

The cwsl_sample project enables offline command word self-learning by default.

Under CI130X_SDK\system, open the sdk_default_config.h file.

#define CICWSL_TOTAL_TEMPLATE      6 //离线命令词自学习可存储模板数量,默认6个

illustrate

Currently, the maximum number of test templates is 20.

5.2 Offline command word self-learning wake-up word and command word number configuration¶¶

Under CI130X_SDK\projects\cwsl_sample\src, open the cwsl_app_sample1.c file.

#define CWSL_WAKEUP_NUMBER         2 // 可学习的唤醒词数量
#define WAKE_UP_ID                 1 // 学习的唤醒词对应的命令词ID
#define CWSL_CMD_NUMBER            ((sizeof(reg_cmd_list) / sizeof(reg_cmd_list[0]))) // 可学习的命令词数量

5.3 Offline command word self-learning command word learning broadcast tone configuration¶¶

Configure reg_cmd_list structure array

Under CI130X_SDK\projects\cwsl_sample\src, open the cwsl_app_sample1.c file.

//左边一列表示的是要学习的命令词ID,右边一列表示需要播报的ID
const cwsl_reg_asr_struct_t reg_cmd_list[]=
{ //命令词ID       //学习提示播报音ID
    {2,             1001},
    {3,             1002},
    {4,             1003},
    {5,             1004},
    {6,             1005},
    {7,             1006},        
};

Configure command word list

Under CI130X_SDK\projects\cwsl_sample\firmware\user_file\cmd_info, open the [60000]{xxxx}cmd_info.xls file.

illustrate

The purpose of the column of command words in the table below is to indicate which word the article is broadcasting.

command word	command word ID	command word semantic ID	Confidence	wake word	compound words	expect word	unexpected word	Announcement tone type	Announcement tone 1ID	Announcement tone 2ID
[1001] Please tell me the learning command to turn on the air conditioner [103] Congratulations [2001] You have successfully learned to turn on the air conditioner	1001	0x00	39	NO	NO	NO	NO	customize	1001	103+2001
[1002] Please tell me the learning command to turn off the air conditioner [103] Congratulations [2002] Successfully learned to turn off the air conditioner	1002	0x00	39	NO	NO	NO	NO	customize	1002	103+2002
[1003] Please tell me the learning command to increase the wind speed [103] Congratulations [2003] You succeeded in learning to increase the wind speed	1003	0x00	39	NO	NO	NO	NO	customize	1003	103+2003
[1004] Please tell me the learning command to reduce the wind speed [103] Congratulations [2004] You succeeded in learning to reduce the wind speed	1004	0x00	39	NO	NO	NO	NO	customize	1004	103+2004
[1005] Please tell me the learning instructions for raising one degree [103] Congratulations [2005] You have successfully studied for raising one degree	1005	0x00	39	NO	NO	NO	NO	customize	1005	103+2005
[1006] Please tell me the instruction for lowering one degree [103] Congratulations [2006] Successfully lowered one degree	1006	0x00	39	NO	NO	NO	NO	customize	1006	103+2006

Increase the corresponding broadcast tone

Add it to the CI130X_SDK\projects\cwsl_sample\firmware\voice\src folder.

5.4 Other parameter configuration¶¶

Under CI130X_SDK\projects\cwsl_sample\src, open the cwsl_app_sample1.c file.

#define CWSL_REG_TIMES       (1)     // Each word needs to be said several times when learning. It can be said one time by default, 1 or 2 times are supported, and the maximum is 2 times;
#define CWSL_TPL_MINWORD      (2)    // The minimum number of words in the self-learning template is 2 words by default, and the minimum number of words in the template can be set to 2 or 3 words;
#define CWSL_WAKEUP_THRESHOLD (37)   // The threshold of wake-up words for learning, the smaller the threshold, the more sensitive. The default value is 37, and the minimum value can be configured to 32;
#define CWSL_CMD_THRESHOLD     (35)  // The threshold value of the command word for learning, the smaller the threshold value, the more sensitive. The default value is 35, and the minimum value can be configured to 30;
#define MAX_LEARN_REPEAT_NUMBER     ( CWSL_REG_TIMES + 2 )  // Total number of repetitions during learning, recommended range CWSL_REG_TIMES + 1、CWSL_REG_TIMES + 2、CWSL_REG_TIMES + 3

5.5 Offline command words self-learning default command words¶¶

The current demo of self-learning command words has a standard process for wake-up words/command words. If the user needs to use the command word self-learning function, the language model (ASR), command word table and announcement sound need to be updated. The specific method is as follows:

The language model (ASR) has added a list of command words:

learn wake word

learn command words

study again

quit studying

i want to delete

remove wake word

delete command word

exit delete

delete all

illustrate

The language model (ASR) currently only supports the Chinese model (except for the low misrecognition model).

Command words added to the command word table:

learn wake word	200	0x00	39	NO	NO	NO	NO	customize	200
learn command words	201	0x00	39	NO	NO	NO	NO	customize	201
study again	202	0x00	41	NO	NO	NO	NO	customize	200	201
quit studying	203	0x00	41	NO	NO	NO	NO	customize	203
i want to delete	204	0x00	41	NO	NO	NO	NO	customize	204
remove wake word	205	0x00	39	NO	NO	NO	NO	customize	205
delete command word	206	0x00	39	NO	NO	NO	NO	customize	206
exit delete	207	0x00	41	NO	NO	NO	NO	customize	207
delete all	208	0x00	41	NO	NO	NO	NO	customize	208

If you modify the command word ID in the command word list under CI130X_SDK\projects\cwsl_sample\firmware\user_file\cmd_info, you also need to modify the enumeration cicwsl_func_index in the CI130X_SDK\projects\cwsl_sample\src\cwsl_app_sample1.h file , the enumeration is as follows:

typedef enum
{
    CWSL_REGISTRATION_WAKE          = 200,      ///< Command words: learn wake-up words
    CWSL_REGISTRATION_CMD           = 201,      ///< Command words: learn command words
    CWSL_REGISTER_AGAIN             = 202,      ///< Command word: relearn
    CWSL_EXIT_REGISTRATION          = 203,      ///< Command word: quit learning
    CWSL_DELETE_FUNC                = 204,      ///< Command word: I want to delete
    CWSL_DELETE_WAKE                = 205,      ///< Command word: delete wake-up word
    CWSL_DELETE_CMD                 = 206,      ///< Command words: deleting command words
    CWSL_EXIT_DELETE                = 207,      ///< Command word: exit delete mode
    CWSL_DELETE_ALL                 = 208,      ///< Command word: delete all

    CWSL_DATA_ENTERY_SUCCESSFUL     = 209,      ///< Announce: Successful learning
    CWSL_DATA_ENTERY_FAILED         = 210,      ///< Announce: learning failure
    CWSL_REGISTRATION_SUCCESSFUL    = 211,      ///< Announce: Successful learning
    CWSL_TEMPLATE_FULL              = 212,      ///< Announce: Learning template exceeds the upper limit
    CWSL_SPEAK_AGAIN                = 1007,     ///< Announcer: Please say it again
    CWSL_TOO_SHORT                  = 1008,     ///< Announcer: The voice length is not enough, please say it again

    CWSL_DELETE_SUCCESSFUL          = 213,      ///< Advertisement: deleted successfully
    CWSL_DELETE_FAILED              = 214,      ///< Advertisement: deletion failed
    CWSL_DELETING                   = 215,      ///< Broadcast: Deleting
    CWSL_DONT_FIND_WORD             = 216,      ///< Advertisement: No deleted words found
    CWSL_REGISTRATION_ALL           = 217,      ///< Broadcast: learning completed
    CWSL_REG_FAILED                 = 218,      ///< Announce: learning failure
}cicwsl_func_index;

Need to increase the broadcast tone:

5.6 Steps for offline command word self-learning¶¶

5.6.1 Learning function¶¶

The learning function can set one learning and two learning, and the function logic is as follows.

Learn once:

Wake-up word learning: When the system is in the wake-up state, the user enters the learning mode by saying the command word “learn wake-up word”. According to the broadcast voice prompts, read a custom wake-up word to complete the study. After the study is completed, the voice will announce “successful study”.

Command word learning: When the system is awake, the user enters the learning mode by saying the command word “Learn command words”. According to the broadcast voice prompt, read a custom command word to complete the learning, and the system will continue to learn the next command word until the learning is completed. If you exit the learning state halfway, the next time you study again, you will continue to study at the position where you exited last time.

Learn twice :

Wake-up word learning: When the system is in the wake-up state, the user enters the learning mode by saying the command word “learn wake-up word”. According to the broadcast voice prompts, read the custom wake-up word for the first time, and read it the second time after the learning is successful, and compare it with the first time. If they are the same, the learning is successful. If it is different, it will prompt learning failure and re-learn the word.

Command word learning: When the system is awake, the user enters the learning mode by saying the command word “Learn command words”. According to the broadcast voice prompts, read the custom command words for the first time, and read the second time after the learning is successful. It will be compared with the first time. If they are the same, the learning is successful. If it is different, it will prompt learning failure and re-learn the word.

5.6.2 Relearn function¶¶

Learn once:

When the last word was learned incorrectly or was mistakenly learned by the disturbing voices around, you can read the command word “re-learn”, and the system will broadcast a voice prompt to re-learn the last word.

Learn twice:

When the second pass of learning is different from the first pass, the system will automatically relearn the word without voice control.

5.6.3 Exit learning function¶¶

After entering the learning mode, the user can read the command word “exit learning” to exit the learning mode.

5.6.4 Delete function¶¶

The user enters the deletion mode by saying the command word “I want to delete”. At this time, three methods are supported for deletion. Method 1: Delete all. When the user reads the command word, all registered wake-up words and command words will be deleted. Method 2: Delete the wake-up word. When the user reads the command word, all registered wake-up words will be deleted. Method 3: Delete the command word. When the user reads the command word, all the registered command words will be deleted.

5.6.5 Exit delete function¶¶

When entering the delete mode, the user can read the command word “exit delete” to exit the delete mode.

5.7 Precautions for offline command word self-learning¶¶

Learning must be done in a quiet environment, at a normal speed, with a loud voice, and within 1 meter of the distance .

The number of words learned is 2 to 5 .

A maximum of 20 learning templates are supported .

The language model (ASR) currently only supports the Chinese model (except for the low misrecognition model)

6 Offline command word self-learning Q&A¶¶

Q1: How to enter the learning mode?

A1: After waking up (the wake-up word of the standard demo is “smart housekeeper”), read the command word “learn wake-up word” or “learn command word” to enter the learning mode.

Q2: How to exit the learning mode?

A2: There are two ways to exit. The first is the user’s active exit. After waking up and entering the learning mode, you can exit the registration mode by saying the command word “exit learning”. The second type: After each successful learning, the system automatically exits the learning mode and enters the recognition mode.

Q3: If you enter the learning mode and there is no feedback when reading the voice, what should you do?

A3: Reason 1: It may be that the sound is too low, and the threshold of VAD is increased during learning. The solution: keep the distance from the device within 0.5 meters, and learn with a loud voice. Reason 2: The system has exited the wake-up state. Solution: Re-enter the learning mode after waking up to solve the problem.

Instructions for Offline Command Word Self-learning¶¶

1 Content overview¶¶

2 Basic concepts of offline command word self-learning¶¶

3 Applicable scenarios for offline command word self-learning¶¶

4 Offline command word self-learning project description¶¶

5 Instructions for Offline Command Word Self-learning¶¶

5.1 Macro definitions related to offline command word self-learning¶¶

5.2 Offline command word self-learning wake-up word and command word number configuration¶¶

5.3 Offline command word self-learning command word learning broadcast tone configuration¶¶

5.4 Other parameter configuration¶¶

5.5 Offline command words self-learning default command words¶¶

5.6 Steps for offline command word self-learning¶¶

5.6.1 Learning function¶¶

5.6.2 Relearn function¶¶

5.6.3 Exit learning function¶¶

5.6.4 Delete function¶¶

5.6.5 Exit delete function¶¶

5.7 Precautions for offline command word self-learning¶¶

6 Offline command word self-learning Q&A¶¶