Microsoft Speech SDK
SAPI 5.1
3.1 SAPI Compliance Required Tests
3.2 SAPI Compliance Feature List Tests
4 Using the compliance testing tool
6 Compliance Testing Configuration Options
6.1 SAPI 5.0 Compliance Testing Application Toolbar
6.2 SAPI 5.0 Compliance Testing Application Menu Choices
6.3 SAPI 5.0 Compliance Testing Logging Options
6.4 SAPI 5.0 Compliance Testing Run Options
6.5 SAPI 5.0 Compliance Test Selection Options
7.4 Compliance Test Customization
7.6 OS Language Incompatibility
8.5 OS Language Incompatibility
Table 1: Events Compliance Test
Table 2: Lexicon Compliance Test
Table 3: Command and Control Compliance Test
Table 4: Required Compliance Tests
Table 5: Events Feature Compliance Test
Table 6: Grammar Feature Compliance Test
Table 7: Feature Compliance Tests
Table 8: Sample Engine Required Compliance Test results
Table 9: Sample Engine Feature Compliance Test results
Table 10: Strings to be localized
Table 17: Sample Engine Required Test Results
Table 18: Sample Engine Feature List Test Results
Table 19: Strings to be localized for compliance tests
Table 21: Required Compliance Tests Failed
Table 22: Feature Compliance Tests Not Supported
This paper, directed toward engine vendors, describes the SAPI 5.0 compliance testing tool by answering the following questions:
· What does SAPI compliance for SAPI 5.0 imply?
· What are the SAPI compliance tests?
· What does each test look for?
The goals of the compliance tool are to help engine vendors test their speech engines for SAPI compliance and port these speech engines to SAPI 5.0. The tests also help vendors to support various SAPI features that are not required for compliance. These tests do not test the speech or performance quality of the engines. All compliance tests assume that SAPI will do parameter validation, and as such, they do not check the engine’s ability to handle invalid parameters such as null, bad pointers, or values out of range.
To run the compliance tests, the SPcomp.exe tool is used and either the Text-to-Speech (TTS) or the Speech Recognition (SR) test suite is selected. This tool generates a log report indicating the results of the compliance tests.
There are two types of SAPI 5 compliance tests:
1) required tests
2) feature list tests
The compliance tests do not necessarily test the DDI directly, instead, the use the SAPI API function calls to test the engine’s response to the DDI. The default engine is always used as the engine in the compliance test. Currently, the supported languages for the compliance tests are English, Japanese and Simplified Chinese[1]. Please check Microsoft® Speech.NET Technologies for language pack updates and information.
The results of the required tests are of a pass/fail nature. These tests were designed to help an engine reach a minimal amount of functionality with the SAPI DDI layer. In order to be SAPI compliant, the engine must pass all required SAPI tests.
The results of the feature list tests are either “Supported” or “Unsupported”. Feature list tests were designed to help engine vendors port advanced features to SAPI 5.0. To be SAPI compliant, the engine does not need to pass any feature list test, although it is recommended that all features be implemented if possible.
The minimum requirements for speech recognition for dictation are a 200 mhz Pentium with 64 MB for win 95/98 or 96 MB for NT. The recommended computer is a 300 Mhz Pentium II with 128 megs of RAM or better.
The SAPI 5.0 compliance test tool, SPcomp.exe, enables you to load compliance test suites and determine the test result logging options. Please see Compliance Tests for more options.
The SPcomp.exe test tool creates the .pro file for a given test suite. Perform compliance tests by starting the SPcomp.exe application and loading a test suite from a .pro file. The .pro file loads the associated dynamic link library (.dll), which contains the SR or TTS compliance tests.
To start the SAPI 5.0 compliance test tool SPcomp.exe from Windows Explorer, double-click the compliance tool icon. Alternatively, you can perform each compliance test from the command line by running the compliance test tool and command line syntax.
For example, the command line syntax for running the SRcomp compliance test in the srcompreq.pro test suite is as follows:
C:> SPcomp.exe srcompreq.proSPcomp.exe srcompreq.pro
Starts the compliance test tool and loads the speech recognition (SR) required tests from the SRcomp.dll.
SPcomp.exe srcompopt.pro
Starts the compliance test tool and loads the speech recognition (SR) feature list tests from the SRcomp.dll.
SPcomp.exe ttscompreq.pro
Starts the compliance test tool and loads the text-to-speech (TTS) required tests from the TTScomp.dll.
SPcomp.exe ttscompopt.pro
Starts the compliance test tool and loads the text-to-speech (TTS) feature list tests from the TTScomp.dll.
The SAPI 5.0 compliance tests verify that you have successfully implemented the required features to be considered compatible with SAPI 5.0. Your engine must successfully complete each of the following four compliance tests with 100 percent pass rate to be compliant with SAPI 5.0.
1. srcompreq.bat
—Speech Recognition (SR) required test batch file.
2. srcompopt.bat
—Speech Recognition (SR) feature list test batch file.
2. ttscompreq.bat
—Text-to-speech (TTS) required test batch file.
4. ttscompopt.bat
—Text-to-speech (TTS) feature list test batch file.
The SAPI 5.0 compliance tool generates a result log and you can configure it to display the result log information or you can save it to a file. The test result log contains pass or fail state information for each segment of the test suite. If a compliance test fails, you can review the result log to determine the origin of the failure.
Example result log with a 100% pass rate for all tests.
Total:============================================================================= PASS FAIL SUPPORTED UNSUPPORTED ABORTED SKIPPED----------------------------------------------------------------------------- 1 0 0 0 0 0=============================================================================Status: PASS
The SAPI 5.0 compliance test application user interface (UI) enables you to configure the testing options. The following section provides additional compliance test configuration information.
· SAPI 5.0 Compliance Testing Application Toolbar
· SAPI 5.0 Compliance Testing Application Menu Choices
· SAPI 5.0 Compliance Testing Logging Options
· SAPI 5.0 Compliance Testing Run Options
· SAPI 5.0 Compliance Test Selection Options
The main window of the SAPI 5.0 compliance testing application contains a toolbar from which you can access the configuration options. Additionally, the configuration options are also available from the menu bar located at the top of the compliance testing application window.
Pause on an icon to display tooltip text. Click an icon to view the information associated with the feature.
Load the test DLL
Loads a test dynamic-link library (DLL).
You can run compliance tests using one of the following methods:
1. From the SAPI Engine Compliance Tool, click File, and then click Load Test DLL.
2. Load
the test DLL into SPcomp.exe from the command line.
For more information, see Using
the compliance testing tool.
Note: loading a compliance test with either method results in automatically unloading any previously loaded compliance tests.
Load the test settings
Loads one of the pre-configured test suites.
You can run compliance tests using one of the following methods:
1. From the SAPI Engine Compliance Tool, click File, and then click Load Settings.
2. Load
the test DLL into SPcomp.exe from the command line.
For more information, see Using
the compliance testing tool.
Note: loading a compliance test with either method results in automatically unloading any previously loaded compliance tests.
Save settings
Saves the configuration settings for the compliance test application.
Copy
Selects and copies content from the display log.
Clear Window
Clears the display contents of the result log.
Find
Searches for a specific word or phrase within the result log.
Find Next
Searches for the next occurrence of a specific word or phrase within the result log.
Run Test
Begins the compliance test.
Stop Test
Stops the compliance test.
Set Run Options
Configures the compliance test options.
Select Tests
Chooses which compliance test contained in the test suite to.
Set Logging
Determines location of the compliance test log information.
The SAPI 5.0 compliance testing application configuration choices are accessible through the menu system. The following items are covered in this section:
· File menu
· Edit menu
· Test menu
· Options menu
· Help menu
Click File to set configuration options to load settings, save settings, or load the appropriate test DLL. Use the arrow keys to view various menu choices. Press ENTER to select a menu choice.
Click Edit to copy text from the result log and search for text within the result log. Use the arrow keys to view various menu choices. Press ENTER to select a menu choice.
Click Test to run the test or select a test. Use the arrow keys to view various menu choices. Press ENTER to select a menu choice.
Click Options to view the various configuration settings. Use the arrow keys to view various menu choices. Press ENTER to select a menu choice.
Click Help and then click About to display the SAPI 5.0 Engine Compliance Tool Version dialog box. Use the arrow keys to view the various menu choices. Press ENTER to select a menu choice.
From the Options menu, choose Logging Settings to set SAPI 5.0 compliance test result log configuration options.
Window
Displays the test result information in the main window of the compliance testing application.
Log File
Saves the test result information as text in a log file.
The log file is located at the same directory as SPcomp.exe tool and the file name will be the following style:
spcomp@442.logThe numbers "442" in the file name are generated by the SPcomp.exe tool and will be incremented by one each time you restart SPcomp.exe tool and run the test. A new log file is generated each time you start SPcomp.exe tool and run a compliance test.
Detailed
Specifies detailed result log information.
Summary
Specifies summary result log information.
From the Options menu, click Run Options to configure SAPI 5.0 compliance testing run options.
Random
Randomizes the test order.
Close after execution
Closes the compliance testing application after the test sequence.
Stress
This option should not be selected for compliance tests.
Run count
Specifies the number of interactions the selected test should run.
Disable screen saver
Disable the screen saver.
Quiet
Runs the selected test in quiet mode.
Random Seed
The random seed value set here is used for the next time you run the compliance test.
Note: When troubleshooting a failed compliance test, you need to enter the same seed value information that was used for the failed compliance test before you repeat the compliance test procedure.
You can obtain the compliance test seed value from the "Random Seed" field information in the SPcomp@xxx.log file that was generated during the unsuccessful compliance test.
From the Test menu, click Select Test to configure SAPI 5.0 compliance test choices.
Test Cases
Displays the current test suite.
Selected Test Cases
Displays the current selected tests.
Add Case(s)
Adds test items to the list of
selected test cases.
Alternatively, to add test cases, right-click the test case in the test case
display window and click Add Item.
Remove Case
Removes the selected test case from the current test. However, removing the selected test does not affect the need to successfully pass this test case to satisfy SAPI compliancy.
Alternatively, to remove test cases, right-click the test cases in the selected test case display window and click Remove Case.
Remove All
Removes all test cases.
SAPI compliant SR engines must be able to perform the following[2]:
§ Generate certain SR events
§ Interact with the SAPI lexicon
§ Handle Command and Control (C&C) grammars
§ Generate Phrase Elements
§ Support auto pause on recognition
§ Support rule synchronization
§ Support multiple instances of the engine
§ Support multiple application contexts
Events will be checked for with .wav files. The test will feed the wav file to the engine and expect a specific event notification to occur. Please note that whether or not the engine can fire a specific event depends on the confidence threshold of the engine. Engine vendors could change the .wav quality to meet their requirement.
For English:
|
Test |
Description |
Resource IDs |
Description |
|
SoundStart |
Test will check if a sound start event occurs. |
IDS_WAV_SOUNDSTART |
Input .wav file, tag_l.wav |
|
IDR_L_GRAMMAR |
Input CFG grammar |
||
|
SoundEnd |
Test will check if a sound end event occurs. |
IDS_WAV_SOUNDEND |
Input .wav file, tag_l.wav |
|
IDR_L_GRAMMAR |
Input CFG grammar |
||
|
PhraseStart |
A .wav file with audio the engine can do recognition on. Test insures that a phrase start event occurs |
IDS_WAV_PHRASESTART IDR_L_GRAMMAR |
Input .wav file, tag_l.wav Input CFG grammar |
|
Recognition |
A .wav with audio that the engine can do recognition on. Test insures that a recognition event occurs. |
IDS_WAV_RECOGNITION_1 IDR_L_GRAMMAR |
Input .wav file, tag_l.wav Input CFG grammar |
|
False Recognition |
A wav file and a mismatching C&C grammar are loaded. Test insures that false recognition event occurs. |
IDS_WAV_RECOGNITION_1 IDR_RULE_GRAMMAR |
Input .wav file, tag_l.wav Input CFG grammar |
|
SoundStart/ SoundEnd |
Test will check that the sound start event occurs before the sound end event. |
IDS_WAV_SOUNDSTARTEND IDR_L_GRAMMAR |
Input .wav file, tag_l.wav Input CFG grammar |
|
PhraseStart/ Recognition |
Test will check that the phrasestart event occurs before the recognition event. |
IDS_WAV_RECOGNITION_1 IDR_L_GRAMMAR |
Input .wav file, tag_l.wav Input CFG grammar |
|
SoundStart/ PhraseStart/ Recognition/ SoundEnd/ |
A wav file with audio that the engine can do recognition on. Test insures that the audiooffsets of these events are correct in terms of value comparison. |
IDS_WAV_RECOGNITION_1 IDR_L_GRAMMAR |
Input .wav file, tag_l.wav Input CFG grammar |
Table 1: Events Compliance Test
It is expected that changes in the user and application lexicon will be synchronized with the engine both when the engine starts up and after it has loaded a command and control grammar.
|
Test |
Description |
Resource IDs |
Description |
|
User Lexicon Before C&C Grammar Loaded |
A made-up word with its customized pronunciation is added to the user lexicon. After command and control grammar is loaded, audio will be sent with the word added and the expected result is checked for. |
IDS_WAV_SYNCH_BEFORE_LOAD IDR_SNORK_GRAMMAR IDS_RECO_SYNCH_BEFORE_LOAD IDS_RECO_NEWWORD_PRON |
Input .wav file, lexicon.wav Input CFG grammar The lexicon form of new word The pronunciation of the new word in user lexicon |
|
User Lexicon After C&C Grammar Loaded |
After command and control grammar is loaded, a made-up word with its customized pronunciation is added to the user lexicon. Audio will be sent with the word added and the expected result is checked for. |
IDS_WAV_SYNCH_AFTER_GRAM IDR_SNORK_GRAMMAR IDS_RECO_SYNCH_AFTER_GRAM IDS_RECO_NEWWORD_PRON |
Input .wav file, lexicon.wav Input CFG grammar The lexicon form of new word The pronunciation of the new word in user lexicon |
|
Application Lexicon and C&C Grammar |
A made-up word with its customized pronunciation is added to the application lexicon. After command and control grammar is loaded, audio will be sent with the word added and the expected result is checked for. |
IDS_WAV_APPLEX IDR_SNORK_GRAMMAR IDS_APPLEX_WORD IDS_APPLEX_PROP
|
Input .wav file, lexicon.wav Input CFG grammar The lexicon form of new word The pronunciation of the new word in application lexicon |
|
User lexicon before application lexicon |
A made-up word is added to both user lexicon and application lexicon using the different customized pronunciations. After command and control grammar is loaded, audio will be sent with the word’s pronunciation in user lexicon and the expected result is checked for. |
IDS_WAV_USERLEXBEFOREAPPLEX IDR_SNORK_GRAMMAR IDS_USERLEXBEFOREAPPLEX_WORD IDS_USERLEXBEFOREAPPLEX_USERPROP IDS_USERLEXBEFOREAPPLEX_APPPROP |
Input .wav file, lexicon.wav Input CFG grammar The lexicon form of new word The pronunciation of the new word in user lexicon The pronunciation of the new word in application lexicon |
Testing the engine for grammar compliance is perhaps the most complex set of tests. The engine must process a grammar correctly. Each test will use a grammar specifically tailored for the particular feature.
|
Test |
Description |
Resource IDs |
Description |
|
L Tag |
A three-element list grammar is loaded. Audio with the middle item to be recognized with the sent to the engine and the result checked for this item. |
IDS_RECO_L_TAG IDS_WAV_L_TAG IDR_L_GRAMMAR |
Expected Result Input .wav file, tag_l.wav Input CFG grammar |
|
Expected Rule |
A grammar with two identical rules is loaded. The first rule will be activated. Audio that triggers this rule is sent and test verifies that the engine uses the first rule. The first rule is then de-activated and the second rule is activated. The same audio is sent and the test verifies that the engine uses the second rule. |
IDS_RECO_EXPRULE_FIRSTRULE IDS_RECO_EXPRULE_SECONDRULE IDS_WAV_EXPRULE_TAG IDR_EXPRULE_GRAMMAR |
First Rule’s Name Second Rule’s Name Input .wav file, tag_exprule.wav Input CFG grammar |
|
P Tag |
A simple grammar with a single phrase. Audio is sent and recognition is expected. Audio that does not contain the phrase is sent and no recognition is expected. |
IDS_RECO_P_TAG IDS_WAV_P_TAG IDR_P1_GRAMMAR |
Expected Result Input .wav file, tag_p.wav Input CFG grammar
|
|
O Tag |
A grammar will be defined with a phrase and an optional phrase preceding and following it. Three audio streams will be sent. One with the first optional phrase, one for the second, and the third that does not contain any optional phrases. The appropriate recognition result is checked for in each case. |
IDS_RECO_O_TAG_1 IDS_RECO_O_TAG_2 IDS_RECO_O_TAG_3 IDS_WAV_O_TAG_1 IDS_WAV_O_TAG_2 IDS_WAV_O_TAG_3 |
FirstOptionalWord Required word Second Optional word Input .wav file containing the first optional word, tag_o1.wav Input .wav file containing the second optional word, tag_o2.wav Input .wav file without optional words, tag_o3.wav |
|
RULEREF Tag |
A grammar with a phrase with a rule reference and a rule defined will be loaded. Audio that triggers the rule will be sent and the result checked. |
IDS_RECO_RULE_TAG IDS_WAV_RULE_TAG IDR_RULE_GRAMMAR |
Expected Result Input .wav file, tag_rule.wav Input CFG grammar |
|
/Disp/lex/pron format |
Test ensures engine can support customized pronunciation provided in the command and control grammar file. |
IDS_CUSTOMPROP_NEWWORD_PRON IDS_CUSTOMPROP_NEWWORD_DISP IDS_CUSTOMPROP_NEWWORD_LEX IDS_CUSTOMPROP_RULE IDS_WAV_CUSTOMPROP |
The customized pronunciation form of the new word The customized display form of the new word The customized lexicon form of the new word The dynamic grammar rule name Input .wav file, lexicon.wav |
Table 3: Command and Control Compliance Test
|
Test |
Description |
Resource IDs |
Description |
|
Phrase Elements |
The audio offsets of SPPHRASEELEMENTs in one SPPHRASE are correctly filled in, which means that the audio offset of the first SPPHRASEELEMENT is less than the audio offset of the second SPPHRASEELEMENT, the audio offset of the second SPPHRASEELEMENT is less than the third one, etc. |
IDS_WAV_RULE_TAG IDR_RULE_GRAMMAR |
Input .wav file, tag_rule.wav Input CFG grammar |
|
Auto Pause |
The test makes sure engine can support auto pause feature provided by SAPI. |
IDS_AUTOPAUSE_DYNAMICWORD1 IDS_AUTOPAUSE_DYNAMICWORD2 IDS_AUTOPAUSE_DYNAMICRULE1 IDS_AUTOPAUSE_DYNAMICRULE2 IDS_WAV_AUTOPAUSE |
The word in the first rule The word in the second rule The name of the first rule The name of the second rule Input .wav file, autopause.wav |
|
Top-level rule invalidation |
Test verifies that engine can synchronize the rule information after SAPI notifies engine of top-level rule invalidation. |
IDS_INVALIDATETOPLEVEL_DYNAMICWORDS IDS_INVALIDATETOPLEVEL_DYNAMICRULE IDS_WAV_INVALIDATETOPLEVEL_OLD IDS_INVALIDATETOPLEVEL_DYNAMICNEWWORDS IDS_WAV_INVALIDATETOPLEVEL_NEW |
The words in the dynamic grammar The rule name in the dynamic grammar Input .wav file used before invalidation, tag_exprule.wav The new words in the dynamic grammar Input .wav file used after invalidation |
|
None-top-level rule invalidation |
Test verifies that engine can synchronize the rule information after SAPI notifies engine of non-top-level rule invalidation. |
IDS_INVALIDATENONTOPLEVEL_RULE1 IDS_INVALIDATENONTOPLEVEL_RULE2 IDS_INVALIDATENONTOPLEVEL_TOPLEVELRULE IDS_INVALIDATENONTOPLEVEL_OLDWORD1 IDS_INVALIDATENONTOPLEVEL_OLDWORD2 IDS_WAV_INVALIDATENONTOPLEVEL_OLD IDS_INVALIDATENONTOPLEVEL_NEWWORD1 IDS_INVALIDATENONTOPLEVEL_NEWWORD2 IDS_WAV_INVALIDATENONTOPLEVEL_NEW |
The first rule name The second rule name The top-level rule name The word in the first rule used before invalidation The word in the second rule used before invalidation Input .wav file used before invalidation, tag_exprule.wav The word in the first rule used after invalidation The word in the second rule used after invalidation Input .wav file used after invalidation, tag_rule.wav |
|
Multiple recognition contexts |
Multiple recognition contexts will be created with different grammars. The test will verify that the recognition event is generated by the correct recognition contexts. |
IDS_RECO_P_TAG IDS_WAV_MULT_RECO IDR_P1_GRAMMAR IDR_P2_GRAMMAR |
The result expected in the second grammar Input .wav file, multireco.wav The first grammar used by the first recocontext The second grammar used by the second recocontext |
|
Multiple recognition engine instances |
Basic tests are run separately on different threads to see if engine can support multi instances. |
NA |
NA |
Some of the features exposed through SAPI are useful from a competitive advantage point of view. Features are not required by SAPI compliance, but may be an attractive function for engine vendors to implement. SAPI features are:
§ Interference and hypothesis events
§ Dictation functionalities
§ Advanced command and control features
§ Command and control alternate
§ Engine properties
§ Inversed text normalization
Events will be checked for with .wav. The test will feed the .wav to the engine and expect a specific event notification to occur. Please note that whether or not the engine can fire a specific event depends on the confidence threshold of the engine. Engine vendors may change the .wav files if it is felt that the .wav quality does not meet their requirements (Refer to Section 7.4).
|
Test |
Description |
Resource IDs |
Descriptions |
|
Interference |
A wav file with noises. Test will check that an interference event occurs. |
IDS_WAV_INTERFERENCE IDR_L_GRAMMAR |
Input .wav file, tag_l.wav Input CFG grammar |
|
Hypothesis |
A .wav file with audio that engine can do recognition on. Test insures a hypothesis event occurs. |
IDS_WAV_HYPOTHESIS IDR_EXPRULE_GRAMMAR |
Input .wav file, tag_exprule.wav Input CFG grammar |
Table 5: Events Feature Compliance Test
This the required features if Engine wants to support dictation grammar. This include some basic functionalities for dictation grammar. This includes lexicon, dictation tag, dictation alternates.
|
Test |
Description |
Resource IDs |
Descriptions |
|
User Lexicon Before dictation Grammar Loaded |
A made-up word with its customized pronunciation is added to the user lexicon. After dictation grammar is loaded, audio will be sent with the word added and the expected result is checked for. |
IDS_WAV_SYNCH_BEFORE_LOAD IDS_RECO_SYNCH_BEFORE_LOAD IDS_RECO_NEWWORD_PRON |
Input .wav file, lexicon.wav The lexicon form of new word The pronunciation of the new word in user lexicon |
|
User Lexicon After dictation Grammar Loaded |
After dictation grammar is loaded, a made-up word with its customized pronunciation is added to the user lexicon. Audio will be sent with the word added and the expected result is checked for. |
IDS_WAV_SYNCH_AFTER_DICT IDS_RECO_SYNCH_AFTER_DICT IDS_RECO_NEWWORD_PRON |
Input .wav file, lexicon.wav The lexicon form of new word The pronunciation of the new word in user lexicon |
|
Dictation Tag |
A rule with dictation tag is loaded. Audio is feed and the test verifies the recognition event is generated. |
IDS_DICTATIONTAG_WORDS IDS_DICTATIONTAG_RULE IDS_WAV_DICTATIONTAG |
The word before the dictation tag The dynamic grammar rule name Input .wav file, tag_exprule.wav |
|
Dictation alternates |
Test ensures that engine can generate alternate results for dictation grammar. The test makes sure that engine has its own alternate object and the object can generate some alternate results. |
IDS_WAV_EXPRULE_TAG |
Input .wav file, tag_exprule.wav |
Table 5: Dictation Compliance Test
Each test will use a grammar specifically tailored for the particular feature. Some tests would use dynamic grammar instead of the static grammar.
|
Test |
Description |
Resource IDs |
Descriptions |
|
|
WildCard Tag |
A rule with wildcard tag is loaded. Audio is feed and the test verifies the recognition event is generated. |
IDS_WILDCARD_WORDS IDS_WILDCARD_RULE IDS_WAV_WILDCARD
|
The word before the wildcard tag The dynamic grammar rule name Input .wav file, tag_rule.wav |
|
|
TextBuffer Tag |
A grammar with <TextBuffer> tag will be loaded. Test fills in the content of TextBuffer on the fly. Audio with both static part and dynamic part of the grammar would be feed and the result would be checked. |
IDS_CFGTEXTBUFFER_WORDS IDS_CFGTEXTBUFFER_BUFFERWORD IDS_CFGTEXTBUFFER_RULE IDS_WAV_CFGTEXTBUFFER |
The words before TEXTBUFFER tag The word for TEXTBUFFER tag The rule name Input .wav file, tag_exprule.wav |
|
|
Use the correct grammar |
Two unambiguous grammars are loaded to test if engine can use the correct grammar to do recognition. |
IDS_RECO_RULE_TAG IDS_WAV_RULE_TAG IDR_L_GRAMMAR IDR_RULE_GRAMMAR |
Expected result for the second grammar Input .wav file, tag_rule.wav The first grammar The second grammar |
|
|
Use the most recently activated grammar |
Two ambiguous grammars are loaded to test if engine can use the most recently activated grammar to do the recognition. |
IDS_WAV_RULE_TAG IDR_RULE_GRAMMAR
|
Input .wav file, tag_rule.wav Input CFG grammar |
|
Table 6: Grammar Feature Compliance Test
|
Test |
Description |
Resource IDs |
Descriptions |
|
Command and Control alternates |
Test ensures that the engine can generate alternate results for command and control grammar |
IDS_ALTERNATESCFG_BESTWORD IDS_ALTERNATESCFG_ALTERNATE1 IDS_ALTERNATESCFG_ALTERNATE2 IDS_ALTERNATESCFG_WORDS IDS_WAV_ALTERMATESCFG |
The best choice of the CFG grammar Alternate word in CFG grammar Alternate word in CFG grammar Others words in the CFG grammar tag_exprule.wav |
|
Engine numeric properties |
If engine supports the numeric properties specified by SAPI |
NA |
|
|
Engine text properties |
If engine can return S_FALSE on the text properties that are not supported. |
NA |
|
|
Inversed Text Normalization |
The test uses a wav file and expects engine to pass back a result containing digits together with the normal result. Please note that this is a very specific ITN test and is not coverage of ITN related issues. |
IDS_RECO_GETITNRESULT IDS_WAV_GETITNRESULT IDR_RULE_GRAMMAR |
Expected ITN result Input .wav file, tag_rule.wav Input CFG grammar |
The sample engine is not fully SAPI compliant due to the fact that it does not have the full range of functionality that a true SR engine would have. Table 8 indicates which compliance tests will pass. Table 9 indicates which features are supported.
|
Test |
Result |
Description |
|
Events |
|
|
|
SoundStart |
Pass |
|
|
SoundEnd |
Pass |
|
|
PhraseStart |
Pass |
|
|
FalseRecognition |
Fail |
The sample engine doesn’t generate this event based on the real SR job. |
|
Recognition |
Pass |
|
|
SoundStart/SoundEnd order |
Pass |
|
|
PhraseStart/Recognition order |
< |