Windows Speech Recognition

From Infogalactic: the planetary knowledge core
Jump to: navigation, search
Speech Recognition
A component of Microsoft Windows
300px
Windows Speech Recognition tutorial included with Windows Vista.
Details
Included with Windows Vista, Windows 7, Windows 8, Windows 8.1, Windows 10
Related components
Microsoft Speech API

Windows Speech Recognition is a speech recognition component developed by Microsoft and introduced in the Windows Vista operating system that enables the use of voice commands to perform operations, such as the dictation of text, within applications and the operating system itself.[1]

Speech recognition relies on the Speech API developed by Microsoft,[2] and is also present in Windows 7,[3] Windows 8,[4] Windows 8.1,[4] and Windows 10.[5]

History

Precursors

Microsoft has been involved in speech recognition and speech synthesis research for many years. In 1993, Microsoft hired Xuedong Huang from Carnegie Mellon University to lead its speech development efforts. The company's research eventually ultimately led to the development of the Speech API, introduced in 1994.[2] Speech recognition technology has been used in some of Microsoft's products prior to Windows Speech Recognition. Versions of Microsoft Office, including Office XP and Office 2003, included support for speech recognition among Office applications and other applications such as Internet Explorer.[6] Installation of Office would enable limited speech functionality in Windows NT 4.0, Windows 98 and Windows ME.[7] The 2002 edition of Windows XP Tablet PC Edition would also include support within the Tablet PC Input Panel feature,[8][9] and the Microsoft Plus! for Windows XP expansion package enabled voice commands to be used in Windows Media Player.[10] However, this support was limited to individual applications, and prior to Windows Vista, the Windows operating system did not include integrated support for speech recognition.[9]

Development

File:WindowsVistaPreliminaryWSR.PNG
A prototype speech recognition Aero Wizard interface seen in Windows Vista (then known as "Longhorn") build 4093.

At the Windows Hardware Engineering Conference of 2002, Microsoft announced that Windows Vista, then known by its codename "Longhorn," would include advances in speech recognition technology and features such as support for microphone arrays.[11] Bill Gates expanded upon this information during the Professional Developers Conference of 2003 where he stated that the company would "build speech capabilities into the system -- a big advance for that in 'Longhorn,' in both recognition and synthesis, real-time."[12][13] Further reports said that the operating system would include integrated support for speech recognition,[14][15] and certain pre-release builds throughout development of the operating system would include a speech engine with training features.[16][17] In 2003, Microsoft clarified the extent of its intended integration for Windows Vista when the company stated within a pre-release software development kit that "the common speech scenarios, like speech-enabling menus and buttons, will be enabled system-wide" in the operating system.[18]

During WinHEC 2004, Microsoft listed speech recognition as part of its "Longhorn" mobile PC strategy to improve productivity and listed microphone arrays as a hardware opportunity for the operating system.[19][20] At WinHEC 2005, Microsoft would release additional details pertaining to speech recognition in Windows Vista with a focus on accessibility, new mobility scenarios, and improvements to the speech user experience. Unlike the speech support included in Windows XP, which was integrated with the Tablet PC Input Panel and required switching between dictation and command modes, Windows Vista would separate the feature from the Tablet PC Input Panel by including a dedicated interface for speech input on the desktop, and the operating system would also unify the previously separate dictation and command modes. In previous versions of Windows, speech recognition would not allow a user to speak a command after dictation or vice-versa without first switching between these two modes.[21] Microsoft also stated that speech recognition in Windows Vista would improve dictation accuracy, and support additional languages and microphone arrays.[22] A demonstration of the feature at WinHEC 2005 focused on e-mail dictation with correction and editing commands,[22] and a set of slides dedicated to microphone arrays was also released.[23] Windows Vista Beta 1 would include an integrated speech recognition application.[24]

On July 27, 2006, before the operating system's release to manufacturing, a notable incident pertaining to the use of speech recognition occurred during a demonstration by Microsoft at its annual Financial Analyst Meeting. Initially, speech recognition failed to function correctly, which resulted in an unintended output of: "Dear aunt, let's set so double the killer delete select all" when attempts to dictate led to consecutive output errors.[25][26] This incident was a subject of derision among analysts and journalists present in the audience.[27][28] It was later revealed that the feature's failure to function properly during the demonstration was due to an audio gain glitch that caused the feature to distort the dictated commands.[29] The glitch was fixed prior to the operating system's release to manufacturing on November 8, 2006.[29]

In an effort to persuade company employees to interact with Windows Speech Recognition during its development, Microsoft offered an opportunity to win a Premium model of its Xbox 360 video game console.[30]

Security report

Reports surfaced in early 2007 that Windows Speech Recognition may be vulnerable to an attack that could allow attackers to take advantage of its capabilities to perform undesired operations on a targeted computer by playing audio through the targeted computer's speakers;[31][32] it is notable for being the first vulnerability discovered after the operating system's general retail availability.[33] While Microsoft has stated that such an attack is theoretically possible, it would have to meet a number of prerequisites in order to be successful: the targeted system would be required to have the speech recognition feature previously activated and configured, speakers and microphone(s) connected to the targeted system would need to be activated, and the exploit would require the software to interpret commands without a user noticing—an unlikely scenario as the affected system would perform user interface operations and produce audible feedback (as speakers would need to be active). Moreover, mitigating factors would include dictation clarity, and microphone feedback and placement. An exploit of this nature would also not be able to perform privileged operations for users or protected administrators without explicit user consent because of User Account Control.[34]

Overview and features

Windows Speech Recognition allows a user to control a computer, including the operating system desktop user interface, through voice commands. Applications, including most of those that are bundled with Windows, can also be controlled through voice commands.[35] By using speech recognition, users can dictate text within documents and e-mail messages, fill out forms, control the operating system user interface, perform keyboard shortcuts, and move the mouse cursor.[36]

Speech recognition uses a speech profile to store information about a user's voice.[4] Accuracy of speech recognition increases through use, which helps the feature adapt to a user's grammar, speech patterns, vocabulary, and word usage.[1][37] Speech recognition also includes a tutorial to improve accuracy,[1] and can optionally review a user's personal documents, including e-mail messages, to improve its command and dictation accuracy.[38] In Windows 7 and later versions, an additional option is available that allows users to send speech information to Microsoft.[39] Individual speech profiles can be created on a per-user basis,[37] and backups of profiles can be performed via Windows Easy Transfer[40] or through a downloable utility developed by Microsoft.[41] Profiles archived through this utility carry the WSRPROFILE filename extension. Windows Speech Recognition relies on Microsoft Speech API.[2] and third-party applications must support the Text Services Framework.[35] Speech Recognition currently supports the following languages: Chinese (Traditional), Chinese (Simplified), English (U.S.), English (U.K.), French, German, Japanese, and Spanish.[1]

Interface

The interface for Windows Speech Recognition primarily consists of a status area for instructions, for information about commands (e.g., if a command could not be heard by the speech recognizer), and also for information related to the state of the speech recognizer; a voice meter is also provided to display visual feedback to the user about voice volume levels. The status area represents the current state of Windows Speech Recognition in a total of three modes, listed below with their respective meanings:

  • Listening: The speech recognizer is active and waiting for user input
  • Sleeping: The speech recognizer will not listen for or respond to commands other than "Start listening"
  • Off: The speech recognizer will not listen or respond to any commands; this mode can be enabled by speaking "Stop listening"

In addition to the three modes listed above, the status area can also display information about messages that users can customize as part of their own Windows Speech Recognition Macros.[42][43]

Alternates panel

File:WSR-AlternatesPanel.png
The alternates panel in Windows Speech Recognition displaying suggestions for a phrase.

A disambiguation interface, referred to as the alternates panel, displays a list of items interpreted by the recognizer as being relevant to a user's spoken word(s); if the word or phrase that the user desired to insert into an application is listed among results, the user can speak the corresponding number of the word that appears among the results and confirm this choice by speaking "OK" to insert it within the application.[44]

The alternates panel will also appear when launching programs or speaking commands that may refer to more than one item (e.g., speaking "Start Internet Explorer" may list the web browser and an alternate version of the web browser with add-ons disabled). However, a Windows Registry entry, ExactMatchOverPartialMatch, can limit commands to programs or commands with exact names if there is more than one instance of that item included among results.[45]

Common commands

Listed below are common commands available for Windows Speech Recognition. Words in italics indicate a variable that can be substituted for a desired item (e.g., the word "direction" in the "scroll direction" command can be substituted with the word "down" to scroll down).[36] A "start typing" command enables Windows Speech Recognition to interpret dictation commands as keyboard shortcuts.[44]

Dictation commands: "New line," "new paragraph," "tab," "literal word," "numeral number," "go to word," "go after word," "no space," "go to start of sentence," "go to end of sentence," "go to start of paragraph," "go to end of paragraph," "go to start of document," "go to end of document," "go to field name" (e.g., go to address, cc, or subject). Special characters, such as a comma, can be dictated simply by stating the name of the special character.[36]
Navigation commands:
Keyboard shortcuts: "Press keyboard key," "press ⇧ Shift plus a," "press capital b." The NATO phonetic alphabet is also supported. Keys that can be pressed without first giving the press command include: ← Backspace, Delete, End, ↵ Enter, Home, Page Down, Page Up, Tab ↹.[36]
Mouse commands: "Click," "click that," "double-click," "double-click that," "mark," "mark that," "right-click," "right-click that," mousegrid.[36]
Window management commands: "Close (alternatively maximize, minimize, or restore) window," "close that," or "close application name," "switch applications," "switch to program name," "scroll direction," "scroll direction in number of pages," "show desktop," "show numbers."[36]
Speech recognition commands: "Start listening," "stop listening," "show speech options," "open speech dictionary," "move speech recognition," "mimimize speech recognition."[36] A list of applicable commands can be shown by speaking "What can I say?" This command is currently only available in English.[1]

Mousegrid

File:Mousegrid.png
The "mousegrid" command overlaying numbers on the Windows Vista desktop.

A "mousegrid" command enables users to control the mouse cursor by overlaying numbers across nine regions on the screen; these regions narrow as a user speaks—by number(s)—which region to focus on until they reach a desired interface element to interact with. Entire regions can be interacted with by speaking "click number of region," which moves the mouse cursor to the desired region and then clicks it; an individual item within a region, such as a computer icon, can also be selected by speaking "mark number of region" where the item appears. A user can then specify, if desired, where to move the marked item by speaking "click number of region." These commands also work for multiple regions.[36]

Show numbers

File:Show numbers.png
The "show numbers" command overlaying numbers on user interface elements in the Games Explorer.

Applications and operating system user interface elements that do not present obvious commands can still be controlled by asking the system to overlay numbers on top of them through a "show numbers" command. Once active, speaking the overlaid number selects that item so a user can open it or perform other operations.[36]

Dictation

Windows Speech Recognition enables dictation of text in the operating system and applications. For applications that do not automatically support dictation, an option to enable dictation everywhere is available.[46] If a mistake in dictation occurs, a user can correct the mistake by saying "correct word" and an alternate panel interface will appear and provide suggestions for correction; these suggestions can be selected by speaking the number corresponding to the number of the suggestion in the list and by saying "OK." If the desired word is not listed among the suggestions included in the dialog, a user can speak the desired word so that it may appear within the interface. In the event that this is not successful, a "spell it" command allows a user to speak the desired word on a per-letter basis so that may appear among suggestions. Multiple words in a sentence may also be corrected at a time. As an example, if a user states "dictating" but speech recognition recognizes this word as "the thing," a user can state "correct the thing" to correct both words.

Speech dictionary

WSR includes a personal dictionary that allows users to include or exclude certain words or expressions from being dictated.[47] By default, this dictionary includes over 100,000 words in the English language.[48] When a user adds a word beginning with a capital letter to the dictionary, a user can specify whether it should always be capitalized during dictation or if capitalization depends on the context where the word is spoken; users may also record pronunciations for words added to the dictionary to increase recognition accuracy; words written via a stylus on a tablet PC for the Windows handwriting recognition feature are also stored. Most of the information stored within a dictionary is included as part of a user's speech profile.[37]

Macros

File:WSRMacroOptions.png
An Aero Wizard interface displaying options to create speech recognition macros.

Windows Speech Recognition supports custom macros through a separate utility released by Microsoft which extends the component's functionality by enabling the use of commands that are further based on natural language processing.[49][50] As an example of this functionality, an e-mail macro released by Microsoft enables a natural language command where a user can state "send e-mail to contact about subject," which opens Microsoft Outlook to compose a new message with the designated contact and subject automatically inserted within the application.[51] Microsoft has also released sample macros for the speech dictionary,[52] for Windows Media Player,[53] for Microsoft PowerPoint,[54] for speech synthesis,[55] to switch between multiple microphones,[56] to customize various aspects of audio device configuration such as volume levels,[57] and for general natural language queries such as "what is the weather forecast?"[58] "what time is it?"[55] and "what's the date?"[55] Answers to these queries are spoken to the user via a speech synthesizer.

Users and developers can create their own custom macros that can be based on text transcription and substitution, program execution (with support for command-line arguments), keyboard shortcuts, emulation of existing voice commands, or a combination of these items. XML, JScript and VBScript are supported,[59] macros can be limited to individual applications if desired,[60] and rules for macros can be defined programmatically.[51] In order for a macro to be loaded, it must be stored within a Speech Macros folder within the current user's Documents directory. By default, all macros are digitally signed if a user certificate is available in order to ensure that created commands are not loaded or tampered with by third-parties; if one is not available, an administrator can create a certificate for use.[61] The macros utility also includes security levels to prohibit unsigned macros from being loaded, to prompt users to sign macros, and to load unsigned macros if a user desires for this to occur.[60]

See also

References

  1. ↑ 1.0 1.1 1.2 1.3 1.4 Lua error in package.lua at line 80: module 'strict' not found.
  2. ↑ 2.0 2.1 2.2 Lua error in package.lua at line 80: module 'strict' not found.
  3. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  4. ↑ 4.0 4.1 4.2 Lua error in package.lua at line 80: module 'strict' not found.
  5. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  6. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  7. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  8. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  9. ↑ 9.0 9.1 Lua error in package.lua at line 80: module 'strict' not found.
  10. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  11. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  12. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  13. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  14. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  15. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  16. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  17. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  18. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  19. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  20. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  21. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  22. ↑ 22.0 22.1 Lua error in package.lua at line 80: module 'strict' not found.
  23. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  24. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  25. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  26. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  27. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  28. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  29. ↑ 29.0 29.1 Lua error in package.lua at line 80: module 'strict' not found.
  30. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  31. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  32. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  33. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  34. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  35. ↑ 35.0 35.1 Lua error in package.lua at line 80: module 'strict' not found.
  36. ↑ 36.0 36.1 36.2 36.3 36.4 36.5 36.6 36.7 36.8 Lua error in package.lua at line 80: module 'strict' not found.
  37. ↑ 37.0 37.1 37.2 Lua error in package.lua at line 80: module 'strict' not found.
  38. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  39. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  40. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  41. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  42. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  43. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  44. ↑ 44.0 44.1 Lua error in package.lua at line 80: module 'strict' not found.
  45. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  46. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  47. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  48. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  49. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  50. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  51. ↑ 51.0 51.1 Lua error in package.lua at line 80: module 'strict' not found.
  52. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  53. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  54. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  55. ↑ 55.0 55.1 55.2 Lua error in package.lua at line 80: module 'strict' not found.
  56. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  57. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  58. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  59. ↑ Lua error in package.lua at line 80: module 'strict' not found.
  60. ↑ 60.0 60.1 Lua error in package.lua at line 80: module 'strict' not found.
  61. ↑ Lua error in package.lua at line 80: module 'strict' not found.

External links

Retrieved from "https://infogalactic.com/w/index.php?title=Windows_Speech_Recognition&oldid=716297497"