Speech recognition and generation

Speech generation (synthesis) produces spoken output from text or stored data. Speech recognition converts a spoken signal into text or commands. The two are inverse processes and are the basis of voice user interfaces.

Speech generation

1. Time-dependent sound concatenation. Short recorded speech units (phonemes, diphones, syllables or words) are stored and joined in sequence. Joins are smoothed so there are no clicks. Quality is natural but needs a large database and care at the joints (co-articulation).
2. Frequency-dependent sound concatenation (formant synthesis). The vocal tract is modelled by resonance filters. Parameters such as fundamental frequency (pitch) and formants (resonances) are generated for each sound and fed to a filter. It needs little storage but sounds more artificial.

 Text -> Text analysis -> Phoneme/prosody -> Sound
         (numbers,         (pitch, duration)   synthesis
          abbreviations)                         |
                                              Speech

Steps of text-to-speech: text normalisation, conversion to phonemes, adding prosody (stress, intonation, rhythm), then generating the waveform.

Speech recognition

 Speech -> Feature   -> Compare with  -> Recognised
 signal    extraction    reference         word/text
           (spectrum)    patterns +
                         language rules

1. Sampling and pre-processing: the signal is digitised and noise is reduced.
2. Feature extraction: the signal is cut into short frames (about 20 ms) and features such as spectrum or cepstral coefficients are computed.
3. Pattern matching: features are compared with stored reference patterns or statistical models (templates, Hidden Markov Models, neural networks) to find the closest phoneme or word.
4. Linguistic analysis: grammar and word probabilities (syntax and semantics) are used to choose the best sentence.

Recognition may be speaker-dependent (trained to one voice, more accurate) or speaker-independent, and isolated word or continuous speech. Difficulties are noise, accents, speed of speaking and similar-sounding words.