{
  "title": "Speech recognition: Art of the possible",
  "sections": [
    {
      "id": "section-1",
      "title": "Speech recognition: Art of the possible",
      "level": 2,
      "headingHtml": "Slide 1",
      "html": "<h3>Speech recognition: Art of the possible</h3><p>Dominik.Lukes@ctl.ox.ac.uk @techczech</p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=1\">View original slide 1</a></p>"
    },
    {
      "id": "section-2",
      "title": "Dominik’s journey",
      "level": 2,
      "headingHtml": "Slide 2",
      "html": "<h3>Dominik’s journey</h3><p><strong>1990–1995</strong></p><ul><li>Computational linguistics</li><li>Cognitive linguistics</li><li>Language teaching</li></ul><p><strong>1995–2008</strong></p><ul><li>Language teacher training</li><li>Translation</li><li>Metaphor / discourse studies</li></ul><p><strong>2009 – present</strong></p><ul><li>Readability</li><li>Learning / Assistive technology</li><li>Dyslexia teacher training</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=2\">View original slide 2</a></p>"
    },
    {
      "id": "section-3",
      "title": "Bill Gates in 2011",
      "level": 2,
      "headingHtml": "Slide 3",
      "html": "<h3>Bill Gates in 2011</h3><blockquote><p>“The next big thing is definitely speech and voice recognition.”</p></blockquote><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-5dfa3365e4c5cea2.png\" alt=\"Photograph of Bill Gates speaking, beside a large quotation mark.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=3\">View original slide 3</a></p>"
    },
    {
      "id": "section-4",
      "title": "What do we want to know?",
      "level": 2,
      "headingHtml": "Slide 4",
      "html": "<h3>What do we want to know?</h3><ul><li>What is the current state of the art?</li><li>How we got here?</li><li>Where are going?</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=4\">View original slide 4</a></p>"
    },
    {
      "id": "section-5",
      "title": "Are we asking the right questions?",
      "level": 2,
      "headingHtml": "Slide 5",
      "html": "<h3>Are we asking the right questions?</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=5\">View original slide 5</a></p>"
    },
    {
      "id": "section-6",
      "title": "Tasks for speech recognition by difficulty",
      "level": 2,
      "headingHtml": "Slide 6",
      "html": "<h3>Tasks for speech recognition by difficulty</h3><ol><li>Select word from list</li><li>Interpret command</li><li>Type dictation</li><li>Transcribe presentation</li><li>Transcribe conversation</li></ol><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-06.png\" alt=\"A rising curved arrow with five points from easiest to hardest: Select word from list, Interpret command, Type dictation, Transcribe presentation, Transcribe conversation, each with a small icon.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=6\">View original slide 6</a></p>"
    },
    {
      "id": "section-7",
      "title": "How we think of it vs how it is",
      "level": 2,
      "headingHtml": "Slide 7",
      "html": "<h3>How we think of it vs how it is</h3><ul><li>Select word from list</li><li>Interpret command</li><li>Type dictation</li><li>Transcribe presentation</li><li>Transcribe conversation</li></ul><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-07.png\" alt=\"Two diagrams of the same five tasks. On the left, “How we think of it”: nested circles, with Select word from list innermost, then Interpret command, Type dictation, Transcribe presentation and Transcribe conversation outermost. On the right, “how it is”: five separate, overlapping circles of different sizes with the same labels and a few small empty circles around them.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=7\">View original slide 7</a></p>"
    },
    {
      "id": "section-8",
      "title": "Speech recognition approximate timeline",
      "level": 2,
      "headingHtml": "Slide 8",
      "html": "<h3>Speech recognition approximate timeline</h3><ul><li>1950s: Select digit</li><li>1970s: Select from 1000 words</li><li>1980s: Select from large vocabulary</li><li>1990s: Dictate word by word</li><li>1997: Dictate whole sentences</li><li>2012: Transcribe YouTube video</li><li>2019: Transcribe conversation</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=8\">View original slide 8</a></p>"
    },
    {
      "id": "section-9",
      "title": "What is the actual job of speech recognition?",
      "level": 2,
      "headingHtml": "Slide 9",
      "html": "<h3>What is the actual job of speech recognition?</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=9\">View original slide 9</a></p>"
    },
    {
      "id": "section-10",
      "title": "What is this word?",
      "level": 2,
      "headingHtml": "Slide 10",
      "html": "<h3>What is this word?</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-10eea7b77cd7c477.png\" alt=\"A narrow phonetic transcription of the word, with aspiration, a devoiced and labialised r, a glottal stop and a syllabic dark l.\" loading=\"lazy\"></p><p><code>[pʰɹɛtsɫ̩]</code></p><p><code>[pɹɛtsl]</code></p><p><code>/pretsəl/</code></p><p><code>&lt;pretzel&gt;</code></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=10\">View original slide 10</a></p>"
    },
    {
      "id": "section-11",
      "title": "What’s the problem",
      "level": 2,
      "headingHtml": "Slide 11",
      "html": "<h3>What’s the problem</h3><ul><li>aspirated /p/ at start of a stressed syllable</li><li>devoiced /r/ following /p/</li><li>labialised /r/ following /p/</li><li>dark /l/</li><li>syllabic consonant</li><li>glottal stop</li></ul><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-11.png\" alt=\"The narrow phonetic transcription of “pretzel” with six red arrows pointing at its parts, labelled aspirated /p/ at start of a stressed syllable, devoiced /r/ following /p/, labialised /r/ following /p/, dark /l/, syllabic consonant and glottal stop.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=11\">View original slide 11</a></p>"
    },
    {
      "id": "section-12",
      "title": "It gets worse: find the missing sounds",
      "level": 2,
      "headingHtml": "Slide 12",
      "html": "<h3>It gets worse: find the missing sounds</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-3364a4f07219785a.png\" alt=\"A waveform above a spectrogram of a spoken phrase, with a red cursor line. Beside each is a small loudspeaker icon for an audio clip; the audio is not part of this package.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=12\">View original slide 12</a></p>"
    },
    {
      "id": "section-13",
      "title": "Course on speech recognition 1993",
      "level": 2,
      "headingHtml": "Slide 13",
      "html": "<h3>Course on speech recognition 1993</h3><blockquote><p>Faster computers won’t help improve speech recognition. We need a new approach.</p></blockquote><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=13\">View original slide 13</a></p>"
    },
    {
      "id": "section-14",
      "title": "Dragon Naturally Speaking released in 1997. Can recognise whole sentences. What happened?",
      "level": 2,
      "headingHtml": "Slide 14",
      "html": "<h3>Dragon Naturally Speaking released in 1997. Can recognise whole sentences. What happened?</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=14\">View original slide 14</a></p>"
    },
    {
      "id": "section-15",
      "title": "How speech recognition does not work?",
      "level": 2,
      "headingHtml": "Slide 15",
      "html": "<h3>How speech recognition does not work?</h3><p>Finding individual sounds (phonemes) in the speech and matching them to letters.</p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=15\">View original slide 15</a></p>"
    },
    {
      "id": "section-16",
      "title": "How speech recognition actually works?",
      "level": 2,
      "headingHtml": "Slide 16",
      "html": "<h3>How speech recognition actually works?</h3><p>P(W|C)</p><p>What is the likelihood that the next word is X given what came before?</p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=16\">View original slide 16</a></p>"
    },
    {
      "id": "section-17",
      "title": "Actually, it is quite a bit more complicated (Huang and Deng 2009)",
      "level": 2,
      "headingHtml": "Slide 17",
      "html": "<h3>Actually, it is quite a bit more complicated (Huang and Deng 2009)</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-5b291982e5d2f7a4.png\" alt=\"Equation: W-hat equals the arg max over W of P(W given A), which equals the arg max over W of P(W) times P(A given W), divided by P(A).\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-eae855b469738282.png\" alt=\"Block diagram of a speech recogniser: Voice feeds Signal processing, which feeds a Decoder; the Decoder connects to an Adaptation block, to Acoustic models and Language models, and to the Application.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=17\">View original slide 17</a></p>"
    },
    {
      "id": "section-18",
      "title": "Probabilistic (stochastic) ASR enabled the change. Linguistics took the back seat.",
      "level": 2,
      "headingHtml": "Slide 18",
      "html": "<h3>Probabilistic (stochastic) ASR enabled the change. Linguistics took the back seat.</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=18\">View original slide 18</a></p>"
    },
    {
      "id": "section-19",
      "title": "Fred Jelinek (ASR Pioneer - 1988?)",
      "level": 2,
      "headingHtml": "Slide 19",
      "html": "<h3>Fred Jelinek (ASR Pioneer - 1988?)</h3><blockquote><p>\"Every time I fire a linguist, the performance of the speech recognizer goes up\"</p></blockquote><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=19\">View original slide 19</a></p>"
    },
    {
      "id": "section-20",
      "title": "Consequence of probabilistic approach: Worse on words not predictable from context",
      "level": 2,
      "headingHtml": "Slide 20",
      "html": "<h3>Consequence of probabilistic approach: Worse on words not predictable from context</h3><ul><li>Names</li><li>Acronyms</li><li>Specialist Terms</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=20\">View original slide 20</a></p>"
    },
    {
      "id": "section-21",
      "title": "Question in 2011",
      "level": 2,
      "headingHtml": "Slide 21",
      "html": "<h3>Question in 2011</h3><blockquote><p>I recorded a lecture, can I use Dragon to transcribe it?</p></blockquote><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=21\">View original slide 21</a></p>"
    },
    {
      "id": "section-22",
      "title": "“Caption fails” in 2014 provided source for comedy",
      "level": 2,
      "headingHtml": "Slide 22",
      "html": "<h3>“Caption fails” in 2014 provided source for comedy</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-e2ee75a7b739a36a.png\" alt=\"Still frame from a comedy video of a man in a teal T-shirt, with a poorly recognised caption beneath him.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-5f203bea05d50b92.png\" alt=\"Second frame of the same video: the man holding a phone to his ear, with a caption that misrecognises his words.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-92558570929adc89.png\" alt=\"Third frame of the same video with a mis-captioned line.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-e784cf6621bc5d1b.png\" alt=\"Fourth frame of the same video, partly overlapped, with another mis-captioned line.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=22\">View original slide 22</a></p>"
    },
    {
      "id": "section-23",
      "title": "YouTube Captions today are usable and useful",
      "level": 2,
      "headingHtml": "Slide 23",
      "html": "<h3>YouTube Captions today are usable and useful</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-23.png\" alt=\"Original slide 23 as rendered: the poster frame of a screen recording of a YouTube video with automatic captions, in front of a news article headlined “Dua Lipa sued by reggae band claiming ‘Levitating’ ripped off their song”; the caption reads “alipa”. The video itself is not part of this package.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=23\">View original slide 23</a></p>"
    },
    {
      "id": "section-24",
      "title": "So what happened between 2014 and 2022?",
      "level": 2,
      "headingHtml": "Slide 24",
      "html": "<h3>So what happened between 2014 and 2022?</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=24\">View original slide 24</a></p>"
    },
    {
      "id": "section-25",
      "title": "Ingredients of success",
      "level": 2,
      "headingHtml": "Slide 25",
      "html": "<h3>Ingredients of success</h3><ul><li>Larger data sets</li><li>More computing power</li><li>Neural networks</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=25\">View original slide 25</a></p>"
    },
    {
      "id": "section-26",
      "title": "Patrick Winston (2015) MIT Lecture 12a in AI course",
      "level": 2,
      "headingHtml": "Slide 26",
      "html": "<h3>Patrick Winston (2015) MIT Lecture 12a in AI course</h3><blockquote><p>It was in <strong>2010</strong>, yes, that's right. It was in <strong>2010</strong>. We were having our annual discussion about what we would dump from 6034 in order to make room for some other stuff. And <strong>we almost killed off neural nets.</strong> That might seem strange because our heads are stuffed with neurons. … But many of us felt that the <strong>neural models of the day weren't much in the way of faithful models</strong> of what actually goes on inside our heads. And besides that, <strong>nobody had ever made a neural net that was worth a darn for doing anything</strong>.</p></blockquote><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=26\">View original slide 26</a></p>"
    },
    {
      "id": "section-27",
      "title": "2012 – ImageNet showed that Neural Networks are much better at computing the probabilities for co…",
      "level": 2,
      "headingHtml": "Slide 27",
      "html": "<h3>2012 – ImageNet showed that Neural Networks are much better at computing the probabilities for complex data.</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=27\">View original slide 27</a></p>"
    },
    {
      "id": "section-28",
      "title": "Ok, we have neural nets, what does that mean?",
      "level": 2,
      "headingHtml": "Slide 28",
      "html": "<h3>Ok, we have neural nets, what does that mean?</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=28\">View original slide 28</a></p>"
    },
    {
      "id": "section-29",
      "title": "Things to know about Neural Nets",
      "level": 2,
      "headingHtml": "Slide 29",
      "html": "<h3>Things to know about Neural Nets</h3><ul><li>Everything has a probability</li><li>Same input does not produce same output</li><li>They have no ‘sanity check’ or ‘common sense’</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=29\">View original slide 29</a></p>"
    },
    {
      "id": "section-30",
      "title": "What do probabilities look like?",
      "level": 2,
      "headingHtml": "Slide 30",
      "html": "<h3>What do probabilities look like?</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-8c76f5c5170c8399.jpeg\" alt=\"Diagram of an image classifier: a photograph of a white car feeds columns of convolution (CONV), RELU and POOL layers, then a fully connected layer (FC) with scores for car (highest), truck, airplane, ship and horse.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=30\">View original slide 30</a></p>"
    },
    {
      "id": "section-31",
      "title": "BERT Examples",
      "level": 2,
      "headingHtml": "Slide 31",
      "html": "<h3>BERT Examples</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-c0d7642cf2c441ad.png\" alt=\"A table of fill-in-the-blank contexts and the words BERT-large predicts: “A robin is a ” gives bird, robin, person, hunter, pigeon; “A daisy is a ” gives daisy, rose, flower, berry, tree; “A hammer is a ” gives hammer, tool, weapon, nail, device; “A hammer is an ” gives object, instrument, axe, implement, explosive. For the negated contexts, “A robin is not a ” gives robin, bird, penguin, man, fly; “A daisy is not a ” gives daisy, rose, flower, lily, cherry; “A hammer is not a ” gives hammer, tool, weapon, gun, rock; “A hammer is not an ” gives object, instrument, axe, animal, artifact.\" loading=\"lazy\"></p><p><strong>What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models</strong></p><p><a href=\"https://arxiv.org/search/cs?searchtype=author&amp;query=Ettinger%2C+A\" target=\"_blank\" rel=\"noopener noreferrer\">Allyson Ettinger</a> 2019</p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=31\">View original slide 31</a></p>"
    },
    {
      "id": "section-32",
      "title": "XKCD on prediction",
      "level": 2,
      "headingHtml": "Slide 32",
      "html": "<h3>XKCD on prediction</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-32fbf9366b4bb19e.png\" alt=\"A comic strip: a sign reads “OH MY GOD, THE VOLCANO IS ERU_”, and three people look at it; one says “ERUDITE?”.\" loading=\"lazy\"></p><p><a href=\"https://what-if.xkcd.com/34\" target=\"_blank\" rel=\"noopener noreferrer\">https://what-if.xkcd.com/34</a></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=32\">View original slide 32</a></p>"
    },
    {
      "id": "section-33",
      "title": "Output changes as more information is made available. (Not always for the better)",
      "level": 2,
      "headingHtml": "Slide 33",
      "html": "<h3>Output changes as more information is made available. (Not always for the better)</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=33\">View original slide 33</a></p>"
    },
    {
      "id": "section-34",
      "title": "Examples from today’s captions",
      "level": 2,
      "headingHtml": "Slide 34",
      "html": "<h3>Examples from today’s captions</h3><ul><li>Crystal &gt; Chris is</li><li>Am &gt; and</li></ul><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-26208484bf781852.png\" alt=\"Caption screenshot in white on black: “conversation with some incredible experience galaxy brained individuals working at the forefront of SPeech”.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-7b23d377dd568e8f.png\" alt=\"Caption screenshot with the word “experts” highlighted: “with some incredible experts galaxy brained individuals working at the forefront of speech recognition, research, and that will be very much a discussion again lots of opportunity for questions.”\" loading=\"lazy\"></p><ul><li>experts &gt; experience</li><li>AR &gt; a our</li></ul><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-a1fa0e96555cfb93.png\" alt=\"Caption screenshot with the words “a our” highlighted: “On this slide is an image of the the holoLens so microsoft’s a our products that commercial device.”\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=34\">View original slide 34</a></p>"
    },
    {
      "id": "section-35",
      "title": "Different ways of transcribing Dua Lipa",
      "level": 2,
      "headingHtml": "Slide 35",
      "html": "<h3>Different ways of transcribing Dua Lipa</h3><ul><li>alipa</li><li>dualipa</li><li>dua lipa</li><li>lipa</li><li>duda lipa</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=35\">View original slide 35</a></p>"
    },
    {
      "id": "section-36",
      "title": "Rise and mostly fall of Google’s new spell Czech",
      "level": 2,
      "headingHtml": "Slide 36",
      "html": "<h3>Rise and mostly fall of Google’s new spell Czech</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-3db53ac13da4435d.png\" alt=\"Screenshot of Czech text “Před nějakým … Merleau-Pontiho o tom, že pro” with the name Merleau-Pontiho underlined in red and a pop-up suggesting “Pontyho”.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-7c8e2110f723047f.png\" alt=\"Screenshot of an English sentence, “what you can do with the language for the meet your communicative goals.”, with a pop-up suggesting “to”.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-1ad0f4a8be8d4274.png\" alt=\"Screenshot of the Czech text “-Pontyho doporučoval i” with “Pontyho” underlined in red.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=36\">View original slide 36</a></p>"
    },
    {
      "id": "section-37",
      "title": "Tracking faces at the tips of the shoes",
      "level": 2,
      "headingHtml": "Slide 37",
      "html": "<h3>Tracking faces at the tips of the shoes</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-8635b6c76e207454.jpeg\" alt=\"Photograph of a seated man seen from behind, looking at a wall of video-call participants, with a tracking camera in the foreground.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-24f7d8e43d10ba95.png\" alt=\"Close-up of a conference camera on a stand.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=37\">View original slide 37</a></p>"
    },
    {
      "id": "section-38",
      "title": "Hallucination is a big problem",
      "level": 2,
      "headingHtml": "Slide 38",
      "html": "<h3>Hallucination is a big problem</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-de506c74f8fd753c.png\" alt=\"A Wikipedia infobox for Constant Vanden Stock (birth date October 30, 1914; death date April 19, 2008, aged 93) beside the neural baseline output “Constant Vanden Stock (June 13, 1914 – April 19, 2008) was an American Figure Skater”, with “American Figure Skater” highlighted in red.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=38\">View original slide 38</a></p>"
    },
    {
      "id": "section-39",
      "title": "Question asked by faculty member in 2021",
      "level": 2,
      "headingHtml": "Slide 39",
      "html": "<h3>Question asked by faculty member in 2021</h3><blockquote><p>We correct the transcripts, why doesn’t the system learn the correct spelling?</p></blockquote><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=39\">View original slide 39</a></p>"
    },
    {
      "id": "section-40",
      "title": "Adding your own word list just tweaks the probabilities.",
      "level": 2,
      "headingHtml": "Slide 40",
      "html": "<h3>Adding your own word list just tweaks the probabilities.</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=40\">View original slide 40</a></p>"
    },
    {
      "id": "section-41",
      "title": "Setting a genre setting tweaks the probabilities.",
      "level": 2,
      "headingHtml": "Slide 41",
      "html": "<h3>Setting a genre setting tweaks the probabilities.</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=41\">View original slide 41</a></p>"
    },
    {
      "id": "section-42",
      "title": "Another thing to know about NN",
      "level": 2,
      "headingHtml": "Slide 42",
      "html": "<h3>Another thing to know about NN</h3><p>Neural Nets use very large data sets and can take days or weeks to train.</p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=42\">View original slide 42</a></p>"
    },
    {
      "id": "section-43",
      "title": "Consequences of NN size",
      "level": 2,
      "headingHtml": "Slide 43",
      "html": "<h3>Consequences of NN size</h3><ul><li>Speech recognition is often not done on device.</li><li>Individual input often cannot adjust the quality (except in pre-training)</li><li>Most applications use APIs from the big players</li><li>Few open source/free options</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=43\">View original slide 43</a></p>"
    },
    {
      "id": "section-44",
      "title": "Big players in the field",
      "level": 2,
      "headingHtml": "Slide 44",
      "html": "<h3>Big players in the field</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-4c00ded3407890ba.png\" alt=\"Google logo.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-c7641587e0950641.png\" alt=\"Microsoft logo.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-a01dfcfe753069fa.png\" alt=\"Amazon logo.\" loading=\"lazy\"></p><ul><li>Google</li><li>Microsoft (now also Nuance)</li><li>Amazon</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=44\">View original slide 44</a></p>"
    },
    {
      "id": "section-45",
      "title": "Interesting smaller companies",
      "level": 2,
      "headingHtml": "Slide 45",
      "html": "<h3>Interesting smaller companies</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-f15678cb425ed77e.png\" alt=\"Verbit logo.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-0a00406d9cb3cd06.png\" alt=\"Carescribe logo: a teal speech bubble with three dots beside a teal page of text lines.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-cd0be0c4b7e0d392.png\" alt=\"Otter.ai logo: a blue circle and three blue bars above the word Otter.ai.\" loading=\"lazy\"></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-a167d4696c91a6df.png\" alt=\"Rev.ai logo.\" loading=\"lazy\"></p><ul><li>Verbit.ai</li><li>Carescribe.io (Caption.Ed)</li><li>Otter.ai</li><li>Rev.ai</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=45\">View original slide 45</a></p>"
    },
    {
      "id": "section-46",
      "title": "Interesting applications",
      "level": 2,
      "headingHtml": "Slide 46",
      "html": "<h3>Interesting applications</h3><ul><li>Descript</li><li>Microsoft Reading Progress</li><li>Microsoft Presentation Coach</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=46\">View original slide 46</a></p>"
    },
    {
      "id": "section-47",
      "title": "What can we expect in the future",
      "level": 2,
      "headingHtml": "Slide 47",
      "html": "<h3>What can we expect in the future</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=47\">View original slide 47</a></p>"
    },
    {
      "id": "section-48",
      "title": "Cautionary tale by SMBC",
      "level": 2,
      "headingHtml": "Slide 48",
      "html": "<h3>Cautionary tale by SMBC</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-48.png\" alt=\"A comic by SMBC, “How AI reporting works”: a man says a computer using scanned human literature can make a novel statement about life, a grey creature answers with a speech bubble full of scribbles, six repeated panels show it scribbling, and the last panel has a newspaper headline “MACHINE INTELLIGENCE COMPREHENDS HUMAN DESIRES!”\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=48\">View original slide 48</a></p>"
    },
    {
      "id": "section-49",
      "title": "Which Roomba is which? Video comparing 1st and latest.",
      "level": 2,
      "headingHtml": "Slide 49",
      "html": "<h3>Which Roomba is which? Video comparing 1st and latest.</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-49.png\" alt=\"Original slide 49 as rendered: a frame of a video from above of two robot vacuum cleaners working on a table scattered with crumbs. The video itself is not part of this package.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=49\">View original slide 49</a></p>"
    },
    {
      "id": "section-50",
      "title": "The Original Roomba (2002) vs Roomba S9+ (2019) - Wow!",
      "level": 2,
      "headingHtml": "Slide 50",
      "html": "<h3>The Original Roomba (2002) vs Roomba S9+ (2019) - Wow!</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-ea1b1671c8bad14d.jpeg\" alt=\"A video thumbnail with the words “2002 VS 2019” above two robot vacuum cleaners, an old pale one marked Roomba and a newer dark one, with curved blue arrows pointing at each.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=50\">View original slide 50</a></p>"
    },
    {
      "id": "section-51",
      "title": "What happens in speeches",
      "level": 2,
      "headingHtml": "Slide 51",
      "html": "<h3>What happens in speeches</h3><p><strong>Fillers</strong></p><p><strong>Repetition</strong></p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-51.png\" alt=\"Two transcript screenshots. On the left, headed Fillers, a transcript search for “you know” with matches highlighted and an enlarged line “Kind of like, yeah okay okay so”. On the right, headed Repetition, a transcript search for “to to” with matches highlighted.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=51\">View original slide 51</a></p>"
    },
    {
      "id": "section-52",
      "title": "What does conversation actually look like?",
      "level": 2,
      "headingHtml": "Slide 52",
      "html": "<h3>What does conversation actually look like?</h3><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-17830e69efda6434.jpeg\" alt=\"A conversation-analysis transcript headed “Excerpt 1.1, from Heritage, 1984a: 236, NB:VII:2”: turns marked E and M, with square brackets for overlapping speech, colons for stretched sounds, equals signs for latched speech, parentheses for pauses and unclear talk, and underlining for stress. The full text of the turns is given below.\" loading=\"lazy\"></p><p><strong>EXCERPT 1.1, FROM HERITAGE, 1984A: 236 [NB:VII:2]</strong></p><p><strong>E:</strong> <code>Oh honey that was a lovely luncheon I shoulda ca:lled you s:soo[:ner but I:]l:[lo:ved it.</code></p><p><strong>M:</strong> <code>[((f)) Oh:::]  [(    )</code></p><p><strong>E:</strong> <code>It w’s just deli:ghtfu[:l. ]</code></p><p><strong>M:</strong> <code>[Well]=</code></p><p><strong>M:</strong> <code>I w’s gla[d you] (came).]</code></p><p><strong>E:</strong> <code>[‘nd yer f:] friends] ’re so da:rli:ng,=</code></p><p><strong>M:</strong> <code>=Oh::: [: it w’z ]</code></p><p><strong>E:</strong> <code>[e-that P]a:t isn’she a do:[ :ll?,]</code></p><p><strong>M:</strong> <code>[iYe]h isn’t she pretty,</code></p><p><code>(.)</code></p><p><strong>E:</strong> <code>Oh: she’s a beautiful girl.=</code></p><p><strong>M:</strong> <code>=Yeh I think she’s a pretty gir[l.=</code></p><p><strong>E:</strong> <code>[En’ that Reinam’n::</code></p><p><code>(.)</code></p><p><strong>E:</strong> <code>She SCA:RES me.</code></p><p>The image prints the excerpt on 16 lines, and each turn starts on the left under its speaker label. The overlapping turns are indented so that the overlapping words stand under each other, and some letters are underlined to mark stress. The turns above keep every character and symbol as printed. The column positions and the underlining are recorded in the two tables below, which count the printed lines from the top: line 2 is the second printed line of the first E turn, and lines 11 and 15 hold only a pause.</p><p>Underlined letters. Only the letters named are underlined; the colons that follow them are not.</p><table><thead><tr><th>Line</th><th>Speaker</th><th>Word</th><th>Underlined letters</th></tr></thead><tbody><tr><td>1</td><td>E</td><td>honey</td><td>h, the first letter</td></tr><tr><td>1</td><td>E</td><td>luncheon</td><td>l, the first letter</td></tr><tr><td>1</td><td>E</td><td>ca:lled</td><td>c, the first letter</td></tr><tr><td>2</td><td>E</td><td>the I before the colon in <code>I:]</code></td><td>I</td></tr><tr><td>7</td><td>E</td><td>friends (the f of <code>f:]</code> is not underlined)</td><td>f, the first letter</td></tr><tr><td>7</td><td>E</td><td>da:rli:ng</td><td>i, the letter after <code>rl</code></td></tr><tr><td>8</td><td>M</td><td>Oh, after the <code>=</code></td><td>O and h</td></tr><tr><td>9</td><td>E</td><td>Pa:t</td><td>P and a, as two separate underlines with the <code>]</code> between them</td></tr><tr><td>10</td><td>M</td><td>isn’t</td><td>i, the first letter</td></tr><tr><td>12</td><td>E</td><td>Oh:</td><td>O and h</td></tr><tr><td>13</td><td>M</td><td>I think</td><td>I</td></tr><tr><td>16</td><td>E</td><td>She</td><td>S, h and e</td></tr></tbody></table><p>Overlap brackets. Each position is the distance in pixels from the left edge of the image, which is 728 pixels wide; two brackets are described as lining up when they are within 6 pixels of each other.</p><table><thead><tr><th>Lines</th><th>Brackets and what lines up</th></tr></thead><tbody><tr><td>2 and 3</td><td>E <code>[:ner but I:]</code> and M <code>[((f)) Oh:::]</code> open at the same place (positions 122 and 119). M closes at 234, just after E’s underlined I and before E’s colon; E closes at 247, after that colon.</td></tr><tr><td>2 and 3</td><td>E <code>[lo:ved it.</code> and M <code>[(    )</code> open at the same place (266 and 265). Neither has a closing bracket in the image, so this overlap is left open.</td></tr><tr><td>4 and 5</td><td>E <code>[:l. ]</code> and M <code>[Well]=</code> open at the same place (288 and 289). E closes at 330 and M closes at 348, about one and a half characters to the right of E.</td></tr><tr><td>4 and 6</td><td>M <code>you]</code> on line 6 closes at 331, level with E’s closing bracket on line 4 (330); M’s closing bracket on line 5 (348) falls between them.</td></tr><tr><td>6 to 10</td><td>Five opening brackets stand in one column (154 to 157): M <code>[d</code> on line 6, E <code>[‘nd</code> on line 7, M <code>[:</code> on line 8, E <code>[e-that</code> on line 9 and M <code>[iYe]</code> on line 10.</td></tr><tr><td>7 to 10</td><td>The closing brackets on these lines are at 268 and 364 (E <code>f:]</code> and <code>friends]</code> on line 7), 252 (M <code>w’z ]</code> on line 8), 254 (E <code>P]</code> on line 9) and 198 (M <code>[iYe]</code> on line 10). Only the closers on lines 8 and 9 line up; line 7’s first closer is 14 pixels to their right and line 10’s is 54 pixels to their left.</td></tr><tr><td>6</td><td>M <code>(came).]</code> closes at 434; no bracket on the lines next to it lines up with it.</td></tr><tr><td>9</td><td>E <code>do:[ :ll?,]</code> opens at 451 and closes at 513; no bracket on lines 8 or 10 lines up with either.</td></tr><tr><td>13 and 14</td><td>M <code>gir[l.=</code> and E <code>[En’ that Reinam’n::</code> open at the same place (402 and 407). Neither has a closing bracket in the image, so this last overlap is also left open.</td></tr></tbody></table><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=52\">View original slide 52</a></p>"
    },
    {
      "id": "section-53",
      "title": "Possible futures?",
      "level": 2,
      "headingHtml": "Slide 53",
      "html": "<h3>Possible futures?</h3><p><strong>Incremental improvement</strong>, similar to Roomba in 17 years</p><ul><li>Accurate lecture transcripts</li><li>Fluent dictation with pauses</li><li>Better meeting transcription</li></ul><p><strong>Revolutionary change</strong>, similar to change in speech recognition in 6 years</p><ul><li>Informal conversation transcription</li><li>Interactive dictation</li><li>Multilingual speech transcription</li></ul><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-53.png\" alt=\"Two large arrows pointing outwards from the middle: on the left Incremental improvement, with Fluent dictation with pauses, Accurate lecture transcripts and Better meeting transcription; on the right Revolutionary change, with Interactive dictation, Multilingual speech transcription and Informal conversation transcription.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=53\">View original slide 53</a></p>"
    },
    {
      "id": "section-54",
      "title": "How should we think about accuracy?",
      "level": 2,
      "headingHtml": "Slide 54",
      "html": "<h3>How should we think about accuracy?</h3><ul><li>We speak 120-180 words per minute</li><li>99% accurate = 2 errors per minute</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=54\">View original slide 54</a></p>"
    },
    {
      "id": "section-55",
      "title": "From Sept 2014 xkcd.com/1425",
      "level": 2,
      "headingHtml": "Slide 55",
      "html": "<h3>From Sept 2014 xkcd.com/1425</h3><p>Sometimes it is hard to judge how much effort will be needed to solve a seemingly easy problem.</p><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/img-db72921f1a38661a.png\" alt=\"An xkcd comic: “When a user takes a photo, the app should check whether they’re in a national park…” “Sure, easy GIS lookup. Gimme a few hours.” “… and check whether the photo is of a bird.” “I’ll need a research team and five years.” Caption: “In CS, it can be hard to explain the difference between the easy and the virtually impossible.”\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=55\">View original slide 55</a></p>"
    },
    {
      "id": "section-56",
      "title": "Wishlist (a few hours of coding)",
      "level": 2,
      "headingHtml": "Slide 56",
      "html": "<h3>Wishlist (a few hours of coding)</h3><ul><li>Transcripts indicate level of confidence</li><li>Benchmarks for lecture transcripts</li><li>Better manual control of transcripts (like Descript)</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=56\">View original slide 56</a></p>"
    },
    {
      "id": "section-57",
      "title": "Dreamlist (5 years and a research team)",
      "level": 2,
      "headingHtml": "Slide 57",
      "html": "<h3>Dreamlist (5 years and a research team)</h3><ul><li>Multilingual transcription (identify change in language)</li><li>Multimodal transcription (use information from video)</li><li>Raw to readable transcript</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=57\">View original slide 57</a></p>"
    },
    {
      "id": "section-58",
      "title": "Welcome to the panel",
      "level": 2,
      "headingHtml": "Slide 58",
      "html": "<h3>Welcome to the panel</h3><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=58\">View original slide 58</a></p>"
    },
    {
      "id": "section-59",
      "title": "Panel",
      "level": 2,
      "headingHtml": "Slide 59",
      "html": "<h3>Panel</h3><ul><li>Kate Knill, Machine Intelligence Lab, University of Cambridge</li><li>Richard Cave, MND Association (and formerly Google project Euphonia)</li><li>Richard Purcell, Caption.Ed</li><li>Irit Opher, Head of Research at Verbit.ai</li></ul><p><img src=\"/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/media/page-59.png\" alt=\"Four portrait photographs in a row, each above a panellist’s name and affiliation: Kate Knill, Richard Cave, Richard Purcell and Irit Opher.\" loading=\"lazy\"></p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=59\">View original slide 59</a></p>"
    },
    {
      "id": "section-60",
      "title": "Questions",
      "level": 2,
      "headingHtml": "Slide 60",
      "html": "<h3>Questions</h3><ul><li>What is the current state of the art of speech recognition in general and in the transcription of recorded speech in particular?</li><li>What are the current quality metrics and how much do they tell us about suitability of models? Do we need better ones?</li><li>After the big recent jump in performance, are we seeing a plateau with incremental growth or can we expect another step change in quality?</li><li>Where can we see the most innovation? What are the research and development blind spots where more effort is needed?</li><li>What are the currently unsolved problems for which we do not have a solution?</li><li>What is the space for smaller players to innovate in this space? How much do they have to rely on pre-trained models from big providers? Is there space for open source?</li></ul><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=60\">View original slide 60</a></p>"
    },
    {
      "id": "section-61",
      "title": "Creative Commons License – CC BY",
      "level": 2,
      "headingHtml": "Slide 61",
      "html": "<h3>Creative Commons License – CC BY</h3><p>This presentation is licensed under Creative Commons By Attribution license except where otherwise noted.</p><p>Icons and stock images from Microsoft Office 365 creative premium. They cannot be distributed separately from this document.</p><p><a href=\"/speech-recognition-art-of-the-possible-digifest-2022-251369803?mode=originals&amp;page=61\">View original slide 61</a></p>"
    }
  ],
  "pages": [
    {
      "number": 1,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-01.png",
      "text": "Speech recognition: Art of the possible\nDominik.Lukes@ctl.ox.ac.uk @techczech"
    },
    {
      "number": 2,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-02.png",
      "text": "Dominik’s journey\n1990–1995\nComputational linguistics\nCognitive linguistics\nLanguage teaching\n1995–2008\nLanguage teacher training\nTranslation\nMetaphor / discourse studies\n2009 – present\nReadability\nLearning / Assistive technology\nDyslexia teacher training"
    },
    {
      "number": 3,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-03.png",
      "text": "Bill Gates in 2011\n“The next big thing is definitely speech and voice recognition.”"
    },
    {
      "number": 4,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-04.png",
      "text": "What do we want to know?\nWhat is the current state of the art?\nHow we got here?\nWhere are going?"
    },
    {
      "number": 5,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-05.png",
      "text": "Are we asking the right questions?"
    },
    {
      "number": 6,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-06.png",
      "text": "Tasks for speech recognition by difficulty\n1. Select word from list\n2. Interpret command\n3. Type dictation\n4. Transcribe presentation\n5. Transcribe conversation"
    },
    {
      "number": 7,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-07.png",
      "text": "How we think of it vs how it is\nSelect word from list\nInterpret command\nType dictation\nTranscribe presentation\nTranscribe conversation"
    },
    {
      "number": 8,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-08.png",
      "text": "Speech recognition approximate timeline\n1950s: Select digit\n1970s: Select from 1000 words\n1980s: Select from large vocabulary\n1990s: Dictate word by word\n1997: Dictate whole sentences\n2012: Transcribe YouTube video\n2019: Transcribe conversation"
    },
    {
      "number": 9,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-09.png",
      "text": "What is the actual job of speech recognition?"
    },
    {
      "number": 10,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-10.png",
      "text": "What is this word?\n[pʰɹɛtsɫ̩]\n[pɹɛtsl]\n/pretsəl/\n<pretzel>"
    },
    {
      "number": 11,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-11.png",
      "text": "What’s the problem\naspirated /p/ at start of a stressed syllable\ndevoiced /r/ following /p/\nlabialised /r/ following /p/\ndark /l/\nsyllabic consonant\nglottal stop"
    },
    {
      "number": 12,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-12.png",
      "text": "It gets worse: find the missing sounds"
    },
    {
      "number": 13,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-13.png",
      "text": "Course on speech recognition 1993\nFaster computers won’t help improve speech recognition. We need a new approach."
    },
    {
      "number": 14,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-14.png",
      "text": "Dragon Naturally Speaking released in 1997. Can recognise whole sentences. What happened?"
    },
    {
      "number": 15,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-15.png",
      "text": "How speech recognition does not work?\nFinding individual sounds (phonemes) in the speech and matching them to letters."
    },
    {
      "number": 16,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-16.png",
      "text": "How speech recognition actually works?\nP(W|C)\nWhat is the likelihood that the next word is X given what came before?"
    },
    {
      "number": 17,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-17.png",
      "text": "Actually, it is quite a bit more complicated (Huang and Deng 2009)"
    },
    {
      "number": 18,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-18.png",
      "text": "Probabilistic (stochastic) ASR enabled the change. Linguistics took the back seat."
    },
    {
      "number": 19,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-19.png",
      "text": "Fred Jelinek (ASR Pioneer - 1988?)\n\"Every time I fire a linguist, the performance of the speech recognizer goes up\""
    },
    {
      "number": 20,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-20.png",
      "text": "Consequence of probabilistic approach: Worse on words not predictable from context\nNames\nAcronyms\nSpecialist Terms"
    },
    {
      "number": 21,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-21.png",
      "text": "Question in 2011\nI recorded a lecture, can I use Dragon to transcribe it?"
    },
    {
      "number": 22,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-22.png",
      "text": "“Caption fails” in 2014 provided source for comedy"
    },
    {
      "number": 23,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-23.png",
      "text": "YouTube Captions today are usable and useful"
    },
    {
      "number": 24,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-24.png",
      "text": "So what happened between 2014 and 2022?"
    },
    {
      "number": 25,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-25.png",
      "text": "Ingredients of success\nLarger data sets\nMore computing power\nNeural networks"
    },
    {
      "number": 26,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-26.png",
      "text": "Patrick Winston (2015) MIT Lecture 12a in AI course\nIt was in 2010, yes, that's right. It was in 2010. We were having our annual discussion about what we would dump from 6034 in order to make room for some other stuff. And we almost killed off neural nets. That might seem strange because our heads are stuffed with neurons. … But many of us felt that the neural models of the day weren't much in the way of faithful models of what actually goes on inside our heads. And besides that, nobody had ever made a neural net that was worth a darn for doing anything."
    },
    {
      "number": 27,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-27.png",
      "text": "2012 – ImageNet showed that Neural Networks are much better at computing the probabilities for complex data."
    },
    {
      "number": 28,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-28.png",
      "text": "Ok, we have neural nets, what does that mean?"
    },
    {
      "number": 29,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-29.png",
      "text": "Things to know about Neural Nets\nEverything has a probability\nSame input does not produce same output\nThey have no ‘sanity check’ or ‘common sense’"
    },
    {
      "number": 30,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-30.png",
      "text": "What do probabilities look like?"
    },
    {
      "number": 31,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-31.png",
      "text": "BERT Examples\nWhat BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models\nAllyson Ettinger 2019"
    },
    {
      "number": 32,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-32.png",
      "text": "XKCD on prediction\nhttps://what-if.xkcd.com/34"
    },
    {
      "number": 33,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-33.png",
      "text": "Output changes as more information is made available. (Not always for the better)"
    },
    {
      "number": 34,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-34.png",
      "text": "Examples from today’s captions\nCrystal > Chris is\nAm > and\nexperts > experience\nAR > a our"
    },
    {
      "number": 35,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-35.png",
      "text": "Different ways of transcribing Dua Lipa\nalipa\ndualipa\ndua lipa\nlipa\nduda lipa"
    },
    {
      "number": 36,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-36.png",
      "text": "Rise and mostly fall of Google’s new spell Czech"
    },
    {
      "number": 37,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-37.png",
      "text": "Tracking faces at the tips of the shoes"
    },
    {
      "number": 38,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-38.png",
      "text": "Hallucination is a big problem"
    },
    {
      "number": 39,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-39.png",
      "text": "Question asked by faculty member in 2021\nWe correct the transcripts, why doesn’t the system learn the correct spelling?"
    },
    {
      "number": 40,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-40.png",
      "text": "Adding your own word list just tweaks the probabilities."
    },
    {
      "number": 41,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-41.png",
      "text": "Setting a genre setting tweaks the probabilities."
    },
    {
      "number": 42,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-42.png",
      "text": "Another thing to know about NN\nNeural Nets use very large data sets and can take days or weeks to train."
    },
    {
      "number": 43,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-43.png",
      "text": "Consequences of NN size\nSpeech recognition is often not done on device.\nIndividual input often cannot adjust the quality (except in pre-training)\nMost applications use APIs from the big players\nFew open source/free options"
    },
    {
      "number": 44,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-44.png",
      "text": "Big players in the field\nGoogle\nMicrosoft (now also Nuance)\nAmazon"
    },
    {
      "number": 45,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-45.png",
      "text": "Interesting smaller companies\nVerbit.ai\nCarescribe.io (Caption.Ed)\nOtter.ai\nRev.ai"
    },
    {
      "number": 46,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-46.png",
      "text": "Interesting applications\nDescript\nMicrosoft Reading Progress\nMicrosoft Presentation Coach"
    },
    {
      "number": 47,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-47.png",
      "text": "What can we expect in the future"
    },
    {
      "number": 48,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-48.png",
      "text": "Cautionary tale by SMBC"
    },
    {
      "number": 49,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-49.png",
      "text": "Which Roomba is which? Video comparing 1st and latest."
    },
    {
      "number": 50,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-50.png",
      "text": "The Original Roomba (2002) vs Roomba S9+ (2019) - Wow!"
    },
    {
      "number": 51,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-51.png",
      "text": "What happens in speeches\nFillers\nRepetition"
    },
    {
      "number": 52,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-52.png",
      "text": "What does conversation actually look like?\nEXCERPT 1.1, FROM HERITAGE, 1984A: 236 [NB:VII:2]\nE: Oh honey that was a lovely luncheon I shoulda ca:lled you s:soo[:ner but I:]l:[lo:ved it.\nM: [((f)) Oh:::] [( )\nE: It w’s just deli:ghtfu[:l. ]\nM: [Well]=\nM: I w’s gla[d you] (came).]\nE: [‘nd yer f:] friends] ’re so da:rli:ng,=\nM: =Oh::: [: it w’z ]\nE: [e-that P]a:t isn’she a do:[ :ll?,]\nM: [iYe]h isn’t she pretty,\n(.)\nE: Oh: she’s a beautiful girl.=\nM: =Yeh I think she’s a pretty gir[l.=\nE: [En’ that Reinam’n::\n(.)\nE: She SCA:RES me.\nThe image prints the excerpt on 16 lines, and each turn starts on the left under its speaker label. The overlapping turns are indented so that the overlapping words stand under each other, and some letters are underlined to mark stress. The turns above keep every character and symbol as printed. The column positions and the underlining are recorded in the two tables below, which count the printed lines from the top: line 2 is the second printed line of the first E turn, and lines 11 and 15 hold only a pause.\nUnderlined letters. Only the letters named are underlined; the colons that follow them are not.\nLine | Speaker | Word | Underlined letters\n1 | E | honey | h, the first letter\n1 | E | luncheon | l, the first letter\n1 | E | ca:lled | c, the first letter\n2 | E | the I before the colon in I:] | I\n7 | E | friends (the f of f:] is not underlined) | f, the first letter\n7 | E | da:rli:ng | i, the letter after rl\n8 | M | Oh, after the = | O and h\n9 | E | Pa:t | P and a, as two separate underlines with the ] between them\n10 | M | isn’t | i, the first letter\n12 | E | Oh: | O and h\n13 | M | I think | I\n16 | E | She | S, h and e\nOverlap brackets. Each position is the distance in pixels from the left edge of the image, which is 728 pixels wide; two brackets are described as lining up when they are within 6 pixels of each other.\nLines | Brackets and what lines up\n2 and 3 | E [:ner but I:] and M [((f)) Oh:::] open at the same place (positions 122 and 119). M closes at 234, just after E’s underlined I and before E’s colon; E closes at 247, after that colon.\n2 and 3 | E [lo:ved it. and M [( ) open at the same place (266 and 265). Neither has a closing bracket in the image, so this overlap is left open.\n4 and 5 | E [:l. ] and M [Well]= open at the same place (288 and 289). E closes at 330 and M closes at 348, about one and a half characters to the right of E.\n4 and 6 | M you] on line 6 closes at 331, level with E’s closing bracket on line 4 (330); M’s closing bracket on line 5 (348) falls between them.\n6 to 10 | Five opening brackets stand in one column (154 to 157): M [d on line 6, E [‘nd on line 7, M [: on line 8, E [e-that on line 9 and M [iYe] on line 10.\n7 to 10 | The closing brackets on these lines are at 268 and 364 (E f:] and friends] on line 7), 252 (M w’z ] on line 8), 254 (E P] on line 9) and 198 (M [iYe] on line 10). Only the closers on lines 8 and 9 line up; line 7’s first closer is 14 pixels to their right and line 10’s is 54 pixels to their left.\n6 | M (came).] closes at 434; no bracket on the lines next to it lines up with it.\n9 | E do:[ :ll?,] opens at 451 and closes at 513; no bracket on lines 8 or 10 lines up with either.\n13 and 14 | M gir[l.= and E [En’ that Reinam’n:: open at the same place (402 and 407). Neither has a closing bracket in the image, so this last overlap is also left open."
    },
    {
      "number": 53,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-53.png",
      "text": "Possible futures?\nIncremental improvement, similar to Roomba in 17 years\nAccurate lecture transcripts\nFluent dictation with pauses\nBetter meeting transcription\nRevolutionary change, similar to change in speech recognition in 6 years\nInformal conversation transcription\nInteractive dictation\nMultilingual speech transcription"
    },
    {
      "number": 54,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-54.png",
      "text": "How should we think about accuracy?\nWe speak 120-180 words per minute\n99% accurate = 2 errors per minute"
    },
    {
      "number": 55,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-55.png",
      "text": "From Sept 2014 xkcd.com/1425\nSometimes it is hard to judge how much effort will be needed to solve a seemingly easy problem."
    },
    {
      "number": 56,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-56.png",
      "text": "Wishlist (a few hours of coding)\nTranscripts indicate level of confidence\nBenchmarks for lecture transcripts\nBetter manual control of transcripts (like Descript)"
    },
    {
      "number": 57,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-57.png",
      "text": "Dreamlist (5 years and a research team)\nMultilingual transcription (identify change in language)\nMultimodal transcription (use information from video)\nRaw to readable transcript"
    },
    {
      "number": 58,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-58.png",
      "text": "Welcome to the panel"
    },
    {
      "number": 59,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-59.png",
      "text": "Panel\nKate Knill, Machine Intelligence Lab, University of Cambridge\nRichard Cave, MND Association (and formerly Google project Euphonia)\nRichard Purcell, Caption.Ed\nIrit Opher, Head of Research at Verbit.ai"
    },
    {
      "number": 60,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-60.png",
      "text": "Questions\nWhat is the current state of the art of speech recognition in general and in the transcription of recorded speech in particular?\nWhat are the current quality metrics and how much do they tell us about suitability of models? Do we need better ones?\nAfter the big recent jump in performance, are we seeing a plateau with incremental growth or can we expect another step change in quality?\nWhere can we see the most innovation? What are the research and development blind spots where more effort is needed?\nWhat are the currently unsolved problems for which we do not have a solution?\nWhat is the space for smaller players to innovate in this space? How much do they have to rely on pre-trained models from big providers? Is there space for open source?"
    },
    {
      "number": 61,
      "image": "/readers/speech-recognition-art-of-the-possible-digifest-2022-251369803/pages/page-61.png",
      "text": "Creative Commons License – CC BY\nThis presentation is licensed under Creative Commons By Attribution license except where otherwise noted.\nIcons and stock images from Microsoft Office 365 creative premium. They cannot be distributed separately from this document."
    }
  ],
  "pageKind": "PowerPoint slides rendered with LibreOffice · local format pilot"
}