---
title: "How to Teach and Learn with ChatGPT:"
subtitle: "AI and new horizons for assistive technologies"
author: "Dominik Lukes, EduTools.fyi"
outline_version: 2
auto_title_slide: false
auto_thanks_slide: false
---

## How to Teach and Learn with ChatGPT: AI and new horizons for assistive technologies
{id=pptx-27c6e7c544b2-001} {title}

Dominik Lukes, EduTools.fyi

## Lots of developments since we scheduled the session
{id=pptx-27c6e7c544b2-002}

![A rocket icon beside the slide’s statement.](assets/media/img-0e87b9594ee3535e.png)

## What has changed
{id=pptx-27c6e7c544b2-003}

- Bing Chat
- GPT-4
- Google Bard
- Office ‘Co-Pilot’s
- GPT-4 Plugins

## bit.ly/academic-ai-newsletter
{id=pptx-27c6e7c544b2-004}

[https://bit.ly/academic-ai-newsletter](https://bit.ly/academic-ai-newsletter)

![Screenshot of the LinkedIn newsletter “AI in Academic Practice” with a cartoon computer with a puzzled face and question marks, beside a QR code.](assets/pages/page-04.png)

## Poll: What is your experience with ChatGPT?
{id=pptx-27c6e7c544b2-005}

![A raised-hand icon beside the poll question.](assets/media/img-0397ac4c9fdd92a0.png)

## Poll: Experience?
{id=pptx-27c6e7c544b2-006}

- Use it every day
- Played around with it
- Heard a LOT about it
- I heard something about it

## Comparison from a recent talk
{id=pptx-27c6e7c544b2-007}

![A poll result from an earlier talk: a pie chart beside the answers Use ChatGPT in my work (5), Have used once or twice (32), Have heard a lot and seen examples (21), Have heard something but not sure (23) and Other (share in chat) (1).](assets/media/img-03089ab548b5a99a.png)

## Poll: What do you think AI might be best at?
{id=pptx-27c6e7c544b2-008}

![A raised-hand icon beside the poll question.](assets/media/img-0397ac4c9fdd92a0.png)

## What do you think AI might be best at?
{id=pptx-27c6e7c544b2-009}

- Looking up facts
- Performing complex calculations
- Writing computer code based on text
- Making complex decisions from data
- Writing poetry
- Making beautiful art

## Comparison with a recent talk
{id=pptx-27c6e7c544b2-010}

![A horizontal bar chart of poll results for what AI might be best at, listing Looking up facts, Performing complex calculations, Writing computer code based on text, Making complex decisions based on data, Writing poetry and Creating beautiful photos, with a pop-up for “Creating beautiful photos”: 1st choice 2 (2%), 2nd choice 3 (4%), 3rd choice 6 (7%), 4th choice 7 (8%), 5th choice 18 (21%), 6th choice 48 (57%).](assets/media/img-16b83dfda6118351.png)

## What it is actually good at?
{id=pptx-27c6e7c544b2-011}

![A superhero-flying icon beside the question.](assets/media/img-a6cf7f656bfc570d.png)

## Nobody expected AI to be any good at this
{id=pptx-27c6e7c544b2-012}

![Two ChatGPT conversations side by side. Left: the prompt asks for a poem to remember the bones in the hand and the answer is a four-stanza rhyming poem. Right: the prompt asks for a mnemonic for the carpals and the answer is the sentence “She Left The Party To Take Cathy Home” with each carpal bone matched to a word.](assets/pages/page-12.png)

**Prompt (left):** Write a poem to help me remember the bones in the hand.

In the realm of hands, a tale unfolds,  
Of bones and joints, a story told,  
Listen close, my friend, and hear,  
The names of bones, so you'll endear.

Twenty-seven bones, a framework grand,  
Form the structure of our hand,  
Three groups of bones to memorize,  
Carpals, metacarpals, phalanges arise.

Begin with carpals, eight in sum,  
In the wrist, they're not cumbersome,  
Proximal row, four in row,  
From lateral to medial, here's how they go:

Scaphoid, lunate, triquetrum align,  
Pisiform, a small pea, so benign,  
Distal row, to continue the tour,  
Trapezium, trapezoid, capitate, hamate ensure.

**Prompt (right):** Make a mnemonic for the carpals

Recall the carpals, eight in all,  
With this mnemonic, you'll never stall:

She Left The Party  
To Take Cathy Home

- Scaphoid - She
- Lunate - Left
- Triquetrum - The
- Pisiform - Party
- Trapezium - To
- Trapezoid - Take
- Capitate - Cathy
- Hamate - Home

## Or this
{id=pptx-27c6e7c544b2-013}

![A collage of AI-generated pictures: a man sitting beside a hole in a wall, colourful retro robot and computer characters, a smiling person with a tree of leaves for hair, and a robot reading in a hollow.](assets/pages/page-13.png)

## We thought AI could do this
{id=pptx-27c6e7c544b2-014}

![A ChatGPT conversation about the arithmetic “4^2 - 234 * (-23) * 329870134987”: it works through the steps and its answer reads 10825535722618; beside it a calculator shows 1775361066500050.](assets/pages/page-14.png)

## But it is also unexpectedly good at this
{id=pptx-27c6e7c544b2-015}

![A ChatGPT conversation. The prompt is “Write code to add two numbers in python”. The answer explains the + operator and gives a short Python program with an add_numbers function, followed by an explanation of the code.](assets/media/img-61cd9a2a20779e93.png)

**Prompt:** Write code to add two numbers in python

In Python, you can add two numbers by simply using the `+` operator. Here's a simple function to add two numbers:

`def add_numbers(a, b):`

    `return a + b`

`number1 = float(input("Enter the first number: "))`

`number2 = float(input("Enter the second number: "))`

`result = add_numbers(number1, number2)`

`print("The sum of the two numbers is:", result)`

In this code, the `add_numbers` function takes two arguments, `a` and `b`, and returns their sum. The user is prompted to enter two numbers, which are then passed to the function to perform the addition. The result is then printed.

## How do we know what modern AI will be good or bad at?
{id=pptx-27c6e7c544b2-016}

![A puzzle-piece icon beside the question.](assets/media/img-55c2aba161dd1561.png)

## We have evaluate it systematically. Almost as a new organism.
{id=pptx-27c6e7c544b2-017}

![A microscope icon beside the statement.](assets/media/img-ee366269d242127c.png)

## Hofstadter, 1979: Questions
{id=pptx-27c6e7c544b2-018}

**Question:** Will a thinking computer be able to add fast?

![Cover of “Gödel, Escher, Bach: an Eternal Golden Braid” by Douglas R. Hofstadter, orange with a block-letter title.](assets/media/img-1336fbc549ff9bd3.jpeg)

## Hofstadter, 1979, Speculations
{id=pptx-27c6e7c544b2-019}

> **Speculation:** Perhaps not. We ourselves are composed of hardware which does fancy calculations but that doesn't mean that our symbol level, where "we" are, knows how to carry out the same fancy calculations.

## Let’s keep this in mind
{id=pptx-27c6e7c544b2-020}

![A tweet by Sam Altman: “here is GPT-4, our most capable and aligned model yet. it is available today in our API (with a waitlist) and in ChatGPT+. openai.com/research/gpt-4 … it is still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it.”](assets/media/img-9f25bd3f7f44c539.png)

## What ChatGPT can do for YOU?
{id=pptx-27c6e7c544b2-021}

![A light-bulb-in-head icon beside the question.](assets/media/img-37d01dc27e04f77c.png)

## Better at poetry than math still means it can be incredibly useful.
{id=pptx-27c6e7c544b2-022}

## Anyone can use it to help with reading
{id=pptx-27c6e7c544b2-023}

![Screenshots of a university web page with a chat panel: the user selects text and asks the assistant to explain, simplify it with easier language and explain it to a 12-year-old; the assistant’s replies appear in the side panel.](assets/pages/page-23.png)

## A student can use this for help
{id=pptx-27c6e7c544b2-024}

![A ChatGPT conversation in which a student asks the model to explain a line of NumPy array code and it explains each element.](assets/media/img-ad8300321e79657b.png)

## A teacher can use it to create tools
{id=pptx-27c6e7c544b2-025}

![A teacher’s prompt asking for a tool that compares two translations of the same text and a web form with two text boxes below it, with the differences between the two texts highlighted in yellow.](assets/pages/page-25.png)

## There are limits but also, we can use it to assist us in real ways. But we should understand what it is doing.
{id=pptx-27c6e7c544b2-026}

## 5 Things about ChatGPT in 15(ish) Minutes
{id=pptx-27c6e7c544b2-027}

- Dominik.Lukes@ctl.ox.ac.uk

## 5 things about ChatGPT in 15 minutes
{id=pptx-27c6e7c544b2-028}

1. It has many building blocks
2. It does not reason, it predicts
3. It is limited by the context window
4. It has no database, copy or paste or memory
5. It does not learn but you can correct it

## Building blocks of modern AI
{id=pptx-27c6e7c544b2-029}

### What does ChatGPT stand for?
{id=pptx-27c6e7c544b2-030}

- Chat: answers questions rather than continues
- G - generative: creates strings rather than labels
- P - pretrained: can be fine-tuned on much less data
- T - transformer: can be trained to predict from long context

Foundation model

### Building blocks of the AI revolution
{id=pptx-27c6e7c544b2-031}

- Foundation Models
- Products: ChatGPT, Consensus, …
- Features: Notion …, PowerPoint…, …

### Language model universe in 2022
{id=pptx-27c6e7c544b2-032}

![A bubble chart, “Language model sizes to Dec/2022”, of large language models by parameter count: from BERT and GPT-2 to GPT-3 (175 billion), Jurassic-1, MT-NLG (530 billion), PaLM (540 billion) and BLOOM, with the source LifeArchitect.ai/models.](assets/media/img-fdd923e5a326859e.png)

### Building blocks of a product
{id=pptx-27c6e7c544b2-033}

- Foundation Model API
- Fine tuning on custom data
- Finetuning by human feedback
- Custom Workflow / Interface
- Consensus (OpenAI)
- Notion (Anthropic)
- Smart Reply (Google)
- ChatGPT (Open AI)
- Claude (Anthropic)
- Bard (Google)
- TeacherMatic
- LangChain…

![Original slide 33 as rendered: a wide dark blue base bar “Foundation Model API”; above it three orange boxes, Fine tuning on custom data, Finetuning by human feedback and Custom Workflow / Interface; above those a row of eight small green boxes, Consensus (OpenAI), Notion (Anthropic), Smart Reply (Google), ChatGPT (Open AI), Claude (Anthropic), Bard (Google), TeacherMatic and LangChain…](assets/pages/page-33.png)

### For example Consensus (now with GPT-4)
{id=pptx-27c6e7c544b2-034}

![Two screenshots of Consensus, a search tool: a question about whether it is possible to improve working memory, with a summary of the evidence and a list of papers with their findings.](assets/pages/page-34.png)

### ChatGPT is just another product on top of GPT
{id=pptx-27c6e7c544b2-035}

![Two screenshots: on the left, the TeacherMatic tool with a learning objectives generator; on the right, ChatGPT asked to write 10 learning objectives for a beginner course about web design, which it lists.](assets/pages/page-35.png)

### All of these work the same way
{id=pptx-27c6e7c544b2-036}

![Two overlapping news articles: “Introducing Microsoft 365 Copilot—A whole new way to work” and “Google announces AI features in Gmail, Docs, and more to rival Microsoft”.](assets/pages/page-36.png)

### Big API providers
{id=pptx-27c6e7c544b2-037}

- OpenAI
- Anthropic
- Google
- Amazon
- Microsoft
- Meta

### Although there are many smaller ones on HuggingFace
{id=pptx-27c6e7c544b2-038}

![The Hugging Face model listing page with filters on the left and a list of model names such as bert-base-uncased and gpt2.](assets/media/img-940aaee695c85b36.png)

### Thousands API customers
{id=pptx-27c6e7c544b2-039}

![The website “There’s an AI for that”: “2,675 AIs for 603 tasks. Updated daily”, “Just launched” and a grid of small tool cards on a dark page.](assets/media/img-25aadc0873eab158.png)

### And thousands more
{id=pptx-27c6e7c544b2-040}

![The website FutureTools: “FutureTools Collects & Organizes All The Best AI Tools So YOU Too Can Become Superhuman!”, with a search box and a grid of category filters and tool cards.](assets/media/img-f6b5cd10ea4fd651.png)

### Always ask what API a product uses!
{id=pptx-27c6e7c544b2-041}

## It does not reason, it predicts
{id=pptx-27c6e7c544b2-042}

### It is easy to be deceived by appearances.
{id=pptx-27c6e7c544b2-043}

### ChatGPT can do this
{id=pptx-27c6e7c544b2-044}

![Two ChatGPT answers. First, a list of tropical fruits grouped by region, South America and Central America & Mexico. Second, the same fruits in a table with columns Fruit Name, Region of Origin and Definition.](assets/pages/page-44.png)

### How do we solve these kinds of problems?
{id=pptx-27c6e7c544b2-045}

### Cognition = Computation
{id=pptx-27c6e7c544b2-046}

**Logic**

- Compare
- Organise
- Transform

**Memory**

- Store
- Retrieve

### Something like this
{id=pptx-27c6e7c544b2-047}

![A flowchart: Find tropical fruits, Look up fruit in memory, Is it tropical? (No loops back; Yes goes to Write to list).](assets/media/img-02b3425b68ba3856.png)

### How does ChatGPT solve these kinds of problems?
{id=pptx-27c6e7c544b2-048}

### ChatGPT does not have cognition or computation
{id=pptx-27c6e7c544b2-049}

~~**Logic**~~

- Compare
- Organise
- Transform

~~**Memory**~~

- Store
- Retrieve

![Original slide 49 as rendered: the Logic (database icon) and Memory (floppy disk icon) columns of the previous slide, each crossed out with a large red X.](assets/pages/page-49.png)

### ChatGPT Process
{id=pptx-27c6e7c544b2-050}

1. Break all preceding text into tokens
2. Assign each token number
3. Predict next token
4. Repeat with the new token

### What and why of tokens? And why it matters. A lot!
{id=pptx-27c6e7c544b2-051}

![A ticket icon beside the question.](assets/media/img-070b5eedaa0d7f30.png)

### How we see text?
{id=pptx-27c6e7c544b2-052}

**Humans**

- Letters
- Words
- Sentences
- Paragraphs
- Texts

**ChatGPT**

- Tokens

### OpenAI: ChatGPT only sees tokens
{id=pptx-27c6e7c544b2-053}

![Two overlapping screenshots of OpenAI’s Tokenizer: the text “Many words map to one token, but some don’t: indivisible.” with 64 tokens and 252 characters, with each token highlighted in a different colour, and an explanation of how the models see tokens.](assets/pages/page-53.png)

### Actually, this is what ChatGPT sees
{id=pptx-27c6e7c544b2-054}

Unicode

![A line of numbers in square brackets, the token IDs of a text, with a large arrow from the word “Unicode” to the group of token numbers 3118, 291 and 1098 that stand for it.](assets/pages/page-54.png)

### Numbers are not digits, just tokens
{id=pptx-27c6e7c544b2-055}

![Two token-ID displays. The sum “1 + 2 * 3654 = 123,565” with each number and sign coloured as its own token, and “2 + 1 / 43^43.245645”; below them a list of token numbers, with arrows from each number to the tokens it is split into.](assets/pages/page-55.png)

### Fun facts about OpenAI tokens
{id=pptx-27c6e7c544b2-056}

- About 50,000 in total
- Only 3,000 English words
- Most parts of words incl. letters, spaces
- About 140 tokens per 100 English words
- Much more for other languages

### Word and tokens are only accidentally identical
{id=pptx-27c6e7c544b2-057}

![A stack of overlapping tokenizer screenshots comparing one sentence in several languages: the token counts shown are 119, 240, 317, 396 and 523, and the last, longest text is split into many small coloured fragments.](assets/pages/page-57.png)

### Unexpected consequence: More tokens more money
{id=pptx-27c6e7c544b2-058}

- English
- French
- Czech
- Chinese
- Arabic

![A rising curved arrow from English, through French, Czech and Chinese, to Arabic: the more tokens the text needs, the higher the cost.](assets/pages/page-58.png)

### Consequences of tokens
{id=pptx-27c6e7c544b2-059}

- Limited rhyming
- Bad at letter tasks
- Unreliable numbers
- More grammar mistakes in some languages

### It’s not good at this BUT it shouldn’t be able to do this at all.
{id=pptx-27c6e7c544b2-060}

![A ChatGPT conversation. The prompt asks for 10 Czech nouns where ‘ů’ changes to ‘o’ as the noun is declined, with the examples ‘stůl’ - ‘stolu’ and ‘dům’ - ‘domu’. The answer is a numbered list of ten nouns with their declined forms and English glosses. The Czech characters in this image are displayed correctly.](assets/media/img-0fde601a3536d25a.png)

**Prompt:** Give me 10 Czech nouns where ‘ů’ changes to ‘o’ as the noun is declined. For example, ‘stůl’ - ‘stolu’, dům - domu.

Certainly! Here are 10 Czech nouns that follow the pattern of the vowel ‘ů’ changing to ‘o’ as the noun is declined:

1. Kůň - koně (horse)
2. Důl - dolu (mine)
3. Hůl - holí (cane)
4. Stroj - stroje (machine)
5. Můstek - mostky (little bridge)
6. Vůz - vozy (carriage)
7. Sůl - soli (salt)
8. Nůž - nože (knife)
9. Kroužek - kroužky (ringlet)
10. Úkol - úkoly (task)

**Tokeniser view on the original slide:** beside the screenshot, the slide shows the same ten answers as coloured tokens in which every accented letter appears as one or two replacement characters: K���� - kon�� (horse); D��l - dolu (mine); H��l - holí (cane); Stroj - stroje (machine); M��stek - mostky (little bridge); V��z - vozy (carriage); S��l - soli (salt); N���� - no��e (knife); Krou��ek - krou��ky (ringlet); ��kol - ��koly (task).

### BTW: GPT-4 is not much better at this.
{id=pptx-27c6e7c544b2-061}

![A ChatGPT answer about Czech noun declension, listing ten nouns with their singular genitive forms and noting that the list is not exhaustive.](assets/media/img-099b295cf7f64657.png)

### Tokens are not graphics, but they can still produce this.
{id=pptx-27c6e7c544b2-062}

![A ChatGPT conversation: asked to convert a text into a Mermaid diagram, it returns Mermaid code, shown beside the resulting flowchart of the fruit-lookup process.](assets/pages/page-62.png)

### Embeddings: Give tokens meaning
{id=pptx-27c6e7c544b2-063}

![A network-of-nodes icon beside the heading.](assets/media/img-e2ec108c23100701.png)

### What are embeddings?
{id=pptx-27c6e7c544b2-064}

Embeddings show how tokens relate to each other in **multidimensional-dimensional** space.

![Icon of a network of connected nodes.](assets/media/img-21ef9e00675b63e2.png)

### Language embeddings
{id=pptx-27c6e7c544b2-065}

![The illustration “The Illustrated Word2vec” showing word embeddings: a list of 50 numbers for the word “king”, and colour strips for king, man and woman.](assets/pages/page-65.png)

### You do maths on matrices and you get
{id=pptx-27c6e7c544b2-066}

king − man + woman = queen

### But all of this is happening in 100s or 1000s or dimensions.
{id=pptx-27c6e7c544b2-067}

![A scatter plot of word embeddings projected into two dimensions: clouds of small labelled words coloured by cluster.](assets/media/img-87eacc341c63f6fa.png)

### More complicated than simple matrix addition
{id=pptx-27c6e7c544b2-068}

![A Dash word-embeddings app: a network of related words around “california − state + country” and a bar chart of nearest neighbours by cosine similarity.](assets/pages/page-68.png)

### Add it all up
{id=pptx-27c6e7c544b2-069}

![An abacus icon beside the heading.](assets/media/img-d9014f29d8875947.png)

### All it is doing on the inside is multiplying matrices.
{id=pptx-27c6e7c544b2-070}

![Four overlapping copies of an icon of a grid of small squares.](assets/pages/page-70.png)

### Really big ones, for every single token in the context window!
{id=pptx-27c6e7c544b2-071}

![An icon of a large grid, beside the statement.](assets/pages/page-71.png)

### On hundreds or thousands of expensive graphics cards.
{id=pptx-27c6e7c544b2-072}

![Photograph of a red-and-black graphics card with two fans.](assets/media/img-4216c01426002066.png)

### This means it costs $$$ every time you ask it to do something.
{id=pptx-27c6e7c544b2-073}

![An icon of banknotes and coins.](assets/media/img-ef32e62f12a77e1c.png)

### Generate predictions
{id=pptx-27c6e7c544b2-074}

![A line-chart-with-points icon beside the heading.](assets/media/img-03b5d0573360a834.png)

### The we predict what comes next (from Wolfram)
{id=pptx-27c6e7c544b2-075}

![A table of the most likely next words after the sentence beginning “The best thing about AI is its ability to”: learn 4.5%, predict 3.5%, make 3.2%, understand 3.1%, do 2.9%.](assets/media/img-a929996d0506a6aa.png)

### The top item is not always selected
{id=pptx-27c6e7c544b2-076}

![A text generator with a pop-up of next-word probabilities after “I went back over to the library and got a book on Civil War”: house 4.35%, other 2.32%, window 1.47%, hospital 1.26%, old 1.17% and library 0.41%, with “library” highlighted.](assets/pages/page-76.png)

### Turn up the temperature: Randomness
{id=pptx-27c6e7c544b2-077}

![A dice icon beside the heading.](assets/media/img-91ee6a99953201c8.png)

### With heat
{id=pptx-27c6e7c544b2-078}

![Overlapping text-generation screenshots: a settings panel with a temperature slider beside three different continuations of text that begin “I went…”.](assets/pages/page-78.png)

### What it looks like with temperature 1
{id=pptx-27c6e7c544b2-079}

![The same next-word probability pop-up, now headed as the case of temperature 1: after “I went back over to the library and got a book on Civil War” the alternatives are house, other, window, hospital, old and library 0.41%.](assets/pages/page-79.png)

### Without heat
{id=pptx-27c6e7c544b2-080}

![Several overlapping text outputs of “I went to the store to buy some groceries. I bought milk, eggs, bread, and some fruit. I also picked up some snacks for later.”, which are the same each time.](assets/pages/page-80.png)

### What it looks like with temperature 0
{id=pptx-27c6e7c544b2-081}

![A text-generation display for “I went to the store”: the next word “store” has probability 98.84 per cent, ahead of grocery 1.10, local 0.05, supermarket 0.01 and convenience 0.00; total logprob -0.01.](assets/media/img-8d39baeeb160b0df.png)

### Remember all of this happens for each new token
{id=pptx-27c6e7c544b2-082}

![The four-step ChatGPT process diagram from an earlier slide: break all preceding text into tokens, assign each token a number, predict the next token, repeat with the new token.](assets/pages/page-82.png)

## Context window is a limit on prediction
{id=pptx-27c6e7c544b2-083}

### GPT-3.5 has a context window of 2,000 tokens. GPT-4 has 8,000 – 32,000.
{id=pptx-27c6e7c544b2-084}

![A round badge of waves and gulls beside the statement.](assets/media/img-34d9a370988f6563.png)

### Every new token generated, eats into the context window!
{id=pptx-27c6e7c544b2-085}

![A Pac-Man-like shape eating dots, beside the statement.](assets/media/img-b877d7e85007a200.png)

### 2,000 word summary by GPT-3.5 with 2k token window
{id=pptx-27c6e7c544b2-086}

![Three overlapping ChatGPT (GPT-3.5) screenshots in which the model lists the people mentioned in an article, first correctly, then when asked for five more it invents names such as Mark Zuckerberg, Jeff Bezos, Elon Musk, Tim Cook and Larry Page.](assets/pages/page-86.png)

### Same summary by GPT-4 with 8k token window
{id=pptx-27c6e7c544b2-087}

![Two ChatGPT (GPT-4) screenshots in which the model lists the people mentioned in the article and then adds a table with name, institution, role and how they were mentioned.](assets/pages/page-87.png)

## There is no database
{id=pptx-27c6e7c544b2-088}

### It looks like it’s looking things up, copying them and pasting them. BUT
{id=pptx-27c6e7c544b2-089}

### It always just predicts
{id=pptx-27c6e7c544b2-090}

https://what-if.xkcd.com/34

![An xkcd comic: a sign reads “OH MY GOD, THE VOLCANO IS ERU_” and three people look at it; one says “ERUDITE?”.](assets/media/img-32fbf9366b4bb19e.png)

### But because it was trained on almost trillion tokens and has 175 billion parameters, it can do this.
{id=pptx-27c6e7c544b2-091}

![A ChatGPT conversation: asked for a table of European countries sorted by their 2018 PISA score, it returns a table with country, reading, mathematics and science scores.](assets/media/img-e24a7c20c639dc21.png)

### It can even do this!
{id=pptx-27c6e7c544b2-092}

![Two ChatGPT screenshots: a story about Captain Smith’s fleet of five ships, asked how many ships did he leave the expedition with and how many returned, and then asked to add a row to a table with the statuses set out, returned and lost at sea.](assets/pages/page-92.png)

### But it because it always just predicts: No fact can be trusted. No common sense can be expected!
{id=pptx-27c6e7c544b2-093}

![A table of capital cities with a distance column, with the rows for Rome (1181) and Vatican City (920) boxed in red.](assets/media/img-4a25feaaf5074f8a.png)

### It really cannot copy and compare any information!!!
{id=pptx-27c6e7c544b2-094}

### Is there a number? DO NOT TRUST?
{id=pptx-27c6e7c544b2-095}

![Two tables of distances of European capitals from Prague: the table produced by ChatGPT with columns for capital, distance, population and local-language name, beside the capitals with distances, and a note that “distances are calculated using Google Maps and may vary based on the mode of transportation used”, boxed in red.](assets/pages/page-95.png)

### Can we trust the results of a search?
{id=pptx-27c6e7c544b2-096}

![A Bing Chat conversation asked for a table of European capitals sorted by distance from Prague, with population and name in the local language; it shows a table and a note that the table was limited to six rows.](assets/media/img-929d5a55e4cafdf4.png)

### It can count some things but not very far
{id=pptx-27c6e7c544b2-097}

![A word-count task: a short story and Word’s Word Count dialog showing 234 words, beside ChatGPT’s answer, “There are 181 words in this story.”](assets/pages/page-97.png)

### Bing will give references but the information is not there
{id=pptx-27c6e7c544b2-098}

![A Bing Chat answer to “Can I have more idioms in Czech featuring animals?”, listing idioms with literal translations and a Learn more row of source links.](assets/media/img-ee727f73e53aa425.png)

### It does not even have access to its own training data!
{id=pptx-27c6e7c544b2-099}

### Bret Devereaux
{id=pptx-27c6e7c544b2-100}

> [ChatGPT] has **read every book in the library once** and distilled the statistical relationships between the words in that library and then **burned the library**.

### ChatGPT does not even try
{id=pptx-27c6e7c544b2-101}

![A ChatGPT answer: “Sure, here are some common Czech idioms involving animals, including cats and dogs”, followed by a numbered list of ten Czech idioms, each with an English translation and an explanation of its meaning.](assets/media/img-6cd1eb8c4fe19701.png)

Sure, here are some common Czech idioms involving animals, including cats and dogs:

1. “Mít husí kůži” - This idiom translates to “to have goosebumps” and is used to describe a feeling of intense fear or anxiety.
2. “Být jako ryba v síti” - This means “to be like a fish in a net” and is used to describe someone who is trapped or stuck in a situation with no way out.
3. “Být jako pes s kočkou” - This translates to “to be like a dog with a cat” and is used to describe two people who cannot get along or are constantly arguing.
4. “Být jako slepý k houslím” - This means “to be like a blind person to a violin” and is used to describe someone who is completely clueless or oblivious to what is going on around them.
5. “Mít oči jako sokol” - This translates to “to have eyes like a hawk” and is used to describe someone who has very sharp or observant eyesight.
6. “Být jako kůň před kostelem” - This means “to be like a horse in front of a church” and is used to describe someone who is nervous or restless, usually due to anticipation or excitement.
7. “Mluvit do větru” - This translates to “to speak into the wind” and is used to describe someone who is talking to no one in particular, or whose words have no effect.
8. “Být zdravý jako ryba” - This means “to be healthy as a fish” and is used to describe someone who is in excellent health.
9. “Mít oči jako kráva” - This translates to “to have eyes like a cow” and is used to describe someone who has a very passive or docile personality.
10. “Mít kočku ve vaku” - This means “to have a cat in a bag” and is used to describe a situation that is not fully understood or is uncertain.

### ChatGPT sometimes hallucinates!
{id=pptx-27c6e7c544b2-102}

![A smiling face with raised eyebrows beside the statement.](assets/media/img-d127ec8cf3afcb4f.png)

### ChatGPT always hallucinates!
{id=pptx-27c6e7c544b2-103}

![A red face with a wide, strained smile beside the statement.](assets/media/img-0efff1bc2503c0fa.png)

### It just happens to match what we need often enough to be really useful.
{id=pptx-27c6e7c544b2-104}

![A green thumbs-up icon beside the statement.](assets/media/img-c3f9f27b71245a86.png)

### How much does predicting the next token get us?
{id=pptx-27c6e7c544b2-105}

![A line-chart-with-points icon beside the question.](assets/media/img-03b5d0573360a834.png)

### Surely, it could not count how many things are mentioned in text just by predicting the next token!
{id=pptx-27c6e7c544b2-106}

### How good is it at counting things
{id=pptx-27c6e7c544b2-107}

![Overlapping ChatGPT screenshots: the ships story and a table of ships set out, returned and lost at sea; asked to write out the reasoning step by step, the model counts the ships in five steps.](assets/pages/page-107.png)

### How good is it at counting intentions
{id=pptx-27c6e7c544b2-108}

![Overlapping ChatGPT screenshots: a thread of email replies to a party invitation and a table of names, reply status and reply text; asked to add a column sorting by who is coming and then by name, it fills a table.](assets/pages/page-108.png)

### Can it take a hint?
{id=pptx-27c6e7c544b2-109}

![A ChatGPT table of invitation replies, with names and reply text; the user asks it to update the table with a new message, “Hi Dominik, I’m travelling on Friday but I’ll see you at the next one.”](assets/media/img-713e4b8d9527c2aa.png)

### How good is it at sorting?
{id=pptx-27c6e7c544b2-110}

![Two ChatGPT tables of party replies: first sorted into a name and status list, then, when asked to use the name as a secondary sort, ordered by status and then name.](assets/pages/page-110.png)

### Can it do metalanguage?
{id=pptx-27c6e7c544b2-111}

![Overlapping ChatGPT screenshots: asked how many words are in a sentence, the model produces a table of words with definitions and miscounts; asked whether the question mark is a word, it explains, then revises its count.](assets/pages/page-111.png)

### GPT-4 Improves things
{id=pptx-27c6e7c544b2-112}

![A ChatGPT (GPT-4) exchange: asked “How many words are in this sentence?”, it says it cannot see the sentence; told the sentence is self-referential, it answers that there are 7 words.](assets/media/img-6e029e93091db1db.png)

### But only up to a point
{id=pptx-27c6e7c544b2-113}

![The word-count task: the short story, Word’s Word Count dialog showing 234 words and 1,168 characters, and ChatGPT’s answer that there are 181 words.](assets/pages/page-113.png)

### But it is close
{id=pptx-27c6e7c544b2-114}

![A ChatGPT (GPT-4) exchange: asked how many characters are in a sentence, it says 39 characters including spaces and 33 without; beside it Word’s Word Count dialog shows 35 characters without spaces and 41 with spaces.](assets/pages/page-114.png)

### Every time you try a number try it more than once
{id=pptx-27c6e7c544b2-115}

![Three overlapping ChatGPT screenshots in which the same questions about the number of characters and words in a sentence get different answers: 49 and 39 characters, and 8, 9 and 7 words.](assets/pages/page-115.png)

### Agrawal
{id=pptx-27c6e7c544b2-116}

![Cover of “Prediction Machines: The Simple Economics of Artificial Intelligence” by Ajay Agrawal, Joshua Gans and Avi Goldfarb, pink with two stylised eyes.](assets/media/img-c8769ef0ef09ee5a.png)

- Computer graphics = Cheap arithmetic
- AI = Cheap prediction

### In 1950s we could not imagine all the things cheap arithmetic would gets. Today, we can’t appreciate what cheap predictions will bring.
{id=pptx-27c6e7c544b2-117}

**Computer room in 1949**

![Black-and-white photograph of a woman working at a large computer in a computer room, captioned Computer room in 1949.](assets/media/img-21017c8a1901eae6.jpeg)

### We did not have good intuitions about what cheap arithmetic will get us. What will cheap prediction?
{id=pptx-27c6e7c544b2-118}

### Four years ago, Google made a big announcement
{id=pptx-27c6e7c544b2-119}

![A page from Google’s blog, “Understanding searches better than ever before”, with several paragraphs of text.](assets/media/img-4f0ac81f092d40f0.png)

### Major progress in 2019
{id=pptx-27c6e7c544b2-120}

![Two phone screenshots, Before and After, for the search “2019 brazil traveler to usa need a visa”: before, a news article about visas for Brazilians; after, a result that explains that a Brazilian traveller needs a visa.](assets/media/img-5a3b6368cafe8436.jpeg)

### And 4 years later
{id=pptx-27c6e7c544b2-121}

![A Bing Chat conversation: asked whether a traveller to the US from Brazil needs a visa, it answers that Brazilians need a visa, with sources listed below.](assets/media/img-ca4c5b95614e3d68.png)

### Also
{id=pptx-27c6e7c544b2-122}

- No idea of spatial relations
- No idea of social relations
- No understanding of physics

### Space is a problem: Some attempts at a bicycle
{id=pptx-27c6e7c544b2-123}

![Four attempts at a bicycle drawn by the model in code: a brown rectangle between two red circles with blue centres; a blue rectangle on two red wheels with a red top; a blue rectangle with two red dots below it; and a red frame with two crossed blue wheels and a green handlebar.](assets/pages/page-123.png)

### Social relations are a problem
{id=pptx-27c6e7c544b2-124}

**Customer:**

Take me downtown Prague.

**Taxi driver:**

How much does the trip cost?

## It does not LEARN live but can be corrected
{id=pptx-27c6e7c544b2-125}

### What it took to train the foundation model
{id=pptx-27c6e7c544b2-126}

- 21 days
- $1+ million

### What it took to fine tune it with human feedback
{id=pptx-27c6e7c544b2-127}

- X months
- $$$ million

### But it is a few-shot learner: It can follow examples
{id=pptx-27c6e7c544b2-128}

### And then… (GPT-3)
{id=pptx-27c6e7c544b2-129}

![Two pages from the paper “Language Models are Few-Shot Learners”: the title page with its abstract, and a figure with zero-shot, one-shot and few-shot prompts for translating English to French.](assets/pages/page-129.png)

### Take it through a problem step by step.
{id=pptx-27c6e7c544b2-130}

![Overlapping ChatGPT screenshots: the model is asked to count characters, says 39, then lists the characters one by one and, asked whether it wants to revise its answer, apologises and says there are actually 41.](assets/pages/page-130.png)

### Time to do some prompt engineering
{id=pptx-27c6e7c544b2-131}

![A scientist-in-a-lab-coat icon beside the statement.](assets/media/img-9e3444c6bcfc6163.png)

### The key lesson for practical use
{id=pptx-27c6e7c544b2-132}

![A teacher-at-a-board icon beside the statement.](assets/media/img-dcb9fef344ee4070.png)

### Keep 2 things in mind at once
{id=pptx-27c6e7c544b2-133}

- Give it prompts as to a human being
- Interpret the results as if autocomplete

![Two large block arrows pointing outwards: an orange one to the left, “Give it prompts as to a human being”, and a green one to the right, “Interpret the results as if autocomplete”.](assets/pages/page-133.png)

### What ChatGPT can do is a matter of experimentation.
{id=pptx-27c6e7c544b2-134}

![A maze icon beside the statement.](assets/media/img-cd9da653c67969dc.png)

### Prompt engineering should aim towards giving more examples and context.
{id=pptx-27c6e7c544b2-135}

![A scientist-in-a-lab-coat icon beside the statement.](assets/media/img-9e9216c9ee6b74f8.png)

### Key lessons from prompt engineers
{id=pptx-27c6e7c544b2-136}

1. Narrow down context
2. Give examples
3. Specify style
4. Specify persona
5. Ask for corrections

### Don’t just ask, don’t just tell. Iterate!
{id=pptx-27c6e7c544b2-137}

![Icon of a cycle of a circle, a diamond and a square joined by arrows.](assets/pages/page-137.png)

### Or in the words of ChatGPT
{id=pptx-27c6e7c544b2-138}

![A GPT-4 conversation: asked to write a limerick about five lessons of prompt engineering (narrow down context, give examples, specify style, specify persona, ask for corrections), it returns a four-line verse.](assets/media/img-da961ebfcf59bfe0.png)

### LearnPrompting
{id=pptx-27c6e7c544b2-139}

![The Learn Prompting website: “Learn Prompting: A Free, Open Source Course on Communicating with Artificial Intelligence”, with Join the Discord and Start Learning buttons and three illustrations.](assets/media/img-7eef45428e0772e8.png)

### Three levels of editing
{id=pptx-27c6e7c544b2-140}

1. Shallow prompt
2. Fine-tuned prompt
3. Co-editing

### Chain of Thought Prompts
{id=pptx-27c6e7c544b2-141}

Chain of Thought =

- more accuracy
- easier review

### But it can do this
{id=pptx-27c6e7c544b2-142}

![Two ChatGPT screenshots: asked for a numbered list of all the characters including spaces and punctuation in a sentence, it lists them; asked whether it wants to revise its answer, it apologises and says there are actually 41 characters.](assets/pages/page-142.png)

### And this
{id=pptx-27c6e7c544b2-143}

![Two ChatGPT screenshots: the model numbers the characters of a sentence up to 49, listing spaces separately; asked how many there are if trailing spaces are not counted, it answers 41.](assets/pages/page-143.png)

### Asking for structure is better than prose!
{id=pptx-27c6e7c544b2-144}

### Outlines are very context-based
{id=pptx-27c6e7c544b2-145}

![A ChatGPT desktop window titled “ChatGPT in Education Poll”, with the chat history in a sidebar on the left. The prompt asks for poll questions and options about educators' existing experience and knowledge of ChatGPT; the reply is a numbered list of poll questions with bulleted answer options, cut off at question 4.](assets/media/img-848e7ffd25d22e64.png)

**Prompt:** I'm doing a presentation about what should educators know about ChatGPT. I want to open with a poll about their existing experience and knowledge. Can you suggest some questions and poll options?

Sure, here are some potential poll questions and options for educators regarding their experience and knowledge of ChatGPT:

1. How familiar are you with ChatGPT?
    - Not at all familiar
    - Somewhat familiar
    - Very familiar
2. Have you used ChatGPT in your teaching or curriculum?
    - No, I have not used it
    - Yes, I have used it in a limited capacity
    - Yes, I have used it extensively
3. In your opinion, what are the potential benefits of using ChatGPT in education?
    - Improved student engagement
    - Increased efficiency of tasks such as grading and providing feedback
    - Enhanced opportunities for personalized learning
    - Other (please specify)
4. Have you encountered any challenges when using ChatGPT in education? (The screenshot ends here; the options for question 4 are not visible.)

**Sidebar chats visible:** ChatGPT in Education Poll (current), People Coming to Party, Party RSVP Confirmations, London To New York Flight Ti…, ASCII car art creation, LLM Language Model Training…, Dagmar Skálová: A Heroine, Story task and activities, Deduplicate Fruits List, Table Words Request Confuse…

### You can convert them into interactive things
{id=pptx-27c6e7c544b2-146}

![A multiple-choice quiz made from an outline: “What type of fruit is grown on the highest mountain?” with options Apples, Pears, Stone fruit and Berries, and a Submit button, followed by further questions.](assets/media/img-6fc28b9f2551d724.png)

### Asking for concepts and examples
{id=pptx-27c6e7c544b2-147}

![Two ChatGPT screenshots: asked for a table with examples of each of Grice’s maxims, it produces a table with columns Maxim, Definition and Example of when the maxim is met; a follow-up adds a column for when the maxim is broken.](assets/pages/page-147.png)

### Asking for timelines better than narratives
{id=pptx-27c6e7c544b2-148}

![Two ChatGPT (GPT-4) answers: a step-by-step account of how hieroglyphics were deciphered, and a timeline of key discoveries and contributions with years and names.](assets/pages/page-148.png)

### Asking to adopt a persona is also very useful,
{id=pptx-27c6e7c544b2-149}

![A ChatGPT role-play. The prompt asks for a conversation in English at level A1 in which ChatGPT is a shopkeeper and the user a student buying bread, with simplified speech. The reply is a short shopkeeper-and-student dialogue about white, whole wheat and rye bread, one loaf at $2.50.](assets/media/img-43d8d5ebdf4bedcb.png)

**Prompt:** Let's have a conversation in English at the level of A1. You are a shopkeeper and I am a student. I'm trying to buy bread. You need to simplify the way you speak so that I can understand.

**Shopkeeper:** Hello, how can I help you?

**Student:** Hi, I want to buy bread.

**Shopkeeper:** Sure, what type of bread would you like? We have white bread, whole wheat bread, and rye bread.

**Student:** What's the difference?

**Shopkeeper:** White bread is made with flour that has been stripped of its nutrients. Whole wheat bread is made with flour that includes the bran and germ, which makes it healthier. Rye bread is made with rye flour and has a distinctive flavor.

**Student:** I want the healthier one. Can I have whole wheat bread?

**Shopkeeper:** Sure, how many loaves would you like?

**Student:** Just one loaf, please.

**Shopkeeper:** That will be $2.50. Would you like anything else?

**Student:** No, that's all. Thank you.

**Shopkeeper:** You're welcome. Have a nice day!

### You can even have a back and forth
{id=pptx-27c6e7c544b2-150}

![Two ChatGPT conversations: a role-play in which the user plays a shopkeeper in a small German village and the model plays a customer in German, then a request to correct the user’s German, answered with a corrected version of what they said.](assets/pages/page-150.png)

### Where to learn more?
{id=pptx-27c6e7c544b2-151}

![A book icon beside the question.](assets/media/img-324ebbc52bee3a4f.png)

### Edutools.FYI
{id=pptx-27c6e7c544b2-152}

![The Edutools.fyi website, a Notion page with “ChatGPT and other Artificial intelligence Tools”, a tools database and readings, beside a QR code.](assets/pages/page-152.png)

### bit.ly/academic-ai-newsletter
{id=pptx-27c6e7c544b2-153}

https://bit.ly/academic-ai-newsletter

![The LinkedIn newsletter “AI in Academic Practice” with a cartoon computer with a puzzled face, beside a QR code.](assets/pages/page-153.png)

### Tools change, people remain the same
{id=pptx-27c6e7c544b2-154}

EduTools.fyi

### This presentation is licensed under Creative Commons By Attribution license except where otherwise noted. Icons and stock photos licensed under Microsoft Premium Content and cannot be reused outside this document.
{id=pptx-27c6e7c544b2-155}

![Creative Commons Attribution licence badge.](assets/media/img-cd64f44f67f59813.png)