All presentations

How to Teach and Learn with ChatGPT: AI and new horizons for assistive technologies

Slide 1: How to Teach and Learn with ChatGPT: AI and new horizons for assistive technologies

How to Teach and Learn with ChatGPT: AI and new horizons for assistive technologies

Dominik Lukes, EduTools.fyi

Original slide 1 (image)

Slide 2: Lots of developments since we scheduled the session

Lots of developments since we scheduled the session

A rocket icon beside the slide’s statement.

Original slide 2 (image)

Slide 3: What has changed

What has changed

  • Bing Chat
  • GPT-4
  • Google Bard
  • Office ‘Co-Pilot’s
  • GPT-4 Plugins

Original slide 3 (image)

Slide 4: bit.ly/academic-ai-newsletter

bit.ly/academic-ai-newsletter

https://bit.ly/academic-ai-newsletter

Screenshot of the LinkedIn newsletter “AI in Academic Practice” with a cartoon computer with a puzzled face and question marks, beside a QR code.

Original slide 4 (image)

Slide 5: Poll: What is your experience with ChatGPT?

Poll: What is your experience with ChatGPT?

A raised-hand icon beside the poll question.

Original slide 5 (image)

Slide 6: Poll: Experience?

Poll: Experience?

  • Use it every day
  • Played around with it
  • Heard a LOT about it
  • I heard something about it

Original slide 6 (image)

Slide 7: Comparison from a recent talk

Comparison from a recent talk

A poll result from an earlier talk: a pie chart beside the answers Use ChatGPT in my work (5), Have used once or twice (32), Have heard a lot and seen examples (21), Have heard something but not sure (23) and Other (share in chat) (1).

Original slide 7 (image)

Slide 8: Poll: What do you think AI might be best at?

Poll: What do you think AI might be best at?

A raised-hand icon beside the poll question.

Original slide 8 (image)

Slide 9: What do you think AI might be best at?

What do you think AI might be best at?

  • Looking up facts
  • Performing complex calculations
  • Writing computer code based on text
  • Making complex decisions from data
  • Writing poetry
  • Making beautiful art

Original slide 9 (image)

Slide 10: Comparison with a recent talk

Comparison with a recent talk

A horizontal bar chart of poll results for what AI might be best at, listing Looking up facts, Performing complex calculations, Writing computer code based on text, Making complex decisions based on data, Writing poetry and Creating beautiful photos, with a pop-up for “Creating beautiful photos”: 1st choice 2 (2%), 2nd choice 3 (4%), 3rd choice 6 (7%), 4th choice 7 (8%), 5th choice 18 (21%), 6th choice 48 (57%).

Original slide 10 (image)

Slide 11: What it is actually good at?

What it is actually good at?

A superhero-flying icon beside the question.

Original slide 11 (image)

Slide 12: Nobody expected AI to be any good at this

Nobody expected AI to be any good at this

Two ChatGPT conversations side by side. Left: the prompt asks for a poem to remember the bones in the hand and the answer is a four-stanza rhyming poem. Right: the prompt asks for a mnemonic for the carpals and the answer is the sentence “She Left The Party To Take Cathy Home” with each carpal bone matched to a word.

Prompt (left): Write a poem to help me remember the bones in the hand.

In the realm of hands, a tale unfolds,
Of bones and joints, a story told,
Listen close, my friend, and hear,
The names of bones, so you'll endear.

Twenty-seven bones, a framework grand,
Form the structure of our hand,
Three groups of bones to memorize,
Carpals, metacarpals, phalanges arise.

Begin with carpals, eight in sum,
In the wrist, they're not cumbersome,
Proximal row, four in row,
From lateral to medial, here's how they go:

Scaphoid, lunate, triquetrum align,
Pisiform, a small pea, so benign,
Distal row, to continue the tour,
Trapezium, trapezoid, capitate, hamate ensure.

Prompt (right): Make a mnemonic for the carpals

Recall the carpals, eight in all,
With this mnemonic, you'll never stall:

She Left The Party
To Take Cathy Home

  • Scaphoid - She
  • Lunate - Left
  • Triquetrum - The
  • Pisiform - Party
  • Trapezium - To
  • Trapezoid - Take
  • Capitate - Cathy
  • Hamate - Home

Original slide 12 (image)

Slide 13: Or this

Or this

A collage of AI-generated pictures: a man sitting beside a hole in a wall, colourful retro robot and computer characters, a smiling person with a tree of leaves for hair, and a robot reading in a hollow.

Original slide 13 (image)

Slide 14: We thought AI could do this

We thought AI could do this

A ChatGPT conversation about the arithmetic “4^2 - 234 * (-23) * 329870134987”: it works through the steps and its answer reads 10825535722618; beside it a calculator shows 1775361066500050.

Original slide 14 (image)

Slide 15: But it is also unexpectedly good at this

But it is also unexpectedly good at this

A ChatGPT conversation. The prompt is “Write code to add two numbers in python”. The answer explains the + operator and gives a short Python program with an add_numbers function, followed by an explanation of the code.

Prompt: Write code to add two numbers in python

In Python, you can add two numbers by simply using the + operator. Here's a simple function to add two numbers:

def add_numbers(a, b):

    return a + b

number1 = float(input("Enter the first number: "))

number2 = float(input("Enter the second number: "))

result = add_numbers(number1, number2)

print("The sum of the two numbers is:", result)

In this code, the add_numbers function takes two arguments, a and b, and returns their sum. The user is prompted to enter two numbers, which are then passed to the function to perform the addition. The result is then printed.

Original slide 15 (image)

Slide 16: How do we know what modern AI will be good or bad at?

How do we know what modern AI will be good or bad at?

A puzzle-piece icon beside the question.

Original slide 16 (image)

Slide 17: We have evaluate it systematically. Almost as a new organism.

We have evaluate it systematically. Almost as a new organism.

A microscope icon beside the statement.

Original slide 17 (image)

Slide 18: Hofstadter, 1979: Questions

Hofstadter, 1979: Questions

Question: Will a thinking computer be able to add fast?

Cover of “Gödel, Escher, Bach: an Eternal Golden Braid” by Douglas R. Hofstadter, orange with a block-letter title.

Original slide 18 (image)

Slide 19: Hofstadter, 1979, Speculations

Hofstadter, 1979, Speculations

Speculation: Perhaps not. We ourselves are composed of hardware which does fancy calculations but that doesn't mean that our symbol level, where "we" are, knows how to carry out the same fancy calculations.

Original slide 19 (image)

Slide 20: Let’s keep this in mind

Let’s keep this in mind

A tweet by Sam Altman: “here is GPT-4, our most capable and aligned model yet. it is available today in our API (with a waitlist) and in ChatGPT+. openai.com/research/gpt-4 … it is still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it.”

Original slide 20 (image)

Slide 21: What ChatGPT can do for YOU?

What ChatGPT can do for YOU?

A light-bulb-in-head icon beside the question.

Original slide 21 (image)

Slide 22: Better at poetry than math still means it can be incredibly useful.

Better at poetry than math still means it can be incredibly useful.

Original slide 22 (image)

Slide 23: Anyone can use it to help with reading

Anyone can use it to help with reading

Screenshots of a university web page with a chat panel: the user selects text and asks the assistant to explain, simplify it with easier language and explain it to a 12-year-old; the assistant’s replies appear in the side panel.

Original slide 23 (image)

Slide 24: A student can use this for help

A student can use this for help

A ChatGPT conversation in which a student asks the model to explain a line of NumPy array code and it explains each element.

Original slide 24 (image)

Slide 25: A teacher can use it to create tools

A teacher can use it to create tools

A teacher’s prompt asking for a tool that compares two translations of the same text and a web form with two text boxes below it, with the differences between the two texts highlighted in yellow.

Original slide 25 (image)

Slide 26: There are limits but also, we can use it to assist us in real ways. But we should understand what it is doing.

There are limits but also, we can use it to assist us in real ways. But we should understand what it is doing.

Original slide 26 (image)

Slide 27: 5 Things about ChatGPT in 15(ish) Minutes

5 Things about ChatGPT in 15(ish) Minutes

Original slide 27 (image)

Slide 28: 5 things about ChatGPT in 15 minutes

5 things about ChatGPT in 15 minutes

  1. It has many building blocks
  2. It does not reason, it predicts
  3. It is limited by the context window
  4. It has no database, copy or paste or memory
  5. It does not learn but you can correct it

Original slide 28 (image)

Slide 29: Building blocks of modern AI

Building blocks of modern AI

Original slide 29 (image)

Slide 30: What does ChatGPT stand for?

What does ChatGPT stand for?

  • Chat: answers questions rather than continues
  • G - generative: creates strings rather than labels
  • P - pretrained: can be fine-tuned on much less data
  • T - transformer: can be trained to predict from long context

Foundation model

Original slide 30 (image)

Slide 31: Building blocks of the AI revolution

Building blocks of the AI revolution

  • Foundation Models
  • Products: ChatGPT, Consensus, …
  • Features: Notion …, PowerPoint…, …

Original slide 31 (image)

Slide 32: Language model universe in 2022

Language model universe in 2022

A bubble chart, “Language model sizes to Dec/2022”, of large language models by parameter count: from BERT and GPT-2 to GPT-3 (175 billion), Jurassic-1, MT-NLG (530 billion), PaLM (540 billion) and BLOOM, with the source LifeArchitect.ai/models.

Original slide 32 (image)

Slide 33: Building blocks of a product

Building blocks of a product

  • Foundation Model API
  • Fine tuning on custom data
  • Finetuning by human feedback
  • Custom Workflow / Interface
  • Consensus (OpenAI)
  • Notion (Anthropic)
  • Smart Reply (Google)
  • ChatGPT (Open AI)
  • Claude (Anthropic)
  • Bard (Google)
  • TeacherMatic
  • LangChain…

Original slide 33 as rendered: a wide dark blue base bar “Foundation Model API”; above it three orange boxes, Fine tuning on custom data, Finetuning by human feedback and Custom Workflow / Interface; above those a row of eight small green boxes, Consensus (OpenAI), Notion (Anthropic), Smart Reply (Google), ChatGPT (Open AI), Claude (Anthropic), Bard (Google), TeacherMatic and LangChain…

Original slide 33 (image)

Slide 34: For example Consensus (now with GPT-4)

For example Consensus (now with GPT-4)

Two screenshots of Consensus, a search tool: a question about whether it is possible to improve working memory, with a summary of the evidence and a list of papers with their findings.

Original slide 34 (image)

Slide 35: ChatGPT is just another product on top of GPT

ChatGPT is just another product on top of GPT

Two screenshots: on the left, the TeacherMatic tool with a learning objectives generator; on the right, ChatGPT asked to write 10 learning objectives for a beginner course about web design, which it lists.

Original slide 35 (image)

Slide 36: All of these work the same way

All of these work the same way

Two overlapping news articles: “Introducing Microsoft 365 Copilot—A whole new way to work” and “Google announces AI features in Gmail, Docs, and more to rival Microsoft”.

Original slide 36 (image)

Slide 37: Big API providers

Big API providers

  • OpenAI
  • Anthropic
  • Google
  • Amazon
  • Microsoft
  • Meta

Original slide 37 (image)

Slide 38: Although there are many smaller ones on HuggingFace

Although there are many smaller ones on HuggingFace

The Hugging Face model listing page with filters on the left and a list of model names such as bert-base-uncased and gpt2.

Original slide 38 (image)

Slide 39: Thousands API customers

Thousands API customers

The website “There’s an AI for that”: “2,675 AIs for 603 tasks. Updated daily”, “Just launched” and a grid of small tool cards on a dark page.

Original slide 39 (image)

Slide 40: And thousands more

And thousands more

The website FutureTools: “FutureTools Collects & Organizes All The Best AI Tools So YOU Too Can Become Superhuman!”, with a search box and a grid of category filters and tool cards.

Original slide 40 (image)

Slide 41: Always ask what API a product uses!

Always ask what API a product uses!

Original slide 41 (image)

Slide 42: It does not reason, it predicts

It does not reason, it predicts

Original slide 42 (image)

Slide 43: It is easy to be deceived by appearances.

It is easy to be deceived by appearances.

Original slide 43 (image)

Slide 44: ChatGPT can do this

ChatGPT can do this

Two ChatGPT answers. First, a list of tropical fruits grouped by region, South America and Central America & Mexico. Second, the same fruits in a table with columns Fruit Name, Region of Origin and Definition.

Original slide 44 (image)

Slide 45: How do we solve these kinds of problems?

How do we solve these kinds of problems?

Original slide 45 (image)

Slide 46: Cognition = Computation

Cognition = Computation

Logic

  • Compare
  • Organise
  • Transform

Memory

  • Store
  • Retrieve

Original slide 46 (image)

Slide 47: Something like this

Something like this

A flowchart: Find tropical fruits, Look up fruit in memory, Is it tropical? (No loops back; Yes goes to Write to list).

Original slide 47 (image)

Slide 48: How does ChatGPT solve these kinds of problems?

How does ChatGPT solve these kinds of problems?

Original slide 48 (image)

Slide 49: ChatGPT does not have cognition or computation

ChatGPT does not have cognition or computation

Logic

  • Compare
  • Organise
  • Transform

Memory

  • Store
  • Retrieve

Original slide 49 as rendered: the Logic (database icon) and Memory (floppy disk icon) columns of the previous slide, each crossed out with a large red X.

Original slide 49 (image)

Slide 50: ChatGPT Process

ChatGPT Process

  1. Break all preceding text into tokens
  2. Assign each token number
  3. Predict next token
  4. Repeat with the new token

Original slide 50 (image)

Slide 51: What and why of tokens? And why it matters. A lot!

What and why of tokens? And why it matters. A lot!

A ticket icon beside the question.

Original slide 51 (image)

Slide 52: How we see text?

How we see text?

Humans

  • Letters
  • Words
  • Sentences
  • Paragraphs
  • Texts

ChatGPT

  • Tokens

Original slide 52 (image)

Slide 53: OpenAI: ChatGPT only sees tokens

OpenAI: ChatGPT only sees tokens

Two overlapping screenshots of OpenAI’s Tokenizer: the text “Many words map to one token, but some don’t: indivisible.” with 64 tokens and 252 characters, with each token highlighted in a different colour, and an explanation of how the models see tokens.

Original slide 53 (image)

Slide 54: Actually, this is what ChatGPT sees

Actually, this is what ChatGPT sees

Unicode

A line of numbers in square brackets, the token IDs of a text, with a large arrow from the word “Unicode” to the group of token numbers 3118, 291 and 1098 that stand for it.

Original slide 54 (image)

Slide 55: Numbers are not digits, just tokens

Numbers are not digits, just tokens

Two token-ID displays. The sum “1 + 2 * 3654 = 123,565” with each number and sign coloured as its own token, and “2 + 1 / 43^43.245645”; below them a list of token numbers, with arrows from each number to the tokens it is split into.

Original slide 55 (image)

Slide 56: Fun facts about OpenAI tokens

Fun facts about OpenAI tokens

  • About 50,000 in total
  • Only 3,000 English words
  • Most parts of words incl. letters, spaces
  • About 140 tokens per 100 English words
  • Much more for other languages

Original slide 56 (image)

Slide 57: Word and tokens are only accidentally identical

Word and tokens are only accidentally identical

A stack of overlapping tokenizer screenshots comparing one sentence in several languages: the token counts shown are 119, 240, 317, 396 and 523, and the last, longest text is split into many small coloured fragments.

Original slide 57 (image)

Slide 58: Unexpected consequence: More tokens more money

Unexpected consequence: More tokens more money

  • English
  • French
  • Czech
  • Chinese
  • Arabic

A rising curved arrow from English, through French, Czech and Chinese, to Arabic: the more tokens the text needs, the higher the cost.

Original slide 58 (image)

Slide 59: Consequences of tokens

Consequences of tokens

  • Limited rhyming
  • Bad at letter tasks
  • Unreliable numbers
  • More grammar mistakes in some languages

Original slide 59 (image)

Slide 60: It’s not good at this BUT it shouldn’t be able to do this at all.

It’s not good at this BUT it shouldn’t be able to do this at all.

A ChatGPT conversation. The prompt asks for 10 Czech nouns where ‘ů’ changes to ‘o’ as the noun is declined, with the examples ‘stůl’ - ‘stolu’ and ‘dům’ - ‘domu’. The answer is a numbered list of ten nouns with their declined forms and English glosses. The Czech characters in this image are displayed correctly.

Prompt: Give me 10 Czech nouns where ‘ů’ changes to ‘o’ as the noun is declined. For example, ‘stůl’ - ‘stolu’, dům - domu.

Certainly! Here are 10 Czech nouns that follow the pattern of the vowel ‘ů’ changing to ‘o’ as the noun is declined:

  1. Kůň - koně (horse)
  2. Důl - dolu (mine)
  3. Hůl - holí (cane)
  4. Stroj - stroje (machine)
  5. Můstek - mostky (little bridge)
  6. Vůz - vozy (carriage)
  7. Sůl - soli (salt)
  8. Nůž - nože (knife)
  9. Kroužek - kroužky (ringlet)
  10. Úkol - úkoly (task)

Tokeniser view on the original slide: beside the screenshot, the slide shows the same ten answers as coloured tokens in which every accented letter appears as one or two replacement characters: K���� - kon�� (horse); D��l - dolu (mine); H��l - holí (cane); Stroj - stroje (machine); M��stek - mostky (little bridge); V��z - vozy (carriage); S��l - soli (salt); N���� - no��e (knife); Krou��ek - krou��ky (ringlet); ��kol - ��koly (task).

Original slide 60 (image)

Slide 61: BTW: GPT-4 is not much better at this.

BTW: GPT-4 is not much better at this.

A ChatGPT answer about Czech noun declension, listing ten nouns with their singular genitive forms and noting that the list is not exhaustive.

Original slide 61 (image)

Slide 62: Tokens are not graphics, but they can still produce this.

Tokens are not graphics, but they can still produce this.

A ChatGPT conversation: asked to convert a text into a Mermaid diagram, it returns Mermaid code, shown beside the resulting flowchart of the fruit-lookup process.

Original slide 62 (image)

Slide 63: Embeddings: Give tokens meaning

Embeddings: Give tokens meaning

A network-of-nodes icon beside the heading.

Original slide 63 (image)

Slide 64: What are embeddings?

What are embeddings?

Embeddings show how tokens relate to each other in multidimensional-dimensional space.

Icon of a network of connected nodes.

Original slide 64 (image)

Slide 65: Language embeddings

Language embeddings

The illustration “The Illustrated Word2vec” showing word embeddings: a list of 50 numbers for the word “king”, and colour strips for king, man and woman.

Original slide 65 (image)

Slide 66: You do maths on matrices and you get

You do maths on matrices and you get

king − man + woman = queen

Original slide 66 (image)

Slide 67: But all of this is happening in 100s or 1000s or dimensions.

But all of this is happening in 100s or 1000s or dimensions.

A scatter plot of word embeddings projected into two dimensions: clouds of small labelled words coloured by cluster.

Original slide 67 (image)

Slide 68: More complicated than simple matrix addition

More complicated than simple matrix addition

A Dash word-embeddings app: a network of related words around “california − state + country” and a bar chart of nearest neighbours by cosine similarity.

Original slide 68 (image)

Slide 69: Add it all up

Add it all up

An abacus icon beside the heading.

Original slide 69 (image)

Slide 70: All it is doing on the inside is multiplying matrices.

All it is doing on the inside is multiplying matrices.

Four overlapping copies of an icon of a grid of small squares.

Original slide 70 (image)

Slide 71: Really big ones, for every single token in the context window!

Really big ones, for every single token in the context window!

An icon of a large grid, beside the statement.

Original slide 71 (image)

Slide 72: On hundreds or thousands of expensive graphics cards.

On hundreds or thousands of expensive graphics cards.

Photograph of a red-and-black graphics card with two fans.

Original slide 72 (image)

Slide 73: This means it costs $$ every time you ask it to do something.

This means it costs $$ every time you ask it to do something.

An icon of banknotes and coins.

Original slide 73 (image)

Slide 74: Generate predictions

Generate predictions

A line-chart-with-points icon beside the heading.

Original slide 74 (image)

Slide 75: The we predict what comes next (from Wolfram)

The we predict what comes next (from Wolfram)

A table of the most likely next words after the sentence beginning “The best thing about AI is its ability to”: learn 4.5%, predict 3.5%, make 3.2%, understand 3.1%, do 2.9%.

Original slide 75 (image)

Slide 76: The top item is not always selected

The top item is not always selected

A text generator with a pop-up of next-word probabilities after “I went back over to the library and got a book on Civil War”: house 4.35%, other 2.32%, window 1.47%, hospital 1.26%, old 1.17% and library 0.41%, with “library” highlighted.

Original slide 76 (image)

Slide 77: Turn up the temperature: Randomness

Turn up the temperature: Randomness

A dice icon beside the heading.

Original slide 77 (image)

Slide 78: With heat

With heat

Overlapping text-generation screenshots: a settings panel with a temperature slider beside three different continuations of text that begin “I went…”.

Original slide 78 (image)

Slide 79: What it looks like with temperature 1

What it looks like with temperature 1

The same next-word probability pop-up, now headed as the case of temperature 1: after “I went back over to the library and got a book on Civil War” the alternatives are house, other, window, hospital, old and library 0.41%.

Original slide 79 (image)

Slide 80: Without heat

Without heat

Several overlapping text outputs of “I went to the store to buy some groceries. I bought milk, eggs, bread, and some fruit. I also picked up some snacks for later.”, which are the same each time.

Original slide 80 (image)

Slide 81: What it looks like with temperature 0

What it looks like with temperature 0

A text-generation display for “I went to the store”: the next word “store” has probability 98.84 per cent, ahead of grocery 1.10, local 0.05, supermarket 0.01 and convenience 0.00; total logprob -0.01.

Original slide 81 (image)

Slide 82: Remember all of this happens for each new token

Remember all of this happens for each new token

The four-step ChatGPT process diagram from an earlier slide: break all preceding text into tokens, assign each token a number, predict the next token, repeat with the new token.

Original slide 82 (image)

Slide 83: Context window is a limit on prediction

Context window is a limit on prediction

Original slide 83 (image)

Slide 84: GPT-3.5 has a context window of 2,000 tokens. GPT-4 has 8,000 – 32,000.

GPT-3.5 has a context window of 2,000 tokens. GPT-4 has 8,000 – 32,000.

A round badge of waves and gulls beside the statement.

Original slide 84 (image)

Slide 85: Every new token generated, eats into the context window!

Every new token generated, eats into the context window!

A Pac-Man-like shape eating dots, beside the statement.

Original slide 85 (image)

Slide 86: 2,000 word summary by GPT-3.5 with 2k token window

2,000 word summary by GPT-3.5 with 2k token window

Three overlapping ChatGPT (GPT-3.5) screenshots in which the model lists the people mentioned in an article, first correctly, then when asked for five more it invents names such as Mark Zuckerberg, Jeff Bezos, Elon Musk, Tim Cook and Larry Page.

Original slide 86 (image)

Slide 87: Same summary by GPT-4 with 8k token window

Same summary by GPT-4 with 8k token window

Two ChatGPT (GPT-4) screenshots in which the model lists the people mentioned in the article and then adds a table with name, institution, role and how they were mentioned.

Original slide 87 (image)

Slide 88: There is no database

There is no database

Original slide 88 (image)

Slide 89: It looks like it’s looking things up, copying them and pasting them. BUT

It looks like it’s looking things up, copying them and pasting them. BUT

Original slide 89 (image)

Slide 90: It always just predicts

It always just predicts

https://what-if.xkcd.com/34

An xkcd comic: a sign reads “OH MY GOD, THE VOLCANO IS ERU_” and three people look at it; one says “ERUDITE?”.

Original slide 90 (image)

Slide 91: But because it was trained on almost trillion tokens and has 175 billion parameters, it can do this.

But because it was trained on almost trillion tokens and has 175 billion parameters, it can do this.

A ChatGPT conversation: asked for a table of European countries sorted by their 2018 PISA score, it returns a table with country, reading, mathematics and science scores.

Original slide 91 (image)

Slide 92: It can even do this!

It can even do this!

Two ChatGPT screenshots: a story about Captain Smith’s fleet of five ships, asked how many ships did he leave the expedition with and how many returned, and then asked to add a row to a table with the statuses set out, returned and lost at sea.

Original slide 92 (image)

Slide 93: But it because it always just predicts: No fact can be trusted. No common sense can be expected!

But it because it always just predicts: No fact can be trusted. No common sense can be expected!

A table of capital cities with a distance column, with the rows for Rome (1181) and Vatican City (920) boxed in red.

Original slide 93 (image)

Slide 94: It really cannot copy and compare any information!!!

It really cannot copy and compare any information!!!

Original slide 94 (image)

Slide 95: Is there a number? DO NOT TRUST?

Is there a number? DO NOT TRUST?

Two tables of distances of European capitals from Prague: the table produced by ChatGPT with columns for capital, distance, population and local-language name, beside the capitals with distances, and a note that “distances are calculated using Google Maps and may vary based on the mode of transportation used”, boxed in red.

Original slide 95 (image)

Slide 96: Can we trust the results of a search?

Can we trust the results of a search?

A Bing Chat conversation asked for a table of European capitals sorted by distance from Prague, with population and name in the local language; it shows a table and a note that the table was limited to six rows.

Original slide 96 (image)

Slide 97: It can count some things but not very far

It can count some things but not very far

A word-count task: a short story and Word’s Word Count dialog showing 234 words, beside ChatGPT’s answer, “There are 181 words in this story.”

Original slide 97 (image)

Slide 98: Bing will give references but the information is not there

Bing will give references but the information is not there

A Bing Chat answer to “Can I have more idioms in Czech featuring animals?”, listing idioms with literal translations and a Learn more row of source links.

Original slide 98 (image)

Slide 99: It does not even have access to its own training data!

It does not even have access to its own training data!

Original slide 99 (image)

Slide 100: Bret Devereaux

Bret Devereaux

[ChatGPT] has read every book in the library once and distilled the statistical relationships between the words in that library and then burned the library.

Original slide 100 (image)

Slide 101: ChatGPT does not even try

ChatGPT does not even try

A ChatGPT answer: “Sure, here are some common Czech idioms involving animals, including cats and dogs”, followed by a numbered list of ten Czech idioms, each with an English translation and an explanation of its meaning.

Sure, here are some common Czech idioms involving animals, including cats and dogs:

  1. “Mít husí kůži” - This idiom translates to “to have goosebumps” and is used to describe a feeling of intense fear or anxiety.
  2. “Být jako ryba v síti” - This means “to be like a fish in a net” and is used to describe someone who is trapped or stuck in a situation with no way out.
  3. “Být jako pes s kočkou” - This translates to “to be like a dog with a cat” and is used to describe two people who cannot get along or are constantly arguing.
  4. “Být jako slepý k houslím” - This means “to be like a blind person to a violin” and is used to describe someone who is completely clueless or oblivious to what is going on around them.
  5. “Mít oči jako sokol” - This translates to “to have eyes like a hawk” and is used to describe someone who has very sharp or observant eyesight.
  6. “Být jako kůň před kostelem” - This means “to be like a horse in front of a church” and is used to describe someone who is nervous or restless, usually due to anticipation or excitement.
  7. “Mluvit do větru” - This translates to “to speak into the wind” and is used to describe someone who is talking to no one in particular, or whose words have no effect.
  8. “Být zdravý jako ryba” - This means “to be healthy as a fish” and is used to describe someone who is in excellent health.
  9. “Mít oči jako kráva” - This translates to “to have eyes like a cow” and is used to describe someone who has a very passive or docile personality.
  10. “Mít kočku ve vaku” - This means “to have a cat in a bag” and is used to describe a situation that is not fully understood or is uncertain.

Original slide 101 (image)

Slide 102: ChatGPT sometimes hallucinates!

ChatGPT sometimes hallucinates!

A smiling face with raised eyebrows beside the statement.

Original slide 102 (image)

Slide 103: ChatGPT always hallucinates!

ChatGPT always hallucinates!

A red face with a wide, strained smile beside the statement.

Original slide 103 (image)

Slide 104: It just happens to match what we need often enough to be really useful.

It just happens to match what we need often enough to be really useful.

A green thumbs-up icon beside the statement.

Original slide 104 (image)

Slide 105: How much does predicting the next token get us?

How much does predicting the next token get us?

A line-chart-with-points icon beside the question.

Original slide 105 (image)

Slide 106: Surely, it could not count how many things are mentioned in text just by predicting the next token!

Surely, it could not count how many things are mentioned in text just by predicting the next token!

Original slide 106 (image)

Slide 107: How good is it at counting things

How good is it at counting things

Overlapping ChatGPT screenshots: the ships story and a table of ships set out, returned and lost at sea; asked to write out the reasoning step by step, the model counts the ships in five steps.

Original slide 107 (image)

Slide 108: How good is it at counting intentions

How good is it at counting intentions

Overlapping ChatGPT screenshots: a thread of email replies to a party invitation and a table of names, reply status and reply text; asked to add a column sorting by who is coming and then by name, it fills a table.

Original slide 108 (image)

Slide 109: Can it take a hint?

Can it take a hint?

A ChatGPT table of invitation replies, with names and reply text; the user asks it to update the table with a new message, “Hi Dominik, I’m travelling on Friday but I’ll see you at the next one.”

Original slide 109 (image)

Slide 110: How good is it at sorting?

How good is it at sorting?

Two ChatGPT tables of party replies: first sorted into a name and status list, then, when asked to use the name as a secondary sort, ordered by status and then name.

Original slide 110 (image)

Slide 111: Can it do metalanguage?

Can it do metalanguage?

Overlapping ChatGPT screenshots: asked how many words are in a sentence, the model produces a table of words with definitions and miscounts; asked whether the question mark is a word, it explains, then revises its count.

Original slide 111 (image)

Slide 112: GPT-4 Improves things

GPT-4 Improves things

A ChatGPT (GPT-4) exchange: asked “How many words are in this sentence?”, it says it cannot see the sentence; told the sentence is self-referential, it answers that there are 7 words.

Original slide 112 (image)

Slide 113: But only up to a point

But only up to a point

The word-count task: the short story, Word’s Word Count dialog showing 234 words and 1,168 characters, and ChatGPT’s answer that there are 181 words.

Original slide 113 (image)

Slide 114: But it is close

But it is close

A ChatGPT (GPT-4) exchange: asked how many characters are in a sentence, it says 39 characters including spaces and 33 without; beside it Word’s Word Count dialog shows 35 characters without spaces and 41 with spaces.

Original slide 114 (image)

Slide 115: Every time you try a number try it more than once

Every time you try a number try it more than once

Three overlapping ChatGPT screenshots in which the same questions about the number of characters and words in a sentence get different answers: 49 and 39 characters, and 8, 9 and 7 words.

Original slide 115 (image)

Slide 116: Agrawal

Agrawal

Cover of “Prediction Machines: The Simple Economics of Artificial Intelligence” by Ajay Agrawal, Joshua Gans and Avi Goldfarb, pink with two stylised eyes.

  • Computer graphics = Cheap arithmetic
  • AI = Cheap prediction

Original slide 116 (image)

Slide 117: In 1950s we could not imagine all the things cheap arithmetic would gets. Today, we can’t appreciate what cheap predictions will bring.

In 1950s we could not imagine all the things cheap arithmetic would gets. Today, we can’t appreciate what cheap predictions will bring.

Computer room in 1949

Black-and-white photograph of a woman working at a large computer in a computer room, captioned Computer room in 1949.

Original slide 117 (image)

Slide 118: We did not have good intuitions about what cheap arithmetic will get us. What will cheap prediction?

We did not have good intuitions about what cheap arithmetic will get us. What will cheap prediction?

Original slide 118 (image)

Slide 119: Four years ago, Google made a big announcement

Four years ago, Google made a big announcement

A page from Google’s blog, “Understanding searches better than ever before”, with several paragraphs of text.

Original slide 119 (image)

Slide 120: Major progress in 2019

Major progress in 2019

Two phone screenshots, Before and After, for the search “2019 brazil traveler to usa need a visa”: before, a news article about visas for Brazilians; after, a result that explains that a Brazilian traveller needs a visa.

Original slide 120 (image)

Slide 121: And 4 years later

And 4 years later

A Bing Chat conversation: asked whether a traveller to the US from Brazil needs a visa, it answers that Brazilians need a visa, with sources listed below.

Original slide 121 (image)

Slide 122: Also

Also

  • No idea of spatial relations
  • No idea of social relations
  • No understanding of physics

Original slide 122 (image)

Slide 123: Space is a problem: Some attempts at a bicycle

Space is a problem: Some attempts at a bicycle

Four attempts at a bicycle drawn by the model in code: a brown rectangle between two red circles with blue centres; a blue rectangle on two red wheels with a red top; a blue rectangle with two red dots below it; and a red frame with two crossed blue wheels and a green handlebar.

Original slide 123 (image)

Slide 124: Social relations are a problem

Social relations are a problem

Customer:

Take me downtown Prague.

Taxi driver:

How much does the trip cost?

Original slide 124 (image)

Slide 125: It does not LEARN live but can be corrected

It does not LEARN live but can be corrected

Original slide 125 (image)

Slide 126: What it took to train the foundation model

What it took to train the foundation model

  • 21 days
  • $1+ million

Original slide 126 (image)

Slide 127: What it took to fine tune it with human feedback

What it took to fine tune it with human feedback

  • X months
  • $$ million

Original slide 127 (image)

Slide 128: But it is a few-shot learner: It can follow examples

But it is a few-shot learner: It can follow examples

Original slide 128 (image)

Slide 129: And then… (GPT-3)

And then… (GPT-3)

Two pages from the paper “Language Models are Few-Shot Learners”: the title page with its abstract, and a figure with zero-shot, one-shot and few-shot prompts for translating English to French.

Original slide 129 (image)

Slide 130: Take it through a problem step by step.

Take it through a problem step by step.

Overlapping ChatGPT screenshots: the model is asked to count characters, says 39, then lists the characters one by one and, asked whether it wants to revise its answer, apologises and says there are actually 41.

Original slide 130 (image)

Slide 131: Time to do some prompt engineering

Time to do some prompt engineering

A scientist-in-a-lab-coat icon beside the statement.

Original slide 131 (image)

Slide 132: The key lesson for practical use

The key lesson for practical use

A teacher-at-a-board icon beside the statement.

Original slide 132 (image)

Slide 133: Keep 2 things in mind at once

Keep 2 things in mind at once

  • Give it prompts as to a human being
  • Interpret the results as if autocomplete

Two large block arrows pointing outwards: an orange one to the left, “Give it prompts as to a human being”, and a green one to the right, “Interpret the results as if autocomplete”.

Original slide 133 (image)

Slide 134: What ChatGPT can do is a matter of experimentation.

What ChatGPT can do is a matter of experimentation.

A maze icon beside the statement.

Original slide 134 (image)

Slide 135: Prompt engineering should aim towards giving more examples and context.

Prompt engineering should aim towards giving more examples and context.

A scientist-in-a-lab-coat icon beside the statement.

Original slide 135 (image)

Slide 136: Key lessons from prompt engineers

Key lessons from prompt engineers

  1. Narrow down context
  2. Give examples
  3. Specify style
  4. Specify persona
  5. Ask for corrections

Original slide 136 (image)

Slide 137: Don’t just ask, don’t just tell. Iterate!

Don’t just ask, don’t just tell. Iterate!

Icon of a cycle of a circle, a diamond and a square joined by arrows.

Original slide 137 (image)

Slide 138: Or in the words of ChatGPT

Or in the words of ChatGPT

A GPT-4 conversation: asked to write a limerick about five lessons of prompt engineering (narrow down context, give examples, specify style, specify persona, ask for corrections), it returns a four-line verse.

Original slide 138 (image)

Slide 139: LearnPrompting

LearnPrompting

The Learn Prompting website: “Learn Prompting: A Free, Open Source Course on Communicating with Artificial Intelligence”, with Join the Discord and Start Learning buttons and three illustrations.

Original slide 139 (image)

Slide 140: Three levels of editing

Three levels of editing

  1. Shallow prompt
  2. Fine-tuned prompt
  3. Co-editing

Original slide 140 (image)

Slide 141: Chain of Thought Prompts

Chain of Thought Prompts

Chain of Thought =

  • more accuracy
  • easier review

Original slide 141 (image)

Slide 142: But it can do this

But it can do this

Two ChatGPT screenshots: asked for a numbered list of all the characters including spaces and punctuation in a sentence, it lists them; asked whether it wants to revise its answer, it apologises and says there are actually 41 characters.

Original slide 142 (image)

Slide 143: And this

And this

Two ChatGPT screenshots: the model numbers the characters of a sentence up to 49, listing spaces separately; asked how many there are if trailing spaces are not counted, it answers 41.

Original slide 143 (image)

Slide 144: Asking for structure is better than prose!

Asking for structure is better than prose!

Original slide 144 (image)

Slide 145: Outlines are very context-based

Outlines are very context-based

A ChatGPT desktop window titled “ChatGPT in Education Poll”, with the chat history in a sidebar on the left. The prompt asks for poll questions and options about educators' existing experience and knowledge of ChatGPT; the reply is a numbered list of poll questions with bulleted answer options, cut off at question 4.

Prompt: I'm doing a presentation about what should educators know about ChatGPT. I want to open with a poll about their existing experience and knowledge. Can you suggest some questions and poll options?

Sure, here are some potential poll questions and options for educators regarding their experience and knowledge of ChatGPT:

  1. How familiar are you with ChatGPT?
    • Not at all familiar
    • Somewhat familiar
    • Very familiar
  2. Have you used ChatGPT in your teaching or curriculum?
    • No, I have not used it
    • Yes, I have used it in a limited capacity
    • Yes, I have used it extensively
  3. In your opinion, what are the potential benefits of using ChatGPT in education?
    • Improved student engagement
    • Increased efficiency of tasks such as grading and providing feedback
    • Enhanced opportunities for personalized learning
    • Other (please specify)
  4. Have you encountered any challenges when using ChatGPT in education? (The screenshot ends here; the options for question 4 are not visible.)

Sidebar chats visible: ChatGPT in Education Poll (current), People Coming to Party, Party RSVP Confirmations, London To New York Flight Ti…, ASCII car art creation, LLM Language Model Training…, Dagmar Skálová: A Heroine, Story task and activities, Deduplicate Fruits List, Table Words Request Confuse…

Original slide 145 (image)

Slide 146: You can convert them into interactive things

You can convert them into interactive things

A multiple-choice quiz made from an outline: “What type of fruit is grown on the highest mountain?” with options Apples, Pears, Stone fruit and Berries, and a Submit button, followed by further questions.

Original slide 146 (image)

Slide 147: Asking for concepts and examples

Asking for concepts and examples

Two ChatGPT screenshots: asked for a table with examples of each of Grice’s maxims, it produces a table with columns Maxim, Definition and Example of when the maxim is met; a follow-up adds a column for when the maxim is broken.

Original slide 147 (image)

Slide 148: Asking for timelines better than narratives

Asking for timelines better than narratives

Two ChatGPT (GPT-4) answers: a step-by-step account of how hieroglyphics were deciphered, and a timeline of key discoveries and contributions with years and names.

Original slide 148 (image)

Slide 149: Asking to adopt a persona is also very useful,

Asking to adopt a persona is also very useful,

A ChatGPT role-play. The prompt asks for a conversation in English at level A1 in which ChatGPT is a shopkeeper and the user a student buying bread, with simplified speech. The reply is a short shopkeeper-and-student dialogue about white, whole wheat and rye bread, one loaf at $2.50.

Prompt: Let's have a conversation in English at the level of A1. You are a shopkeeper and I am a student. I'm trying to buy bread. You need to simplify the way you speak so that I can understand.

Shopkeeper: Hello, how can I help you?

Student: Hi, I want to buy bread.

Shopkeeper: Sure, what type of bread would you like? We have white bread, whole wheat bread, and rye bread.

Student: What's the difference?

Shopkeeper: White bread is made with flour that has been stripped of its nutrients. Whole wheat bread is made with flour that includes the bran and germ, which makes it healthier. Rye bread is made with rye flour and has a distinctive flavor.

Student: I want the healthier one. Can I have whole wheat bread?

Shopkeeper: Sure, how many loaves would you like?

Student: Just one loaf, please.

Shopkeeper: That will be $2.50. Would you like anything else?

Student: No, that's all. Thank you.

Shopkeeper: You're welcome. Have a nice day!

Original slide 149 (image)

Slide 150: You can even have a back and forth

You can even have a back and forth

Two ChatGPT conversations: a role-play in which the user plays a shopkeeper in a small German village and the model plays a customer in German, then a request to correct the user’s German, answered with a corrected version of what they said.

Original slide 150 (image)

Slide 151: Where to learn more?

Where to learn more?

A book icon beside the question.

Original slide 151 (image)

Slide 152: Edutools.FYI

Edutools.FYI

The Edutools.fyi website, a Notion page with “ChatGPT and other Artificial intelligence Tools”, a tools database and readings, beside a QR code.

Original slide 152 (image)

Slide 153: bit.ly/academic-ai-newsletter

bit.ly/academic-ai-newsletter

https://bit.ly/academic-ai-newsletter

The LinkedIn newsletter “AI in Academic Practice” with a cartoon computer with a puzzled face, beside a QR code.

Original slide 153 (image)

Slide 154: Tools change, people remain the same

Tools change, people remain the same

EduTools.fyi

Original slide 154 (image)

Slide 155: This presentation is licensed under Creative Commons By Attribution license except where otherwise noted. Icons and stock photos licensed under Microsoft Premium Content and cannot be reused outside this document.

This presentation is licensed under Creative Commons By Attribution license except where otherwise noted. Icons and stock photos licensed under Microsoft Premium Content and cannot be reused outside this document.

Creative Commons Attribution licence badge.

Original slide 155 (image)