Online Corpus: Literacy Teachers’ Best Friend

On the slide, the word “knowledge” is above the photograph and “language” below it; the words “of” (over the grass) and “about” (over the soil) are written on the photograph itself.

knowledge

language

http://www.flickr.com/photos/missturner/3029700617/

Original slide 4 (image)

Slide 5: Prescriptivism

Prescriptivism

… how language should be used

… how language is used

Descriptivism

Original slide 5 (image)

Slide 6: Slide 6

Cover of “The Language Instinct: The New Science of Language and Mind” by Steven Pinker.

“Most of the prescriptive rules of the language mavens make no sense on any level. They are bits of folklore that originated for screwball reasons several hundred years ago… For as long as they have existed, speakers have flouted them…”
— Steven Pinker, The Language Instinct

Original slide 6 (image)

Slide 7: Slide 7

The slide's original page: the covers of “Politics and the English Language” by George Orwell, “The Elements of Style” by William Strunk Jr. and E. B. White and “Strictly English” by Simon Heffer, a portrait labelled “Grammarian Geoffrey K Pullum on …”, and the quoted comments listed beside the books.

Grammarian Geoffrey K Pullum on …
- Orwell, Politics and the English Language
  - “dishonest and stupid”
  - “More passives in Orwell's pompous essay with the warning about how you mustn't use them than in any periodical you can lay your hands on! “
- Strunk and White, The Elements of Style
  - “vile little compendium of tripe about style”
- Heffer, Strictly English
  - “intellectual abdication”
  - “should be ashamed”
  - “current around 1900”
  - “a perversion of grammatical education”
  - “blind to textual evidence even when he himself exhibits it”

Original slide 7 (image)

Slide 8: Slide 8

http://languagelog.ldc.upenn.edu/nll/?p=2790

Screenshot of a Language Log page headed “Archive for Prescriptivist Poppycock”.

Screenshot of the post “Does it really matter if it dangles?”, November 20, 2010, filed by Geoffrey K. Pullum under Prescriptivist Poppycock, Syntax, Writing and ambiguity.

This usage stuff is not straightforward and easy. If ever someone tells you that the rules of English grammar are simple and logical and you should just learn them and obey them, walk away, because you're getting advice from a fool.

Original slide 8 (image)

Slide 9: Corpus

Corpus

Key modern tool for finding out about how language works…

Original slide 9 (image)

Slide 10: Corpus

Corpus

… is a large database of representative language samples …

Original slide 10 (image)

Slide 11: Corpus

Corpus

… 100s of millions of words from (mostly) written language in different genres in small samples (~2000 words) …

Original slide 11 (image)

Slide 12: Corpus

Corpus

… used for linguistic research, making dictionaries, writing grammars, …

Original slide 12 (image)

Slide 13: Slide 13

The slide's original page: the covers of “Collins COBUILD English Grammar”, “The American Heritage Dictionary” and “A Comprehensive Grammar of the English Language” (Quirk, Greenbaum, Leech and Svartvik); a red arrow points to the words “Collins Corpus: the world's largest database for language study” on the Collins cover.

Original slide 13 (image)

Slide 14: Corpora available for teachers

Corpora available for teachers

Screenshot of the corpus.byu.edu home page, “corpora, size, queries = better resources, more insight”, with a table of English corpora (Hansard, Wikipedia, GloWbE, COCA, COHA, TIME Magazine Corpus, Corpus of American Soap Operas, BYU-BNC) with their sizes, dialects and time periods.

http://corpus.byu.edu

Original slide 14 (image)

Slide 15: BYU corpora available

BYU corpora available

COCA (contemporary Am English)
COHA (historical Am English)
GloWbE (global web English)
Wikipedia
Google Books (BrEng/AmEng)
BNC (British National Corpus)
Hansard (British parliamentary speeches)
Spanish/Portugese

Original slide 15 (image)

Slide 16: Access to COCA and related BYU corpora is free…

Access to COCA and related BYU corpora is free…

…but free registration required for more than ~10 queries a day

Screenshot of a login form with Email and Password fields and a “(HELP) LOG IN” link.

Original slide 16 (image)

Slide 17: Slide 17

Screenshot of a Google search for “byu corpus”; the results list CORPORA: 1.9 billion words - 45 million words each: free online, BYU-BNC, COHA, COCA and GloWbE.

On the slide, a red ellipse is drawn round the search box that holds “byu corpus”.

Original slide 17 (image)

Slide 18: Other resources derived from BYU corpora

Other resources derived from BYU corpora

WordFrequency.info
WordAndPhrase.info
AcademicWords.info
Collocates.info

Original slide 18 (image)

Slide 19: Slide 19

Screenshot of the WebCorp Live search page (“Concordance the web in real-time”) with a search for dys* and options for case, span, search API and language.

http://www.webcorp.org.uk

Original slide 19 (image)

Slide 20: Slide 20

Screenshot of the Leeds “A collection of English corpora” query page: a choice of corpora (BNC, Reuters, British News, Internet, Brown Corpus, ukWac) and parameters for a concordance or collocations query.

http://corpus.leeds.ac.uk

Original slide 20 (image)

Slide 21: Slide 21

Photograph of a yellow roadside sign reading “You think life is full of surprises? Wait till you die!”

http://www.flickr.com/photos/atoach/3900591006/

Searching a corpus early on in the process of making a generalization can save you a lot of unpleasant surprises later.

Original slide 21 (image)

Slide 22: How do we use the word dyslexia?

How do we use the word dyslexia?

We speak more often of dyslexic children than adults.

We speak more often of dyslexia than any other dys- word.

Original slide 22 (image)

Slide 23: Concordance

Concordance

BNC: dyslexic [n*]
COCA: dyslexic [n*]

Screenshot of BNC results for the query above: DYSLEXIC CHILDREN 20, DYSLEXIC CHILD 6, DYSLEXIC PERSON 4, DYSLEXIC BOY 3, DYSLEXIC PEOPLE 3, DYSLEXIC PUPILS 3, DYSLEXIC YOUNGSTERS 1, DYSLEXIC UNIT 1.

Screenshot of COCA results for the query above: DYSLEXIC CHILDREN 15, DYSLEXIC STUDENTS 14, DYSLEXIC KIDS 5, DYSLEXIC POPULATION 4, DYSLEXIC RESEARCH 3, DYSLEXIC PIANIST 2, DYSLEXIC TREATMENT 2, DYSLEXIC READERS 2, DYSLEXIC WRITERS.

http://www.americancorpus.org/

http://corpus.byu.edu/bnc

Original slide 23 (image)

Slide 24: Slide 24

COCA: dys*

Screenshot of COCA results for dys*: DYSFUNCTION 1662, DYSFUNCTIONAL 1565, DYSON 703, DYSLEXIA 386, DYSTROPHY 264, DYSENTERY 258, DYSLEXIC 200, DYSPLASIA 175, DYSPHAGIA 126, DYSPHAGIA 121, DYSPNEA 105, DYSPHORIA 72, DYSTOPIAN 66, DYSPEPTIC 65, DYSPEPSIA 62.

Screenshot of the COCA search form with “dyslexia” in the word box and a list of collocates: FIBROUS, HIP, CERVICAL, TEMPORAL, BONE.

Original slide 24 (image)

Slide 25: Suffixing rules

Suffixing rules

*yed

*ied

Screenshot of COCA results for *yed: PLAYED 48431, STAYED 18470, DESTROYED 12207, ENJOYED 12106, EMPLOYED 10771, DISPLAYED 6737, DELAYED 4976, SURVEYED 4544, PORTRAYED 4101, UNEMPLOYED 3286.

Screenshot of COCA results for *ied: TRIED 68388, DIED 52829, MARRIED 39303, CARRIED 25469, IDENTIFIED 24469, WORRIED 21619, APPLIED 17856, STUDIED 15468, TIED 14965, REPLIED 12514.

Original slide 25 (image)

Slide 26: Suffixing rules

Suffixing rules

*yed

played

stayed

portrayed

enjoyed

unemployed

surveyed

*ied

died

tried

married

worried

identified

applied

Original slide 26 (image)

Slide 27: The Corpus Magic

The Corpus Magic

*

?

[ ]

[n* ]

Different corpora use slightly different codes. Read the manual.

Original slide 27 (image)

Slide 28: The Corpus Magic

The Corpus Magic

* Any number of characters (incl 0)
? Any one character
[ ] Lemma (all inflectional forms of a word)
[n* ] Part of speech tags (e.g. nouns)

Different corpora use slightly different codes. Read the manual.

Original slide 28 (image)

Slide 29: Slide 29

*

*each each, reach, beach, teach, outreach, …, impeach, …
teach* teachers, teaching, …, teachable, teacher-librarians, …
t*ch touch, teach, tech, torch, trench, twitch, …, three-inch, …
teach * teach the, teach us, teach students, …

Original slide 29 (image)

Slide 30: Slide 30

?

?each reach, beach, teach, peach, leach, keach, …
each? each- (1), each# (1) [ie nothing]
?each? peachy, bleachy, teacha, reachs (2) [ie spelling error], …
t?ch tech, tach, toch, tuch, tsch, tich
t??ch touch, teach, torch, tisch, …

Original slide 30 (image)

Slide 31: [Lemma]

`[Lemma]`

Word box: [book]

Screenshot of the COCA search page (Corpus of Contemporary American English, 425 million words, 1990-2011) with the word box set to the query above, outlined in red; the results BOOK, BOOKS, BOOKED and BOOKING are outlined.

Word box: [booking]

Screenshot of the COCA search page with the word box set to the query above, outlined in red; the results BOOKING and BOOKINGS are outlined.

Original slide 31 (image)

Slide 32: Part of speech tags

Part of speech tags

[run].[n*]

[run] [n*]

Screenshot of COCA results for the first query above showing text passages in which the word run is highlighted.

Screenshot of COCA results for the second query above: RUN AFOUL, RUN COUNTER, RUN THINGS, RUN AVERAGE, RUN DEFENSE, RUN ERRANDS, RUNS COUNTER, RUNNING ERRANDS with their counts.

Original slide 32 (image)

Slide 33: Common tags

Common tags

`[n*]`	noun
`[NN2]`	plural nouns
`[v*]`	verb
`[VVD]`	verb past tense
`[aj*]` (BNC)	adjective
`[j*]` (COCA)	adjective
`[av*]` (BNC)	adverb
`[r*]` (COCA)	adverb

On the slide the BNC tag and the COCA tag for adjective, and for adverb, stand on one line, separated by a slash; the table gives each tag its own row with its corpus label.

Original slide 33 (image)

Slide 34: Help

Help

Codes in the panel:

[pos]
[pos*]
[lemma]
[=word]

Examples in the panel:

[vvg]
[v*]
[sing]
[tall]
[=strong]

Screenshot of the COCA search page with a red arrow pointing from the search-string box to the “HELP: SEARCH SYNTAX” panel, which lists search codes with their meanings, examples and matching words (the codes and examples are given in the text above).

Original slide 34 (image)

Slide 35: Slide 35

Search string	Results shown
`*`	strike, strikes, struck, striking
`big [n*]`	look, handbook, shook, textbook
`[love]`	love
`*ook`	look, took, book, cook, rook
`?ook`	big bang, big man, big house
`[strike].[v*]`	anything
`[strike].[n*]`	love, loves, loved, loving
`love`	strike, strikes

Screenshot of a matching exercise: a column of eight search strings beside boxes of results, each pair marked with a red cross. The strings and the results beside them are listed in the table above.

Original slide 35 (image)

Slide 36: Slide 36

Search string	Results shown
`*`	anything
`big [n*]`	big bang, big man, big house
`[love]`	love, loves, loved, loving
`*ook`	look, handbook, shook, textbook
`?ook`	look, took, book, cook, rook
`[strike].[v*]`	strike, strikes, struck, striking
`[strike].[n*]`	strike, strikes
`love`	love

Screenshot of the same matching exercise answered correctly: each of the eight search strings has its matching results beside it and a green tick. The strings and results are listed in the table above.

Original slide 36 (image)

Slide 37: You can also

You can also

cats and dogs search for idioms
?each*s combine wildcards
[=pretty] search for synonyms
car|bike|horse search for alternatives
used -car exclude searches

For more details see:

Screenshot of the COCA search-string box with a red arrow pointing to a small button at its right edge.

Original slide 37 (image)

Slide 38: Concordance + KWIC

Concordance + KWIC

*ies.[N*]

Screenshot of COCA results, rows 8 to 16 of a list of nouns ending in -ies with their counts (BABIES, SPECIES, PARTIES, LIES, STUDIES, CENTURIES, CITIES, ENEMIES, CRIES), above a block of example passages from fiction in which “cities” is highlighted; red arrows point at CITIES in the list and at passages below. The search string is not visible in the screenshot; it is given in the line above.

Original slide 38 (image)

Slide 39: KWIC – Key-Word In Context

KWIC – Key-Word In Context

*ies.[N*]

Screenshot of a key-word-in-context display for the query shown above: lines of text with the matching words (species, communities, series, movies and others) highlighted in colour in a central column.

Original slide 39 (image)

Slide 40: Limit searches by genre

Limit searches by genre

Screenshot of the COCA sections panel with two lists: the genres SPOKEN, FICTION, MAGAZINE, NEWSPAPER, ACADEMIC and the period 1990-1994, and the fiction sub-genres FIC:Gen (Book), FIC:Gen (Jrnl), FIC:SciFi/Fant, FIC:Juvenile and FIC:Movies.

Original slide 40 (image)

Slide 41: Other questions corpus can answer

Slide 42: Corpus, rules, and regularity

Corpus, rules, and regularity

pre*
*ed
*ies.[V*]

Black-and-white photograph of a hand writing “I will not spend all my spare time on Flickr.” repeatedly on a page of lined paper.

http://www.flickr.com/photos/51505078@N00/352492687

Original slide 42 (image)

Slide 43: Collocations

Collocations

Limits on variability

Screenshot of COCA collocates of “surprising”: the search form with the collocates box set to * and the results THAT 2310, ., ., TO, NUMBER, AND, IS, IN, BECAUSE, GIVEN, THING, HOW; THAT, NUMBER, BECAUSE and GIVEN are outlined in red.

Slide 44: Collocations

Collocations

Limits on variability

Slide 45: Collocations (cont)

Collocations (cont)

[teacher] must [v*]

Screenshot of COCA results for the query above: TEACHERS MUST BE 144, TEACHER MUST BE 53, TEACHERS MUST HAVE 29, TEACHERS MUST LEARN 18, TEACHERS MUST BECOME 16, TEACHER MUST HAVE 12, TEACHERS MUST PROVIDE 10, TEACHERS MUST TAKE 10, TEACHERS MUST UNDERSTAND 10, TEACHERS MUST MAKE 9, TEACHERS MUST REALIZE 8, TEACHER MUST DETERMINE 7 and others.

Original slide 45 (image)

Slide 46: Idioms and set phrases

Idioms and set phrases

275 results

359 results

Screenshot of concordance lines for “cats and dogs” in phrases such as “raining like cats and dogs”, with the words before the phrase outlined in red.

Screenshot of concordance lines for “dogs and cats” in contexts such as “adorable dogs and cats” and “pets”, with the words before the phrase outlined in red.

Original slide 46 (image)

Slide 47: Google as a Corpus

Google as a Corpus

"put the search text in quotes"

use * for the search item

Screenshot of Google's search suggestions for “a teacher is *”: a teacher is like, a teacher is poem, a teacher is like a gardener, a teacher is someone who, a teacher is like a candle quote, a teacher is like a flower, a teacher is like metaphor, a teacher is one who makes, a teacher is a leader, a teacher is a candle.

Original slide 47 (image)

Slide 48: Slide 48

Screenshot of Google results for “a good student always *”, with matching passages outlined in red and two callouts, “Relevant results” beside the genuine matches and “Irrelevant results” beside matches in other contexts.

Original slide 48 (image)

Slide 49: Google as a Corpus

Google as a Corpus

PRO: rare, low frequency usage, up-to-date usage
CON: no sampling, no frequency sort, no genre limit, no part of speech tags

Original slide 49 (image)

Slide 50: Google results counts are only rough estimates…

Google results counts are only rough estimates…

Different people searching in different geographic locations can get different numbers

Sometimes searching for A gives fewer results than searching for A without B

Screenshot of a Google search for “good teachers” with a red arrow pointing to “About 80,600,000 results (0.13 seconds)”.

Screenshot of a Google search for “bad teachers” with a red arrow pointing to “About 44,700,000 results (0.09 seconds)”.

http://searchengineland.com/why-google-cant-count-results-properly-53559

Original slide 50 (image)

Slide 51: …but Google fights can be fun

…but Google fights can be fun

Screenshot of the Googlefight web page comparing “good teachers” (13300000 results) with “bad teachers” (4660000 results).

Original slide 51 (image)

Slide 52: WebCorp is makes Google search results linguist-friendly

WebCorp is makes Google search results linguist-friendly

Screenshot of a page of lines of text in which “good teachers” is highlighted in colour.

Screenshot of the WebCorp Live search form with “good teachers” in the Search box; Case Insensitive is ticked, Span is 50 characters, Search API is Google, Language is Not specified, Show URLs is unticked, Pages is 64, One concordance line per web page is ticked and the Site box is empty.

On the slide, three red arrows are drawn over the WebCorp form: one pointing at the Span box, one at the Show URLs checkbox and one at the “One concordance line per web page” checkbox.

Original slide 52 (image)

Slide 53: Avoid Common Corpus Errors

Avoid Common Corpus Errors

Photograph of graffiti-covered wall with a pasted-up image of an Internet Explorer error dialog.

http://www.flickr.com/photos/andreassolberg/433734311

Original slide 53 (image)

Slide 54: Slide 54

Be aware of limitations: sampling, coverage, size, presence of typos and errors, bad part of speech tagging
Beware of low frequency results
Beware of homographs

Original slide 54 (image)

Slide 55: Slide 55

Check results come from multiple sources
Check KWIC to confirm relevance
Limit search by genre

Original slide 55 (image)

Slide 56: Check examples and sources

Check examples and sources

Click on concordance results for KWIC view
…to check relevance of result (e.g. part of speech tag)

Screenshot of COCA concordance results listing wordsa list of words (TIED, DRED, SLED, NEED and others) with counts above lines of context in which the word TIED is outlined in red.

Original slide 56 (image)

Slide 57: Always check low frequency results

Always check low frequency results

must [v*] [n*]

Screenshot of COCA results for the query above: MUST TAKE CARE 5, MUST BE FRANKIE 3, MUST BE KING 3, with a red arrow pointing at MUST BE FRANKIE.

Screenshot of context lines from a transcript in which “Must be Frankie” appears three times in the same passage, with red arrows.

…sometimes they come from the same source

Original slide 57 (image)

Slide 58: False roots

False roots

Screenshot of COCA results for *face: FACE, SURFACE, INTERFACE, FACE-TO-FACE, PREFACE, CLUBFACE, SUBSURFACE, IN-YOUR-FACE, ABOUT-FACE, BABYFACE, with PREFACE outlined in red.

Screenshot of the Online Etymology Dictionary entry for “preface (n.)”, from late 14th century Old French preface, with the Latin praefatio outlined in red.

http://etymonline.com

corner, silly, preface, cockroach, protest, stable …

Original slide 58 (image)

Slide 59: Make your own corpus with TextSTAT

Make your own corpus with TextSTAT

Screenshot of the TextSTAT program window showing a concordance for the word “test” in a corpus of text files.

http://neon.niederlandistik.fu-berlin.de/en/textstat

Original slide 59 (image)

Slide 60: Make your own corpus with AntConc

Make your own corpus with AntConc

Screenshot of the AntConc program window showing a concordance with the search term in the centre column of each line and tabs for Concordance, Concordance Plot, File View, Clusters, Collocates, Word List and Keyword List.