Searching POS-tagged corpora

Formatting conventions for POS-tagged corpora

CorpusSearch can search files that are POS-tagged but not further annotated for syntactic structure. In this section, the special formatting conventions for POS-tagged files are first discussed and then illustrated in a template file.

Because CorpusSearch expects its input to be in Penn Treebank format by default, POS-tagged files must be specially flagged. This is done by including a format flag as the first line of the POS-tagged file, as illustrated below: 

#!FORMAT=POS_1
This first line may be followed by philological or other header information. The beginning of the text to be searched is indicated by the following delimiter: 
<text>
Thereafter, the file must contain word/tag pairs formatted according to standard conventions. Space functions as the delimiter between word/tag pairs, so neither words nor tags can themselves contain spaces. The delimiter between a word and its tag is "/" (forward slash).

The delimiter between sentences is a blankline. In addition, every sentence must end with a sentence-final punctuation tag. By default, this tag is "." (period). Alternatively, the sentence-final punctuation tag may be "PONFP". The PONFP alternative must be indicated in the file-initial format line by replacing the default "POS_1" format by "POS_0". The terminal node ("word") associated with the sentence-final punctuation tag need not be a period. See the template file for examples.

Sentence-internal punctuation is also treated as a word/tag pair, but the tag must differ from the sentence-final punctuation tag, as illustrated in the following examples, which distinguish "," (comma) and "." (period) or PON and PONFP. 

After/P the/D party/N ,/, we/PRO all/Q went/VBD back/ADV ./.

Après/P la/D fête/N ,/PON nous/PRO sommes/BEP tous/Q retornés/VBN ./PONFP

Finally, the end of the text is indicated by the closing counterpart of the opening <text> delimiter: 

</text>

Template file

The following template file illustrates the formatting conventions just discussed, using the POS_0 alternative. 
#!FORMAT=POS_0

This file contains the best edition of the "Chanson de Roland",
with emendations by Christiane Marchello-Nizia as noted.

<text>

word1/tag1 word2/tag2 word3/tag3 .... ./PONFP

word1/tag1 word2/tag2 word3/tag3 .... ?/PONFP

word1/tag1 word2/tag2 word3/tag3 .... ,/PONFP

.
.
.

</text>

Search functions

CorpusSearch treats POS-tagged files as containing sentences parsed with a completely flat structure, with every word/tag pair as an immediate daughter of the root node. Query files for POS-tagged corpora are therefore essentially the same as for parsed corpora. In particular:

exists (variant: Exists)

Exists searches for a POS tag or text anywhere in the sentence. For instance, this query: 
query: (MD0 exists)
finds this sentence: 
/~*
I shal not conne wel goo thyder ./. (ID CMREYNAR,14.261)
*~/

/*
    4 MD0 conne
*/

( (PRO I) (MD shal) (NEG not) (MD0 conne) (ADV wel) (VB goo) (ADV thyder)
  (ID CMREYNAR,14.261) )

iDominates (variants: idominates, iDoms, idoms)

As mentioned above, POS tags immediately dominate the words of the text. So this query: 
query:     (PRO iDominates he)
       AND (FP iDominates ane)
finds this sentence: 
/~*
Sythen he ledes +tam by +tar ane,
(CMROLLEP,118.978)
*~/

/*
    2 PRO he, 7 FP ane
*/
( (ADV Sythen) (PRO he) (VBP ledes) (PRO +tam) (P by) (PRO$ +tar) (FP ane) (. ,) 
  (CMROLLEP,118.978) )

/*

iPrecedes (variants: iprecedes, iPres, ipres)

"iPrecedes" is true if and only if its first argument immediately precedes its second argument in the sentence. So this query: 
query:     (as iPrecedes sone) 
       AND (sone iPrecedes P)
finds this sentence: 
/~*
and as sone as he myght he toke his horse .
(CMMALORY,206.3401)
*~/
/*
2 as, 3 sone, 4 P as
*/

( (CONJ and) (ADVR as) (ADV sone) (P as) (PRO he) (MD myght) (PRO he)
  (VBD toke) (PRO$ his) (N horse) (. .)
  (CMMALORY,206.3401) )

neighborhood (variant: Neighborhood)

The "neighborhood" function is available only for POS-tagged files. It takes three arguments - two words or tags and a number - and searches for sentences in which the two words/tags are separated from each other by the specified number of words (or fewer). For instance, this query: 
query: (whoreson neighborhood 2 wilt)
returns all tokens in the corpus in which the word "whoreson" is separated from the word "wilt" by at most two words, as in the following sentence: 
/~*
why thou whoreson when wilt thou be maried?
(DELONEY,79.296)
*~/
/*
3 whoreson,  5 wilt
*/

( (WADV why) (PRO thou) (N whoreson) (WADV when) (MD wilt) (PRO thou) (BE be) (VAN maried) (. ?)
  (ID DELONEY,79.296) )

precedes (variants: Precedes, Pres, pres)

"Precedes" imposes a less strict condition than "iPrecedes". "x precedes y" means "x comes before y in the sentence but not necessarily immediately".

Example query: 

query: (VB precedes N)

Example output: 

/~*
thenne have ye cause to make myghty werre upon hym.
(CMMALORY,2.25)
*~/

/*
    6 VB make, 8 N werre
*/

( (ADV thenne) (HV have) (PRO ye) (N cause) (TO to) (VB make) (ADJ myghty) (N werre) (P upon)
  (PRO hym) (. .)
  (ID CMMALORY,2.25))