Natural Language Processing with Java by Richard M. Reese

Author:Richard M. Reese
Language: eng
Format: epub
Tags: COM051280 - COMPUTERS / Programming Languages / Java, COM042000 - COMPUTERS / Natural Language Processing, COM044000 - COMPUTERS / Neural Networks
Publisher: Packt Publishing
Published: 2018-07-31T06:08:23+00:00

A maximum entropy tagger uses statistics to determine the POS for a word and often uses a corpus to train a model. A corpus is a collection of words marked up with POS tags. Corpora exist for a number of languages. These take a lot of effort to develop. Frequently used corpora include the Penn Treebank (https://www.seas.upenn.edu/~pdtb//) or Brown Corpus (http://www.essex.ac.uk/linguistics/external/clmt/w3c/corpus_ling/content/corpora/list/private/brown/brown.html).

A sample from the Penn Treebank corpus, which illustrates POS markup, is as follows:

Well/UH what/WP do/VBP you/PRP think/VB about/IN the/DT idea/NN of/IN ,/, uh/UH ,/, kids/NNS having/VBG to/TO do/VB public/JJ service/NN work/NN for/IN a/DT year/NN ?/.

There are traditionally nine parts of speech in English: nouns, verbs, articles, adjectives, prepositions, pronouns, adverbs, conjunctions, and interjections. However, a more complete analysis often requires additional categories and subcategories. There have been as many as 150 different parts of speech identified. In some situations, it may be necessary to create new tags. A short list is shown in the following table. These are the tags we will be using frequently in this chapter:

Part

Meaning

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.