Thoughtful Machine Learning in Python by Kirk Matthew

Author:Kirk, Matthew
Language: eng
Format: epub, mobi
Publisher: O'Reilly Media
Published: 2017-01-16T05:00:00+00:00

from collections import defaultdict dictionary = {'a': 'b'} dictionary['b'] # Yields KeyError default_dictionary = defaultdict(lambda: 0, dictionary) default_dictionary['b'] == 0

You’ll notice that we’re doing a bit of error handling for the case when zeros happen, because we will throw a divide-by-zero error. Next, we need to address the probability of word tag combinations, which we can do by introducing the following to our existing test:

import StringIO class TestPOSTagger(unittest.TestCase): def setUp(): self.stream = StringIO("A/B C/D C/D A/D A/B ./.") self.pos_tagger = POSTagger([StringIO.StringIO(self.stream)]) self.pos_tagger.train() def it_calculates_probability_of_word_and_tag(self): self.assertEqual(self.pos_tagger.word_tag_probability("Z", "Z"), 0) # A and B happens 2 times, count of b happens twice therefore 100% self.assertEqual(self.pos_tagger.word_tag_probability("A", "B"), 1) # A and D happens 1 time, count of D happens 3 times so 1/3 self.assertEqual(self.pos_tagger.word_tag_probability("A", "D"), 1.0/3.0) # START and START happens 1, time, count of start happens 1 so 1 self.assertEqual(self.pos_tagger.word_tag_probability("START", "START"), 1) self.assertEqual(self.pos_tagger.word_tag_probability(".", "."), 1)

To make this work in the POSTagger class, we need to write the following:

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.