Natural Language Processing Fundamentals by Sohom Ghosh

Author:Sohom Ghosh
Language: eng
Format: epub
Publisher: Packt Publishing
Published: 2020-01-15T00:00:00+00:00

Topic Discovery

The main goal of topic modeling is to find a set of topics that can be used to classify a set of documents. These topics are implicit because we do not know what they are beforehand, and they are unnamed. We just generally assume that some documents are similar to each other and that we can organize them into topics.

The number of topics is usually small; that is, from 2 to 10. However, there are some use cases in which you may want to have up to 100 (or even more) topics. Since it is the computer algorithm that discovers the topics, the number is generally arbitrary. These topics may not always directly correspond to topics a human would identify. In practice, the number of topics should be much smaller than the number of documents. This helps the topic modeling algorithm in the sorting process. The more examples of documents that we provide, the better the accuracy with which the algorithm can sort and place the documents into categories.

The number of topics chosen depends on the documents and the objectives of the project. You may want to increase the number of topics, if you have a large number of documents or if the documents are fairly diverse. Conversely, if you are analyzing a narrow set of documents, you may want to decrease the number of topics. This generally flows from your assumptions about the documents. If you think that the document set might inherently contain a large number of topics, you should configure the algorithm to look for a similar number of topics. Essentially, here, you are guiding the algorithm to discover what is already inherent in the documents, and you may have already gained a fair idea of that from sampling a few documents and seeing what types of topics they contain.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.