Conversational AI: Chatbots that work by Andrew R. Freed

Author:Andrew R. Freed [Freed, Andrew R.]
Language: eng
Format: epub
Publisher: Manning Publications Co.
Published: 2021-09-13T22:00:00+00:00

The second dimension lists the predicted intents.

Each square in the grid is the number of times the correct and predicted intent combination occurred.

Any number off the main diagonal represents confusion.

Optionally, the squares are color-coded such that the opacity matches the relative frequency of that occurrence, which makes the sources of confusion easier to spot. Figure 7.6 shows an initial confusion matrix.

Figure 7.6 A confusion matrix visualizes accuracy testing results.

With the confusion matrix, insights jump right off the page. The matrix is easily scannable. Most of the color opacity is on the main diagonal, and it’s easy to find the intent pair with confusion.

Fictitious Inc. can now test multiple utterances, create a table of the test results, and visualize the results with a confusion matrix. These tools are foundational in the quest to assess the accuracy of the assistant. This first test data set included only four utterances, which are not nearly enough to give a reliable estimate of the assistant’s accuracy. The company needs to build a good test data set so that it can understand how accurate the assistant will be when it is deployed to production.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.