Multiple Departments Chatlog Review

1. Every institution will go through the chatlog review process on their respective institution’s chatlog data. Refer to “Excel chatlog” page for this.
2. All departments will come together to merge the reviewed examples.
   1. For the first few weeks of production launch, or pilot launch, it is recommended that all institutions meet weekly for this exercise.
   2. After the stabilization period, all institutions should come monthly for this exercise.
3. During the combined session, all institutions will come together and merge the reviewed examples. 1 department will lead the unifying exercise each week.

How to merge the worksheet?

1. The leading institution will create an online shared excel file and put in the headers (top row).
   1. Please note that every institution will have to come prepared with the completed review, and given access to the shared excel file.
2. Double-check if your reviewed chatlog file has the same columns. If not, adjust your columns accordingly to make the transfer process more manageable.
   1. Please note that the header should follow the sequence downloaded from KeyReply dashboard (Home page > Daily Training Data)
   2. A “correct_intent” column should be added in between the “prediction” and “question” column, or column E
3. All institutions will filter out the blank cells of the “correct_intent” column.
   1. The purpose behind this is because these rows are considered as correctly answered or KIV or ignored (e.g. should go to fallback or out of topic)
   2. Also, during the unifying process, the process will be more focused on those that require training.
4. All institutions will copy the filtered rows and paste them into the merged file.

What should the leading department do next?

1. The leading department will check through the file and filter out blank cells of “correct_intent” column.
2. Sort the “correct_intent” column by alphabetical order.
3. Check through all institutions’ classifications to make sure that the understanding of the intent is correct.
   1. There should not be any keywords added – keywords should be added to entities or triggers.
   2. This column should only have the intent name (refer to FAQ in the dashboard), and not content node name.
   3. If any institutions understood the intent wrongly, re-classify it to the correct intent by updating the “correct_intent” column.
   4. If new intent is requested, all institutions must agree with the new intent and prepare answers for it. Please name the intent meaningfully.
4. Once the “correct_intent” column has been checked through, the leading department will add the examples in the dashboard’s FAQ. You may copy the rows from Excel to the examples field in the FAQ.

Creating answers

1. If there is a new intent created (step 3d above), all departments will need to prepare the intent.
2. The leading department will create an intent with a meaningful name. The content node should be added, having the same name.
3. A leading department can add the main placeholder answer. With “Please select the corresponding answer” and button options to the versions. For example:
4. And connect each button to the corresponding version. For example:
5. All departments will proceed to add your version and fill in the answers. Please add corresponding App Source.

For more information about this article, please reach out to us at cspm@keyreply.com