Chatbot Training Guide
Debug And Optimization Chatbots
It's always possible that your data might not cover all possible scenarios. Consequently, ongoing enhancements are essential to improve your chatbot's response quality.
Identify Issues
Regularly review the conversation history in the Chat Logs section to monitor the chatbot's interactions in production. If the chatbot's response does not meet your expectations, use the corresponding user query to investigate and debug the issue. 
Debugging Issues
To debug an issue, use the complete chat history by reproducing the problematic response through our debugging interface at https://www.chat-data.com/chatbot/{chatbotId}. Although a direct query may yield a satisfactory answer, altering the context and repeating the query can result in inappropriate responses. To investigate this, replicate the full conversation to examine the sources that influenced the incorrect response.
The discrepancy between responses from direct and threaded queries arises because the system uses past interactions to generate an embedding vector. This vector assists in identifying relevant text chunks for responses, ensuring the chatbot recalls previous conversations. In instances where past messages influence current topics, the context lookup system may fail to locate the most relevant texts, especially when training with unstructured data. If such issues arise, consider using the Q&A data type, which is a structured data type and has higher chance to be selected as relevant content for responses. 
If the expected Q&A is not visible in the sources, consider these possibilities:
- The Q&A may not be saved in our database—verify that it is indeed stored.
- If you tested the chatbot immediately after uploading data, the training might not have completed.
- If past conversations covered various topics, our lookup tool might not retrieve the expected Q&A, as it only presents the five most pertinent text chunks. In such cases, refine your Q&A to be more specific or remove irrelevant text chunks that could skew the results. For assistance, contact us via email at [email protected].
Should the appropriate Q&As be sourced, ensure your base prompt is configured correctly and verify whether you are using the GPT-4.0 model. The GPT-4o mini model may exhibit inaccuracies due to 'hallucinations' and might not adhere strictly to the provided context and base prompt. For further assistance, reach out via email at [email protected].
Fix Issues
Utilize the "Revise" feature to directly update the response. The question and its corrected response will automatically be converted into a Q&A pair.
Frequently Asked Questions
How should I start debugging a chatbot response that looks wrong?
Starting in `Chat Logs` by reviewing production conversations and locating the exact user query that produced the bad response. That query then becomes the starting point for reproducing and investigating the issue.
Why does the guide tell me to reproduce the full conversation when debugging?
It explains that the chatbot uses past conversation context to build the embedding query, so a direct one-off question can behave differently from the threaded conversation. Reproducing the full chat helps you inspect the actual sources and retrieval behavior that led to the incorrect answer.
What should I check if the expected Q&A is not showing in the sources?
Several checks: confirm the Q&A is actually saved in the database, confirm training had enough time to finish after upload, and consider whether previous messages or irrelevant text chunks are crowding out the expected result from the top retrieved sources.
How do I fix a bad chatbot answer after identifying the problem?
The guide says to use the `Revise` feature in Chat Logs to directly correct the response. That revision is then turned into a Q&A pair automatically, which helps improve future answers for similar cases.