The problem with virtual conversations
To recap the three avatar communication options are:
- Person to Person
- Conversation Tree
- AI Based
Unfortunately all of three of these options are less than perfect. Person to person is great for roleplaying games but in a learning environment it can add co-worker biases and get’s expensive quickly.
Conversation tree’s are time consuming to design and implement. Plus, no matter how much work you put in selecting options will always feel limiting.
Lastly, AI based sounds great but the technology is not currently ready to have full on conversations. Instead different AI or chatbots specialize in specific areas so they can answer specific types of questions.
XpertVR’s solution
While we wait for the future where fully Artificial Intelligent avatars exist, sound natural and communicate like a regular person, XpertVR decided to build a solution for today. What we call it is the Conversation Matrix Tool (CMT). To put it simply CMT is a conversation tree creation tool but with some exciting functionality and whole lot of reusability to build on.
At the moment CMT is an internal tool we have built for Unity Engine so that our team can easily create avatar conversations in virtual reality. And by easy we mean so simple that someone with no Unity skills can jump in and start making conversations from the get go. But that’s not all, because we built CMT on Unity we are able to automatically attach it different 3D avatars, events, character animations and more.
This means instead of just having a character respond to you, it could respond to something you do during your training. As an example let’s pretend you’re a police officer in one of our police training simulations. In the simulation you need to investigate a crime scene and then question witnesses. With the CMT if you collect the right evidence the CMT will unlock new options for you to choose, plus possibly change the witnesses reactions, animations and responses. Not to mention you can swap out different avatars (currently using Ready Player Me avatars) with the click of a button, while transferring facial animations and other information.
Having a dynamic conversation tree like this, that is directly tied into a greater simulation means that organizations looking at VR training options can now mix their hard and soft skills training so that it feels more natural compared to the real world.


Next Steps
We have working on our Conversation Matrix Tool for almost a year now but we know there is so much further we can take it. In the short-term we are integrating saving functionality so that users can save where they were in a conversation. Plus we are adding data collection features and improving animations. The data collection allows teachers to help give direction to students in class and helps students learn by themselves while practicing at home. As for the animations, this is an area we will always be improving because it leads to higher realism and sense of immersion, especially when it comes to facial animations.
In the long-term we are looking into how we can integrate AI into CMT, possibly starting with Amazon Lex or something similar. That being said we won’t stop there and are always looking for suggestions for our partners and anyone with new ideas!