Innovative avatars conversations in virtual reality
Learn how we talk to virtual avatars at XpertVR
An Introduction:
If you have played any video game with characters you are likely familiar with at least one type of virtual avatar conversation. The most common being a conversation with another person. In this situation, you have an avatar that represents who you are, and so does the other person. You communicate through voice, text chat, or sometimes emojis. In professional training environments, these avatars are typically paid actors or coworkers who contribute to the realism of the virtual training.
The second type of avatar conversation is based on a conversation tree. In these conversations, you select what you want to say from options presented to you (i.e. “Hello”, “Don’t talk to me”, “How is your day going”, etc) and then the conversation follows the path or branch you select. The third type of avatar conversations is AI-based. In these conversations, an Artificial Intelligence is able to understand what you say/type and gives a response based on what it has learned is likely to be the right answer.
The problem:
To recap, the three avatar communication options are:
- Person to Person
- Conversation Tree
- AI Based
Unfortunately, all three of these options are less than perfect. Person-to-person is great for role-playing games, but in a learning environment, it can add co-worker biases and get expensive quickly.
Conversation tree’s are time consuming to design and implement. Plus, no matter how much work you put in selecting options will always feel limiting.
Lastly, AI-based technology sounds promising, but it is not yet ready to engage in full-on conversations. Instead, different AI systems or chatbots specialize in specific areas, enabling them to answer questions of a particular type.
"The data collection enables teachers to provide direction to students in class and helps students learn independently while practicing at home."
Our solution:
While we wait for the future where fully Artificial Intelligent avatars exist, sound natural and communicate like a regular person, XpertVR decided to build a solution for today. What we call it is the Conversation Matrix Tool (CMT). To put it simply, CMT is a conversation tree creation tool, but with some exciting functionality and a great deal of reusability to build upon.
Currently, CMT is an internal tool we have developed for the Unity Engine, enabling our team to easily create avatar conversations in virtual reality. And by easy, we mean so simple that someone with no Unity skills can jump in and start making conversations from the get-go. However, that’s not all. Since we built CMT on Unity, we can automatically attach it to various 3D avatars, events, character animations, and more.
This means instead of just having a character respond to you, it could respond to something you do during your training. As an example, let’s pretend you’re a police officer in one of our police training simulations. In the simulation, you need to investigate a crime scene and then question witnesses. With the CMT, if you collect the right evidence, the CMT will unlock new options for you to choose, plus possibly change the witnesses’ reactions, animations and responses.
Not to mention, you can swap out different avatars (currently using Ready Player Me avatars) with the click of a button, while transferring facial animations and other information.
Having a dynamic conversation tree like this, that is directly tied into a greater simulation, means that organizations looking at VR training options can now mix their hard and soft skills training so that it feels more natural compared to the real world.
Next steps:
We have been working on our Conversation Matrix Tool for almost a year now, but we know there is so much further we can take it. In the short term, we are integrating a save functionality so that users can save their current position in a conversation. Additionally, we are introducing data collection features and enhancing animations.
The data collection enables teachers to provide direction to students in class and helps students learn independently while practicing at home. As for the animations, this is an area we will continually improve, as it leads to higher realism and a greater sense of immersion, especially when it comes to facial animations.
In the long term, we are exploring how to integrate AI into CMT, possibly starting with Amazon Lex or a similar platform. That being said, we won’t stop there and are always looking for suggestions for our partners and anyone with new ideas!
The bottom line:
As you can see, there are a lot of ways to communicate with avatars in VR and they all have their pros and cons. We would love to hear what solutions have worked for you and how they could be improved!
Plus, if you are looking for virtual solutions, reach out to our team. We would be happy to offer some insights and see how we can support you and your goals.