The Virtual Third Man in the Room

Creating more effective advice conversation with intelligent virtual agents

Imagine yourself in an advice situation where a banking consultant is giving you advice on how to wisely invest. Giving both you and the consultant more information on certain scenarios during the conversation would have the potential to make the meeting more effective. With existing technology, this problem could be solved by introducing an intelligent virtual agent. But what is exactly the role of the agent and how does it interact with the customer and the consultant? This unique question gave rise to an innovation project in the group of Information Management Research Group (IMRG). Within this project, Heinzelmännli was created, an intelligent virtual agent specifically designed to fulfill this desire.

Concretely, the Heinzelmännli understands what happens in the conversation between a banking consultant and the client of a bank. The virtual agent then assists in creating a risk profile for the client and autonomously recognizes too riskyinvestment options when they occur in the conversation between the client and the consultant. Hence, the Heinzelmännli acts as a 3rd and independent agent in the consulting session. The overall goal of this technology is to reduce the information asymmetry between consultants and clients in the negotiation process.

By chance, the project also pioneers Swiss-German Language Recognition

This joint project between multiple partners got started after observing that consultants wish for a system that can recognize and transfer information you speak out loud. Starting at a time where language recognition systems such as Siri and Alexa, were suddenly improving immensely, this seemed like a fruitful project. However, one of the main technological challenges included the understanding of the Swiss-German dialect, such that Heinzelmännli can understand the statements of the client and the consultant. To tackle this, the project leaders collaborated with recapp, a company which is focused on Swiss-German language recognition technology. Having understood the speech, Heinzelmännli needs to recognize how to react to the message. This is where Ergon and the Fachhochschule Nordwestschweiz (FHNW), two other project partners, come into play to provide Heinzelmännli with the ability to handle requests and react to them in a way that is pleasant for humans. Finally, Heinzelmännli needs to connect with banking systems to check and execute operations on the client’s data. Innofactory, the fourth project partner, takes care of this.

Trial phase supported by Innosuisse

One key challenge which arose during the research project was that the interests of the firms, partners, and researchers did not always align. For instance, for the researchers it was extremely important to make decisions which are all evidence based and backed by previous research on advisory services, while the industrial partners were more focused on addressing the technical challenges as early as possible. The researchers were able to overcome this by creating many artifacts, prototypes, and storyboards to demonstrate the functionality of certain elements. In the end, all parties were able to profit and learn from each other.

Currently, the researchers and all involved parties in the project are preparing for a testing phase, which will begin this July. Due to the pandemic, face-2-face interaction, which is the setting for which the Heinzelmännli was developed for, has been significantly reduced. If this Innosuisse funded project succeeds in its initial trial phase, the developers hope to be able to hit the market in the upcoming years.