Conversational UI: Don’t count on getting a second chance to make a first impression

Science tells us we have just seven seconds of meeting someone to make a positive first impression. In real life, it’s all about body language, eye contact, tone of voice, volume, active listening, personality, positivity, memorability, and good manners!

It’s not much different in today’s digital world. Advancements in artificial intelligence (AI) have enabled a new generation of chatbots to permeate our everyday lives.  Whether in finance, banking, healthcare, hospitality or retail, the old cliché of “never getting a second chance to make a first impression” is as apt as ever. And, most of us can relate to the reasons why chatbots can fail us at times.

In my last blog, I talked about the importance of matching tone with intention in text- and voice-based communications to create customer experiences that are not only accurate, but likeable and well-matched to the situation. In other words, good news feels like good news, concerning news feels concerning, and so on.

Now I’d like to explore a few tools that can help designers gain a better understanding of the sentiment they’re conveying with conversational user interface (CUI) technology to create positive, authentic and relatable customer experiences—and avoid the common pitfalls involved with chatbot design. We’ve integrated each of the open source tools described below into an all-purpose sentiment analysis tool we call Mumbler (https://expo.futures.a.intuit.com/mumbler) that’s publicly available to anyone that registers for a free Intuit Mint account.

It’s not just what you say, but how you say it

Our journey began in 2017 with a search for open source tool kits and libraries that could help us to analyze sentiment in text.

We found two particularly good ones: 1) Stanford Sentiment Analysis, created by Stanford University’s Natural Language Processing Group, and 2) VADER (Valence-Aware Dictionary and sEntiment Reasoner), which is optimized for social media (for instance, to identify sentiment in Twitter). Each tool ranks sentences or paragraphs on a five-point scale from very positive to very negative. In our initial tests (using the text of a handful of modern novels), we found them to correlate strongly with the reactions of people in real-life situations.

Using these tools to analyze copy blocks I’d written for several of our chatbots was an eye-opening experience. Every single block came back with a negative rating!

Clearly there was much more to learn.

So, we delved more deeply into exactly how language conveys a given sentiment, turning to SentiWordNet (a lexical resource for opinion mining), WordNet-Affect (an extension of WordNet Domains), and the revised Revised Dictionary of Affect in Language (DAL) (an instrument for analyzing individual words according to the way they “feel” rather than their meaning alone).

For example, the DAL was developed by a large group of volunteers who rated approximately 8700 words according to three factors:

• Pleasantness – whether the word inspires an agreeable, pleasing, welcoming feeling
• Activation – whether the word connotes an active, energetic, dynamic feeling
• Imagery – how easily the word calls an image to mind

When tested against 350,000 words of English text, the DAL demonstrated a matching rate of 90 percent, meaning that 9 out of 10 words in most English texts could be accurately correlated in the DAL to a feeling or mood.  The way these words conjure affect is fascinating.  For example, people find that pleasant-active words are the most fun and words that are more easily visualized are more memorable.

Next, we looked at another dimension: readability. By calculating and showing six different readability indices, we were able to measure whether our words and sentence structures were simple, easy to understand and accessible to a broad base of Intuit customers. The indices included: Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, Gunning Fog Score, Coleman Liau Index, and Smog Index.

A visual guide to emotional affect in language

Tools like Stanford Sentiment Analysis, VADER, SentiWordNet, WordNet-Affect and the DAL offer tremendous insight. But, they’re not exactly at-a-glance dashboards. That’s why we integrated all these tools into an all-in-one sentiment analyzer to help our designers visualize, and finetune, the sentiment of their CUI copy blocks with Mumbler.

Here’s what it  had to say about a less-than-ideal sentence I wrote for Mint a few years ago:

As you can see, there’s a lot of red here. Larger words are more active, like “significantly” and “spent,” while smaller ones like “in” and “your” play a more passive role. The boldness of each word corresponds to its imagery. In this case, the word restaurant is highly visual, but words like “also” and “all” don’t bring anything in particular to mind. The readability index for this sentence is relatively high, meaning that it’s less accessible than it could be.

Finding better ways to say things

Lastly, we turned to a couple of other open source tools called WordNet and SentiWordNet to give designers a simple way to take action on the results of sentiment analysis. Developed at universities and powered by machine learning, WordNet lists all the synonyms for a given word, and SentiWordnet shows whether the synonym is objective, positive, or negative in connotation.

Now integrated into Mumbler, these tools show designers a whole range of alternatives for each word they’ve used, styled to reflect their relative pleasantness, activation, and imagery. It’s not perfect—not every synonym works in a given context, so it’s not as simple as moving a slider to change words—but it’s way better than starting from scratch.

As a designer works to refine a copy block, the Mumbler experience looks something like this:

When they swap-in various synonyms, the indices shown across the bottom change in real-time. While some interactions are bound to be less than pleasant, but authentic (such as being told that you’ve been overspending), we can reduce the number of red callouts to create a more positive customer experience. By using active voice, more active words, and visual words, we can make it more memorable. And, by simplifying the words and sentence structure, we can improve readability. Altogether, the new sentence might look something like this.

I’m proud of the progress we’ve made using Mumbler to design chatbots that make excellent first impressions with customers of Intuit’s QuickBooks, TurboTax, and Mint products. It’s proven to be an effective tool for helping our engineers, designers and content writers verify that the tone of their text- and voice-based communications matches their desired intent, resulting in an awesome customer experience.

In my next blog, I’ll take you through the next phase of our CUI journey into voice-driven customer interactions, where an entirely new dimension of affect and sentiment comes into play.