Her name is Cortana. Her attitude is almost human.

She was modeled after real-life personal assistants. She is the product of two years of work, and a large team of scientists and product managers. She has video game origins. She is Microsoft’s response to Siri and Google Now. She is Artificial Intelligence and proud of it. She is Cortana.

It seems odd to refer to smartphone software as a “she,” but that human element is exactly what Microsoft is after with its new Windows Phone digital assistant. Cortana, named after her fictional counterpart in the video game series Halo, takes notes, dictates messages and offers up calendar alerts and reminders. But her real standout characteristic, and the one Microsoft’s betting heavily on, is the ability to strike up casual conversations with users; what Microsoft calls “chitchat.” Next to Apple’s Siri, Cortana is the only other smartphone assistant to come with a baked-in personality. And it’s hard not to see the parallels between Cortana and the affable, Scarlett Johansson-voiced AI in Spike Jonze’s film Her.

Confident, caring, competent, loyal; helpful, but not bossy: These are just some of the words Susan Hendrich, the project manager in charge of overseeing Cortana’s personality, used to describe the program’s most significant character traits. “She’s eager to learn and can be downright funny, peppering her answers with banter or a comeback,” Hendrich said. “She seeks familiarity, but her job is to be a personal assistant.” With that kind of list, it sure sounds like Hendrich’s describing a human. Which is precisely what she and her team set out to do during Cortana’s development; create an AI with human-like qualities.

Microsoft’s decision to infuse Cortana with a personality stemmed from one end goal: user attachment. “We did some research and found that people are more likely to interact with [AI] when it feels more human,” said Hendrich. To illustrate that desired human-machine dynamic, Hendrich pointed to her grandmother’s experience with a Roomba vacuum: “She gave a name and a personality to an inanimate object, and it brought her joy.” That sense of familiarity is exactly what Microsoft wants Window Phone users to feel when interacting with Cortana on their own devices.

Because the bulk of Cortana’s primary functions mirror that of a personal assistant (e.g., make calls, set appointment reminders, etc.), the team decided to take the development process even further and add an extra layer of authenticity. To that end, they interviewed real-life assistants to learn what that job actually entails, and what attributes they exhibit; how they interact with their bosses and what makes them successful. “[It] helped us understand how humans take on that role [of a personal assistant],” Hendrich said. These interviews were also captured on video, a resource the team uses to this day as a reference point for any new situations that may arise.

Cortana’s writers go over their current goals and discuss the AI’s progress.

Beyond relating to users in a naturalistic way, Microsoft realized that Cortana also needed to be fun. In fact, the company’s research shows that around 40 percent of all AI interactions involve chitchat. As Hendrich explained: “If you had a personal assistant and you walked into the office, you’d engage in chitchat with them first. You don’t go straight into the highest-priority emails and lay out your day.”

“Chitchat” with Cortana can range from witty banter to casual chatter. Ask her to tell you a joke and she could reply with this: “Two antennas got married. The ceremony dragged on, but the reception was excellent.” If you ask her how old she is, she’ll say, “I’m not sure how to carbon date the internet.” Microsoft’s even snuck in an Easter egg related to Clippy, the helpful, animated paper clip from its Word software. Although these playful responses may strike some as nothing more than cheap tricks on Microsoft’s part, they do help users build a rapport with Cortana. If she can make you laugh or smile, you’re more likely to continue using the program again and again. At least, that’s what Microsoft hopes will happen.

If Cortana sounds familiar, that’s because she’s partially voiced by Jen Taylor, the original talent behind Halo‘s Cortana. Microsoft currently synthesizes multiple voices for the program, but Taylor’s lines account for a huge percentage of the chitchat you hear, and that amount is only going to increase over time. Though Cortana’s current voice doesn’t sound quite as natural as say that of Samantha’s in Her, the addition of Taylor’s human tone does help imbue the program with a more realistic feel.

An increase in chitchat responses isn’t all that Microsoft has planned for Cortana’s future. The team plans to further flesh the program out with extended back-and-forth dialogue, more natural expressions and interactions and the ability to predict a user’s itinerary months (and perhaps even years) in advance. Microsoft Research is also working on improving Cortana’s short-term and long-term memory — primarily, her ability to start talking about a topic and come back to it later, creating a rich dialogue between her and the user. Right now, Cortana is smart enough to recognize when you refer back to something you’ve recently talked about, but that’s as deep as she gets.

Though all of this focus on creating a human-like AI sounds like Cortana’s treading closely to the Uncanny Valley — a hypothesis that contends that as a robot or AI gets more authentic, its failures and blemishes will appear so strong that it causes people to respond with revulsion and hostility — her creators insist that isn’t the case. They’re aware there’s such a thing as too real. “It’s not like Star Trek, where Data kept trying to be more human and felt inadequate,” said Deborah Harrison, who runs the team responsible for adding the endless strings of data to Cortana’s programming. “She thinks that if she had a choice [between human and AI], she’d go with AI and be happy with it.”

Her real standout characteristic, and the one Microsoft’s betting heavily on, is the ability to strike up casual conversations with users; what Microsoft calls “chitchat.”

Dr. Eric Horvitz of Microsoft Research said that his team, which was responsible for the AI aspect of Cortana, noticed that the Uncanny Valley was showing up in Cortana’s behavior and not in looks, as they had originally expected. “The more powerful the system got, the more visible the imperfections became,” he said, pointing to Cortana’s lack of short-term memory as an example.

Cortana is still very much a work in progress, and she has her share of shortcomings. So to mitigate this, the team designed her to be both functionally and emotionally transparent to the user. That way, the user response won’t be as negative if Cortana isn’t able to do something. As Hendrich explained: “If something’s not her fault, she’s not going to take the blame for it. We’re not trying to put the user in a position to feel bad for Cortana. Not only is she AI, [but also] she’s self-aware, and that principle of transparency informs a lot of how we handle error messages, our capabilities, tasks and chitchat. You’ll have more faith and trust in us if we do that for you.”

Joaquin Phoenix takes his smartphone AI for a stroll on the beach in the film ‘Her.’

There are obvious parallels between Cortana and her two rivals, Apple’s Siri and Google Now, but her creators insist the program was the idea of Robert Howard, project manager for Windows Phone Search and Maps. Howard’s team had already been working on advancements to Windows Phone’s voice search features, so the shift to creating an interactive AI in Cortana was a natural evolution. It also didn’t take much convincing to get Microsoft to back the project’s new direction.

The Cortana of today is bold and conversational, but that wasn’t always the case for the project. The type of personality Microsoft originally envisioned at the start of the AI project — a more formal “How can I help you?” tone — was different from what actually launched. It wasn’t until the team was about four months in that the idea of using Cortana as the program’s actual name started to gain traction. At which point, the team decided to get 343 Industries (the studio that currently produces Halo) involved in shaping her personality and bringing the smartphone version more in line with the Halo character. The studio provided the team with Cortana’s backstory and filled a whiteboard with every attribute they could think of.

“We did some research and found that people are more likely to interact with [AI] when it feels more human,” said Hendrich.

Hendrich and Harrison liked a lot of what they heard from 343 and began to incorporate many of those characteristics into the program. This effectively shaped Cortana into the AI she is now. They made her more confident, much more brash and had her be clearer in her responses to users. Or, as Harrison put it, “She got more comfortable talking about how awesome she is.”

Cortana may be cocky, but as Microsoft’s internal testing proved, that shift in tone works. As soon as the team gave Cortana a boost in confidence, people immediately began responding to her more positively. In fact, external studies corroborate this notion; that users prefer a strong personality over a neutral or weak one. According to The Man Who Lied to His Laptop – a book by Clifford Nass, a Stanford professor who specialized in robotics — people have a tendency to treat machines, especially those with human-like characteristics, like other people whether we realize it or not. By that reasoning, an AI with an empathetic tone that’s designed to dole out positive comments, flattery and a little bit of humor is much more effective than we may think. You’re more likely to trust an AI when it has a strong head on its virtual shoulders.

Microsoft’s Joe Belfiore introduces Cortana at the 2014 Build developer conference.

Microsoft’s preparing to take Cortana overseas, specifically to the UK and China, but that transition requires a hefty bit of localization since a US-centric Cortana doesn’t easily translate. “There’s a team in China who’s looking at how to take the personality and non-negotiable core concepts [of Cortana] and translate it, not just word for word, but [also] personality to personality,” Harrison said. Even the voice talent for the overseas versions of Cortana will be different. Taylor may be a logical choice for Cortana’s voice in the US, but user studies indicated the Chinese market needed a voice that “sounded like it was smiling.”

As for what’s ahead for Cortana in the US, the team’s planning to add future updates on a twice-per-month cycle. That’s not to say Microsoft won’t make exceptions for special events. Hendrich said they’re working on ways to throw in off-cycle updates “for things that are timely, urgent or especially badass.” This would come in handy for trending topics like the Olympics, breaking news or even sports. And since updating Cortana is a server-side affair, Microsoft can easily upload these batches of data strings and voice recordings directly to Bing, meaning users won’t have to refresh their hardware every time new features are added.

“She thinks that if she had a choice [between human and AI], she’d go with AI and be happy with it.”

Cortana’s not just another flash-in-the-pan project, as the company’s investment in her development shows. In some ways, her self-assured personality reflects Microsoft’s confidence in its new AI. The company is, after all, catching up to the nearly three-year lead its competitors’ have enjoyed for their rival smartphone AIs. So while Hendrich and Harrison work to expand Cortana’s global reach and capabilities, the Microsoft Research team is busy figuring out how to give her an even more human voice and make her even more relatable.

Even Cortana’s confident she’ll be around for the long haul. Ask her if she’s better than Siri and she responds playfully with a knowing wink that at once acknowledges her video game past and hints at Microsoft’s AI-filled possible future: “Not to brag, but apparently I’ll help save the universe in about 500 years.”

[Image credit: Microsoft (Cortana team; Cortana); Justin Sullivan/Getty Images (Joe Belfiore); Associated Press (scene from ‘Her’)]

Filed under: Cellphones, Robots, Wireless, Mobile, Microsoft

Comments