An AI that understands common sense, skilled technology, and even emotions! Going “Beyond the Challenge”

Text: Toru Uesaka / Editing: Kanae Maruyama

What values do people working on the global stage have? What do they look for in the places where they work and the people with whom they work? These interviews focus on overseas growth companies and cutting-edge Japanese venture companies that are currently gaining momentum. This time, we’ll be hearing from Luminoso Technologies, a leading company in natural language understanding that makes full use of AI analysis.

We spoke with David Smith, representative of Luminoso Japan LLC’s Japanese corporation, about the company’s business, which makes full use of cutting-edge technology in natural language understanding, especially to understand context.

Programming the common sense of society in advance

For example, let’s say we have a computer that can understand tens of thousands of customer requests based on nuances in their language. Collecting, collating, and reporting data on various industry trend topics and ongoing issues typically takes weeks, but this can now be down in hours. From here, customers can generate useful insights. A company that provides technology that makes this possible is located in Global Business Hub Tokyo.

Luminoso Technologies is based in Massachusetts, USA. David Smith, a representative of Luminoso Japan, a Japanese corporation, says, “The company called ‘Luminoso’ provides technology that offers insights into the data of conversations that people are having.”

The press release states that the company is a leader in natural language understanding, providing data insights using AI. However, this is not one of the current state-of-the-art AI mechanisms, like deep learning.

“Today’s AI has computers understanding patterns. That’s what we call deep learning. However, what Luminoso is doing is, so to speak, programming the common sense of society in advance. So you can gain insight into specific words. Because it understands the surrounding context like a human being, it can make inferences. That’s the kind of engine we use.” This approach is apparently called “transfer learning.” “That’s a pretty maniacal approach. I’ve never come across another company with similar AI.”

Natural language understanding software that enables the proactive prediction of claims and recalls

Luminoso Technologies began in 1999 at the MIT Media Lab. “There was a very sharp engineer who started a project to teach language to a computer.” In fact, he had been doing research and development for 11 years. Then, in 2010, he spun out, and Luminoso Technologies was born. “I don’t think there is any other startup that has been doing R&D for 10 years. During this time, 15 languages were taught to the computer. The precision of the Japanese is improved through collaboration with large Japanese companies and universities, but the big feature is that you can see the context.”

He explains that the computer understands not only Japanese but also the common sense of Japanese society. This is what it means to know the context. “For example, here’s a practical example. You pair the voice of the customer with the voice of the employee. The voice of the customer, the voice of the employee, right?” An easy way to understand the voice of the employee is that it’s the technical information left by employees. In any company, employees leave a variety of reports and documents. That’s what they use as input. “On the other hand, let’s say the voice of the customer comes in and says that the equipment they’re using is broken. That’s brought to the store to explain how it broke or it’s put in writing.”

This is compared with the technical information of the voice of the employee. Then you can see whether it is an unknown phenomenon or a known phenomenon. “That’s what AI insights mean. As a result, you might find out that there’s a high probability that it’s a new and unknown problem. That way, you can treat it as a prelude to complaints and recalls.”

Technology that understands “society’s common sense” and “corporate culture”

Many overseas customers are pharmaceutical companies and game companies, but, in Japan, a lot of interest seems to come from the manufacturing industry. This is not often the case in the West. “One trend is the problem of technology transfer. Many companies have enormous quantities of intelligence in the form of textual information left behind by past skilled workers. However, once they retire or otherwise leave the workplace, the reality is that you cannot make use of the information in those documents.” That’s where Luminoso’s natural language technology comes in. Enormous amounts of text data are put into a computer. “Luminoso understands proprietary terminology within the company. If we provide an engine that can load and do vague searches—for example, if a young engineer has a problem, they can just type in the phenomenon and it will be easier to get search results that identify the cause. It increases the likelihood that countermeasures can be found in past documents.”

Just as you understand society’s common sense, the Luminoso system can understand the common sense of the company and individual words. This means that the vast amount of past technical information that has been dormant can be utilized again. Solutions like this are created by thinking through the challenges with each individual company, and a key point is that the AI doesn’t analyze everything.

“The analysis is conducted by humans in the end. The media have high hopes that AI will solve everything, but our more realistic approach is that it’s best for humans to do the analysis,” he says, adding that it can be an effective tool for that. That’s why leading Japanese companies are being listed as business partners one after another.

The decisive factor was the “good aura” created by the people working in “Otemachi”

Smith, who joined Luminoso Japan in 2017, has a unique background. He studied Japanese at the University of Arizona and suddenly got a job at an investment bank in Tokyo. “That was in 2008. I was born in the United States, but I have never worked in the United States. At the Japanese branch of an investment bank, I started my work by receiving phone calls from clients under the supervision of Japanese supervisors. All my Japanese was obtained just from studying at university. It was really hard at first.”

After that, he was involved in financial projects at GREE and was in charge of startup investment at Digital Garage. He has also experienced entrepreneurship in Japan. “A Japanese car manufacturer was interested in Luminoso, but there was no staff in Japan at the time. So I joined because they wanted someone to in-between Japanese customers and the US headquarters.”

Initially, he thought that coordinating translations would be the main job, but he realized that he needed to communicate with customers on the basis of a firm understanding of technology. After that, he was also put in charge of sales, with which he had no experience at all. The Japanese staff is now composed of four people. “The customers in Japan are all really talented. I get to hear so many different things from them, so it’s also an important role to listen to them and convey what I hear to the US headquarters. That can then lead to work that makes new products.”

They moved into Global Business Hub Tokyo in October 2018. When Luminoso Japan started operating at full-scale, it became necessary to hire staff and rent an office. “I was introduced to Global Business Hub Tokyo by a friend who travels back and forth between the United States and Japan. The first thing I liked more than anything else was the place, Otemachi.” He adds that, at the time, it was very important to be close to Tokyo Station because it was necessary to meet customers in Aichi Prefecture once a week using the Shinkansen. “Sometimes customers come to Tokyo, so then the access is really convenient.” He went on a viewing and had a look around, and he liked it at first sight.

“It made a very good impression, including the people who showed me around. I felt a very good aura from the people already working in the office. When I first came here, I thought this is where we should be.” He remembers liking the rows of company nameplates at the entrance. “When I saw the company names there, I was immediately relieved. They included companies that are very famous now, and I hope that Luminoso can do the same.”

Life is full of contradictions

In terms of improving individual and team performance, he says that there were things he noticed after he started working remotely. “Until then, I would get a change of pace by strolling around the Imperial Palace, which is right next door, going on a run, and going to the sauna at the spa in the building, but now I realized that the best change of pace actually comes from chatting with people.” With remote work, face-to-face communication has decreased. This actually became a big stress factor. “So now I’m actively chatting with everybody. Just talking a little bit about the news or sports can give such a change of pace. Even remotely, we have one-on-one meetings and phone calls. That cheers me up.”

As an individual, he tries to be flexible. He has convictions, but he’s not too bound by them. If there is new information, he will accept it and update his beliefs. “It’s contradictory, but life is full of contradictions [laughs].” Another thing is to try out what you think is difficult. “Three or five years from now, I’m sure I’ll be glad I did it. It’s because I’ll inevitably be growing without even being aware of it.” The same is true for when he thought about working in Japan as soon as he graduated from college. “I thought that, if I went somewhere unknown where they speak a different language, I’d definitely be glad I did five years onward. Because it was a huge challenge. But I’m actually glad I did it.”

On July 28, 2020, Luminoso unveiled a deep learning model capable of advanced sentiment analysis of documents in unstructured text data. Five years from now, in an ever-evolving environment, Smith will undoubtedly have undergone unanticipated growth.

Photography: Tomoyasu Osakabe