Building unique AI models and mass-producing the Tensai-kun Series

―――SHIFT is very active in generative AI-related initiatives. First, can you give us a rough idea of what projects are underway within the company?

Matsunaga: Broadly speaking, these initiatives fall into two categories: R&D-focused activities, led by us in the DAAE Strategy Unit, and external sales of services, which are primarily handled by the AI Services Division or other business units.

―――Can you share some details about the R&D projects you’re working on?

Matsunaga: Essentially, it’s about using AI to improve work efficiency and create products that can be developed into businesses.

There are two main approaches. The first is to collaborate with group companies to develop proprietary AI models specialized in testing and apply them in practical scenarios.

The second involves mass-producing lightweight AI products, collectively called the Tensai-kun Series, which are primarily designed for our internal back-office operations.

―――Which of the two approaches did you begin with?

Matsunaga: Our first step was building proprietary models. We’re working closely with DeMiA, an AI-focused IT startup from Kyoto University that has recently joined the SHIFT Group, to drive our R&D efforts.

―――I see. Before we dive into the details of the proprietary models, what was the catalyst for starting this project?

Matsunaga: The start was quite straightforward. Since test operations are one of SHIFT’s core services, the idea was raised internally that generative AI could be applied to this area.

This happened around the time generative AI started to become an enormous topic in Japan.

Sakase: While models like GPT-4 are LLMs (Large Language Models) that have undergone extensive training, several open-source models have gained Japanese language capabilities at a more preliminary stage.

For this project, we used those models as a base to build our proprietary models, which we tuned by training them with additional SHIFT test cases.

Matsunaga: At SHIFT, we use our original test design support tool called TD (Test Designer).

By combining our proprietary models with GPT-4, the AI can make various recommendations on TD that align with SHIFT’s test design methodology.

A proprietary model that exceeds GPT-4’s accuracy across several metrics

―――What kind of accuracy have you achieved with the proprietary model?

Matsunaga: Although only some testing operations are being practically validated, the original model has shown higher accuracy than GPT-4 across multiple indicators, indicating higher effectiveness.

―――What were some of the difficulties you faced?

Matsunaga: Currently, we are using a combination of GPT-4 and our own model for each process, but while GPT-4 is highly accurate from the start, there are issues in terms of processing speed.

Even if it requires initial costs and time for training, we want to design and build our own unique model with faster processing speeds.

Sakase: The proprietary models themselves are mainly being developed by DeMiA, but there is an unavoidable gap between what they are working on and what the TD team is creating.

The TD team provides test data and text, but this cannot be passed directly to the AI, so we are developing an intermediate API that allows this information to be passed from TD to AI or vice versa.

―――What are the plans for future initiatives in this area?

Sakase: GPT-based systems inevitably take time, so to shorten the time required, we would like to continue to build our proprietary model.

Matsunaga: From a project management perspective, we’re only using the proprietary model for about half of the process right now, but we aim to fully transition the remaining GPT-4 parts to the proprietary model.

Right now, we have to create the input test specifications in a set format, but the reality is that specifications come in many different forms. When we specify a particular format, it ends up creating extra work to adapt to it.

We plan to create a system that automatically converts those formats while also advancing various other smaller tasks in parallel.

―――Can you tell me more about the other initiative, the Tensai-kun Series? I understand it focuses on mass-producing lightweight AI products for your in-house back-office functions—what exactly does that mean?

Matsunaga: At its core, this initiative is about building a system that empowers our team members to take the lead in driving business improvements themselves.

Specifically, we provide an admin interface called “Urakata-san” that allows internal teams such as HR, public relations, and even sales (not part of the back office) to easily create products that are linked to AI and APIs.

By configuring a few settings, an AI tool can be created that automatically supports the relevant tasks.

A generative AI tool for internal DX, developed in one week

―――I see; so end users can tune it themselves. When did this project start?

Matsunaga: Around March 2024, we discussed the future direction of AI utilization with the management team and decided to move forward with mass-producing lightweight products.

Since then, we have been moving forward at a rapid pace.

Sakase: I set up the backend system, and in about a week, we had the basic version ready.

Right now, we are in the process of giving the product to the recruiting team for verification and having them try it out.

―――A week is incredibly fast! What kind of tasks is the tool designed for?

Matsunaga: For example, you can input a question, and the tool acts like a secretary, answering queries about SHIFT and SHIFT Group’s company information.

Sakase: We extract text from a PDF, optimize it, and then pass it to the AI. It’s not a complicated task.

Initially, we went with GPT-4V (Vision), thinking it was the best option, and implemented it. But now, we’ve switched to the latest model, GPT-4o.

We will continue reviewing and tuning the model to ensure smooth processing in the future.

―――This also means that end users don’t need to be aware of which LLM is being used.

Sakase: Considering the current state of AI development, I don’t think it’s a good idea to lock in on a specific LLM just yet. We should be ready to switch to better models as they become available.

As end users, I believe we will enter a world where AI is used in low-code tools, and AI will simply be just one piece of the puzzle.

SHIFT’s “productive flexibility” is empowering a culture of innovation

―――From each of your perspectives, what are the defining features and appeal of the DAAE environment?

Matsunaga: I feel grateful that we have a generous budget for R&D.

A 30% batting average is good enough; the important thing is to keep swinging. The company encourages this mindset, so I’m given the freedom to go all out.

Sakase: Since our department reports directly to the executive team, we can quickly act on requests, saying: “Let’s try it.” And we also have the freedom to experiment with new technologies without restrictions.

I’m actually a returnee who left SHIFT once, but I don’t think other companies would offer the same level of freedom.

―――Sakase-san, you mentioned that you left SHIFT once. What made you decide to leave and what brought you back?

Sakase: I was here for about two years last time, and I felt like I had done everything I wanted to do, so I went out into the world.

Although I don’t consider it a bad decision, I realized that to fully pursue my goals, I needed to be part of a company with SHIFT’s resources. And more than anything, I felt that “productive flexibility” is essential.

At that time, my predecessor had been developing products using generative AI at DAAE and asked me to return, which is how I came back.

―――What do you mean by “productive flexibility?”

Sakase: In a typical company, there are various pressures, such as sales goals, and once you take on a task, there is often a strong sense of responsibility to finish it.

However, in the case of DAAE, we have a cultural foundation that allows us to stop and move on to the next challenge if we don’t get results.

Of course, many codes have been left dormant because of this (laughs), but every day, I feel that it is an environment where it is easy to take on new challenges.

Matsunaga: I’ve been with SHIFT since I graduated, so I don’t know about other companies. But even though I tend to get bored easily, I’ve never felt like quitting in the last six years.

I believe that this is definitely an environment that allows us to take on new challenges.

I don’t think we have shown the world that SHIFT is strong in AI yet, so I would like to communicate this through our efforts at DAAE.

(*The content of this article and the affiliations of the interviewees are as of the time of the interview.)