Telexistence Deploys AI and Remote-Controlled Robots in 300 Convenience Stores, Utilizing Azure

Providing AI and Remote-Controlled Robots for Retail and Logistics Industries Based on Azure

Telexistence Inc. (hereinafter Telexistence) deploys remote-controlled and artificial intelligence robots for retail and logistics industries under the corporate vision of "Grasping every single physical object in the world with our ‘hands’" and the mission of "Creating the world where every single person on every planet will benefit from the robotic revolution." The company name, Telexistence, refers to a concept first proposed by Professor Emeritus Susumu Tachi of the University of Tokyo in 1980, describing a vision and technological system that expands human presence utilizing remote control and artificial intelligence.

Telexistence is a startup company established in 2017, gathering highly specialized talent from around the world and undertaking integrated in-house development of hardware, software, AI, and remote-control technology. In November 2021, they began the social implementation of TX SCARA, an artificial intelligence and remote-controlled robot for the retail industry. Currently, it is being introduced in about 300 convenience stores. Also, in October 2022, they conducted a demonstration experiment of Telexistence-manufactured robots for logistics facilities in collaboration with a logistics company, and since July 2024, they have begun phased operations of robots in actual operations at a combined normal and low-temperature center established within a large logistics facility.

Genki Sano, CTO of Telexistence, who oversees the company's technology and solution development, describes the features of their robots as follows:

"Telexistence is a technology that allows you to operate a remote robot as if it were yourself. By combining this core technology with AI-based automatic control, we provide robot solutions using automation and teleoperation. TX SCARA, which is installed in the backrooms of convenience stores, is a robot that automatically detects when beverages run out on the shelves and automatically replenishes them. Our staff can also operate the robot remotely, and if AI-based stocking fails due to unexpected environmental changes, it is possible to complete the stocking task 100% through remote operation by humans via the internet," says Sano.

Telexistence’s service provision platform is based on Microsoft Azure (hereinafter Azure). When adding new features to robot solutions, they also utilize Azure for cost-efficient development. As part of these efforts, they have recently begun providing a solution that visualizes store operations using Azure.

Store Operations Visualization Solution for Analyzing In-Store Information

The Store Operations Visualization Solution is a service offered as an additional feature to the beverage display solution powered by TX SCARA. Sano explains the purpose behind its development:

"The beverage display solution is a service where robots relieve humans of the labor-intensive tasks of displaying and restocking beverages inside refrigerators. In addition to addressing labor shortages, it allows staff to focus on in-store tasks that only humans can do, such as customer service, by dividing tasks with robots during peak times. However, for robots and humans to collaborate effectively, it's not enough to simply enhance efficiency through robots. We need to understand what kind of tasks are actually consuming human time and determine the optimal timing for utilizing robots. We developed the Store Operations Visualization Solution to meet these needs," says Sano.

The Store Operations Visualization Solution is a service that analyzes data on the location and movements of staff, gathered from sensors installed in stores, and displays a variety of information on a dashboard. Store managers, area managers, headquarters analysts, and management teams can use this dashboard to explore patterns of collaboration between humans and robots and leverage the data to drive operational improvements.

Quick Prototype Development on a Different Cloud Leads to Operational Cost Challenges

For the Store Operations Visualization Solution, Telexistence initially created a prototype using a different cloud service.

"The other cloud service was a familiar tool, which made it ideal for quickly developing and testing the prototype. However, we realized that migrating this prototype directly to a production environment would lead to unexpectedly high operational costs. After actually implementing the service, we found that the cost of processing large amounts of data was enormous, making it difficult to continue operations this way," says Sano.

The prototype involved real-time transfer of store data from sensors to the cloud, where it was stored in a cloud DWH for analysis, with the results displayed on a web dashboard. To accelerate development, Mobile Backend as a Service (MBaaS) was also utilized. The issues that arose included the cost of storing large volumes of data, the cost of analytical processing, and the cost of data transfer.

"Using multiple cloud services posed an additional challenge of increasing future operational burdens in the future. The need for data integration and system integration also raised concerns about performance," says Sano.

Decision to Migrate to Azure and Adoption of Azure Data Explorer as the Optimal Service

To address these issues, Telexistence worked on the migration of the system infrastructure used for the Store Operations Visualization Solution to Azure.

"When faced with challenges related to cost, performance, and operations, I was informed by a Microsoft representative that migrating to Azure services would enable us to build a cost-efficient, flexible, and scalable service infrastructure. However, I wasn't very familiar with Azure's data and analytics services myself. I thought we could probably use services like Azure IoT Hub and Azure Synapse Analytics, but I hadn't considered how to use them to build the optimal system," says Sano.

After repeated discussions with Microsoft representatives, Azure Data Explorer was ultimately adopted as the optimal service for the Store Operations Visualization Solution system. Azure Data Explorer is a service that quickly executes queries and analysis on large volumes of log and telemetry data.

"We were able to rebuild the service with a simple configuration where data is sent directly from sensors installed in stores to Azure Storage and analyzed with Azure Data Explorer. In the previous configuration, we incurred costs every time we stored large amounts of data in databases, or queried and transferred data to the cloud DWH. By consolidating these functions into Azure Data Explorer, we have been able to analyze data in the shortest time possible. Additionally, migrating the system to the Azure platform allowed us to enhance integration with our beverage display solution," says Sano.

Telexistence has been building their systems utilizing Microsoft's startup support program. In the beverage display solution using TX SCARA, Azure has been fully adopted. For example, it utilizes Azure Cosmos DB for MongoDB, a managed service for the MongoDB database, and Azure Cache for Redis, a managed Redis service for high-performance real-time data processing. Additionally, Gordon, the proprietary AI system developed by Telexistence, runs on Azure VM.

By being able to provide both the beverage display solution and the Store Operations Visualization Solution from a single cloud platform, they were able to reduce the overall system management costs. (This project involved technical cooperation from ZEAL Corporation, a Microsoft partner.)

Realizing Benefits in Cost Optimization, Data Integration, and Knowledge Acquisition

The benefits and effects gained from migrating to Azure can be summarized in three main points.

The first is cost optimization. While specific cost reductions will become clear with future production operations, Telexistence has been able to reduce costs for storing large amounts of data, DWH analysis, data transfer, and managing multiple clouds, and has established a scheme that optimizes costs as the business and systems scale.

"As cloud service usage increases, costs rise, requiring continuous review. Gaining knowledge and know-how to build systems at optimal costs for actual production operations, not just quick prototype development, is a significant benefit," says Sano.

The second is the opening up of new options for data utilization by consolidating onto Azure.

"By bringing together the cloud platform that controls robots and the analysis platform that visualizes store operations on Azure, we have been able to easily integrate their respective data. Previously, the prototype used a different cloud, which required data conversion and, at times, daily batch processing of CSV files for data integration. Now, more real-time data integration is possible, allowing quick confirmation of robot and staff status on the web dashboard. Specifically, we can see what tasks the staff were performing at what time, and what the robot’s operational status was during that time. Data display response and performance have also improved," says Sano.

The third is know-how gained for utilizing Azure. By receiving support from Microsoft representatives, we gained knowledge of new services like Data Explorer and tips for reviewing existing architectures.

“Microsoft provided a variety of knowledge and know-how, ranging from reviewing architectures to selecting specific functions and services. We have already been receiving support through Microsoft’s startup support program, so we would like to continue utilizing Microsoft's support as we work on revising existing services and developing new functions. For data utilization, we can also consider integration functions with POS and ordering systems,” says Sano.

Spreading Intelligent Robots to People's Living Environments Outside of Factories

Data integration with POS and ordering systems is an area where the benefits of utilizing robots become particularly evident. Sano explains:

"Previously, it was necessary to manually check in-store inventory. However, if robots are used for automatic restocking, we can accurately grasp when, what, and how much was sold. Our customers in retail are required to develop strategies for ordering and inventory management. By providing our data obtained through robot displays and store operations visualization, we believe they can achieve more strategic store and product development. In the future, we think this will lead to data utilization across the entire supply chain," says Sano.

Sano envisions the future as follows:

"As a company, we are focusing on three pillars to grow our business. The first is to introduce robots to more stores, primarily domestic convenience stores. The second is to expand outside Japan, especially to North America. The third is to introduce robots to industries other than retail. From a system perspective, for domestic operations, we need to scale up and out server processing capacity as the business grows. It's important to work with Microsoft to enhance servers while optimizing costs. Also, for North America, we need to reconsider robot production and create new solutions to account for differences in store layouts and business practices. The key to deploying the core systems across regions and collaborating with local staff will also lie in our cooperation with Microsoft. For expansion beyond retail, such as our existing logistics case, we need to consider how to manage resources in servers and cloud, so we would like Microsoft's support," says Sano.

Telexistence aims to spread intelligent robots to people's living environments outside of factories. As services expand, robots and the cloud will become more widely and closely connected. Sano expresses his expectations for Microsoft:

"We want Microsoft to provide highly reliable services at optimal costs and walk alongside us as a partner. Also, the areas where AI can be leveraged have been increasing in recent years. We want to advance more in-depth AI development while utilizing Microsoft's AI technologies together with our in-house developed AI," says Sano.

Microsoft will continue to support Telexistence’s efforts moving forward.