As an employee, an easy-seeming task like booking a meeting room can quickly become difficult if you cannot see which rooms are available—that is where taking a transformed approach to load testing can help.
Employees booking rooms and dozens of other similar activities in Microsoft buildings generate significant Internet of Things (IoT) sensor data. All these spatial data readings must be processed in near real-time so that the information remains relevant and significant. If you are looking or searching for a meeting room and cannot figure out whether it is occupied or not within seconds, not only will you walk out the door but also waste your time looking for other rooms, losing productivity. And if the IoT motion sensor data isn’t processed as fast as the employee is entering or leaving the room, you are left with incorrect and outdated information. That lag in information retrieval—even just a few milliseconds—can significantly degrade an employee’s experience.
The way to power employee experiences is to understand the data behind occupancy, room temperatures, hot desking, and more. The question then becomes, ‘How do you power all these employee experiences that are a part of your workplace?’
—Apoorv Gupta, senior software engineer, Microsoft Digital Employee Experience
Sensors in Microsoft buildings take more than two million IoT readings per day, with an expected traffic growth of around 30 million readings per day.
“The way to power employee experiences is to understand the data behind occupancy, room temperatures, hot desking, and more,” says Apoorv Gupta, a senior software engineer for Microsoft Digital Employee Experience, the team building a new load testing service for the Microsoft Global Workplace Services (GWS) team that manages the company’s buildings. “The question then becomes, ‘How do you power all of these employee experiences that are a part of your workplace?’”
Microsoft’s solution for tracking all these sensor readings was to deploy a Digital Integration Platform that brings all those people, places, and devices together in new and better ways. This solution can simultaneously process data from multiple employee experiences in over 300 buildings across the globe—without compromising system stability, reliability, and load performance. Microsoft Azure Digital Twins powers the Digital Integration Platform on the backend, storing all the sensor data and using it for processing. To load test the solution the team decided to build a full-blown load testing solution leveraging open-source technologies.
I challenged the team to prove that our Digital Integration Platform can scale to integrate with a host of new sensor types, and scale to enable real-time IoT-powered experiences for our global workforce. The MS Digital team met the challenge, leveraging configurable cloud automation to validate the platform at scale, for a remarkably low cost.
—Emmanuel Daniel, director of Digital Transformation and Smart Buildings, Global Workplace Services
“Azure Digital Twins makes it easy to merge the physical and digital worlds by allowing us to map data to device to building,” says Emmanuel Daniel, director of Digital Transformation and Smart Buildings for GWS. “This enables our employees to have great experiences in the physical space.”
According to Daniel, employee experiences are dependent on multiple data points, from many sensors producing a large volume of messages. And for the experiences to successfully work, the platform needs to scale. For example, the size and scale of Microsoft’s Puget Sound-area East Campus makes it very difficult to mirror the environment.
“I challenged the team to prove that our Digital Integration Platform can scale to integrate with a host of new sensor types, and scale to enable real-time IoT-powered experiences for our global workforce,” Daniel says. “The Microsoft Digital team met the challenge, leveraging configurable cloud automation to validate the platform at scale, for a remarkably low cost.”
[Check out inside Microsoft’s smart building in Herzliya, Israel. Read more about accelerating Microsoft’s global real estate transformation with Microsoft Digital. Discover how Microsoft operations evolve with Azure. Look into building Microsoft’s employee-centric experience.]
Leveraging open-source technology for faster deployment
To start the testing process, the team gathered three months of baseline load data from GWS’ previous tool and used that as a benchmark. Then the team created tests that measured how well the system could handle the number of readings/requests per second (RPS) received and overall load distribution. For example, if you have a continuous stream of sensor readings being ingested across hundreds of buildings all at the same time on a global scale, that is a massive load of requests that need to be processed in near real-time between all the types of IoT data that is being processed.
The team ran a complete cycle of load tests, including X load (peak load), twice the peak (2X), and a long-haul endurance run that tracked performance over 24-48 hours. These tests are crucial to ensure that the components as well as the entire system can function at that high-intensity level. They also prepare the system to handle higher loads as additional employee use cases continue to be added.
Once you test the system, you need to determine whether it behaves how you want it to, and how it behaves under load. You need to generate the load and simulate it to mimic the real-life scenario.
—Mridul Verma, senior software engineer, Microsoft Digital Employee Experience
“We already knew our solution was going to need to support millions of readings in the future, so we needed to be ready for that by making sure it’s stable under that load,” says Shuvajyoti Deb, a software engineer for the Microsoft Digital Employee Experience team. “We helped create the methodology and implementation plan that we used to deploy the load testing solution.”
The team opted to use Apache JMeter and Microsoft Azure Kubernetes Service (AKS) clusters to run the load testing solution. The load testing solution mimics the people, places, devices, sensor types and their locations in all the buildings so the team can see how the Digital Integration Platform would operate in real life.
“Once you test the system, you need to determine whether it behaves how you want it to, and how it behaves under load. You need to generate the load and simulate it to mimic the real-life scenario,” says Mridul Verma, a senior software engineer on the Microsoft Digital Employee Experience team who helped set load testing benchmarks and identified bottlenecks.
Throughout the course of testing, the team focused on various metrics related to reliability, performance, and availability to ensure that the solution met the benchmarks and expectations that were set at the beginning of the project. Upon completing each round of testing, the team ran an analysis and determined where it could alter the instances or configurations to better optimize the system. Once the alterations were made the team ran the same set of tests again to see if performance improved and continued to optimize until it met expectations.
The last stage of the project was creating automated pipelines and making them available on GitHub so anyone in GWS can perform load testing. The availability of the solution on GitHub makes it a lot easier and faster for teams to conduct the load testing for their respective services and platforms. Additionally, the availability of the load testing solution on GitHub makes it open to customization as required.
“GitHub makes it easy for users to access the load testing solution quickly contribute to it to meet their needs,” says Amit Agrawal, another software engineer on the Microsoft Digital Employee Experience team. His role was to develop the methodology and process for load testing.
Optimizing today for future growth tomorrow
Load testing helps Microsoft gain insight into the IoT sensor data it collects to give employees a better experience in its buildings. With a more stable and reliable platform, more employee experiences can be added, more facilities can be onboarded, and more teams can use this tool to make data quickly available. As a result, employees are more productive in their day-to-day tasks, saving them time during the week.
Looking forward, the team will onboard the company’s new Eastside Campus in the Puget Sound region to the Digital Integration Platform, where the company will be opening 17 new smart buildings starting in late 2023.
- Load testing is essential for any scalable platform, as it gives teams confidence in the scalability and load it can take in the future.
- Determine benchmarks for the system and run tests to ensure the system can meet these expectations.
- Performance testing is necessary to determine whether your system is behaving as expected to handle the current load and be optimized to handle extra load requests in the future.
- Identify your peak load and 3-5x of that load to see how the system performs under that stress. Perform stress tests on the whole platform and each component.
