Shadow IT is the set of applications, services, and infrastructure that are developed and managed outside of defined company standards. These line-of-business-built solutions (aka Shadow IT) have always existed at Microsoft and are a common industry problem.
Over the years, corporate function teams—including business development, legal, finance, human resources, marketing and sales, support, and consulting—have looked to alternative engineering solutions for many different reasons. Some examples include a lack of IT engineering capacity or prioritization of the business need, historically decentralized budgets, a lack of trust between IT and shadow teams, the need for specialized domain solutions, and the availability of modern tools that enable no-code/low-code solutions to be stood up by citizen developers.
Many of these reasons make strong business sense, if it can be done securely. However, because Shadow IT solutions are often built outside of the guardrails of the company’s engineering systems, they pose a potential compliance risk to the enterprise, specifically in the areas of security, privacy, data governance, and accessibility.
At Microsoft, we needed to first understand if applications built by shadow teams met our security compliance standards. In 2019, we conducted a security assessment on a small random sampling built by shadow teams that showed that all the Shadow apps failed to meet at least two out of three of the key security requirements, and one Shadow app failed all key security requirement areas. This presents a huge and unnecessary risk to the whole company.
Ensuring we address our biggest security vulnerabilities has been our first priority internally at Microsoft in our Shadow IT journey, as the risk in today’s environment is huge. The average data breach in the United States costs $4.2 million (2021 IBM), and cybercrime costs the world $6-7 trillion annually (2020 Annual Cybercrime Report).
[Learn how you can start reducing your organization’s Shadow IT risk in 3 steps. Learn more about Microsoft Azure Tenant Security Solutions (scanner) available on GitHub. Learn why Microsoft is doing more with less: Optimizing shadow IT through Microsoft Azure best practices.]
Vision
Rather than centralizing all applications into IT, our goal is to reduce or eliminate Microsoft risk by enabling teams to self-manage their assets and ensure that they adhere to the compliance standards set forth by Microsoft. Teams must not only get clean, but also stay clean.
Compliance standards
Microsoft compliance standards are typically defined as four areas of focus, which are all supported by our set of Engineering Fundamentals:
Security: To ensure that the confidentiality, integrity, and availability of the data and systems of an organization is maintained.
Privacy: To ensure control over the collection, use, and distribution of information.
Data Governance: To ensure that the organizational roles and responsibilities by which information is retrieved is captured and maintained appropriately.
Accessibility: To ensure that our products or services are usable by everyone.
Of note, Engineering Fundamentals is seen as an enabler to many compliance areas. Solid engineering fundamentals enables teams with the data, processes, and tools to build solutions that are compliant by design. Retrofitting compliance requirements after a solution has been designed creates additional risk and more work for Microsoft. Additionally, engineering fundamentals enable compliance scale.
Engineering maturity
Given the size and scope of this program, we approached the journey as if we were running a marathon, not a sprint. We kicked off this program in 2020 and have been operating on a multi-year time horizon. Our work has involved and impacted many people, processes, and technology across the enterprise.
Initially, it was important for us to recognize that not all teams were at the same level of maturity. As such, we use the following model to ensure a consistent set of criteria is used to measure engineering maturity, which allowed us to engage with teams at the right level and provide the resources they needed to advance.
Over time, shadow teams matured their level of engineering fundamentals and ability to adhere to compliance requirements. Most teams started their journey with manual efforts, and have made progress over time, but to date are not fully mature yet. We’re continuing to work toward scaling our efforts, especially as the work gets more complex.
Customized support
Likewise, each division had specific needs for the amount and kind of engagement we provided them, depending on the size, scope, and nature of the team. At Microsoft, we customized the approach based on the nature of the team to successfully move the shadow teams forward in their journey.
| Pattern | Characteristics | Approach |
|---|---|---|
| Small teams with small asset footprints |
|
|
| Medium to large asset footprints |
|
|
| Large to very-large asset footprint |
|
|
While we recognize the difference in approach required for each pattern, the intent of the program remains the same, albeit the timing and approach to the work may be different. Eventually, we plan for this program to become a standard operating principle that is absorbed within normal business functions, instead of being managed as a separate program.
Program approach
We prioritized addressing cloud-based solutions because most Shadow applications existed in the cloud, and the digital environment allowed us to scale the program. We developed a three-step approach to guide our work: visibility, controls, and enforcement.
- Visibility: Understanding all the assets, devices, identities, cloud tenants and subscriptions, and applications allowed us to create an inventory with clear ownership. To help with visibility in our cloud assets, we built a scanner that inventories Microsoft Azure assets and reads their configurations. Once we identified the assets, we were able to clean up by ensuring each asset was aligned to an appropriate division and eliminate assets that were empty or unused. This helped reduce our scope for moving onto the next phase. The Microsoft Azure Tenant Security Solutions scanner is available on GitHub.
- Controls: We used information from our scanner to compare the remaining assets’ configurations to our defined controls and create reports for all configurations that were out of compliance.
- Enforcement: We used our inventory and controls reports to start enforcing security and engineering compliance. In many cases, we were able to prevent misconfigurations from the start. When that wasn’t possible, we worked to auto-remediate the non-compliant items to quickly resolve existing issues at scale. To date, we’ve been able to auto-remediate about half of the Microsoft Azure controls we enforce. When auto-remediation wasn’t possible, we employed manual remediation. To manage all this activity, we use a central notification tool that tracks action items and notifies owners of pending deliverables. The tool also allows us to create executive-level reporting to bring awareness of our security risk across all levels of the company.
Lessons learned
Over the past two years, we’ve made a lot of progress, but also encountered many roadblocks. One important discovery is that in specific cases, there may be valid business reasons why an engineering asset may not be able to comply with a security control, and we continue to work with those teams to work around individual parameters to ensure both business and security priorities are met. We also know that this work is never “complete” because security is never-ending; we will continue to update our compliance requirements and approaches as the threat landscape and our technology evolve.
Looking back, there are a few key elements of our Shadow program that enabled our success so far:
Build a team: We funded a central Shadow team within the security organization, led by a dedicated Shadow IT program manager who is fully dedicated to this program. We also obtained program support from the security, IT, and finance departments, and worked together to ensure there were enough IT resources dedicated to this effort to assist with inventory, drive engineering tooling adoption, and provide engineering guidance to the shadow teams. Finally, it was critical to build accountability across the business divisions by appointing one “Directly Responsible Individual” (also known as a DRI) within each participating team, who was accountable for helping their teams work toward compliance, and served as our primary contact and for engaging executive support from those teams.
Drive culture change: While the leaders within the space are important, we quickly realized that we needed to reach the individuals who own and run the Shadow solutions across the company. They needed to understand the importance of security and how to ensure security as a part of their day-to-day actions. We began educating our employees by sharing real security events and highlighting the impacts of these events to emphasize the importance of the actions people take.
We have also adopted a culture to “embrace the red” metrics on the scorecards. We shifted our mindsets to understand that “red” or uncompliant metrics help guide our priorities and work. Once we addressed specific security gaps, those specific metrics turned green, and we immediately replaced the “good” metrics with another “red” metric so that we can continually see progress and address new gaps.
We also provided training, support, and best practice guidance to the shadow teams, including:
- Gathering compliance activities into requirements in quarterly asks
- Providing guidance on funding and skills needs in the first year
- Catering to the lowest knowledge state in wikis and trainings
Be data driven: Managing our reporting process was critical in our ability to drive progress and show the importance of this work. In the early stages, we frequently reviewed our status with executives across the company, and took advantage of our executive sponsor to facilitate these conversations, which helped build momentum. We learned quickly that it was important for us to engage the middle management layer in addition to executives. Our DRIs typically sat two to three layers below the executives, so we needed to ensure there was support for the DRIs between them and the executive.
We also learned over time how to interpret our reporting. We started out reporting on compliance, which worked well until a team had an exception against a control. The exceptions would show up green on our reports. However, an exception is an acceptance of risk, not a sign of compliance. So, we made a plan to start reducing exceptions and began reporting on risk instead of compliance. Reporting on risk aligns well with our Zero Trust reporting, so this was a natural way to drive alignment and create clarity across the company.
What’s next
Our Shadow journey is far from over. We will continue expanding our technology controls and governance to ensure all new solutions and cloud tenants meet compliance standards, and work toward securing the developer pipeline. As for the future of the program, we will reduce custom support processes and enable all teams to adopt our standard enterprise-wide security practices, like the enterprise scorecard and the risk committee. Once teams have met the agreed-upon threshold, the security work will transition from a program into the normal operations of the business.
Addressing Shadow IT risk at any company can feel overwhelming at first. Here are a few things that we learned along the way that can help you get started:
Build a team
- Designate a Shadow IT security program manager
- Obtain a Directly Responsible Individual (DRI) and executive sponsor from all targeted divisions
- Engage your CIO and finance partner for sponsorship
Define the scope
- Scan cloud inventory and configurations within your organization
- Define cloud security controls
Support
- Expand engineering and security capabilities to support additional services
- Develop a communication plan for driving compliance
- Implement a reporting process to identify focus areas and show progress
