Data Platform Virtual Summit 2022 (DPS 2022) is just a few weeks away, running from September 19–23. A free global learning event for data professionals, DPS 2022 features a keynote from Bob Ward (Principal Architect, Microsoft) and Buck Woody (Applied Data Scientist, Microsoft), as well as more than 200 breakout sessions delivered by Azure Data engineering, partner organizations, and community leaders. With content delivered almost around the clock in five editions, DPS 2022 empowers Azure Data professionals worldwide with the deep technical skills they need to move ahead in their careers and digitally transform their organizations.
This year, DPS 2022 features eight tracks focusing on Azure Data:
The virtual platform offers a live Q and A session, a networking lounge, and a community zone. Additionally, attendees will receive lifetime on-demand access to session recordings.
DPS 2022 offers an incredible opportunity to learn directly from our engineering teams, who will share the latest advances and insights on the Azure Data platform.
Register now for a week of free training at the Data Platform Virtual Summit and receive all the session recordings also for free (streaming access).
Today, July 12, 2022, marks the end of support for SQL Server 2012. This version has had an amazing 10-year run and we want to thank our customers for your investment in learning and using Microsoft SQL Server. Looking forward, there are four great options for data estate migration and modernization that will continue to provide security and product support for SQL Server 2012 instances. Microsoft is here to help: the Azure Migration and Modernization Program provides assistance with planning and carrying out your journey to the cloud with migration guidance and cost-saving offers for your move.
There are four options to keep your SQL Server 2012 secure and supported:
As you plan to protect SQL Server 2012 workloads, you should also plan to do the same for Windows Server 2012 and 2012 R2 as they reach end of support next year on October 10, 2023. You can continue to protect these workloads by upgrading to Windows Server 2022 or migrating to the cloud.
With Azure, you can focus on modernization, knowing your Windows Server 2012/R2 legacy workloads are seamlessly secured with free Extended Security Updates. Take advantage of unique capabilities for Windows Server in Azure Virtual Machines or hybrid management through Azure Arc. Solutions such as Azure Automanage simplify management of your Windows Server Virtual Machines, while you can accelerate application modernization with PaaS solutions such as Azure App Service and Azure Kubernetes Service.
For customers that need to run business-critical applications on-premises, you can continue to protect your workloads by upgrading to Windows Server 2022 with the latest innovation in security, hybrid capabilities, and application development.
Microsoft is here to help you get started. We recently announced an expansion to the Azure Migration and Modernization Program, which has helped thousands of customers unlock the value of the cloud. We’re now investing significantly more to support your largest Windows Server and SQL Server migration and modernization projects, up to 2.5 times more than previous investments, based on project eligibility. This investment will help with your migration in two ways: partner assistance with planning and moving your workloads, and Azure credits that offset transition costs during your move to Azure Virtual Machines, Azure SQL Managed Instance, and Azure SQL Database.
Learn more about end of support and migration resources:
Today, we are announcing changes to SQL Server analytics which includes:
We continue to see increased migration to the cloud, with analytical workloads leading that charge.
Customers have indicated that analytics in the cloud best aligns to employee skillsets, deployment simplicity and manageability, and cloud flexibility and scalability.
When we first introduced cloud analytics in 2017, many were still investing in on-premises analytical workloads. Today, we offer a wealth of cloud-based services that provide users with similar functionality, including Azure Data Lake Storage (ADLS), Azure Synapse Analytics, Azure SQL, and Azure Machine Learning.
According to the Gartner® 2020 Data and Analytics survey:
Today, we are announcing the retirement of SQL Server 2019 Big Data Clusters. All existing users of SQL Server 2019 with Software Assurance will be fully supported on the platform for the next three years, through February 28, 2025. This software will continue to be maintained through SQL Server cumulative updates until that time. In the latest version of SQL Server, we are engineering the best mix of on-premises and in-cloud relational workloads and connectivity to Azure Synapse Analytics for advanced analytics in a flexible, scalable, and integrated environment. Please see below and read our documentation on SQL Server Big Data Clusters to learn more.
Today, we are announcing the retirement of PolyBase scale-out groups in Microsoft SQL Server. Scale-out group functionality will be removed from the product in SQL Server 2022. In-market SQL Server 2019, 2017, and 2016 will continue to support the functionality to the end of support for those products.
PolyBase data virtualization will continue to be fully supported as a scale-up feature in SQL Server.
Secondly, Cloudera (CDP) and Hortonworks (HDP) external data sources will also be retired for all in-market versions of SQL Server and will not be included in SQL Server 2022. Moving forward, support for external data sources will be limited to product versions in mainstream support by the respective vendor. You are encouraged to use the new object storage integration functionality available in SQL Server 2022.
In SQL Server 2022, users will need to configure their external data sources to use new connectors when connecting to Azure Storage. The table below summarizes the change:
| External Data Source | From | To |
| Azure Blob Storage | wasb[s] | abs |
| ADLS Gen 2 | abfs[s] | adls |
If you wish to run analytics on-premises, SQL Server 2022 also provides important new capabilities, building upon its data virtualization suite of connectors by providing object storage integration over REST APIs. We will also continue to invest in the Spark SQL connector to ensure first-class connectivity from Apache Spark to all our SQL products. Additionally, we continue to invest in expanding hybrid capabilities with Azure Arc-enabled data services.
Integrating SQL Server with cloud analytics solutions is a critical capability, which is why we are introducing Azure Synapse Link for SQL Server 2022, the latest release of SQL Server, which will be generally available to purchase later this year. This is a major investment in helping you realize cloud-scale analytics in near real-time on your operational data.
Our priority is to empower you with the tools and services that ensure SQL Server integrates seamlessly into the world of analytic workloads in the cloud by blending operational, analytical, and virtual use cases in our flagship database engine. Please contact your Microsoft account manager if you need assistance in exploring how Microsoft can best empower your analytical needs.
¹Gartner Inc.: Use Cloud to Compose Analytics, BI and Data Science Capabilities for Reusability and Resilience, Julian Sun, Joao Tapadinhas, June 10, 2021.
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
It is almost time for PASS Data Community Summit 2021, a free online conference for the Microsoft data platform professional. The conference, hosted by Redgate, will include the latest SQL Server and Azure data innovations, practical training, and networking to empower you to transform your career and your organization. This year’s event is coming to you online for free from November 8 – 12, 2021, and we will continue the tradition of a Microsoft day one keynote.
Hear directly from Microsoft’s Rohan Kumar and senior Microsoft engineering leaders during the day one kick-off keynote as they take you on a journey to a new universe shaped by our past—and built to take us into a limitless future. The cloud has created a whole new universe and advancements in Microsoft data products and services are your bridge.
You’ll see how you can use your existing SQL Server and Azure skills, and learn about new tools and platforms available from Microsoft to deliver faster performance than ever before. You’ll see how to shape your data so you can harness its power to find a new galaxy of insights, answers, and predictions. And you will hear about new innovations that continue Microsoft’s rich heritage of data integrity and governance.
Additionally, in the special on-demand keynote, Microsoft Azure Data CTO Raghu Ramakrishnan and team will share a technical keynote and demos showing Azure Purview and SQL.
Don’t miss this opportunity to see how Microsoft is uniquely positioned to provide you with an end-to-end data platform seamlessly integrating limitless database scale and performance, unmatched analytics and intelligence, and unified data governance.
After the keynotes, ground your learning with in-depth training in one of more than two dozen sessions Microsoft will be delivering. Hear the latest from the Engineering teams who develop the tools you use every day. After your sessions, don’t forget to visit the virtual exhibit hall where you can connect with our team across SQL Server 2022, Azure SQL, Azure Synapse Analytics, Microsoft Power BI, Azure Arc, and more.
Register for PASS Data Community Summit today.
Data Platform Virtual Summit 2021 (DPS 2021) is just a few days away. A free, global learning event for the data professionals, DPS 2021 will feature over 150 breakout sessions and 15 training classes. This content will be delivered by Azure Data Engineering, partner organizations, community leaders, and Data Platform MVPs. The event is fine-tuned for global time zones, AMERICA, EMEA, and APAC making it a truly global and inclusive learning event. Attendees will get to learn about the latest SQL Server and Azure Data innovations and gain deep technical skills to move ahead in your careers.
DPS 2021 will feature five parallel tracks focusing on Azure Data (Development and Administration), Advanced Analytics, Power BI, and Artificial Intelligence. The virtual platform will give a truly immersive experience to the attendees by offering live Q&A, networking lounge, Azure Data + AI Gurukul (Technical Round Tables), and community zone. Attendees can attend sessions and network amongst peers and speakers, convenient to their time zones. Delegates will get the session recordings for 12-month on-demand access. DPS 2021 offers an incredible opportunity to learn directly from our engineering teams, who will be sharing the latest advances and insights on the data platform, providing in-depth training across key products and technologies.
Bob Ward, Anna Hoffman, and Buck Woody will be the keynote speakers for DPS 2021. The trio will showcase SQL from edge to cloud.
Microsoft Azure Data Teams are delivering over 40 sessions at DPS 2021. Hear the latest from the teams who develop the tools you use every day, and engage in live discussions. Visit the virtual expo hall where you can connect with our team across SQL Server, Azure SQL, Azure Synapse Analytics, Power BI, and more.
DPS 2021 features a variety of learning formats including breakouts, deep-dives, short-drives, and demo-only sessions.
Register for Data Platform Virtual Summit today and be part of an amazing week of training convenient to your time zone and receive 12 months of on-demand access to world-class learning.
Many organizations have either moved or extended their SQL Server operations from an on-premises data center into Azure. There are many reasons why you might be considering this move for your organization as well. Hosting SQL Server on Azure lets you flexibly scale as needed well beyond what’s possible in the physical data center. Often, the big question is simply how best to get your data into the cloud.
There are several possible methodologies to choose from, and Microsoft provides tools to help you decide what’s right for your organization. For instance, you might use the Azure Database Migration Service to automate the process and ensure that you are compliant with Microsoft’s best practices from the start. Or you might simply leverage the Data Migration Assistant to assess your migration and identify any blocking issues or incompatibilities before executing one of the many strategies for migration, including:
With Pure Cloud Block Store™ for Azure from our partner Pure Storage, there is another easy option which warrants your consideration. With built-in replication capability from an on-premises FlashArray, it’s an easy way to replicate your SQL databases directly into Azure and then simply attach them to a SQL Server instance running on Azure VM.
Running SQL Server on an Azure Virtual Machine is one of the quickest and easiest ways to lift and shift your SQL workloads into the cloud. And with Cloud Block Store handling the data movement, it can be even easier. One of the great things about SQL Server on Azure Virtual Machines is that it is the same SQL Server database engine you are used to running on-premises. For the most part, configuration, settings, maintenance, and performance tuning techniques are all going to be the same. Likewise, Cloud Block Store is running the same Purity operating environment as an on-premises FlashArray. Together, the two combine to give you a familiar management experience right from the start.
As a block storage service created from Azure VMs and Azure Managed Disks, Cloud Block Store extends these cloud-native building blocks to provide a vast array of rich data services which make it not only a great way to migrate data but also a compelling option for storing and managing SQL Server data long term. Let’s take a look at a few of them.
Whether it’s replicating data from an on-premises array or among multiple Cloud Block Store instances, you can use the data replication functionality that comes natively with Cloud Block Store for easy data migration. That means you can asynchronously transfer database-consistent point-in-time snapshots directly into Azure, all while the on-premises production database stays up and running, then use the Azure copy for several additional purposes.
First and foremost, the data replication capability of Cloud Block Store can be used as the primary means to migrate data from an on-premises array into Azure. It’s fast, efficient, and completely offloads the task of data transfer from the SQL Server hosts, letting the storage array do all the heavy lifting. As a bonus, all the previous snapshots that were taken on the on-premises side become potential recovery or cloning points within Azure.
It’s also incredibly easy to set up with just a few simple steps:
Once your database is in Azure, Cloud Block Store provides further options, including:
With Pure Cloud Block Store on Azure, you get fast, flexible storage from a foundation built on Azure disk, and building upon that, a host of advanced data features and functionality that enterprise customers have come to depend on, all provided directly in Azure. It provides a fast and efficient means to transport your data into the cloud and makes a number of options available to you once it’s there—from dev/test, to disaster recovery, to full-on production in the cloud with the highest levels of performance and data protection. Try Cloud Block Store out today and discover what Pure Cloud Block Store on Azure can do for you and your business.
SQL Server Big Data Clusters (BDC) is a new capability brought to market as part of the SQL Server 2019 release. BDC extends SQL Server’s analytical capabilities beyond in-database processing of transactional and analytical workloads by uniting the SQL engine with Apache Spark and Apache Hadoop to create a single, secure, and unified data platform. BDC is available exclusively to run on Linux containers, orchestrated by Kubernetes, and can be deployed in multiple-cloud providers or on-premises.
Today, we’re announcing the release of the latest cumulative update (CU9) for SQL Server Big Data Clusters, which includes important capabilities:
This announcement highlights some of the major improvements, provides additional context to better understand the design behind these capabilities, and points you to relevant resources to learn more and get started.
SQL Server Big Data Clusters, a feature released as part of SQL Server 2019, is a data platform for operational and analytical workloads. We are announcing new configuration management functionality as part of today’s CU9 release. Workload requirements are constantly changing and these enhancements will help customers ensure that their Big Data Cluster is always prepared for their needs.
Configuration management is the ability to alter or tune various parts of the Big Data Cluster after deployment and to provide users with clarity into the cluster’s configurations. This allows administrators to configure the Big Data Cluster configurations to meet their workload’s needs. Whether an administrator wants to turn on SQL Agent, define the baseline resources for their organization’s Spark jobs, or even see what settings are configurable at each scope—configuration management is the one-stop solution to meet these needs.
To enable this functionality, we are exposing new commands to the azdata command line interface (CLI). Azdata, an interface to manage a BDC, now includes post-deployment configuration functionality to set, diff, and apply configuration settings. To start, customers can configure settings at the cluster, service, and resource scope and then commit them for change. After applying pending configuration changes, customers can monitor the process through azdata or Azure Data Studio. Once the update is completed, the Big Data Cluster is ready for the next workload.
Learn more and get started with configuration management.
Data engineers and data scientists often want to experiment with and use a variety of different libraries and packages as part of their workflows. There are separate ways to do this for each language including importing from Maven, installing from Python Package Index (PyPi) or conda, or installing from Microsoft R Application Network (MRAN). Before today, customers could import jars from Maven or reference custom packages stored in Hadoop Distributed File System (HDFS) through Spark job configurations.
Starting in CU9, data engineers and data scientists now have added flexibility for their PySpark jobs through job-level virtual environments. They can easily configure a conda virtual environment and get to work with their favorite Python libraries.
Learn how to configure a job-level Spark environment.
In SQL Server Big Data Clusters CU8, we introduced a comprehensive encryption at rest feature set that focused on system-managed keys. This enabled application-level encryption capabilities to all data stored in the platform, on both SQL Server and HDFS. The HDFS experience provided at that time for administrators was centered on usage of Azure Data Studio Notebooks to control all aspects of the feature. Starting with CU9, in addition to expanding the Notebook experience, we are enabling HDFS encryption zones and HDFS key management through azdata. This enables the automation of encryption at rest administrative tasks for HDFS administrators, a much desirable and consistent feature of the SQL Server Big Data Clusters platform.
To learn more about the new notebooks and the new azdata commands, visit the release notes.
Check out the SQL Server CU9 release notes for Big Data Clusters to learn more about all of the improvements available with the latest update. For a technical deep-dive on Big Data Clusters, read the documentation and visit our GitHub repository.
Follow the instructions on our documentation page to get started and deploy Big Data Clusters.
On February 3-4, the Azure SQL and Azure Data Factory teams are partnering on a learning event for the community to come together to learn about Azure SQL and Azure Data Factory, meet the engineering teams behind these services, and learn how you can get started with using Azure Data Factory and Azure SQL together.
The digital event occurs across two time zones:
Register now for the free digital event today! We can’t wait to meet everyone at the virtual event.
Many people are familiar with SQL Server and SQL Server Integration Services (SSIS). These are services that global enterprise customers love for all their data and data integration scenarios.
Azure is a great place for SQL Server workloads. The Azure SQL family of SQL cloud databases provides flexible options for application migration, modernization, and development. The Azure SQL family is comprised of SQL Server on Azure Virtual Machine, Azure SQL Managed Instance, Azure SQL Database, and Azure SQL Edge, all of which are based on the same core SQL Server engine. Migrate your SQL Server workloads to Azure SQL and improve your total cost of ownership (TCO), accelerate your time to market, and operate more efficiently.
Azure Data Factory is a fully managed, serverless data integration solution for ingesting, preparing, and transforming all your data at scale. In addition, you can re-host your SSIS packages using Azure Data Factory, enabling you to realize significant costs savings. Azure Data Factory enables you to bring data from diverse and multiple data sources to Azure SQL and more.
At the digital event, occurring around the clock, you will meet and interact live with many members of the Microsoft Azure Data product group including industry experts Bob Ward, Buck Woody, Davide Mauri, Julie Koesmarno, Silvano Coriani, Mara-Florina Steiu, Arvind Shyamsundar, Anna Hoffman, Srini Acharya, Drew Skwiers-Koballa, Andreas Wolter, Mark Kromer, Sandy Winarko, Wee Hyong Tok, Ye Xu, Abhishek Narain, Linda Wang, Shirley Wang and more!
Data Exposed Live is coming to you on all major streaming platforms, namely Learn TV, Microsoft’s around the clock, free channel. The show, hosted by Anna Hoffman, airs live every Wednesday at 9:00 AM PT.
This digital event is a Data Exposed Special—kicking off with a special episode of Data Exposed Live at 9:00 AM PT. The event is packed with an exciting agenda, which also includes two Ask the Experts sessions.
We are looking forward to meeting the Azure SQL and Azure Data Factory community.
Register now for the free digital event today! We can’t wait to meet everyone.
Register today to:
Update: In January 2021, Microsoft publicly launched the new Azure Data Community webpage, began onboarding qualified user groups to the new Azure Data Meetup network, and began providing Microsoft Teams access to qualified user groups. The blog post below has been updated accordingly. If you are a leader who is interested in adding your existing user group to our network, sign up here.
This past year has been marked by a combination of challenges and change. Like so many people and organizations across the globe, we have all faced uncertainly and the need to adapt.
Recently, the PASS organization, formerly known as the Professional Association for SQL Server, announced that it ceased operations on January 15, 2021. We would like to thank PASS for its work to grow and support our SQL family over the last two decades.
Although Microsoft built SQL Server, it’s clear that the passion and dedication from each of you is what makes it thrive.
To empower the community, we are providing you with some resources to help you continue to connect with one another, learn and develop your skills, and grow in your careers.
Check out the Azure Data Community landing page today to join local user groups and access SQL Server and Azure Data content.
If you are a user group leader interested in finding out more about how to access and use these resources, or if you have additional feedback on how we can best support the SQL community, please let us know.
Data Platform Virtual Summit 2020 (DPS 2020) is just a couple of weeks away. A global learning event for data professionals, DPS 2020 features a keynote from Rohan Kumar, Microsoft Corporate Vice President of Azure Data, as well as 200 breakout sessions and 30 training classes delivered by Azure Data engineering, partner organizations, and community leaders. With content delivered around-the-clock, DPS 2020 empowers Azure Data professionals worldwide with the deep technical skills they need to move ahead in their careers and digitally transform their organizations.
This year, DPS 2020 features five parallel tracks focusing on Azure Data:
The virtual platform offers live Q and A, a networking lounge, a community zone, and technical round tables. Additionally, attendees will receive 12 month on-demand access to session recordings.
DPS 2020 offers an incredible opportunity to learn directly from our engineering teams, who will share the latest advances and insights on the Azure Data platform.
Register now for a week of training at Data Platform Virtual Summit and receive twelve months of on-demand access to the DPS 2020 sessions.
It is almost time for PASS Virtual Summit 2020, the premier learning event for the data professional. Similar to the in-person event, this virtual conference will include the latest SQL Server and Azure Data innovations, practical training, and networking—to empower you to transform your career and your company.
This year, PASS is taking full advantage of the virtual platform and partnering with Microsoft to make your PASS Virtual Summit experience second to none. Learn directly from our engineering team, who will be sharing the latest insights and advances on the data platform, providing in depth training across key tools and topics, and engaging live for one to one support to help you solve your challenges in real time.
If you haven’t already heard, Rohan Kumar, Microsoft Corporate Vice President of Azure Data, will be back this year for the Day 1 keynote. Rohan will showcase the latest innovation across the Microsoft Data Platform, including exclusive announcements at PASS Virtual Summit. Additionally, in the Day 2 keynote, Technical Fellow, Hanuma Kodavalla will discuss SQL eco-system transformation and innovation at scale.
After the keynotes, ground your learning with in-depth training in one of the over thirty sessions Microsoft will be delivering. Hear the latest from the teams who develop the tools you use every day, and ask or upvote questions through the Interactive Platform. After your sessions, don’t forget to visit the virtual exhibit hall where you can connect with our team across SQL Server, Azure SQL, Azure Synapse Analytics, Power BI, and more.
The Microsoft Azure Data Clinic will be back again this year, providing you with one to one access to the engineers behind the products you love. This year we are taking full advantage of the virtual platform, to allow you to pre-book, scheduling your clinic appointment around those must-attend networking activities, and ensuring that you are connected to the right expert for your challenge.
Register for PASS Virtual Summit today for an incredible week of training, and 12 months on-demand access to world class learning.
For the past decade, data science has become integral to many enterprise applications. Languages such as R and Python have left the realm of data scientists and are being used more frequently by data engineers supporting them. R and Python are the most popular data science languages currently for creating, training, and scoring models. Modernization has also accelerated the use of these languages leveraging the benefits of the cloud to enable in-database processing of machine learning algorithms and models.
In SQL Server 2016 through 2019, we added R and Python language support, which enable secure execution of R and Python programs in the context of a SQL Server query. This enables a wide range of scenarios such as performing advanced text and data preparation tasks, and reaching out to external APIs to get data and also training machine learning models and model scoring.
Previously, we announced a Java extension. Today, we are sharing that we are open sourcing the R and Python language extensions for SQL Server for both Windows and Linux on GitHub.
These extensions are the latest examples using an evolved programming language extensibility architecture which allows integration with a new type of language extension. This new architecture gives customers the freedom to bring their own runtime and execute programs using that runtime in SQL Server, while leveraging the existing security and governance that the SQL Server programming language extensibility architecture provides.
Choosing which runtime to use does provide the flexibility to choose different distributions of R and Python, and as newer versions of the R and Python runtimes get released, this architecture will make it easier to upgrade the R and Python runtime. Enterprises need to have a support contract in place for their R and Python runtime.
Now that support is not an issue, let’s look at what use cases R and Python can enable inside SQL Server. Bringing R and Python workloads closer to the data opens a variety of possibilities:
The R and Python language extension leverages the Extensibility Framework API for SQL Server to communicate and exchange data with SQL Server. This API has been publicly documented. The API in combination with the open source code of the R and Python language extension provides an end to end example implementation of how a programming language extension can be built. This makes it easier for additional programming language extensions to be built for SQL Server by the community. What language extensions would you like to see?
Whether you are interested in creating your own language extension or just using the R and Python language extension for SQL Server, here is some more information to get you started.
