Nicholas Hurt articles

Secure Access to Storage: Azure Databricks and Azure Data Lake Storage Gen2 Patterns 

In a previous article we covered six access control patterns, the advantages and disadvantages of each, and the scenarios in which they would be most appropriate.

Just-in-time Azure Databricks access tokens and instance pools for Azure Data Factory pipelines using workspace automation 

Using AAD tokens it is now possible to generate an Azure Databricks personal access token programmatically, and provision an instance pool using the Instance Pools API. The token can be generated and utilised at run-time to provide “just-in-time” access to the Databricks workspace.

Configure access to Azure Data Lake Gen 2 from Azure Databricks 

This article provides customers of Azure Data Lake (ADLS) and Azure Databricks with an introduction of the common approaches in which data can be accessed and secured in ADLS from Databricks.

Building your Data Lake on Azure Data Lake Storage gen2 – Part 2 

Part 1 of this blog covered some fundamental data lake topics such as planning, design and structure. In Part 2, Nicholas Hurt forcuses on key aspects of ADLS gen2 such as implementation, security and optimisation.

Building your Data Lake on Azure Data Lake Storage gen2 – Part 1 

In part one of this two-part series, Nicholas Hurt will provide guidance to those planning their data lake and addresses some key considerations around planning, design and structure.

Enterprise-wide orchestration using multiple data factories 

Building simple data engineering pipelines with a single Azure Data Factory (ADF) is easy, and multiple activities can be orchestrated within a single pipeline. More sophisticated data engineering patterns require flexibility and reusability through Pipeline Orchestration.