Phil Woubshet Key2 ConsultingBy: Phil Woubshet
 
 
In part 1 of this series, we implemented a solution within the Azure Portal to copy multiple tables from an on-premise SQL Server database to Azure Synapse Analytics (formerly Azure SQL Data Warehouse).

Today’s exercise will be to implement the same solution programmatically using PowerShell.  We will skip the Azure Portal interface entirely.

A PowerShell solution could help augment an existing solution that you build in the Azure Portal that has repeatable elements to it.  You could also utilize a PowerShell solution to launch similar configurations to different resources, servers, data factories, etc.

Here’s an outline of what we will accomplish:

  1. Connect Azure Account
  2. Create Resource Group
  3. Create Data Factory
  4. Create Linked Service
  5. Create Pipeline
  6. Execute Pipeline

 

PowerShell Module installation

This is a one-time task per machine where you want to run the PowerShell script:

 

Create Configuration file:

 

Main Script Components

A. Connect Azure Account

 

B. Create Resource Group

 

C. Create Data Factory

 

D. Create Linked Service to On-Premise SQL Server Database

 

E. Create Datasets

 

F. Create Pipelines

 

G. Execute the Pipeline

 

References

https://docs.microsoft.com/en-us/azure/data-factory/quickstart-create-data-factory-powershell
https://docs.microsoft.com/en-us/azure/data-factory/monitor-programmatically
 

Questions?

Thanks for reading. We hope you found this blog post to be useful. Do let us know if you have any questions or topic ideas related to BI, analytics, the cloud, machine learning, SQL Server, Star Wars, or anything else of the like that you’d like us to write about. Simply leave us a comment below, and we’ll see what we can do!
 

Keep Your Business Intelligence Knowledge Sharp by Subscribing to our Email List

Get fresh Key2 content around Business Intelligence, Data Warehousing, Analytics, and more delivered right to your inbox!

 


Key2 Consulting is a data warehousing and business intelligence company located in Atlanta, Georgia. We create and deliver custom data warehouse solutions, business intelligence solutions, and custom applications.