A Quick Look at Azure Databricks Secret Scopes

November 29, 2021
Azure Databricks secret scopes is an excellent tool for creating effective data security measures and protecting sensitive data.

By: Phillip Sharpless

When building solutions with Azure Databricks (see my previous blog post on Databricks for a quick overview of the analytics service), it’s more than likely that the need to access secure data will arise.

Access to some data may require sensitive information, such as passwords, access tokens, or account names. These are things you really should not have hardcoded within an application or have sitting around in a file somewhere.

How you handle securely accessing and using credentials within any application is always a major security concern. Azure Databricks has an excellent construct for such situations known as secret scopes.

A secret scope serves as secure storage for sensitive information. It also represents a logical container, such that sensitive information can be logically grouped and organized within different scopes for different audiences. Secret scopes store secrets, which are key-value pairs that contain the sensitive information.

Secret scopes can be managed in one of two ways:

  • they can be stored directly within an encrypted database managed by Azure Databricks
  • they can be managed by the Azure Key Vault

Secret scope permissions are managed by something known as Access Control Lists (ACLs). Different people or groups can be assigned to only the scopes they need. There are also varying levels of access permission.

Manage, the highest level of access, gives users the ability to fully read from and write to the scope as well as manage the ACLs associated with it. Write gives users the ability to read from and write to the scope but no ACL access. And finally, Read gives users the ability to read from the scope with no write access.

In summary, secret scopes is an excellent tool for securing and storing sensitive information in Azure Databricks and/or Azure Key Vault. If you’re looking to enhance your data security and are currently working with either of these Azure tools, we highly recommend secret scopes.

Thanks for Reading! Questions?

Thanks for reading! We hope you found this blog post useful. Feel free to let us know if you have any questions about this article by simply leaving a comment below. We will reply as quickly as we can.

Keep Your Data Analytics Knowledge Sharp

Get fresh Key2 content and more delivered right to your inbox!


About Us

Key2 Consulting is a boutique data analytics consultancy that helps business leaders make better business decisions. We are a Microsoft Gold-Certified Partner and are located in Atlanta, Georgia. Learn more here.

Related Content
How to Use Azure AI Language for Sentiment Analysis

How to Use Azure AI Language for Sentiment Analysis

By: Jay Clegg Intro - NLP & Sentiment Analysis Although research in Natural Language Processing (NLP) dates back many decades, recent advancements in both computing hardware and NLP architectures have produced incredibly useful results that have attracted intense...

GitHub Source Control Integration with Azure Synapse Workspace

GitHub Source Control Integration with Azure Synapse Workspace

By: Syed Islam   GitHub source control integration with Azure Synapse workspace allows data professionals to manage scripts, notebooks, and pipelines in a version-controlled environment. This integration provides a centralized repository where teams can...