Securing Connections

Datafold supports multiple options to secure connections between your resources (e.g., databases and cloud storage) and Datafold.

​

Encryption

Traffic to the Datafold app and API is HTTPS-only. Connections from Datafold to your databases use each database driver’s TLS encryption.

​

IP whitelisting

If access to your data connection is restricted to IP addresses on an allowlist, you will need to manually add Datafold’s addresses in order to use our product. Otherwise, you will receive a connection error when setting up your data connection.

Contact [e-mail] for the egress IP addresses of your Datafold host (us.datafold.com, eub.datafold.com, or your dedicated deployment). For customer-hosted deployments, allow the static egress IP of your own deployment.

On Datafold Classic (app.datafold.com), see the Classic IP list.

​

Private Link

Private connectivity (PrivateLink, Private Service Connect, VPC peering) is set up case by case. For multi-tenant organizations on us.datafold.com or eub.datafold.com, contact [e-mail] to confirm which options are available for your region and database.

AWS

​

AWS PrivateLink

AWS PrivateLink allows you to connect Datafold to your databases without exposing data to the internet. Contact support to confirm availability for your deployment.

The following diagram shows the architecture for a customer with a High Availability RDS setup:

PrivateLink to an HA RDS setup

​

Setup

Supported databases

The following setup assumes you have an RDS/Aurora database you want to connect to. Datafold also supports PrivateLink connections to other databases such as Snowflake, which should only be accessed from your VPC. Please contact [e-mail] to get assistance with connecting to your specific database.

Our support team will send you the following:

• The role ARN to establish the PrivateLink connection.
• The Datafold VPC CIDR range.

You need to do the following steps:

1. Send us the region(s) where your database(s) are located.
2. Create a VPC Endpoint Service and NLB.
   • The core concepts of this setup are described in this AWS blog: Access Amazon RDS across VPCs using AWS PrivateLink and Network Load Balancer.
   • If your databases are HA, please implement the failover mechanics described in the blog.
     • A CloudFormation template for inspiration can be found here.
   • You’ll need to create a Network Load Balancer that points to your database and a VPC Endpoint Service that exposes the NLB.
   • Configure security groups to allow traffic from Datafold’s VPC to your database.
   • If your databases are HA (High Availability), implement automatic failover mechanics to ensure the NLB routes to the active database instance.
   • For detailed step-by-step instructions, see our AWS PrivateLink Setup Guide.
3. Add the provided role ARN as ‘Allowed Principal’ on the VPC Endpoint Service.
4. Allow ingress from the Datafold VPC CIDR range.
5. Send us the:
   • Service name(s), e.g. com.amazonaws.vpce.us-west-2.vpce-svc-0cfd2f258c4395ad6.
   • Availability Zone ID(s) used in the VPCE Service(s), e.g. use1-az6 or usw2-az3.
   • RDS/Aurora hostname(s), e.g. datafold.c2zezoge6btk.us-west-2.rds.amazonaws.com.

At the end, the database hostname used to configure the data source will be the original RDS/Aurora hostname. But with private DNS resolution, we will resolve the hostname to the VPC Endpoint. Our support team will let you know when everything is set up and you can accept the PrivateLink connection and start configuring the data source.

Detailed Instructions

For comprehensive step-by-step instructions including security group configuration, target group setup, and troubleshooting, see our AWS PrivateLink Setup Guide.

​

Cross-Region PrivateLink

If your databases are in a different region from your Datafold deployment, Datafold can set up cross-region PrivateLink. Contact support with the regions involved. The setup is similar to the regular PrivateLink setup.

GCP

​

Private Service Connect

Google Cloud’s Private Service Connect is only available if both parties are in the same cloud region. This option is only available for Datafold Dedicated Cloud customers. The diagram below illustrates how the solution works:

The basics of Private Service Connect are available here.

Azure

​

Azure Private Link

Azure Private Link is only available if both parties are in the same cloud region. This option is only available for Datafold Dedicated Cloud customers. The diagram below illustrates how the solution works:

The basics of Private Link are available here.

For Customer-Hosted Dedicated Cloud, achieving cross-tenant access requires using Private Link. The documentation can be accessed here.

Connecting to Snowflake or Databricks?

Both platforms offload large query results to cloud storage rather than the query engine itself, which usually means a second private endpoint is needed alongside the main account/workspace connection. See the Snowflake and Databricks sections below.

​

Before you approve the first endpoint

Attaching a private endpoint to an Azure Storage account causes Azure to create a public CNAME redirecting the storage hostname to *.privatelink.blob.core.windows.net (or the dfs equivalent). If you already run private DNS zones for blob.core.windows.net or dfs.core.windows.net, enable “Fallback to Internet” on those zones before approving the first endpoint — otherwise this can break DNS resolution for other, unrelated storage accounts in your environment that still rely on the public endpoint.

This applies whether the private endpoint you’re approving is for Snowflake’s internal stage or a Databricks workspace storage account — both sit on Azure Blob Storage under the hood.

​

Connecting to Snowflake

​

Prerequisites

• Your Snowflake account must be on Business Critical Edition or higher. This gates both the account-level Private Link and the internal stage private endpoint.
• Your Azure subscription must be registered with the Azure Storage resource provider before you can connect to a Snowflake internal stage from a private endpoint.

​

Step 1: Retrieve your Private Link configuration

As ACCOUNTADMIN, run:

USE ROLE ACCOUNTADMIN;
SELECT SYSTEM$GET_PRIVATELINK_CONFIG();

Send us:

• privatelink-pls-id
• Your account URL
• The OCSP URLs in the output (we need these for our firewall and DNS configuration)

​

Step 2: Enable Private Link for internal stages

ALTER ACCOUNT SET ENABLE_INTERNAL_STAGES_PRIVATELINK = true;

Re-run SYSTEM$GET_PRIVATELINK_CONFIG() and send us the ResourceID under the privatelink-internal-stage key.

Skipping this step is the most common cause of “connection works, large queries time out.” Without it, there’s no private path for downloading results once they’re written to the internal stage — only the query itself travels over the account endpoint.

​

Step 3: We create your private endpoints

We’ll create one private endpoint for your account and a separate one for the internal stage, and send you both resource IDs.

​

Step 4: Authorize both endpoints

As ACCOUNTADMIN, authorize each endpoint. They use different functions and can’t both be approved from the Azure portal, since the internal stage’s storage account is Snowflake-owned, not yours.

For the account endpoint, SYSTEM$AUTHORIZE_PRIVATELINK also requires a federated token, not just the endpoint resource ID. We’ll generate this token against our own Azure subscription and send it to you along with the resource ID — treat it like a credential (don’t paste it into a shared ticket or channel).

SELECT SYSTEM$AUTHORIZE_PRIVATELINK('<account_private_endpoint_resource_id>', '<federated_token>');

For the internal stage endpoint:

SELECT SYSTEM$AUTHORIZE_STAGE_PRIVATELINK_ACCESS('<stage_private_endpoint_resource_id>');

​

Step 5: Verify

-- Confirms account-level authorization
SELECT SYSTEM$GET_PRIVATELINK();

-- Lists authorized stage endpoints and their approval status
SELECT SYSTEM$GET_STAGE_PRIVATELINK_AUTHORIZED_ENDPOINTS();

​

Network policies and public access

The account endpoint and the internal stage are governed by separate public-access controls:

• Account endpoint — governed by your account-level network policy, if you have one. If so, add Datafold’s private endpoint IP range to the allowed list.
• Internal stage — governed independently, via:
  • SYSTEM$BLOCK_INTERNAL_STAGES_PUBLIC_ACCESS() — blocks all public access to internal stages. All-or-nothing; no partial IP allowlist.
  • SYSTEM$BLOCK_INTERNAL_STAGES_PUBLIC_ACCESS_WITH_EXCEPTION('<cidr_or_ip_list>') — same, but with an IP/CIDR allowlist exception. Only available on Azure, not AWS or GCP.
  • SYSTEM$INTERNAL_STAGES_PUBLIC_ACCESS_STATUS() — check current status.

Let us know which of these (if any) is active so we can confirm whether our private endpoint’s IP range needs to be added explicitly.

​

Connecting to Databricks

​

Prerequisites

• Workspace on the Premium plan.
• VNet injection and secure cluster connectivity (No Public IP) enabled, for connections from the classic compute plane.
• A dedicated subnet (minimum /28) for the storage account’s private endpoints — separate from your two standard workspace subnets.

​

Step 1: Check whether the workspace storage firewall is enabled

By default, a Databricks workspace’s storage account (the one behind the DBFS root and any Unity Catalog workspace catalog) accepts connections from all networks — nothing further is needed for Datafold’s query result downloads to work.

If your organization has enabled the storage firewall to block public network access, continue with the steps below.

​

Step 2: We create two private endpoints for Cloud Fetch

Datafold’s driver uses Cloud Fetch to download query results larger than 1 MB directly from your workspace storage account rather than through the SQL warehouse — results are written as Arrow files (up to 20 MB each) and handed to us as presigned URLs. If the storage account is firewalled, Cloud Fetch needs its own private path from Datafold’s VNet.

These private endpoints live in Datafold’s VNet, so we create them from our own subscription and you approve them in Step 3. We’ll create two against your workspace storage account, using two different Target sub-resource values:

• dfs
• blob

We create these in a resource group other than your workspace’s managed resource group — Azure rejects private endpoints created inside the managed resource group with a deny assignment error. If the error shows up even outside the managed resource group, your workspace may predate the current managed-resource-group permissions model; check with your Databricks account team.

​

Step 3: Approve the endpoints

Go to the workspace storage account (inside the managed resource group, named dbstorage...) → Networking → Private endpoint connections, and confirm both endpoints show Approved. They may auto-approve, or need manual approval.

Cross-tenant approval

Datafold’s private endpoints are created from our own Azure subscription, in our own tenant. The standard “Connect to an Azure resource in my directory” flow assumes same-tenant access — we’ll connect using the resource ID method instead. Confirm with your Azure team that cross-tenant private endpoint approval is enabled in your environment before starting.

​

Serverless SQL warehouses

If you use serverless SQL warehouses, you’ll also need a private endpoint rule for the storage account in your workspace’s Network Connectivity Config (NCC), in addition to the VNet-based private endpoints above.

​

Alternatives to a private endpoint

A private endpoint is the supported path. If one isn’t feasible on your side, contact [e-mail] before planning around an alternative — each one is a change to how Datafold connects, not a setting you can apply on your workspace:

• Disabling Cloud Fetch makes results come back through the SQL warehouse instead of directly from storage. It’s noticeably slower on large result sets, so we treat it as a fallback rather than a default.
• Routing Cloud Fetch through a proxy isn’t a setting Datafold offers today. Ask us rather than designing around it.

Also send us your SQL warehouse hostname, and confirm whether front-end Private Link (the connection Datafold uses to submit queries) is already in place for the workspace — that’s a separate endpoint from the storage path described here.

​

Troubleshooting

Connection succeeds, but queries with any real amount of data time out or hang: This almost always means the account/workspace endpoint is set up correctly, but the storage-side path isn’t. For Snowflake, check that ENABLE_INTERNAL_STAGES_PRIVATELINK is set and the stage endpoint is authorized. For Databricks, check that both the dfs and blob private endpoints exist and are approved. If you reproduce it with your own JDBC-based tooling, a Snowflake error like Timeout waiting for the download of #chunk0 is a direct symptom; Datafold connects with the Python connector, which reports the same failure differently.

Azure rejects the private endpoint with a deny assignment error: You’re likely creating it inside the Databricks-managed resource group. Create it in a separate resource group instead, or contact your Databricks account team if the error persists.

Other resolvers or clients in your Azure environment start failing DNS lookups after you approve a private endpoint: This is the public CNAME behavior described above. Enable “Fallback to Internet” on your private DNS zone for the affected privatelink.*.core.windows.net zone.

​

VPC Peering (multi-tenant)

VPC peering joins both networks, so the IP ranges must not overlap. For multi-tenant organizations, contact [e-mail] with the region, VPC ID, and CIDR of your VPC to confirm whether peering is available.

​

VPC Peering (Dedicated Cloud)

VPC Peering is a supported option for all cloud providers, both for Datafold-hosted and customer-hosted deployments. Basic information for each cloud provider can be found here:

• AWS
• GCP
• Azure

VPC vs VNet

We use the term VPC across all major cloud providers. However, Azure calls this concept a Virtual Network (VNet).

​

SSH Tunnel

To set up a tunnel, please contact our team at [e-mail] and provide the following information:

• Hostname of your bastion host and port number used for SSH service.
• Hostname and port number of your database.
• SSH fingerprint of the bastion host (optional).

We’ll get back to you with:

• SSH public key that you need to add to ~/.ssh/authorized_keys of the tunnel user on your bastion host.
• IP address and port to use for data connection configuration in the Datafold application.

​

IPSec tunnel

Please contact our team at [e-mail] for more information.