To send data to Microsoft Fabric OneLake in Azure IoT Operations, you can configure a data flow endpoint. This configuration allows you to specify the destination endpoint, authentication method, table, and other settings.
Prerequisites
The Azure CLI installed on your development machine. Check Available Azure CLI extensions for the minimum required version to use the azure-iot-ops extension. Use az --version to check your version and az upgrade to update if necessary. For more information, see Install the Azure CLI.
The Azure IoT Operations extension for the Azure CLI. Use the following command to add the extension or update it to the latest version:
az extension add --upgrade --name azure-iot-ops
The Azure CLI examples in this article use environment variables so that you can set each value once and then copy and paste the commands as-is. If you're using the Azure IoT Operations Codespaces environment from the quickstart, these variables are already set for you and you can skip this step. Otherwise, set the following environment variables in your shell before you run the commands.
The following scripts set the most commonly used environment variables:
| Environment variable |
Description |
SUBSCRIPTION_ID |
The ID of the subscription that contains your Azure IoT Operations instance. |
RESOURCE_GROUP |
The name of the resource group that contains your Azure IoT Operations instance. |
AIO_INSTANCE_NAME |
The name of your Azure IoT Operations instance. To list your instances, run az iot ops list -o table. |
CLUSTER_NAME |
The name of the Azure Arc-enabled Kubernetes cluster that hosts your instance. |
LOCATION |
The Azure region to use for new resources, for example eastus. |
SUBSCRIPTION_ID=<subscription-id>
RESOURCE_GROUP=<resource-group-name>
AIO_INSTANCE_NAME=<instance-name>
CLUSTER_NAME=<cluster-name>
LOCATION=<region>
$SUBSCRIPTION_ID = "<subscription-id>"
$RESOURCE_GROUP = "<resource-group-name>"
$AIO_INSTANCE_NAME = "<instance-name>"
$CLUSTER_NAME = "<cluster-name>"
$LOCATION = "<region>"
You only need to set the variables that this article uses. This article might use additional environment variables for resource names that you choose. The article explains how to set them where they're introduced.
This article also uses the following environment variables for values that you choose: ENDPOINT, FABRIC_WORKSPACE_NAME, LAKEHOUSE_NAME, PATH_TYPE, ONELAKE_ACCOUNT_NAME, CLIENT_ID, TENANT_ID, and SCOPE. Set each one before you run the related commands.
Assign permission to managed identity
To configure a data flow endpoint for Microsoft Fabric OneLake, we recommend using either a user-assigned or system-assigned managed identity. This approach is secure and eliminates the need for managing credentials manually.
Important
If you're using a data flow graph and a user-assigned managed identity, make sure to assign the identity with the --usage wasm-graph flag when you run the az iot ops identity assign command. This ensures that the identity has the correct permissions for data flow graphs.
After the Microsoft Fabric OneLake is created, you need to assign a role to the Azure IoT Operations managed identity that grants permission to write to the Fabric lakehouse.
If using system-assigned managed identity, in Azure portal, go to your Azure IoT Operations instance and select Overview. Copy the name of the extension listed after Azure IoT Operations Arc extension. For example, azure-iot-operations-xxxx7. Your system-assigned managed identity can be found using the same name of the Azure IoT Operations Arc extension.
Go to Microsoft Fabric workspace you created, select Manage access > + Add people or groups.
- Search for the name of your user-assigned managed identity set up for cloud connections or the system-assigned managed identity. For example, azure-iot-operations-xxxx7.
- Select Contributor as the role, then select Add. This gives the managed identity the necessary permissions to write to the Fabric lakehouse. To learn more, see Roles in workspaces in Microsoft Fabric.
Create data flow endpoint for Microsoft Fabric OneLake
In the operations experience, select the Data flow endpoints tab.
Under Create new data flow endpoint, select Microsoft Fabric OneLake > New.
Enter the following settings for the endpoint:
| Setting |
Description |
| Host |
The hostname of the Microsoft Fabric OneLake endpoint in the format onelake.dfs.fabric.microsoft.com. |
| Lakehouse name |
The name of the lakehouse where the data should be stored. |
| Workspace name |
The name of the workspace associated with the lakehouse. |
| OneLake path type |
The type of path used in OneLake. Select Files or Tables. |
| Authentication method |
The method used for authentication. Choose System assigned managed identity or User assigned managed identity. |
| Client ID |
The client ID of the user-assigned managed identity. Required if using User assigned managed identity. |
| Tenant ID |
The tenant ID of the user-assigned managed identity. Required if using User assigned managed identity. |
Select Apply to provision the endpoint.
Create or replace
Use the az iot ops dataflow endpoint create fabric-onelake command to create or replace a Microsoft Fabric OneLake data flow endpoint.
az iot ops dataflow endpoint create fabric-onelake --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --workspace $FABRIC_WORKSPACE_NAME --lakehouse $LAKEHOUSE_NAME --path-type $PATH_TYPE
The --workspace parameter is the name of the Microsoft Fabric workspace. The --lakehouse is the name of the Microsoft Fabric lakehouse within the workspace. The --path-type parameter specifies the OneLake path type, which can be either Tables or Files.
Here's an example command to create or replace a Microsoft Fabric OneLake data flow endpoint named fabric-endpoint:
az iot ops dataflow endpoint create fabric-onelake --resource-group myResourceGroup --instance myAioInstance --name fabric-endpoint --workspace myWorkspace --lakehouse myLakehouse --path-type Tables
Create or change
Use the az iot ops dataflow endpoint apply command to create or change a Microsoft Fabric OneLake data flow endpoint.
az iot ops dataflow endpoint apply --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --config-file config.json
The --config-file parameter is the path and file name of a JSON configuration file containing the resource properties.
In this example, assume a configuration file named fabric-endpoint.json with the following content stored in the user's home directory:
{
"endpointType": "FabricOneLake",
"fabricOneLakeSettings": {
"authentication": {
"method": "SystemAssignedManagedIdentity",
"systemAssignedManagedIdentitySettings": {}
},
"batching": {
"latencySeconds": 60,
"maxMessages": 100000
},
"host": "https://onelake.dfs.fabric.microsoft.com",
"oneLakePathType": "Tables",
"names": {
"workspaceName": "<WorkspaceName>",
"lakehouseName": "<LakehouseName>"
}
}
}
Here's an example command to create a new Microsoft Fabric OneLake data flow endpoint named fabric-endpoint:
az iot ops dataflow endpoint apply --resource-group myResourceGroupName --instance myAioInstanceName --name fabric-endpoint --config-file ~/fabric-endpoint.json
Create a Bicep .bicep file with the following content.
param aioInstanceName string = '<AIO_INSTANCE_NAME>'
param customLocationName string = '<CUSTOM_LOCATION_NAME>'
param endpointName string = '<ENDPOINT_NAME>'
param workspaceName string = '<WORKSPACE_NAME>'
param lakehouseName string = '<LAKEHOUSE_NAME>'
resource aioInstance 'Microsoft.IoTOperations/instances@2026-03-01' existing = {
name: aioInstanceName
}
resource customLocation 'Microsoft.ExtendedLocation/customLocations@2021-08-31-preview' existing = {
name: customLocationName
}
resource oneLakeEndpoint 'Microsoft.IoTOperations/instances/dataflowEndpoints@2026-03-01' = {
parent: aioInstance
name: endpointName
extendedLocation: {
name: customLocation.id
type: 'CustomLocation'
}
properties: {
endpointType: 'FabricOneLake'
fabricOneLakeSettings: {
// The default Fabric OneLake host URL in most cases
host: 'https://onelake.dfs.fabric.microsoft.com'
authentication: {
// See available authentication methods section for method types
// method: <METHOD_TYPE>
}
oneLakePathType: 'Tables'
names: {
workspaceName: workspaceName
lakehouseName: lakehouseName
}
}
}
}
Then, deploy via Azure CLI.
az deployment group create --resource-group $RESOURCE_GROUP --template-file main.bicep
Important
The use of Kubernetes deployment manifests isn't supported in production environments and should only be used for debugging and testing.
Create a Kubernetes manifest .yaml file with the following content.
apiVersion: connectivity.iotoperations.azure.com/v1
kind: DataflowEndpoint
metadata:
name: <ENDPOINT_NAME>
namespace: azure-iot-operations
spec:
endpointType: FabricOneLake
fabricOneLakeSettings:
# The default Fabric OneLake host URL in most cases
host: https://onelake.dfs.fabric.microsoft.com
authentication:
# See available authentication methods section for method types
# method: <METHOD_TYPE>
oneLakePathType: Tables
names:
workspaceName: <WORKSPACE_NAME>
lakehouseName: <LAKEHOUSE_NAME>
Then apply the manifest file to the Kubernetes cluster.
kubectl apply -f main.yaml
OneLake path type
The oneLakePathType setting determines the type of path to use in the OneLake path. The default value is Tables, which is the recommended path type for the most common use cases. The Tables path type is a table in the OneLake lakehouse that is used to store the data. It can also be set as Files, which is a file in the OneLake lakehouse that is used to store the data. The Files path type is useful when you want to store the data in a file format that isn't supported by the Tables path type.
The OneLake path type is set in the Basic tab for the data flow endpoint.
If you use the az iot ops dataflow endpoint create fabric-onelake command, the --path-type parameter specifies the OneLake path type, which can be either Tables or Files.
az iot ops dataflow endpoint create fabric-onelake --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --workspace $FABRIC_WORKSPACE_NAME --lakehouse $LAKEHOUSE_NAME --path-type $PATH_TYPE
If you use the az iot ops dataflow endpoint apply, the oneLakePathType property is set in the JSON configuration file.
{
"endpointType": "FabricOneLake",
"fabricOneLakeSettings": {
"oneLakePathType": "Tables"
}
}
fabricOneLakeSettings: {
oneLakePathType: 'Tables' // Or 'Files'
}
Important
The use of Kubernetes deployment manifests isn't supported in production environments and should only be used for debugging and testing.
fabricOneLakeSettings:
oneLakePathType: Tables # Or Files
Available authentication methods
The following authentication methods are available for Microsoft Fabric OneLake data flow endpoints.
System-assigned managed identity
Before you configure the data flow endpoint, assign a role to the Azure IoT Operations managed identity that grants permission to write to the Fabric lakehouse:
- In the Azure portal, go to your Azure IoT Operations instance and select Overview.
- Copy the name of the extension listed after Azure IoT Operations Arc extension. For example, copy azure-iot-operations-xxxx7.
- In Microsoft Fabric, go to your workspace and select Manage access > Add people or groups.
- Search for the Azure IoT Operations Azure Arc extension identity that you copied. For example, azure-iot-operations-xxxx7.
- Assign workspace permission of Contributor or higher to the identity.
Then, configure the data flow endpoint with system-assigned managed identity settings.
In the operations experience data flow endpoint settings page, select the Basic tab then choose Authentication method > System assigned managed identity.
Create or replace
Use the az iot ops dataflow endpoint create fabric-onelake command to create or replace a Microsoft Fabric OneLake data flow endpoint.
az iot ops dataflow endpoint create fabric-onelake --auth-type SystemAssignedManagedIdentity --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --workspace $FABRIC_WORKSPACE_NAME --lakehouse $LAKEHOUSE_NAME --path-type $PATH_TYPE
The --workspace parameter is the name of the Microsoft Fabric workspace. The --lakehouse is the name of the Microsoft Fabric lakehouse within the workspace. The --path-type parameter specifies the OneLake path type, which can be either Tables or Files.
Here's an example command to create or replace a Microsoft Fabric OneLake data flow endpoint named fabric-endpoint:
az iot ops dataflow endpoint create fabric-onelake --resource-group myResourceGroup --instance myAioInstance --name fabric-endpoint --workspace myWorkspace --lakehouse myLakehouse --path-type Tables
Create or change
Use the az iot ops dataflow endpoint apply command to create or change a Microsoft Fabric OneLake data flow endpoint.
az iot ops dataflow endpoint apply --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --config-file config.json
The --config-file parameter is the path and file name of a JSON configuration file containing the resource properties.
In this example, assume a configuration file named fabric-endpoint.json with the following content stored in the user's home directory:
{
"endpointType": "FabricOneLake",
"fabricOneLakeSettings": {
"authentication": {
"method": "SystemAssignedManagedIdentity",
"systemAssignedManagedIdentitySettings": {}
},
"batching": {
"latencySeconds": 60,
"maxMessages": 100000
},
"host": "https://onelake.dfs.fabric.microsoft.com",
"oneLakePathType": "Tables",
"names": {
"workspaceName": "<WorkspaceName>",
"lakehouseName": "<LakehouseName>"
}
}
}
Here's an example command to create a new Microsoft Fabric OneLake data flow endpoint named fabric-endpoint:
az iot ops dataflow endpoint apply --resource-group myResourceGroupName --instance myAioInstanceName --name fabric-endpoint --config-file ~/fabric-endpoint.json
fabricOneLakeSettings: {
authentication: {
method: 'SystemAssignedManagedIdentity'
systemAssignedManagedIdentitySettings: {}
}
}
Important
The use of Kubernetes deployment manifests isn't supported in production environments and should only be used for debugging and testing.
fabricOneLakeSettings:
authentication:
method: SystemAssignedManagedIdentity
systemAssignedManagedIdentitySettings:
{}
If you need to override the system-assigned managed identity audience, you can specify the audience setting.
In most cases, you don't need to specify a service audience. Not specifying an audience creates a managed identity with the default audience scoped to your storage account.
Create or replace
Use the az iot ops dataflow endpoint create fabric-onelake command to create or replace a Microsoft Fabric OneLake data flow endpoint.
az iot ops dataflow endpoint create fabric-onelake --auth-type SystemAssignedManagedIdentity --audience https://$ONELAKE_ACCOUNT_NAME.onelake.dfs.fabric.microsoft.com --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --workspace $FABRIC_WORKSPACE_NAME --lakehouse $LAKEHOUSE_NAME --path-type $PATH_TYPE
The --audience parameter specifies the audience for the system-assigned managed identity. The default audience is https://<account>.onelake.dfs.fabric.microsoft.com.
Here's an example command to create or replace a Microsoft Fabric OneLake data flow endpoint named fabric-endpoint:
az iot ops dataflow endpoint create fabric-onelake --auth-type SystemAssignedManagedIdentity --audience https://account.onelake.dfs.fabric.microsoft.com --resource-group myResourceGroup --instance myAioInstance --name fabric-endpoint --workspace myWorkspace --lakehouse myLakehouse --path-type Tables
Create or change
Use the az iot ops dataflow endpoint apply command to create or change a Microsoft Fabric OneLake data flow endpoint.
az iot ops dataflow endpoint apply --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --config-file config.json
The --config-file parameter is the path and file name of a JSON configuration file containing the resource properties.
In this example, assume a configuration file named fabric-endpoint.json with the following content stored in the user's home directory:
{
"endpointType": "FabricOneLake",
"fabricOneLakeSettings": {
"authentication": {
"method": "SystemAssignedManagedIdentity",
"systemAssignedManagedIdentitySettings": {
"audience": "https://<account>.onelake.dfs.fabric.microsoft.com"
}
},
"batching": {
"latencySeconds": 60,
"maxMessages": 100000
},
"host": "https://onelake.dfs.fabric.microsoft.com",
"oneLakePathType": "Tables",
"names": {
"workspaceName": "<WorkspaceName>",
"lakehouseName": "<LakehouseName>"
}
}
}
Here's an example command to create a new Microsoft Fabric OneLake data flow endpoint named fabric-endpoint:
az iot ops dataflow endpoint apply --resource-group myResourceGroupName --instance myAioInstanceName --name fabric-endpoint --config-file ~/fabric-endpoint.json
fabricOneLakeSettings: {
authentication: {
method: 'SystemAssignedManagedIdentity'
systemAssignedManagedIdentitySettings: {
audience: 'https://<ACCOUNT>.onelake.dfs.fabric.microsoft.com'
}
}
}
Important
The use of Kubernetes deployment manifests isn't supported in production environments and should only be used for debugging and testing.
fabricOneLakeSettings:
authentication:
method: SystemAssignedManagedIdentity
systemAssignedManagedIdentitySettings:
audience: https://<ACCOUNT>.onelake.dfs.fabric.microsoft.com
User-assigned managed identity
To use a user-assigned managed identity for authentication, you must first deploy Azure IoT Operations with secure settings enabled. Then you need to set up a user-assigned managed identity for cloud connections. To learn more, see Enable secure settings in Azure IoT Operations deployment.
To grant the user-assigned managed identity access to your Microsoft Fabric workspace:
- In Microsoft Fabric, go to your workspace and select Manage access > Add people or groups.
- Search for your user-assigned managed identity.
- Assign workspace permission of Contributor or higher to the identity.
This permission grants the managed identity the access it needs to write to the Fabric lakehouse.
Then, configure the data flow endpoint with user-assigned managed identity settings.
In the operations experience data flow endpoint settings page, select the Basic tab then choose Authentication method > User assigned managed identity.
Enter the user assigned managed identity client ID and tenant ID in the appropriate fields.
Create or replace
Use the az iot ops dataflow endpoint create fabric-onelake command to create or replace a Microsoft Fabric OneLake data flow endpoint with user-assigned managed identity.
az iot ops dataflow endpoint create fabric-onelake --auth-type UserAssignedManagedIdentity --client-id $CLIENT_ID --tenant-id $TENANT_ID --scope $SCOPE --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --workspace $FABRIC_WORKSPACE_NAME --lakehouse $LAKEHOUSE_NAME --path-type $PATH_TYPE
The --auth-type parameter specifies the authentication method, which is UserAssignedManagedIdentity in this case. The --client-id, --tenant-id, and --scope parameters specify the user-assigned managed identity client ID, tenant ID, and scope respectively.
Here's an example command to create or replace a Microsoft Fabric OneLake data flow endpoint named fabric-endpoint:
Example command:
az iot ops dataflow endpoint create fabric-onelake --auth-type UserAssignedManagedIdentity --client-id ClientId --tenant-id TenantId --scope https://storage.azure.com/.default --resource-group myResourceGroup --instance myAioInstance --name fabric-endpoint --workspace myWorkspace --lakehouse myLakehouse --path-type Tables
Create or change
Use the az iot ops dataflow endpoint apply command to create or change a Microsoft Fabric OneLake data flow endpoint with user-assigned managed identity.
az iot ops dataflow endpoint apply --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --config-file config.json
The --config-file parameter is the path and file name of a JSON configuration file containing the resource properties.
Example fabric-endpoint.json:
{
"endpointType": "FabricOneLake",
"fabricOneLakeSettings": {
"authentication": {
"method": "UserAssignedManagedIdentity",
"userAssignedManagedIdentitySettings": {
"clientId": "<ClientId>",
"scope": "<Scope>",
"tenantId": "<TenantId>"
}
},
"batching": {
"latencySeconds": 60,
"maxMessages": 100000
},
"host": "https://onelake.dfs.fabric.microsoft.com",
"oneLakePathType": "Tables",
"names": {
"workspaceName": "<WorkspaceName>",
"lakehouseName": "<LakehouseName>"
}
}
}
Example command:
az iot ops dataflow endpoint apply --resource-group myResourceGroupName --instance myAioInstanceName --name fabric-endpoint --config-file ~/fabric-endpoint.json
fabricOneLakeSettings: {
authentication: {
method: 'UserAssignedManagedIdentity'
userAssignedManagedIdentitySettings: {
clientId: '<ID>'
tenantId: '<ID>'
// Optional, defaults to 'https://storage.azure.com/.default'
// scope: 'https://<SCOPE_URL>'
}
}
}
Important
The use of Kubernetes deployment manifests isn't supported in production environments and should only be used for debugging and testing.
To use a user-assigned managed identity, specify the UserAssignedManagedIdentity authentication method and provide the clientId and tenantId of the managed identity.
fabricOneLakeSettings:
authentication:
method: UserAssignedManagedIdentity
userAssignedManagedIdentitySettings:
clientId: <ID>
tenantId: <ID>
# Optional, defaults to 'https://storage.azure.com/.default'
# scope: https://<SCOPE_URL>
Here, the scope is optional and defaults to https://storage.azure.com/.default. If you need to override the default scope, specify the scope setting using Bicep or Kubernetes.
Advanced settings
You can set advanced settings for the Fabric OneLake endpoint, such as the batching latency and message count. You can set these settings in the data flow endpoint Advanced portal tab or within the data flow endpoint custom resource.
Batching
Use the batching settings to configure the maximum number of messages and the maximum latency before the messages are sent to the destination. This setting is useful when you want to optimize for network bandwidth and reduce the number of requests to the destination.
| Field |
Description |
Required |
latencySeconds |
The maximum number of seconds to wait before sending the messages to the destination. The default value is 60 seconds. |
No |
maxMessages |
The maximum number of messages to send to the destination. The default value is 100000 messages. |
No |
For example, to configure the maximum number of messages to 1000 and the maximum latency to 100 seconds, use the following settings:
In the operations experience, select the Advanced tab for the data flow endpoint.
Use the az iot ops dataflow endpoint apply command to create or change Microsoft Fabric OneLake data flow endpoint advanced settings.
az iot ops dataflow endpoint apply --resource-group $RESOURCE_GROUP --instance $AIO_INSTANCE_NAME --name $ENDPOINT --config-file config.json
The --config-file parameter is the path and file name of a JSON configuration file containing the resource properties.
In this example, assume a configuration file named fabric-endpoint.json with the following content stored in the user's home directory:
{
"endpointType": "FabricOneLake",
"fabricOneLakeSettings": {
"authentication": {
"method": "SystemAssignedManagedIdentity",
"systemAssignedManagedIdentitySettings": {}
},
"batching": {
"latencySeconds": 100,
"maxMessages": 1000
},
"host": "https://onelake.dfs.fabric.microsoft.com",
"oneLakePathType": "Tables",
"names": {
"workspaceName": "<WorkspaceName>",
"lakehouseName": "<LakehouseName>"
}
}
}
Here's an example command to create a new Microsoft Fabric OneLake data flow endpoint named fabric-endpoint:
az iot ops dataflow endpoint apply --resource-group myResourceGroupName --instance myAioInstanceName --name fabric-endpoint --config-file ~/fabric-endpoint.json
fabricOneLakeSettings: {
batching: {
latencySeconds: 100
maxMessages: 1000
}
}
Important
The use of Kubernetes deployment manifests isn't supported in production environments and should only be used for debugging and testing.
fabricOneLakeSettings:
batching:
latencySeconds: 100
maxMessages: 1000
Next steps
To learn more about data flows, see Create data flow.