Connectors Reference

Azure Data Factory

Azure Data Factory is a hybrid data integration service that allows you to create, schedule and orchestrate your ETL/ELT workflows at scale wherever your data lives, in cloud or self-hosted network.

 

Status: Preview

Tier: Standard

Version: 1.0.0

 

Actions:

Name

Summary

GetDataFactory (string subscriptionId, string resourceGroupName, string dataFactoryName, [internal]string x-ms-api-version)

Get Data Factory

ListPipelines (string subscriptionId, string resourceGroupName, string dataFactoryName, [internal]string x-ms-api-version)

List pipelines

CreatePipelineRun (string subscriptionId, string resourceGroupName, string dataFactoryName, string pipelineName, [internal]string x-ms-api-version)

Create a pipeline run

CancelPipelineRun (string subscriptionId, string resourceGroupName, string dataFactoryName, string pipelineRunName, [internal]string x-ms-api-version)

Cancel a pipeline run

GetPipeline (string subscriptionId, string resourceGroupName, string dataFactoryName, string pipelineName, [internal]string x-ms-api-version)

Get pipeline

ListDataFactories (string subscriptionId, string resourceGroupName, [internal]string x-ms-api-version)

List Factories

GetPipelineRun (string subscriptionId, string resourceGroupName, string dataFactoryName, string pipelineRunName, [internal]string x-ms-api-version)

Get a pipeline run

ListSubscriptions ([internal]string x-ms-api-version)

List subscriptions

ListResourceGroups ([internal]string x-ms-api-version, string subscriptionId, [advanced][Optional]string $filter, [advanced][Optional]integer $top)

List resource groups

 

Triggers:

Name

Summary

 

Objects:

Name

Summary

Activity

 

CreatePipelineRunResponse

 

DataFactory

 

DataFactoryListResult

 

Pipeline

 

PipelineListResult

 

PipelineRun

 

ResourceGroup

 

ResourceGroupListResult

 

ResourceGroupProperties

 

Subscription

 

SubscriptionListResult

 

SubscriptionPolicies

Subscription Policies

 

Actions:

GetDataFactory

Summary: Get Data Factory

Description: Get a particular Data Factory

 

Syntax:

AzureDataFactory.GetDataFactory (string subscriptionId, string resourceGroupName, string dataFactoryName, [internal]string x-ms-api-version)

 

Parameters:

Name

Type

Summary

Required

Related Action

subscriptionId

string

(Subscription)

The unique identifier for the  Microsoft Azure subscription. The subscription ID forms part of the ID for every Azure resource.

True

ListSubscriptions

resourceGroupName

string

(Resource Group)

Resource group name.

True

ListResourceGroups

dataFactoryName

string

(Data Factory Name)

The name of the Data Factory.

True

ListDataFactories

x-ms-api-version

string

(API Version)

 

True

 

Returns:

          Type:DataFactory

          Description: DataFactory information.

 

ListPipelines

Summary: List pipelines

Description: List the list of pipelines from a particular factory

 

Syntax:

AzureDataFactory.ListPipelines (string subscriptionId, string resourceGroupName, string dataFactoryName, [internal]string x-ms-api-version)

 

Parameters:

Name

Type

Summary

Required

Related Action

subscriptionId

string

(Subscription)

The unique identifier for the  Microsoft Azure subscription. The subscription ID forms part of the ID for every Azure resource.

True

ListSubscriptions

resourceGroupName

string

(Resource Group)

Resource group name.

True

ListResourceGroups

dataFactoryName

string

(Data Factory Name)

The name of the Data Factory.

True

ListDataFactories

x-ms-api-version

string

(API Version)

 

True

 

Returns:

          Type:PipelineListResult

          Description: List of pipelines.

 

CreatePipelineRun

Summary: Create a pipeline run

Description: This operation creates a new pipeline run in your factory

 

Syntax:

AzureDataFactory.CreatePipelineRun (string subscriptionId, string resourceGroupName, string dataFactoryName, string pipelineName, [internal]string x-ms-api-version)

 

Parameters:

Name

Type

Summary

Required

Related Action

subscriptionId

string

(Subscription)

The unique identifier for the  Microsoft Azure subscription. The subscription ID forms part of the ID for every Azure resource.

True

ListSubscriptions

resourceGroupName

string

(Resource Group)

Resource group name.

True

ListResourceGroups

dataFactoryName

string

(Data Factory Name)

The name of the Data Factory.

True

ListDataFactories

pipelineName

string

(Data Factory Pipeline Name)

The name of the Data Factory pipeline.

True

ListPipelines

x-ms-api-version

string

(API Version)

 

True

 

Returns:

          Type:CreatePipelineRunResponse

 

CancelPipelineRun

Summary: Cancel a pipeline run

Description: This operation cancels a pipeline run by its run ID.

 

Syntax:

AzureDataFactory.CancelPipelineRun (string subscriptionId, string resourceGroupName, string dataFactoryName, string pipelineRunName, [internal]string x-ms-api-version)

 

Parameters:

Name

Type

Summary

Required

Related Action

subscriptionId

string

(Subscription)

The unique identifier for the  Microsoft Azure subscription. The subscription ID forms part of the ID for every Azure resource.

True

ListSubscriptions

resourceGroupName

string

(Resource Group)

Resource group name.

True

ListResourceGroups

dataFactoryName

string

(Data Factory Name)

The name of the Data Factory.

True

ListDataFactories

pipelineRunName

string

(Data Factory Pipeline Run Id)

The id of the Data Factory pipeline run.

True

x-ms-api-version

string

(API Version)

 

True

 

Returns:

 

GetPipeline

Summary: Get pipeline

Description: Get a particular pipeline

 

Syntax:

AzureDataFactory.GetPipeline (string subscriptionId, string resourceGroupName, string dataFactoryName, string pipelineName, [internal]string x-ms-api-version)

 

Parameters:

Name

Type

Summary

Required

Related Action

subscriptionId

string

(Subscription)

The unique identifier for the  Microsoft Azure subscription. The subscription ID forms part of the ID for every Azure resource.

True

ListSubscriptions

resourceGroupName

string

(Resource Group)

Resource group name.

True

ListResourceGroups

dataFactoryName

string

(Data Factory Name)

The name of the Data Factory.

True

ListDataFactories

pipelineName

string

(Data Factory Pipeline Name)

The name of the Data Factory pipeline.

True

ListPipelines

x-ms-api-version

string

(API Version)

 

True

 

Returns:

          Type:Pipeline

          Description: Pipeline information.

 

ListDataFactories

Summary: List Factories

Description: Get the list of Data Factories

 

Syntax:

AzureDataFactory.ListDataFactories (string subscriptionId, string resourceGroupName, [internal]string x-ms-api-version)

 

Parameters:

Name

Type

Summary

Required

Related Action

subscriptionId

string

(Subscription)

The unique identifier for the  Microsoft Azure subscription. The subscription ID forms part of the ID for every Azure resource.

True

ListSubscriptions

resourceGroupName

string

(Resource Group)

Resource group name.

True

ListResourceGroups

x-ms-api-version

string

(API Version)

 

True

 

Returns:

          Type:DataFactoryListResult

          Description: List of Data Factories.

 

GetPipelineRun

Summary: Get a pipeline run

Description: Get a particular pipeline run execution

 

Syntax:

AzureDataFactory.GetPipelineRun (string subscriptionId, string resourceGroupName, string dataFactoryName, string pipelineRunName, [internal]string x-ms-api-version)

 

Parameters:

Name

Type

Summary

Required

Related Action

subscriptionId

string

(Subscription)

The unique identifier for the  Microsoft Azure subscription. The subscription ID forms part of the ID for every Azure resource.

True

ListSubscriptions

resourceGroupName

string

(Resource Group)

Resource group name.

True

ListResourceGroups

dataFactoryName

string

(Data Factory Name)

The name of the Data Factory.

True

ListDataFactories

pipelineRunName

string

(Data Factory Pipeline Run Id)

The id of the Data Factory pipeline run.

True

x-ms-api-version

string

(API Version)

 

True

 

Returns:

          Type:PipelineRun

          Description: Information about a pipeline run.

 

ListSubscriptions

Summary: List subscriptions

Description: Gets a list of all the subscriptions to which the principal has access.

 

Syntax:

AzureDataFactory.ListSubscriptions ([internal]string x-ms-api-version)

 

Parameters:

Name

Type

Summary

Required

Related Action

x-ms-api-version

string

Values: [2017-09-01-preview]

Client Api Version.

True

 

Returns:

          Type:SubscriptionListResult

          Description: Subscription list operation response.

 

ListResourceGroups

Summary: List resource groups

Description: Lists all the resource groups within the subscription. The results are paginated at 1,000+ records.

 

Syntax:

AzureDataFactory.ListResourceGroups ([internal]string x-ms-api-version, string subscriptionId, [advanced][Optional]string $filter, [advanced][Optional]integer $top)

 

Parameters:

Name

Type

Summary

Required

Related Action

x-ms-api-version

string

Values: [2017-09-01-preview]

Client Api Version.

True

subscriptionId

string

(Subscription)

The unique identifier for the  Microsoft Azure subscription. The subscription ID forms part of the ID for every Azure resource.

True

ListSubscriptions

$filter

string

(Filter)

The filter to apply on the operation.

False

$top

integer(int32)

(Top)

Query parameters. If nothing is passed returns all values.

False

 

Returns:

          Type:ResourceGroupListResult

          Description: List of resource groups.

 


 

Activity

Summary:

Description: Pipeline activity.

 

          Properties:

Name

Type

Summary


 

CreatePipelineRunResponse

Summary:

Description:

 

          Properties:

Name

Type

Summary

runId

string

runId

runId


 

DataFactory

Summary:

Description: DataFactory information.

 

          Properties:

Name

Type

Summary

name

string

Name

The resource name.

tags

Tags

Tags

The resource tags.

properties

Properties

 

Data Factory additional properties.

identity

Identity

 

Managed service identity of the factory.

id

string

Id

The resource identitfier.

type

string

Type

The resource type.

location

string

Location

The resource location

 

Tags

Summary: Tags

Description: The resource tags.

 

          Properties:

Name

Type

Summary

 

Properties

Summary:

Description: Data Factory additional properties.

 

          Properties:

Name

Type

Summary

provisioningState

string

Provisioning State

Factory provisioning state.

createTime

string(date-time)

Create Time

Time the factory was created in ISO8601 format.

version

string

Version

Version of the factory.

 

Identity

Summary:

Description: Managed service identity of the factory.

 

          Properties:

Name

Type

Summary

type

string

Type

The identity type.

principalId

string

Principal Id

The principal id of the identity.

tenantId

string

Tenant Id

The client tenant id of the identity.

 


 

DataFactoryListResult

Summary:

Description: List of Data Factories.

 

          Properties:

Name

Type

Summary

value

array of (DataFactory)

 

The list of Data Factories.

nextLink

string

Next Link

The URL to get the next set of results.


 

Pipeline

Summary:

Description: Pipeline information.

 

          Properties:

Name

Type

Summary

id

string

Id

The resource identifier.

name

string

Name

The resource name of the pipeline.

properties

Properties

Properties

The pipeline properties.

etag

string

etag

Etag identifies change in the resource.

 

Properties

Summary: Properties

Description: The pipeline properties.

 

          Properties:

Name

Type

Summary

activities

array of (Activity)

Activities

Pipeline activities.

concurrency

integer

Concurrency

The max number of concurrent runs for the pipeline.

description

string

Description

The description of the pipeline.

parameters

Parameters

Parameters

The parameters of the pipeline.

 

Parameters

Summary: Parameters

Description: The parameters of the pipeline.

 

          Properties:

Name

Type

Summary

defaultValue

DefaultValue

Default Value

Default value of parameter.

type

string

Type

Parameter type.  Values: [Array, Bool, Float, Int, Object, String]

 

DefaultValue

Summary: Default Value

Description: Default value of parameter.

 

          Properties:

Name

Type

Summary

 


 

PipelineListResult

Summary:

Description: List of pipelines.

 

          Properties:

Name

Type

Summary

value

array of (Pipeline)

 

The list of Data Factories.

nextLink

string

Next Link

The URL to get the next set of results.


 

PipelineRun

Summary:

Description: Information about a pipeline run.

 

          Properties:

Name

Type

Summary

runId

string

Run Id

Identifier of a run

pipelineName

string

Pipeline Name

The pipeline name

parameters

Parameters

Parameters

The full of partial list of parameter name, value pair used in the pipeline run.

invokedBy

InvokedBy

Invoked By

Entity that started the pipeline.

runStart

string(date-time)

Run Start

The start time of a pipeline run in ISO8601 format.

runEnd

string(date-time)

Run End

The end time of a pipeline run in ISO8601 format.

durationInMs

integer(int32)

Duration

The duration of a pipeline run in milliseconds.

status

string

Status

The status of the pipeline run.

message

string

Message

The message from a pipeline run.

lastUpdated

string(date-time)

Last Updated

The last updated timestamp for the pipeline run event in ISO8601 format.

annotations

array of (string)

Annotations

The annotations for the pipeline run.

 

Parameters

Summary: Parameters

Description: The full of partial list of parameter name, value pair used in the pipeline run.

 

          Properties:

Name

Type

Summary

 

InvokedBy

Summary: Invoked By

Description: Entity that started the pipeline.

 

          Properties:

Name

Type

Summary

id

string

ID

The ID of the entity that started the run.

name

string

Name

Name of the entity that started the pipeline run.

 


 

ResourceGroup

Summary:

Description: Resource group information.

 

          Properties:

Name

Type

Summary

id

string

Id

The ID of the resource group (e.g. /subscriptions/XXX/resourceGroups/YYY).

name

string

Name

The Name of the resource group.

location

string

Location

The location of the resource group. It cannot be changed after the resource group has been created. Has to be one of the supported Azure Locations, such as West US, East US, West Europe, East Asia, etc.

managedBy

string

Managed By

Id of the resource that manages this resource group.

tags

Tags

Tags

The tags attached to the resource group.

properties

ResourceGroupProperties

 

The resource group properties.

 

Tags

Summary: Tags

Description: The tags attached to the resource group.

 

          Properties:

Name

Type

Summary

 


 

ResourceGroupListResult

Summary:

Description: List of resource groups.

 

          Properties:

Name

Type

Summary

value

array of (ResourceGroup)

 

The list of resource groups.

nextLink

string

Next Link

The URL to get the next set of results.


 

ResourceGroupProperties

Summary:

Description: The resource group properties.

 

          Properties:

Name

Type

Summary

provisioningState

string

Provisioning State

The provisioning state.


 

Subscription

Summary:

Description: Subscription information.

 

          Properties:

Name

Type

Summary

id

string

Id

The fully qualified Id. For example, /subscriptions/00000000-0000-0000-0000-000000000000.

subscriptionId

string

Subscription Id

The subscription Id.

tenantId

string

Tenant Id

The tenant Id.

displayName

string

Display Name

The subscription display name.

state

string

State

The subscription state.  Values: [Enabled, Warned, PastDue, Disabled, Deleted]

subscriptionPolicies

SubscriptionPolicies

Subscription Policies

Subscription policies.

authorizationSource

string

Authorization Source

The authorization source of the request. Valid values are one or more combinations of Legacy, RoleBased, Bypassed, Direct and Management. For example, 'Legacy, RoleBased'.


 

SubscriptionListResult

Summary:

Description: Subscription list operation response.

 

          Properties:

Name

Type

Summary

value

array of (Subscription)

 

The subscriptions.

nextLink

string

Next Link

The URL to get the next set of results.


 

SubscriptionPolicies

Summary: Subscription Policies

Description: Subscription policies.

 

          Properties:

Name

Type

Summary

locationPlacementId

string

Location Placement Id

The subscription location placement Id. The Id indicates which regions are visible for a subscription. For example, a subscription with a location placement Id of Public_2014-09-01 has access to Azure public regions.

quotaId

string

Quota Id

The subscription quota Id.

spendingLimit

string

Spending Limit

The subscription spending limit.  Values: [On, Off, CurrentPeriodOff]