Orchestration¶
ADF¶
pyatlan.model.assets.core.a_d_f.ADF(__pydantic_self__, **data: Any)
¶
Bases: Catalog
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
ADF_ASSET_FOLDER_PATH: TextField = TextField('adfAssetFolderPath', 'adfAssetFolderPath')
class-attribute
¶
Defines the folder path in which this ADF asset exists.
ADF_FACTORY_NAME: TextField = TextField('adfFactoryName', 'adfFactoryName')
class-attribute
¶
Defines the name of the factory in which this asset exists.
AdfActivity¶
pyatlan.model.assets.core.adf_activity.AdfActivity(__pydantic_self__, **data: Any)
¶
Bases: ADF
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
ADF_ACTIVITY_BATCH_COUNT: NumericField = NumericField('adfActivityBatchCount', 'adfActivityBatchCount')
class-attribute
¶
Defines the batch count of activity to runs in ForEach activity.
ADF_ACTIVITY_FIRST_ROW_ONLY: BooleanField = BooleanField('adfActivityFirstRowOnly', 'adfActivityFirstRowOnly')
class-attribute
¶
Indicates whether to import only first row only or not in Lookup activity.
ADF_ACTIVITY_IS_SEQUENTIAL: BooleanField = BooleanField('adfActivityIsSequential', 'adfActivityIsSequential')
class-attribute
¶
Indicates whether the activity processing is sequential or not inside the ForEach activity.
ADF_ACTIVITY_MAIN_CLASS_NAME: TextField = TextField('adfActivityMainClassName', 'adfActivityMainClassName')
class-attribute
¶
Defines the main class of the databricks spark activity.
ADF_ACTIVITY_NOTEBOOK_PATH: TextField = TextField('adfActivityNotebookPath', 'adfActivityNotebookPath')
class-attribute
¶
Defines the path of the notebook in the databricks notebook activity.
ADF_ACTIVITY_POLICT_RETRY_INTERVAL: NumericField = NumericField('adfActivityPolictRetryInterval', 'adfActivityPolictRetryInterval')
class-attribute
¶
The retry interval in seconds for the ADF activity.
ADF_ACTIVITY_POLICY_TIMEOUT: TextField = TextField('adfActivityPolicyTimeout', 'adfActivityPolicyTimeout')
class-attribute
¶
The timout defined for the ADF activity.
ADF_ACTIVITY_PRECEDING_DEPENDENCY: TextField = TextField('adfActivityPrecedingDependency', 'adfActivityPrecedingDependency')
class-attribute
¶
The list of ADF activities on which this ADF activity depends on.
ADF_ACTIVITY_PYTHON_FILE_PATH: TextField = TextField('adfActivityPythonFilePath', 'adfActivityPythonFilePath')
class-attribute
¶
Defines the python file path for databricks python activity.
ADF_ACTIVITY_REFERENCE_DATAFLOW: TextField = TextField('adfActivityReferenceDataflow', 'adfActivityReferenceDataflow')
class-attribute
¶
Defines the dataflow that is to be used in dataflow activity.
ADF_ACTIVITY_RUNS: KeywordField = KeywordField('adfActivityRuns', 'adfActivityRuns')
class-attribute
¶
List of objects of activity runs for a particular activity.
ADF_ACTIVITY_SINKS: TextField = TextField('adfActivitySinks', 'adfActivitySinks')
class-attribute
¶
The list of names of sinks for the ADF activity.
ADF_ACTIVITY_SINK_TYPE: TextField = TextField('adfActivitySinkType', 'adfActivitySinkType')
class-attribute
¶
Defines the type of the sink of the ADF activtity.
ADF_ACTIVITY_SOURCES: TextField = TextField('adfActivitySources', 'adfActivitySources')
class-attribute
¶
The list of names of sources for the ADF activity.
ADF_ACTIVITY_SOURCE_TYPE: TextField = TextField('adfActivitySourceType', 'adfActivitySourceType')
class-attribute
¶
Defines the type of the source of the ADF activtity.
ADF_ACTIVITY_STATE: KeywordField = KeywordField('adfActivityState', 'adfActivityState')
class-attribute
¶
Defines the state (Active or Inactive) of an ADF activity whether it is active or not.
ADF_ACTIVITY_SUB_ACTIVITIES: TextField = TextField('adfActivitySubActivities', 'adfActivitySubActivities')
class-attribute
¶
The list of activities to be run inside a ForEach activity.
ADF_ACTIVITY_TYPE: KeywordField = KeywordField('adfActivityType', 'adfActivityType')
class-attribute
¶
The type of the ADF activity.
ADF_DATAFLOW: RelationField = RelationField('adfDataflow')
class-attribute
¶
TBC
ADF_DATASETS: RelationField = RelationField('adfDatasets')
class-attribute
¶
TBC
ADF_LINKEDSERVICES: RelationField = RelationField('adfLinkedservices')
class-attribute
¶
TBC
ADF_PIPELINE: RelationField = RelationField('adfPipeline')
class-attribute
¶
TBC
ADF_PIPELINE_QUALIFIED_NAME: KeywordTextField = KeywordTextField('adfPipelineQualifiedName', 'adfPipelineQualifiedName', 'adfPipelineQualifiedName.text')
class-attribute
¶
Unique name of the pipeline in which this activity exists.
PROCESSES: RelationField = RelationField('processes')
class-attribute
¶
TBC
AdfDataflow¶
pyatlan.model.assets.core.adf_dataflow.AdfDataflow(__pydantic_self__, **data: Any)
¶
Bases: ADF
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
ADF_ACTIVITIES: RelationField = RelationField('adfActivities')
class-attribute
¶
TBC
ADF_DATAFLOW_SCRIPT: TextField = TextField('adfDataflowScript', 'adfDataflowScript')
class-attribute
¶
The gererated script for the dataflow.
ADF_DATAFLOW_SINKS: TextField = TextField('adfDataflowSinks', 'adfDataflowSinks')
class-attribute
¶
The list of names of sinks for this dataflow.
ADF_DATAFLOW_SOURCES: TextField = TextField('adfDataflowSources', 'adfDataflowSources')
class-attribute
¶
The list of names of sources for this dataflow.
ADF_DATASETS: RelationField = RelationField('adfDatasets')
class-attribute
¶
TBC
ADF_LINKEDSERVICES: RelationField = RelationField('adfLinkedservices')
class-attribute
¶
TBC
ADF_PIPELINES: RelationField = RelationField('adfPipelines')
class-attribute
¶
TBC
AdfDataset¶
pyatlan.model.assets.core.adf_dataset.AdfDataset(__pydantic_self__, **data: Any)
¶
Bases: ADF
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
ADF_ACTIVITIES: RelationField = RelationField('adfActivities')
class-attribute
¶
TBC
ADF_DATAFLOWS: RelationField = RelationField('adfDataflows')
class-attribute
¶
TBC
ADF_DATASET_ANNOTATIONS: TextField = TextField('adfDatasetAnnotations', 'adfDatasetAnnotations')
class-attribute
¶
The list of annotation assigned to a dataset.
ADF_DATASET_COLLECTION_NAME: TextField = TextField('adfDatasetCollectionName', 'adfDatasetCollectionName')
class-attribute
¶
Defines the name collection in the cosmos dataset.
ADF_DATASET_CONTAINER_NAME: TextField = TextField('adfDatasetContainerName', 'adfDatasetContainerName')
class-attribute
¶
Defines the container or bucket name in the storage file system dataset.
ADF_DATASET_DATABASE_NAME: TextField = TextField('adfDatasetDatabaseName', 'adfDatasetDatabaseName')
class-attribute
¶
Defines the name of the database used in the azure delta lake type of dataset.
ADF_DATASET_FILE_FOLDER_PATH: TextField = TextField('adfDatasetFileFolderPath', 'adfDatasetFileFolderPath')
class-attribute
¶
Defines the folder path of the file in the storage file system dataset.
ADF_DATASET_FILE_NAME: TextField = TextField('adfDatasetFileName', 'adfDatasetFileName')
class-attribute
¶
Defines the name of the file in the storage file system dataset.
ADF_DATASET_LINKED_SERVICE: KeywordField = KeywordField('adfDatasetLinkedService', 'adfDatasetLinkedService')
class-attribute
¶
Defines the name of the linked service used to create this dataset.
ADF_DATASET_SCHEMA_NAME: TextField = TextField('adfDatasetSchemaName', 'adfDatasetSchemaName')
class-attribute
¶
Defines the name of the schema used in the snowflake, mssql, azure sql database type of dataset.
ADF_DATASET_STORAGE_TYPE: TextField = TextField('adfDatasetStorageType', 'adfDatasetStorageType')
class-attribute
¶
Defines the storage type of storage file system dataset.
ADF_DATASET_TABLE_NAME: TextField = TextField('adfDatasetTableName', 'adfDatasetTableName')
class-attribute
¶
Defines the name of the table used in the snowflake, mssql, azure sql database type of dataset.
ADF_DATASET_TYPE: KeywordField = KeywordField('adfDatasetType', 'adfDatasetType')
class-attribute
¶
Defines the type of the dataset.
ADF_LINKEDSERVICE: RelationField = RelationField('adfLinkedservice')
class-attribute
¶
TBC
ADF_PIPELINES: RelationField = RelationField('adfPipelines')
class-attribute
¶
TBC
AdfLinkedservice¶
pyatlan.model.assets.core.adf_linkedservice.AdfLinkedservice(__pydantic_self__, **data: Any)
¶
Bases: ADF
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
ADF_ACTIVITIES: RelationField = RelationField('adfActivities')
class-attribute
¶
TBC
ADF_DATAFLOWS: RelationField = RelationField('adfDataflows')
class-attribute
¶
TBC
ADF_DATASETS: RelationField = RelationField('adfDatasets')
class-attribute
¶
TBC
ADF_LINKEDSERVICE_ACCOUNT_NAME: TextField = TextField('adfLinkedserviceAccountName', 'adfLinkedserviceAccountName')
class-attribute
¶
Defines the name of the account used in the cosmos linked service.
ADF_LINKEDSERVICE_ANNOTATIONS: TextField = TextField('adfLinkedserviceAnnotations', 'adfLinkedserviceAnnotations')
class-attribute
¶
The list of annotation assigned to a linked service.
ADF_LINKEDSERVICE_AZURE_CLOUD_TYPE: TextField = TextField('adfLinkedserviceAzureCloudType', 'adfLinkedserviceAzureCloudType')
class-attribute
¶
Defines the type of cloud being used in the ADLS linked service.
ADF_LINKEDSERVICE_CLUSTER_ID: TextField = TextField('adfLinkedserviceClusterId', 'adfLinkedserviceClusterId')
class-attribute
¶
Defines the cluster id in the Azure databricks delta lake linked service.
ADF_LINKEDSERVICE_CREDENTIAL_TYPE: TextField = TextField('adfLinkedserviceCredentialType', 'adfLinkedserviceCredentialType')
class-attribute
¶
Defines the type of credential, authentication being used in the ADLS, snowflake, azure sql linked service.
ADF_LINKEDSERVICE_DATABASE_NAME: TextField = TextField('adfLinkedserviceDatabaseName', 'adfLinkedserviceDatabaseName')
class-attribute
¶
Defines the name of the database used in the cosmos, snowflake linked service.
ADF_LINKEDSERVICE_DOMAIN_ENDPOINT: TextField = TextField('adfLinkedserviceDomainEndpoint', 'adfLinkedserviceDomainEndpoint')
class-attribute
¶
Defines the url, domain, account_identifier, server in the ADLS, Azure databricks delta lake, snowflake, azure sql linked service.
ADF_LINKEDSERVICE_RESOURCE_ID: TextField = TextField('adfLinkedserviceResourceId', 'adfLinkedserviceResourceId')
class-attribute
¶
Defines the resource id in the Azure databricks delta lake linked service.
ADF_LINKEDSERVICE_ROLE_NAME: TextField = TextField('adfLinkedserviceRoleName', 'adfLinkedserviceRoleName')
class-attribute
¶
Defines the name of the role in the snowflake linked service.
ADF_LINKEDSERVICE_TENANT: TextField = TextField('adfLinkedserviceTenant', 'adfLinkedserviceTenant')
class-attribute
¶
Defines the tenant of cloud being used in the ADLS linked service.
ADF_LINKEDSERVICE_TYPE: KeywordField = KeywordField('adfLinkedserviceType', 'adfLinkedserviceType')
class-attribute
¶
Defines the type of the linked service.
ADF_LINKEDSERVICE_USER_NAME: TextField = TextField('adfLinkedserviceUserName', 'adfLinkedserviceUserName')
class-attribute
¶
Defines the name of the db user in the snowflake linked service.
ADF_LINKEDSERVICE_VERSION: TextField = TextField('adfLinkedserviceVersion', 'adfLinkedserviceVersion')
class-attribute
¶
Defines the version of the linked service in the cosmos linked service.
ADF_LINKEDSERVICE_VERSION_ABOVE: BooleanField = BooleanField('adfLinkedserviceVersionAbove', 'adfLinkedserviceVersionAbove')
class-attribute
¶
Indicates whether the service version is above 3.2 or not in the cosmos linked service.
ADF_LINKEDSERVICE_WAREHOUSE_NAME: TextField = TextField('adfLinkedserviceWarehouseName', 'adfLinkedserviceWarehouseName')
class-attribute
¶
Defines the name of the warehouse in the snowflake linked service.
ADF_PIPELINES: RelationField = RelationField('adfPipelines')
class-attribute
¶
TBC
AdfPipeline¶
pyatlan.model.assets.core.adf_pipeline.AdfPipeline(__pydantic_self__, **data: Any)
¶
Bases: ADF
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
ADF_ACTIVITIES: RelationField = RelationField('adfActivities')
class-attribute
¶
TBC
ADF_DATAFLOWS: RelationField = RelationField('adfDataflows')
class-attribute
¶
TBC
ADF_DATASETS: RelationField = RelationField('adfDatasets')
class-attribute
¶
TBC
ADF_LINKEDSERVICES: RelationField = RelationField('adfLinkedservices')
class-attribute
¶
TBC
ADF_PIPELINE_ACTIVITY_COUNT: NumericField = NumericField('adfPipelineActivityCount', 'adfPipelineActivityCount')
class-attribute
¶
Defines the count of activities in the pipline.
ADF_PIPELINE_ANNOTATIONS: TextField = TextField('adfPipelineAnnotations', 'adfPipelineAnnotations')
class-attribute
¶
The list of annotation assigned to a pipeline.
ADF_PIPELINE_RUNS: KeywordField = KeywordField('adfPipelineRuns', 'adfPipelineRuns')
class-attribute
¶
List of objects of pipeline runs for a particular pipeline.
Airflow¶
pyatlan.model.assets.core.airflow.Airflow(__pydantic_self__, **data: Any)
¶
Bases: Catalog
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
AIRFLOW_RUN_END_TIME: NumericField = NumericField('airflowRunEndTime', 'airflowRunEndTime')
class-attribute
¶
End time of the run.
AIRFLOW_RUN_NAME: KeywordField = KeywordField('airflowRunName', 'airflowRunName')
class-attribute
¶
Name of the run.
AIRFLOW_RUN_OPEN_LINEAGE_STATE: KeywordField = KeywordField('airflowRunOpenLineageState', 'airflowRunOpenLineageState')
class-attribute
¶
State of the run in OpenLineage.
AIRFLOW_RUN_OPEN_LINEAGE_VERSION: KeywordField = KeywordField('airflowRunOpenLineageVersion', 'airflowRunOpenLineageVersion')
class-attribute
¶
Version of the run in OpenLineage.
AIRFLOW_RUN_START_TIME: NumericField = NumericField('airflowRunStartTime', 'airflowRunStartTime')
class-attribute
¶
Start time of the run.
AIRFLOW_RUN_TYPE: KeywordField = KeywordField('airflowRunType', 'airflowRunType')
class-attribute
¶
Type of the run.
AIRFLOW_RUN_VERSION: KeywordField = KeywordField('airflowRunVersion', 'airflowRunVersion')
class-attribute
¶
Version of the run in Airflow.
AIRFLOW_TAGS: TextField = TextField('airflowTags', 'airflowTags')
class-attribute
¶
Tags assigned to the asset in Airflow.
SPARK_ORCHESTRATED_ASSETS: RelationField = RelationField('sparkOrchestratedAssets')
class-attribute
¶
TBC
AirflowDag¶
pyatlan.model.assets.core.airflow_dag.AirflowDag(__pydantic_self__, **data: Any)
¶
Bases: Airflow
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
AIRFLOW_DAG_SCHEDULE: KeywordField = KeywordField('airflowDagSchedule', 'airflowDagSchedule')
class-attribute
¶
Schedule for the DAG.
AIRFLOW_DAG_SCHEDULE_DELTA: NumericField = NumericField('airflowDagScheduleDelta', 'airflowDagScheduleDelta')
class-attribute
¶
Duration between scheduled runs, in seconds.
AIRFLOW_TASKS: RelationField = RelationField('airflowTasks')
class-attribute
¶
TBC
AirflowTask¶
pyatlan.model.assets.core.airflow_task.AirflowTask(__pydantic_self__, **data: Any)
¶
Bases: Airflow
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
AIRFLOW_DAG: RelationField = RelationField('airflowDag')
class-attribute
¶
TBC
AIRFLOW_DAG_NAME: KeywordTextField = KeywordTextField('airflowDagName', 'airflowDagName.keyword', 'airflowDagName')
class-attribute
¶
Simple name of the DAG this task is contained within.
AIRFLOW_DAG_QUALIFIED_NAME: KeywordField = KeywordField('airflowDagQualifiedName', 'airflowDagQualifiedName')
class-attribute
¶
Unique name of the DAG this task is contained within.
AIRFLOW_TASK_CONNECTION_ID: KeywordTextField = KeywordTextField('airflowTaskConnectionId', 'airflowTaskConnectionId.keyword', 'airflowTaskConnectionId')
class-attribute
¶
Identifier for the connection this task accesses.
AIRFLOW_TASK_GROUP_NAME: KeywordField = KeywordField('airflowTaskGroupName', 'airflowTaskGroupName')
class-attribute
¶
Group name for the task.
AIRFLOW_TASK_OPERATOR_CLASS: KeywordTextField = KeywordTextField('airflowTaskOperatorClass', 'airflowTaskOperatorClass.keyword', 'airflowTaskOperatorClass')
class-attribute
¶
Class name for the operator this task uses.
AIRFLOW_TASK_POOL: KeywordField = KeywordField('airflowTaskPool', 'airflowTaskPool')
class-attribute
¶
Pool on which this run happened.
AIRFLOW_TASK_POOL_SLOTS: NumericField = NumericField('airflowTaskPoolSlots', 'airflowTaskPoolSlots')
class-attribute
¶
Pool slots used for the run.
AIRFLOW_TASK_PRIORITY_WEIGHT: NumericField = NumericField('airflowTaskPriorityWeight', 'airflowTaskPriorityWeight')
class-attribute
¶
Priority of the run.
AIRFLOW_TASK_QUEUE: KeywordField = KeywordField('airflowTaskQueue', 'airflowTaskQueue')
class-attribute
¶
Queue on which this run happened.
AIRFLOW_TASK_RETRY_NUMBER: NumericField = NumericField('airflowTaskRetryNumber', 'airflowTaskRetryNumber')
class-attribute
¶
Retry count for this task running.
AIRFLOW_TASK_SQL: TextField = TextField('airflowTaskSql', 'airflowTaskSql')
class-attribute
¶
SQL code that executes through this task.
AIRFLOW_TASK_TRIGGER_RULE: KeywordField = KeywordField('airflowTaskTriggerRule', 'airflowTaskTriggerRule')
class-attribute
¶
Trigger for the run.
INPUTS: RelationField = RelationField('inputs')
class-attribute
¶
TBC
OUTPUTS: RelationField = RelationField('outputs')
class-attribute
¶
TBC
PROCESS: RelationField = RelationField('process')
class-attribute
¶
TBC
Fivetran¶
pyatlan.model.assets.core.fivetran.Fivetran(__pydantic_self__, **data: Any)
¶
Bases: Catalog
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
FIVETRAN_LAST_SYNC_RECORDS_UPDATED: NumericField = NumericField('fivetranLastSyncRecordsUpdated', 'fivetranLastSyncRecordsUpdated')
class-attribute
¶
Number of records updated in the latest sync on Fivetran
FIVETRAN_LAST_SYNC_STATUS: KeywordField = KeywordField('fivetranLastSyncStatus', 'fivetranLastSyncStatus')
class-attribute
¶
Status of the latest sync on Fivetran.
FIVETRAN_WORKFLOW_NAME: KeywordField = KeywordField('fivetranWorkflowName', 'fivetranWorkflowName')
class-attribute
¶
Name of the atlan fivetran workflow that updated this asset
FivetranConnector¶
pyatlan.model.assets.core.fivetran_connector.FivetranConnector(__pydantic_self__, **data: Any)
¶
Bases: Fivetran
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
FIVETRAN_CONNECTOR_CREDITS_USED: NumericField = NumericField('fivetranConnectorCreditsUsed', 'fivetranConnectorCreditsUsed')
class-attribute
¶
Total credits used by this destination
FIVETRAN_CONNECTOR_DESTINATION_NAME: KeywordField = KeywordField('fivetranConnectorDestinationName', 'fivetranConnectorDestinationName')
class-attribute
¶
Destination name added by the user on Fivetran
FIVETRAN_CONNECTOR_DESTINATION_TYPE: KeywordField = KeywordField('fivetranConnectorDestinationType', 'fivetranConnectorDestinationType')
class-attribute
¶
Type of destination on Fivetran. Eg: redshift, bigquery etc.
FIVETRAN_CONNECTOR_DESTINATION_URL: KeywordField = KeywordField('fivetranConnectorDestinationURL', 'fivetranConnectorDestinationURL')
class-attribute
¶
URL to open the destination details on Fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_EXTRACT_TIME_SECONDS: NumericField = NumericField('fivetranConnectorLastSyncExtractTimeSeconds', 'fivetranConnectorLastSyncExtractTimeSeconds')
class-attribute
¶
Extract time in seconds in the latest sync on fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_EXTRACT_VOLUME_MEGABYTES: NumericField = NumericField('fivetranConnectorLastSyncExtractVolumeMegabytes', 'fivetranConnectorLastSyncExtractVolumeMegabytes')
class-attribute
¶
Extracted data volume in metabytes in the latest sync on Fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_FINISHED_AT: NumericField = NumericField('fivetranConnectorLastSyncFinishedAt', 'fivetranConnectorLastSyncFinishedAt')
class-attribute
¶
Timestamp (epoch) when the latest sync finished on Fivetran, in milliseconds
FIVETRAN_CONNECTOR_LAST_SYNC_ID: KeywordField = KeywordField('fivetranConnectorLastSyncId', 'fivetranConnectorLastSyncId')
class-attribute
¶
ID of the latest sync
FIVETRAN_CONNECTOR_LAST_SYNC_LOAD_TIME_SECONDS: NumericField = NumericField('fivetranConnectorLastSyncLoadTimeSeconds', 'fivetranConnectorLastSyncLoadTimeSeconds')
class-attribute
¶
Load time in seconds in the latest sync on Fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_LOAD_VOLUME_MEGABYTES: NumericField = NumericField('fivetranConnectorLastSyncLoadVolumeMegabytes', 'fivetranConnectorLastSyncLoadVolumeMegabytes')
class-attribute
¶
Loaded data volume in metabytes in the latest sync on Fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_PROCESS_TIME_SECONDS: NumericField = NumericField('fivetranConnectorLastSyncProcessTimeSeconds', 'fivetranConnectorLastSyncProcessTimeSeconds')
class-attribute
¶
Process time in seconds in the latest sync on Fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_PROCESS_VOLUME_MEGABYTES: NumericField = NumericField('fivetranConnectorLastSyncProcessVolumeMegabytes', 'fivetranConnectorLastSyncProcessVolumeMegabytes')
class-attribute
¶
Process volume in metabytes in the latest sync on Fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_REASON: KeywordTextField = KeywordTextField('fivetranConnectorLastSyncReason', 'fivetranConnectorLastSyncReason.keyword', 'fivetranConnectorLastSyncReason')
class-attribute
¶
Failure reason for the latest sync on Fivetran. If status is FAILURE, this is the description of the reason why the sync failed. If status is FAILURE_WITH_TASK, this is the description of the Error. If status is RESCHEDULED, this is the description of the reason why the sync is rescheduled.
FIVETRAN_CONNECTOR_LAST_SYNC_RESCHEDULED_AT: NumericField = NumericField('fivetranConnectorLastSyncRescheduledAt', 'fivetranConnectorLastSyncRescheduledAt')
class-attribute
¶
Timestamp (epoch) at which the latest sync is rescheduled at on Fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_STARTED_AT: NumericField = NumericField('fivetranConnectorLastSyncStartedAt', 'fivetranConnectorLastSyncStartedAt')
class-attribute
¶
Timestamp (epoch) when the latest sync started on Fivetran, in milliseconds
FIVETRAN_CONNECTOR_LAST_SYNC_TABLES_SYNCED: NumericField = NumericField('fivetranConnectorLastSyncTablesSynced', 'fivetranConnectorLastSyncTablesSynced')
class-attribute
¶
Number of tables synced in the latest sync on Fivetran
FIVETRAN_CONNECTOR_LAST_SYNC_TASK_TYPE: KeywordField = KeywordField('fivetranConnectorLastSyncTaskType', 'fivetranConnectorLastSyncTaskType')
class-attribute
¶
Failure task type for the latest sync on Fivetran. If status is FAILURE_WITH_TASK or RESCHEDULED, this field displays the type of the Error that caused the failure or rescheduling, respectively, e.g., reconnect, update_service_account, etc.
FIVETRAN_CONNECTOR_LAST_SYNC_TOTAL_TIME_SECONDS: NumericField = NumericField('fivetranConnectorLastSyncTotalTimeSeconds', 'fivetranConnectorLastSyncTotalTimeSeconds')
class-attribute
¶
Total sync time in seconds in the latest sync on Fivetran
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_CHANGE_PERCENTAGE_FREE: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsChangePercentageFree', 'fivetranConnectorMonthlyActiveRowsChangePercentageFree')
class-attribute
¶
Increase in the percentage of free MAR compared to the previous month
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_CHANGE_PERCENTAGE_PAID: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsChangePercentagePaid', 'fivetranConnectorMonthlyActiveRowsChangePercentagePaid')
class-attribute
¶
Increase in the percentage of paid MAR compared to the previous month
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_CHANGE_PERCENTAGE_TOTAL: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsChangePercentageTotal', 'fivetranConnectorMonthlyActiveRowsChangePercentageTotal')
class-attribute
¶
Increase in the percentage of total MAR compared to the previous month
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_FREE: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsFree', 'fivetranConnectorMonthlyActiveRowsFree')
class-attribute
¶
Free Monthly Active Rows used by the connector in the past month
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_FREE_PERCENTAGE_OF_ACCOUNT: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsFreePercentageOfAccount', 'fivetranConnectorMonthlyActiveRowsFreePercentageOfAccount')
class-attribute
¶
Percentage of the account's total free MAR used by this connector
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_PAID: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsPaid', 'fivetranConnectorMonthlyActiveRowsPaid')
class-attribute
¶
Paid Monthly Active Rows used by the connector in the past month
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_PAID_PERCENTAGE_OF_ACCOUNT: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsPaidPercentageOfAccount', 'fivetranConnectorMonthlyActiveRowsPaidPercentageOfAccount')
class-attribute
¶
Percentage of the account's total paid MAR used by this connector
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_TOTAL: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsTotal', 'fivetranConnectorMonthlyActiveRowsTotal')
class-attribute
¶
Total Monthly Active Rows used by the connector in the past month
FIVETRAN_CONNECTOR_MONTHLY_ACTIVE_ROWS_TOTAL_PERCENTAGE_OF_ACCOUNT: NumericField = NumericField('fivetranConnectorMonthlyActiveRowsTotalPercentageOfAccount', 'fivetranConnectorMonthlyActiveRowsTotalPercentageOfAccount')
class-attribute
¶
Percentage of the account's total MAR used by this connector
FIVETRAN_CONNECTOR_NAME: KeywordField = KeywordField('fivetranConnectorName', 'fivetranConnectorName')
class-attribute
¶
Connector name added by the user on Fivetran
FIVETRAN_CONNECTOR_SYNC_FREQUENCY: KeywordField = KeywordField('fivetranConnectorSyncFrequency', 'fivetranConnectorSyncFrequency')
class-attribute
¶
Sync frequency for the connector in number of hours. Eg: Every 6 hours
FIVETRAN_CONNECTOR_SYNC_PAUSED: BooleanField = BooleanField('fivetranConnectorSyncPaused', 'fivetranConnectorSyncPaused')
class-attribute
¶
Boolean to indicate whether the sync for this connector is paused or not
FIVETRAN_CONNECTOR_SYNC_SETUP_ON: NumericField = NumericField('fivetranConnectorSyncSetupOn', 'fivetranConnectorSyncSetupOn')
class-attribute
¶
Timestamp (epoch) on which the connector was setup on Fivetran, in milliseconds
FIVETRAN_CONNECTOR_SYNC_SETUP_USER_EMAIL: KeywordField = KeywordField('fivetranConnectorSyncSetupUserEmail', 'fivetranConnectorSyncSetupUserEmail')
class-attribute
¶
Email ID of the user who setpu the connector on Fivetran
FIVETRAN_CONNECTOR_SYNC_SETUP_USER_FULL_NAME: KeywordField = KeywordField('fivetranConnectorSyncSetupUserFullName', 'fivetranConnectorSyncSetupUserFullName')
class-attribute
¶
Full name of the user who setup the connector on Fivetran
FIVETRAN_CONNECTOR_TOP_TABLES_BY_MAR: TextField = TextField('fivetranConnectorTopTablesByMAR', 'fivetranConnectorTopTablesByMAR')
class-attribute
¶
Total five tables sorted by MAR synced by this connector
FIVETRAN_CONNECTOR_TOTAL_TABLES_SYNCED: NumericField = NumericField('fivetranConnectorTotalTablesSynced', 'fivetranConnectorTotalTablesSynced')
class-attribute
¶
Total number of tables synced by this connector
FIVETRAN_CONNECTOR_TYPE: KeywordField = KeywordField('fivetranConnectorType', 'fivetranConnectorType')
class-attribute
¶
Type of connector on Fivetran. Eg: snowflake, google_analytics, notion etc.
FIVETRAN_CONNECTOR_URL: KeywordField = KeywordField('fivetranConnectorURL', 'fivetranConnectorURL')
class-attribute
¶
URL to open the connector details on Fivetran
FIVETRAN_CONNECTOR_USAGE_COST: NumericField = NumericField('fivetranConnectorUsageCost', 'fivetranConnectorUsageCost')
class-attribute
¶
Total usage cost by this destination
PROCESSES: RelationField = RelationField('processes')
class-attribute
¶
TBC
Matillion¶
pyatlan.model.assets.core.matillion.Matillion(__pydantic_self__, **data: Any)
¶
MatillionComponent¶
pyatlan.model.assets.core.matillion_component.MatillionComponent(__pydantic_self__, **data: Any)
¶
Bases: Matillion
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
MATILLION_COMPONENT_ID: KeywordField = KeywordField('matillionComponentId', 'matillionComponentId')
class-attribute
¶
Unique identifier of the component in Matillion.
MATILLION_COMPONENT_IMPLEMENTATION_ID: KeywordField = KeywordField('matillionComponentImplementationId', 'matillionComponentImplementationId')
class-attribute
¶
Unique identifier for the type of the component in Matillion.
MATILLION_COMPONENT_LAST_FIVE_RUN_STATUS: KeywordField = KeywordField('matillionComponentLastFiveRunStatus', 'matillionComponentLastFiveRunStatus')
class-attribute
¶
Last five run statuses of the component within a job.
MATILLION_COMPONENT_LAST_RUN_STATUS: KeywordField = KeywordField('matillionComponentLastRunStatus', 'matillionComponentLastRunStatus')
class-attribute
¶
Latest run status of the component within a job.
MATILLION_COMPONENT_LINKED_JOB: KeywordField = KeywordField('matillionComponentLinkedJob', 'matillionComponentLinkedJob')
class-attribute
¶
Job details of the job to which the component internally links.
MATILLION_COMPONENT_SQLS: TextField = TextField('matillionComponentSqls', 'matillionComponentSqls')
class-attribute
¶
SQL queries used by the component.
MATILLION_JOB: RelationField = RelationField('matillionJob')
class-attribute
¶
TBC
MATILLION_JOB_NAME: KeywordTextField = KeywordTextField('matillionJobName', 'matillionJobName.keyword', 'matillionJobName')
class-attribute
¶
Simple name of the job to which the component belongs.
MATILLION_JOB_QUALIFIED_NAME: KeywordTextField = KeywordTextField('matillionJobQualifiedName', 'matillionJobQualifiedName', 'matillionJobQualifiedName.text')
class-attribute
¶
Unique name of the job to which the component belongs.
MATILLION_PROCESS: RelationField = RelationField('matillionProcess')
class-attribute
¶
TBC
MatillionGroup¶
pyatlan.model.assets.core.matillion_group.MatillionGroup(__pydantic_self__, **data: Any)
¶
MatillionJob¶
pyatlan.model.assets.core.matillion_job.MatillionJob(__pydantic_self__, **data: Any)
¶
Bases: Matillion
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
MATILLION_COMPONENTS: RelationField = RelationField('matillionComponents')
class-attribute
¶
TBC
MATILLION_JOB_COMPONENT_COUNT: NumericField = NumericField('matillionJobComponentCount', 'matillionJobComponentCount')
class-attribute
¶
Number of components within the job.
MATILLION_JOB_PATH: KeywordTextField = KeywordTextField('matillionJobPath', 'matillionJobPath', 'matillionJobPath.text')
class-attribute
¶
Path of the job within the project. Jobs can be managed at multiple folder levels within a project.
MATILLION_JOB_SCHEDULE: KeywordField = KeywordField('matillionJobSchedule', 'matillionJobSchedule')
class-attribute
¶
How the job is scheduled, for example: weekly or monthly.
MATILLION_JOB_TYPE: KeywordField = KeywordField('matillionJobType', 'matillionJobType')
class-attribute
¶
Type of the job, for example: orchestration or transformation.
MATILLION_PROJECT: RelationField = RelationField('matillionProject')
class-attribute
¶
TBC
MATILLION_PROJECT_NAME: KeywordTextField = KeywordTextField('matillionProjectName', 'matillionProjectName.keyword', 'matillionProjectName')
class-attribute
¶
Simple name of the project to which the job belongs.
MATILLION_PROJECT_QUALIFIED_NAME: KeywordTextField = KeywordTextField('matillionProjectQualifiedName', 'matillionProjectQualifiedName', 'matillionProjectQualifiedName.text')
class-attribute
¶
Unique name of the project to which the job belongs.
MatillionProject¶
pyatlan.model.assets.core.matillion_project.MatillionProject(__pydantic_self__, **data: Any)
¶
Bases: Matillion
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
MATILLION_ENVIRONMENTS: TextField = TextField('matillionEnvironments', 'matillionEnvironments')
class-attribute
¶
List of environments in the project.
MATILLION_GROUP: RelationField = RelationField('matillionGroup')
class-attribute
¶
TBC
MATILLION_GROUP_NAME: KeywordTextField = KeywordTextField('matillionGroupName', 'matillionGroupName.keyword', 'matillionGroupName')
class-attribute
¶
Simple name of the Matillion group to which the project belongs.
MATILLION_GROUP_QUALIFIED_NAME: KeywordTextField = KeywordTextField('matillionGroupQualifiedName', 'matillionGroupQualifiedName', 'matillionGroupQualifiedName.text')
class-attribute
¶
Unique name of the Matillion group to which the project belongs.
MATILLION_JOBS: RelationField = RelationField('matillionJobs')
class-attribute
¶
TBC
MATILLION_PROJECT_JOB_COUNT: NumericField = NumericField('matillionProjectJobCount', 'matillionProjectJobCount')
class-attribute
¶
Number of jobs in the project.
MATILLION_VERSIONS: TextField = TextField('matillionVersions', 'matillionVersions')
class-attribute
¶
List of versions in the project.
Spark¶
pyatlan.model.assets.core.spark.Spark(__pydantic_self__, **data: Any)
¶
Bases: Catalog
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
SPARK_ORCHESTRATED_BY_AIRFLOW_ASSETS: RelationField = RelationField('sparkOrchestratedByAirflowAssets')
class-attribute
¶
TBC
SPARK_RUN_END_TIME: NumericField = NumericField('sparkRunEndTime', 'sparkRunEndTime')
class-attribute
¶
End time of the Spark Job eg. 1695673598218
SPARK_RUN_OPEN_LINEAGE_STATE: KeywordField = KeywordField('sparkRunOpenLineageState', 'sparkRunOpenLineageState')
class-attribute
¶
OpenLineage state of the Spark Job run eg. COMPLETE
SPARK_RUN_OPEN_LINEAGE_VERSION: KeywordField = KeywordField('sparkRunOpenLineageVersion', 'sparkRunOpenLineageVersion')
class-attribute
¶
OpenLineage Version of the Spark Job run eg. 1.1.0
SPARK_RUN_START_TIME: NumericField = NumericField('sparkRunStartTime', 'sparkRunStartTime')
class-attribute
¶
Start time of the Spark Job eg. 1695673598218
SPARK_RUN_VERSION: KeywordField = KeywordField('sparkRunVersion', 'sparkRunVersion')
class-attribute
¶
Spark Version for the Spark Job run eg. 3.4.1
SparkJob¶
pyatlan.model.assets.core.spark_job.SparkJob(__pydantic_self__, **data: Any)
¶
Bases: Spark
Description
Source code in pyatlan/model/assets/core/referenceable.py
Attributes¶
INPUTS: RelationField = RelationField('inputs')
class-attribute
¶
TBC
OUTPUTS: RelationField = RelationField('outputs')
class-attribute
¶
TBC
PROCESS: RelationField = RelationField('process')
class-attribute
¶
TBC
SPARK_APP_NAME: KeywordTextField = KeywordTextField('sparkAppName', 'sparkAppName.keyword', 'sparkAppName')
class-attribute
¶
Name of the Spark app containing this Spark Job For eg. extract_raw_data
SPARK_MASTER: KeywordField = KeywordField('sparkMaster', 'sparkMaster')
class-attribute
¶
The Spark master URL eg. local, local[4], or spark://master:7077