Mastering

Jobs

Tasks related to running jobs for Tamr Mastering projects

tamr_toolbox.project.mastering.jobs.run(project, *, run_estimate_pair_counts=False, run_apply_feedback=False, run_update_realtime_match=False, process_asynchronously=False)[source]

Run the existing pipeline without training

Parameters

  • project (MasteringProject) – Target mastering project

  • run_estimate_pair_counts (bool) – Whether an estimate pairs job should be run

  • run_apply_feedback (bool) – Whether train should be called on the pair matching model

  • run_update_realtime_match (bool) – Whether to update RealTime match data after publishing clusters

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.update_unified_dataset(project, *, process_asynchronously=False)[source]

Updates the unified dataset for a mastering project

Parameters

  • project (MasteringProject) – Target mastering project

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.estimate_pair_counts(project, *, process_asynchronously=False)[source]

Estimates the number of pairs for a mastering project

Parameters

  • project (MasteringProject) – Target mastering project

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.generate_pairs(project, *, process_asynchronously=False)[source]

Generates the pairs for a mastering project

Parameters

  • project (MasteringProject) – Target mastering project

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.apply_feedback(project, *, process_asynchronously=False)[source]

Applies feedback to update the model for a mastering project

Parameters

  • project (MasteringProject) – Target mastering project

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.update_pair_predictions(project, *, process_asynchronously=False)[source]

Updates pair predictions only.

Parameters

  • project (MasteringProject) – Target mastering project

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.update_clusters(project, *, process_asynchronously=False)[source]

Re-runs clustering only.

Parameters

  • project (MasteringProject) – Target mastering project

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.apply_feedback_and_update_results(project, *, process_asynchronously=False)[source]

Trains the model, predicts the pair labels, and updates the draft clusters of a mastering project

Parameters

  • project (MasteringProject) – Target mastering project

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.update_results_only(project, *, process_asynchronously=False)[source]

Predicts the pair labels based on the existing pair model and updates the draft clusters of a mastering project

Parameters

  • project (MasteringProject) – Target mastering project

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

tamr_toolbox.project.mastering.jobs.publish_clusters(project, *, run_update_realtime_match=False, process_asynchronously=False)[source]

Publishes the clusters of a mastering project

Parameters

  • project (MasteringProject) – Target mastering project

  • run_update_realtime_match (bool) – whether to update RealTime match data after publishing clusters

  • process_asynchronously (bool) – Whether or not to wait for the job to finish before returning - must be set to True for concurrent workflow

Return type

List[Operation]

Returns

The operations that were run

Schema

tamr_toolbox.project.mastering.schema.map_attribute(project, *, source_attribute_name, source_dataset_name, unified_attribute_name)

Maps source_attribute in source_dataset to unified_attribute in unified_dataset. If the mapping already exists it will log a warning and return the existing AttributeMapping from the project’s collection.

Parameters

  • source_attribute_name (str) – Source attribute name to map

  • source_dataset_name (str) – Source dataset containing the source attribute

  • unified_attribute_name (str) – Unified attribute to which to map the source attribute

  • project (Project) – The project in which to perform the mapping

Return type

AttributeMapping

Returns

The created AttributeMapping

Raises

ValueError – if input variables source_attribute_name or source_dataset_name or unified_attribute_name are set to empty strings; or if the dataset source_dataset_name is not found on Tamr; or if source_attribute_name is missing from the attributes of source_attribute_name

tamr_toolbox.project.mastering.schema.unmap_attribute(project, *, source_attribute_name, source_dataset_name, unified_attribute_name)

Unmaps a source attribute.

Parameters

  • source_attribute_name (str) – the name of the source attribute to unmap

  • source_dataset_name (str) – the name of the source dataset containing that source attribute

  • unified_attribute_name (str) – the unified attribute from which to unmap

  • project (Project) – the project in which to unmap the attribute

Return type

None

Returns

None

tamr_toolbox.project.mastering.schema.bootstrap_dataset(project, *, source_dataset, force_add_dataset_to_project=False)

Bootstraps a dataset (i.e. maps all source columns to themselves)

Parameters

  • source_dataset (Dataset) – the source dataset (a Dataset object not a string)

  • project (Project) – the project to do the mapping ing

  • force_add_dataset_to_project (bool) – boolean whether to add the dataset to the project if it is not already a part of it

Return type

List[AttributeMapping]

Returns

List of the AttributeMappings generated

Raises

RuntimeError – if source_dataset is not part of the given project, set ‘force_add_dataset_to_project’ flag to True to automatically add it

tamr_toolbox.project.mastering.schema.unmap_dataset(project, *, source_dataset, remove_dataset_from_project=False, skip_if_missing=False)

Wholly unmaps a dataset and optionally removes it from a project.

Parameters

  • source_dataset (Dataset) – the source dataset (Dataset object not a string) to unmap

  • project (Project) – the project in which to unmap the dataset

  • remove_dataset_from_project (bool) – boolean to also remove the dataset from the project

  • skip_if_missing (bool) – boolean to skip if dataset is not in project. If set to false and dataset is not in project will raise a RuntimeError

Return type

None

Returns

None

Raises

RuntimeError – if source_dataset is not in project and skip_if_missing not set to True

Transformations

class tamr_toolbox.project.mastering.transformations.InputTransformation(transformation, datasets=<factory>)

A transformation scoped to input datasets

Version:

Requires Tamr 2020.009.0 or later

Parameters

  • transformation (str) – The text of a transformations script

  • datasets (List[Dataset]) – The list of input datasets that the script should be applied to

class tamr_toolbox.project.mastering.transformations.TransformationGroup(input_scope=<factory>, unified_scope=<factory>)

A group of input transformations and unified transformations

Version:

Requires Tamr 2020.009.0 or later

Parameters

  • input_scope (List[InputTransformation]) – A list of transformation to apply to input datasets

  • unified_scope (List[str]) – A list of transformation scripts to apply to the unified dataset

tamr_toolbox.project.mastering.transformations.get_all(project)

Get the transformations of a Project

Version:

Requires Tamr 2020.009.0 or later

Parameters

project (Project) – Project containing transformations

Return type

TransformationGroup

Returns

All input transformations and unified transformations of a project

tamr_toolbox.project.mastering.transformations.set_all(project, tx, *, allow_overwrite=True)

Set the transformations of a Project

Version:

Requires Tamr 2020.009.0 or later

Parameters

  • project (Project) – Project to place transformations within

  • tx (TransformationGroup) – Transformations to put into project

  • allow_overwrite – Whether existing transformations can be overwritten

Return type

Response

Returns

Response object created when transformations of a project are replaced

Raises

  • RuntimeError – if allow_overwrite is set to False but transformations already exists in project

  • ValueError – if provided tx are invalid

tamr_toolbox.project.mastering.transformations.get_all_unified(project)

Get the unified transformations of a Project

Version:

Requires Tamr 2020.009.0 or later

Parameters

project (Project) – Project containing transformations

Return type

List[str]

Returns

All unified transformations of a project

tamr_toolbox.project.mastering.transformations.set_all_unified(project, tx, *, allow_overwrite=True)

Set the unified transformations of a Project. Any input transformations will not be altered

Version:

Requires Tamr 2020.009.0 or later

Parameters

  • project (Project) – Project to place transformations within

  • tx (List[str]) – Unified transformations to put into project

  • allow_overwrite – Whether existing unified transformations can be overwritten

Return type

Response

Returns

Response object created when transformations of a project are replaced

Raises

RuntimeError – if allow_overwrite is set to False but transformations already exists in project