How does SynchUP match existing records between two different systems?

Understand the workflow that SynchUP follows to determine whether it needs to update an existing record or create a new record in a destination system.

Before you read

Prior to reading this article, it is highly recommended that you possess a clear understanding of the concepts of internalId and externalId, as well as how SynchUP leverages these identifiers to establish relationships between records in different systems. If you require a refresher on these concepts, please refer to the following resource: SynchUP internalId & externalId

Why record matching?

When synchronizing data between two different systems, the most straightforward scenario is when the record exists in only one system, allowing you to create a duplicate in the second system. By using internalId and externalId, you can establish a relationship between these two records, simplifying future updates and ensuring reliability.

However, there are situations where the record already exists in both systems, and a relationship has not been established using internalId and externalId. A common example is for Customer records. In cases where you are syncing customers between an ERP system and a web platform, customer records can be created in both systems. For instance, your call center may create a customer record when the customer calls in, while the same customer may have created an online account in your web platform.

In this scenario, the two records would already exist in both systems and would not have an internalId and externalId linking them. Ideally, if a SynchUP job is set up to retrieve Customer records from one system and create or update them in another, SynchUP would update the matching records instead of creating new ones. To achieve this, SynchUP utilizes record matching techniques. For example, when it comes to matching Customer records, SynchUP will attempt to match them based on email address. If a match is found, it will link the records together using internalId and externalId, and then update the record accordingly. Each record type has different matching criteria, and furthermore each system has different capabilities when it comes to matching. Matching criteria and system capabilities are detailed in the respective SynchUP Data Pusher knowledge base article.

SynchUP Data Pusher matching workflow

All SynchUP Data Pushers follow the same workflow, with some variation on what data matching techniques are performed. The workflow is as follows: 

Let's go into more detail about each of these steps in the workflow.

1. Does the record contain an externalId?

Focusing on this part of the workflow:

If the incoming record from the Data Puller contains an externalId, it signifies that SynchUP has established a bi-directional relationship between two records in both systems. The Data Pusher will verify if the internalId in the destination system exists.

1. If the externalId on the incoming record matches an internalId in the destination system, SynchUP will utilize that record to perform updates.
2. However, if the externalId on the incoming record does not correspond to an internalId in the destination system, SynchUP will classify that record as "Failed" and refrain from creating or updating anything in the destination system.

This specific path in the workflow deviates and does not proceed further. Essentially, if there is an externalId on the incoming record, it implies that there is a record to update in the destination system. If the record cannot be found, it indicates an issue that requires your intervention for resolution.

2. Does Data Pusher have an externalId storage location defined?

Focusing on this part of the workflow:

During the previous step, we verified the presence of an externalId on the incoming record. If an externalId was found, it indicates that a bi-directional relationship has been established between the two records, allowing us to determine which record to update in the destination system.

However, if the incoming record does not have an externalId, it could mean that your SynchUP Jobs are not configured for bi-directional data synchronization, or the record does not exist in the destination system. To determine if a uni-directional data relationship is established, the Data Pusher will perform a check.

To perform this check, the Data Pusher must have the externalId storage location setting configured. This setting specifies where the internalId of the origin record is saved as the externalId on the destination record. If this setting is not configured, SynchUP will proceed to the next step without creating or updating any records in the destination system.

If the Data Pusher has the externalId storage location setting configured, it will verify if the internalId of the incoming record matches a value saved in the defined storage location.

1. If a match is found between the internalId and the externalId of a record in the destination system, SynchUP will utilize that record to perform updates.
2. If no match is found, the Data Pusher will move on to the next step without creating or updating any records in the destination system.

3. Perform record matching techniques

Focusing on this part of the workflow:

If we have reached this stage, for some reason or another SynchUP has been unable to confirm the presence of an existing record in the destination system that has a bi-directional or uni-directional relationship established through internalId and externalId. At this point, SynchUP's final step is to attempt to find a match between some of the data from the incoming record and the data in the destination system.

This step varies significantly depending on the entity type that the Job is processing. For example, if the Job is processing a Customer record, SynchUP will attempt to match the incoming email address on the Customer record to an email address in the destination system. Alternatively, if the Job is processing a Product record, SynchUP may try to match based on the Item Number. It is important to refer to the specific Data Pusher's knowledge base article to understand the specific data points that the Job is matching on.

Once SynchUP has completed the data matching techniques, there are three possible outcomes: no matches found, a single match found, or multiple matches found. Let's explore each of these scenarios and see how SynchUP handles them.

3a. Perform record matching techniques - No match found

This outcome is clear-cut. If no match is found, SynchUP will generate a new record in the destination system. As long as the Data Pusher has designated a storage location for externalId in the destination system, SynchUP will be capable of updating this newly created record in subsequent Job runs.

3b. Perform record matching techniques - Single match found

This outcome is just as straightforward. If a single match is found, SynchUP will update all data points for the record in the destination system. Similarly to the previous result, as long as the Data Pusher has defined a storage location for externalId in the destination system, SynchUP will save the internalId of the record from the origin system for easier record matching in future Job runs.

3c. Perform record matching techniques - Multiple matches found

This situation can be complex. When SynchUP identifies multiple matches in the destination system, there is no one-size-fits-all solution. Depending on your specific use cases and requirements, you may need to take different actions in this scenario. That's why SynchUP offers you various options to choose from: create a new record, update the last modified record, or fail the record. Each data pusher allows you to configure how you want to handle this scenario to meet your unique business needs.

Let's give a little more details about each option and what it means:

1. Create a new record: Selecting this option in SynchUP will result in the creation of a new record whenever multiple matches are found.
   1. It is important to note that for this option to work effectively, your data pusher should have a storage location for externalId configured. This ensures that a relationship for that record between the two systems is established, allowing the newly created record to be updated in subsequent runs. To avoid the continuous creation of new records in subsequent runs, it is highly recommended to configure a storage location for externalId.
2. Update the last modified record: If you choose this option, SynchUP will determine which of the multiple matches was most recently updated and then update that record.
   1. As long as your data pusher has a storage location for externalId configured, this updated record will establish a relationship between the two systems and update that existing record in subsequent runs.

3. Fail the record: If this option is selected, SynchUP will not perform any updates or creations in the destination system if multiple matches were found. Instead, the record will be marked as "failed".

   1. To ensure data integrity, it is recommended to combine this option with alert emails. This will notify you when a record has encountered multiple matches, allowing you to manually handle the situation and maintain the accuracy of your data.