One of the most useful features when working with Flatfile is the ability to work with data outside of Flatfile. You have the ability to do things such as use Flatfile to clean the user data already in your system, compare new data to the current data in your system during an import and keep data consistent as edits are being made. In this guide, we will show you some of the most useful ways to use Flatfile with your external data source as well as help you avoid some of the common pitfalls when using an external datasource with Flatfile.
Common use cases:
When we talk about initializing the importer with data, we are talking about bypassing the standard, "choose a file" style import to access some external data using Flatfile as a means of editing and submitting the clean data. During this process, you are able to provide feedback in terms of errors and warnings that will indicate to the user what may need to be corrected.
If you check out our
requestDataFromUser() SDK reference, you will notice that there is an optional object (LoadOptionsObject) you can pass into the method to initialize the importer with data instead of a user file. When you pass in this object, the matching stage of the import (which would include field hooks and record init hooks) is skipped and you will go to the Review phase of the import. In all of the common use case scenarios above, this will be the starting point. It is useful to know that within the
InputObject SDK reference, you will pass in the data, but can optionally pass in the sequence (index number) and any error messaging. We allow for different error messaging, and you can learn more about the levels of messaging here.
Let's take a look at an example of how this works. For a typical import, you might call the
requestDataFromUser() method like in the below example:
For the sake of the example, we are taking Flatfile and launching an import using the
click event of the
launchButton which is assumed to be a valid element in your UI.
To make things easier to visualize, we are going to set a variable with some data and pass that into the method. Keep in mind that you can work with external data, but it will need to have the same data structure as what you see here.
Now, we can pass this object into the method as seen below:
And when the importer starts, your user would see the below screenshot.
Common use cases:
Being able to validate your data against a server or use an external resource for formatting data is useful in the process for having clean data. With Field Hooks, you can send the entire dataset of a particular field to your server or some form of external validation. In the below example, we have used typicode's jsonplaceholder API to replicate having some data to validate against on a server. You can check out our data on GitHub, but what we are looking for is to make sure that there are no emails in the following data that are already on our server.
|First Name||User ID|
Below is what the code would look like for validating this against the external datasource.
What happens in the example with the data provided is that the first row comes back showing that the email is already in the data and is a duplicate value.
You can also use record hooks with external data. The key thing to remember when using record hooks with an external datasource is that these do not run as one bulk operation like field hooks, but rather run on each individual record. So while it is absolutely possible to use record hooks on initialization of the importer, it is best practice to use field hooks on initialization for working with external data, and then use record hooks on
change to keep the information updated. In this example, we will add onto the fields hooks example from above, but we will add the record hooks validation to the change event to make sure that when a record is updated, we didn't update an email address to one already listed in the system.
Notice that with the record hooks we added here, we are verifying that the
mode (which can be thought of as the record hook event) is
change before proceeding to perform a server call. Filtering out the
init hooks here prevents the code from running each record individually on initialization which could cause as many server calls as there are records in the file.
If you want to learn more about Data Hooks and their best practices, you can check out our guide on Data Hooks best practices.