Data Shuttle to DataTable: Replace DataTable

Keith Birchfield

I want to create an automated system that maintains my DataTable as an exact replica of a source file. Data Shuttle does not quite do this for a DataTable workflow. In the Data Shuttle workflow for DataTables you can only Add and/or Update data. This can result in rows of data remaining in the DataTable when they are no longer in the source file thereby creating discrepancies between the source file and the DataTable.

In the Data Shuttle workflow for sheets, there are options to Replace the entire data set and to Delete rows that no longer match the filter criteria. I'd like to have the option to Replace the entire DataTable using Data Shuttle, which would ensure that the DataTable always contains the same data as the source file.

JaymeBoyd

This is something I am running into as well and would be very helpful as an update.

Jacob Minnich

I have the same problem. I use Data Shuttle and sheets to replicate source files, but several source files I want to use are too large to make into a sheet (20,000+ rows). If I can't fully replace the Data Table with the updated source file, it's practically useless to me.

Neil Egsgard

Yes, please!

We need the ability to automatically remove bad data from DataTable.

My workaround is to put a "Record date" field on each record in the source. I use the DataTable connection filter to only include records with a "Record date" field = to the update frequency of the data.

For example:

- if the data refreshes daily, the DataTable connection filters records with a "Record date" = today or yesterday.

- if data refreshes every 3 days, the DataTable connection filters records with a "Record date" = 3 days ago or 4 days ago

Including one additional day reduces data accuracy but ensures date mismatches between Smartsheet and the data source don't remove data.