Data Shuttle Execution Duration

Sheryl P

I'm looking for information on how long your Data Shuttles run. I have one offload, and one upload (first ones - very small - under 50 rows added no deleted, skipped, filtered, no drop downs updated) Looking at a month's worth of logs I see a noticeable increase in execution duration for no apparent reason. What have you found?

Jake Gustafson

I've got the same question, but about a larger dataset. I'm on my 2nd day of processing and things will only get larger. I'm just about at 15 minutes now, so looking towards expectation management now. I'm updating and adding rows. Had about 800 records yesterday and now I should be around 1600 today.

Sheryl P

Hi @Jake Gustafson - Thanks for adding your experience to this thread. I did open a support ticket in Dec 2023 - below is the feedback I received. I tend to agree that it is related to the workflow jobs in SmartSheet's queue. Post again and let me know if your run times grow as the sheet grows, or if the extended run times are inconsistent. SS Support also referenced the complexity (or number of formulas/workflows/etc) of the sheets associated the the Data Shuttle workflow.

From Smart Sheet Support:

I looked over the attached screenshot, and in reviewing our documentation Data Shuttle workflows can at times fall back onto a 15 minute interval if the system hits the limit for workflows run. It is likely that your workflow was falling into this situation and therefore caused the workflow at times to run an extended time. It's an intermittent issue, but the workflow will still run as intended. There isn't anything in specific that needs to be done as this is completely system related.

Sometimes the system gets clogged up whilst attempting to run all the workflows (all the existing ones in DataShuttle from all of our customers). When this happens, all workflows will experience longer update times. This usually resolves automatically after the queue is emptied.

Jake Gustafson

Thanks @Sheryl P. That was helpful context. For the most part, it's been processing pretty quick. But as is normal when you're sitting waiting for it to execute, that's when it takes the longest. When I let it do its thing on its normal schedule, then I haven't noticed any delays.

CaraBart28

Does anyone know how long it takes for the destination sheet to be updated after data shuttle is run? I have a doc that is transferring data from Box into a master sheet in SmartSheet but it is taking forever for the master sheet to update.

Sheryl P

@CaraBart28 Is the job running long once started, or is it taking long to start after the Box data is available?