Data Shuttle Data Mesh Schedule Run Times
I have issues with my automated systems failing due to timing both with Data Mesh and Data Shuttle.
It is necessary that six Data Shuttle workflows run the same time otherwise they occasionally fail due to them fighting over the same certificate when obtaining information from my cloud account. This causes a catastrophic cascade of failures in other workflows that occur later in the day. If I could space each Data Shuttle workflow out in smaller increments of 10-15 min they would be far apart enough not to interfere with each other. The selection currently available for timing does not allow updating in a timely manner.
Data Shuttle will also fight over updating the same table as Data Mesh. It is because they are both trying to access the same table at the same time and occasionally they fight of the authorization certificate and it fails. Like Data Shuttle, Data Mesh has very few options for timing also. None of Data Mesh times work to update my tables in a timely manner that works for the operations people. The options for time to run interfere with Data Shuttle. The system often crashes because Data Mesh and Data Shuttle access the same sheet at the same time. Again, if Data Mesh had more timing options I would not have a issue.
To conclude, both Data Mesh and Data Shuttle need to have more options for timing. I know this is a issue experience by many other companies. It seems that allowing more selection in timing is a easy fix.
Answers
-
I'd like to see linking data shuttle/data mesh workflow so you can set up a cascade. Fire off one workflow and when it finishes, the next one starts and so on. Then the workflows don't step on each other and we don't have to worry about one starting later than expected.
/marc
-
Marc4, I had the same thought however just being able to customize the time anything runs to a more granular level would make so many issues disappear.
-
Yes ... this is definitely needed.
I've noticed that sometimes my Shuttle workflows complete in seconds then at other times it can be many minutes for the same workflow. I have 2 banks of 3 workflows, with the last workflow in each bank feeding into the same target.
Ideally I'd like to schedule each bank to begin 10 minutes apart and to have one workflow trigger the next one.
So the two features would be:
- More granularity (10 minute windows would be good)
- Chaining of workflows
That would be amazing 😊
-
Hi Smartsheet team!
Any developments here? Would be so useful ... pretty please 😄
