Bridge Job Run Errors Randomly when Processing Smartsheet w/ 1500+ row changes

cabbsman

I have an issue where my Bridge job flow has an error when processing changes on my workflow.

The Source/trigger Smartsheet is created from a Datashuttle job that converts an uploaded Excel file to a Smartsheet file. This is the trigger file for this workflow. After the upload, there will be 1500 + changes and new rows added at one time to the trigger sheet. The sheet changes are determined by examining the trigger sheet's row against the target sheet's row after looking up and matching a key common to both Smartsheets. If no key match is found, the row from the trigger Smartsheet is added to the target Smartsheet. If there is a match a change is applied.

That means there will be allot of job runs on the workflow as each row is processed as a job run.

I have two Smartsheet triggers set up, one if the source Smartsheet has row changes and one if rows are added to the source Smartsheet. Both will trigger the same job flow.

The Error

An error example: job run 1 will fail at the beginning step of my workflow where I do a search for a string on the target sheet while another job run 2 will error at a step where the target sheet is updated towards the end of the workflow. In the Run Log, when selecting the different jobs, I find that the job errors randomly in different steps of my workflow. All job stops have the same error message:

• "workflow execution failed: Plugin Error INVALID_DATA : failed to execute extension module : Failed to parse module params : An unexpected error has occurred. Please contact the Support team at https://help.smartsheet.com/contact for assistance."

Here's an example of one job error:

Here's an example of another job error

I talked to Smartsheet Support and they explained that the limit to process job runs is 300 per minute. So, this may be the problem.

Has anyone experienced this problem before and is there a way to get around it?

cabbsman

Thanks for your patience on this. After discussing this with our senior resources, the add row/update row modules update one row at a time, so when you run the Bridge workflow and there are 1000 rows to be updated, Bridge will run 1000 times in this scenario.

To work around this, you can set up a Call API tool in your Bridge workflow instead to update these rows in a single run: Configure the Call API tool

However, this method will require you to develop a custom script in the body section of the API call in order to do this. 

Alternatively, you can use a parent/child workflow, where the parent will need to run on schedule and you will need to use a 'Modified date' system column in your sheet: Column Type Reference

In this workflow, you will need to create a filter in the Search Sheet module where the Modified date is greater than the Retrieve Data utility. 

I've attached an example of this scenario. Keep in mind that this scenario isn't guaranteed to work every time due to the large amount of data being updated. Also, in this example, it is using the same sheet to trigger when a change is made, so you will need to fit it within your workflow. I hope that helps! 

(note: the attached file referenced is a screen recording and Smartsheet will not allow that to be uploaded here.)

cabbsman

I came across a MATH:RANDOM NUMBER module and decided to give it a try where I would randomly select numbers (60 second intervals) and pass that on to the DELAY module. Where the DELAY module will wait on processing the job based upon the RANDOM number generated. It mostly worked. Out of approximately 1000 changes, I had about 20 STOPS, all others were completed. To mitigate, I added about 30 more numbers to be picked by the MATH:RANDOM NUMBER module. This should hopefully resolve the remaining STOPS on my next run.

Genevieve P.

Hi @cabbsman

I have seen a similar error when asking Bridge to update or add more than 500+ rows in a Smartsheet sheet. Based on the error, this looks to be a Smartsheet API error and is not necessarily linked to Bridge configuration.

It sounds like you already have a case open with Smartsheet Support which is what I would have suggested - they'll be able to troubleshoot this in more detail and dive into the API as well as the Bridge modules to understand if this is something that can be changed or not.

Thank you!

Genevieve

cabbsman

Thanks for your patience on this. After discussing this with our senior resources, the add row/update row modules update one row at a time, so when you run the Bridge workflow and there are 1000 rows to be updated, Bridge will run 1000 times in this scenario.

To work around this, you can set up a Call API tool in your Bridge workflow instead to update these rows in a single run: Configure the Call API tool

However, this method will require you to develop a custom script in the body section of the API call in order to do this. 

Alternatively, you can use a parent/child workflow, where the parent will need to run on schedule and you will need to use a 'Modified date' system column in your sheet: Column Type Reference

In this workflow, you will need to create a filter in the Search Sheet module where the Modified date is greater than the Retrieve Data utility. 

I've attached an example of this scenario. Keep in mind that this scenario isn't guaranteed to work every time due to the large amount of data being updated. Also, in this example, it is using the same sheet to trigger when a change is made, so you will need to fit it within your workflow. I hope that helps! 

(note: the attached file referenced is a screen recording and Smartsheet will not allow that to be uploaded here.)

cabbsman

I came across a MATH:RANDOM NUMBER module and decided to give it a try where I would randomly select numbers (60 second intervals) and pass that on to the DELAY module. Where the DELAY module will wait on processing the job based upon the RANDOM number generated. It mostly worked. Out of approximately 1000 changes, I had about 20 STOPS, all others were completed. To mitigate, I added about 30 more numbers to be picked by the MATH:RANDOM NUMBER module. This should hopefully resolve the remaining STOPS on my next run.