Data Shuttle Offloading Wrong Data?

calebgardner · 08/25/23

Hello,

I have a main data source which collects project information from different vendors.

Figure 1. Example of main sheet row

To work with outside vendors, we have public facing sheets that we offload data to using data shuttle.

Figure 2. Data Shuttle Offload Criteria

This data is then uploaded to the company's sheet where they can mark the task complete when they have finished their task.

Figure 3. Company Sheet

The problem I'm having is that projects that do not meet the criteria found in Figure 2. Are being found in the company's sheet, often as a repeat. If we look between Figure 1 and 3, these two represent the same project, however because the main sheet is checked in the IP Complete column, it should be filtered out of the data shuttle.

Has anyone else encountered this or know a work around?

Thanks,

Caleb G

Genevieve P. · 08/29/23

Hi @calebgardner

For your checkbox columns (IP Complete and Market Complete), try changing the criteria to be either Is Blank (un-checked - false) or Is Not Blank (checked - true).

Let me know if this helped filter correctly for you!

Cheers,

Genevieve

cjhagerman · 09/12/23

I'm having the same issue. My filter is a check box - is not blank. The offload workflow doesn't remove items that have been unchecked from the offload file; it just continues to build on the file.

Genevieve P. · 09/12/23

Hi @cjhagerman

Data Shuttle will move information across for you, but it won't update the destination if a previously-added row no longer meets a specific criteria.

If I've misunderstood what you're looking for, could you explain your current situation a little further, perhaps with screen captures of how your Workflow is set up?

Cheers,

Genevieve

cjhagerman · 09/12/23

https://community.smartsheet.com/discussion/comment/395014#Comment_395014

Hi Genevieve,

My offload workflow exports the checkbox-filtered data to a CSV file. A separate workflow then pulls the CSV file to update dropdown lists. When a project closes, the checkbox changes, which should remove the project from the dropdown lists. However, the checkbox change isn't reflected in the offload workflow CSV. Do I need to delete the previous CSV every time there's a checkbox change in order for the offload CSV to properly reflect the data? I assumed the offload would create a new CSV with the updated checkbox-filtered data.

Genevieve P. · 09/12/23

Hi @cjhagerman

Thanks for clarifying! You are correct, the offloaded workflow will create a new file each time to replace the previous CSV.

Can you post a screen capture of your filter criteria and a screen capture of the source sheet? (But please block out any sensitive information.)

Thanks,

Genevieve

cjhagerman · 09/14/23

https://community.smartsheet.com/discussion/comment/395049#Comment_395049

https://community.smartsheet.com/discussion/comment/395433#Comment_395433

Screenshot 2023-09-14 at 4.08.55 PM.png

Screenshot 2023-09-14 at 4.09.26 PM.png

cjhagerman · 10/04/23

https://community.smartsheet.com/discussion/comment/395433#Comment_395433

@Genevieve P. , were you able to review the screenshots I attached? This is what I've found:

The system will only remove a row that no longer meets the filter requirements if another row now meets the requirements. If another row now meets the requirements, the system will scan all rows and add and/or delete as appropriate. However, if a row simply changes and no longer meets the filter requirements on its own, it doesn't trigger the system to update the file. There has to be a new "yes" event for the system to recognize and update new "no" events. The system also counts the yes and no event changes as "rows added" instead of 1 row added and 1 row deleted - see image below.

I don't see a fix.

Genevieve P. · 10/10/23

Hey @cjhagerman

My apologies! I didn't receive a notification for this (which I'm looking into so it doesn't happen again).

Thank you for following-up. The first part of what you've described is expected: the filter part of Data Shuttle is a filter that is only applied during the workflow process.

A filter does not trigger the workflow to run, so you'll need to have configured your workflow to run either on a schedule or by attachment in order for the rows to be reviewed and the CSV created and offloaded.

If you mean that when the Offload workflow runs, the UN-Checked boxes aren't being filtered out, could we try the following filter instead to see if this makes a difference:

equals = true (all lowercase)

I tested this and it exported my 12 checked rows (+ 1 header row) correctly for the first export (the bottom run in the image below).

I then only un-checked boxes (without checking any) to ensure that is still filters out rows, and it did in my tests (3 rows + 1 header row added to the CSV, 111 rows filtered).

In regards to what shows up in the Log, you won't see rows deleted since it's creating a new CSV file.

Instead, all of the rows that are no longer going to be exported should show in the "Rows Filtered" section, indicating how many rows remain in the sheet versus how many rows are "added" to your CSV export file. Does that make sense?

You may see +1 extra row in your Log if you have a header row in your export. Let me know if that helps clarify the logs you're seeing, and let me know if "true" worked for you as a filter!

Cheers,

Genevieve

cjhagerman · 10/11/23

https://community.smartsheet.com/discussion/comment/399260#Comment_399260

That seems to have been the trick. I changed all my filter columns to check boxes vs text - yes/no - boxes and changed the filter to equals true. That worked, thanks!

Data Shuttle Offloading Wrong Data?

Figure 1. Example of main sheet row

Figure 2. Data Shuttle Offload Criteria

Figure 3. Company Sheet

Answers

Categories