Data Shuttle loses source when file is updated

MikeS

Hi all,

I've been experiencing an issue with Data Shuttle in which it loses its source connection when the source file (csv) is updated each morning. I set everything up and the source file is highlighted blue:

The next morning, I'll add the updated file to the folder in which the source is located. It's probably important to note that I need to rename the file to match the source name, as it is sent to me with the date and time it was generated: (i.e., training_report-INCOMPLETES_20240115_06_10_45_PM is changed to training_report-INCOMPLETES). I place the newly renamed file in the folder for Data Shuttle to grab, and the source becomes grayed out. I then have the "edit" the data shuttle flow and relocate the source.

The run log doesn't give me much help as to why it can no longer access the source:

Some additional info: The source is located in a OneDrive for Business folder. It is synced when I move it over to the source folder from my desktop. Also - I initially used power automate to grab this file from my email, rename it and place it in the source initially. When I did it that way, everything worked fine. Unfortunately the file became so big that it couldn't be emailed. Now I need to manually pull the report and move it over to the folder.

Any help would be great! Thanks!

GiGiBiGi

I have the exact same problem with Data Shuttle. The Shuttle is set up with an Excel source file and runs ok. When the source Excel file is replaced in the source location by a file with the same name and exact format the Data Shuttle errors out.

Please provide solution!

MikeS

Anyone able to help or have any suggestions to try?

Razetto

@MikeS I have a similar process that works fine. In my case, I created a specific OneDrive folder that I shared with IT so that they could link it to their program, which is a report generated off our ERP, and it runs every 6 hours replacing all the data in the same file. Replacing a file with another one with the same name won't be recognized by DataShuttle which is your problem. The only time when DataShuttle hasn't uploaded the latest data has been because of the multi factor expiring which happens seldomly.

MikeS

Thank you Razetto. Unfortunately I am unable to get this specific report auto-sent to a folder or my email. I have to pull the report manually, and when I do it downloads with the exact date and time included in the filename.

It seems this is the problem - "Replacing a file with another one with the same name won't be recognized by DataShuttle which is your problem." I assume each file has a specific identifier and that's why? Any workarounds for this?