Power BI .csv and/or .xls* files and Incremental Refresh

[iosiflupis]

Hello all.

I am currently working with Power BI Premium User with Version: 2.126.927.0 64-bit (February 2024). I am working with a folder that has 27 individual files. Each file is for a 3-month (quarter) period of a year. Each file is approximately 1.5 MB in size. Using my current PBI file it takes about between 30 and 60 minutes for the refresh to complete.

What I am trying to do is to be able to use incremental refresh. I have done much research and most literature says that it cannot be done. However, I came across a youtube video that showed the possibility of how to append a date-time column from the file metadata into the file data itself. Unfortunately, I did NOT save the video to my playlist or watch later list.

So my question for the world is:
1. Can Incremental Refresh be done with these flat folders?
2. If yes, then could I be shown where to find instructions/directions?

Thank you in advance

Joseph

 

[Matt Allington]

It shouldn’t take that long. Are the files on a server? If so, that can be an issue. Syncing to a PC with OneDrive can fix it.

Regarding “can you use incremental refresh”, I believe so but I’ve never done it. Miguel Escobar wrote an article. Incremental refresh for files in a Folder or SharePoint - Power BI — The Power User

Alternatively you can refresh, then write the finished table to a CSV file using DAX Studio. That becomes the history file to replace the 27 files (which you should archive). Actually, probably best to keep the latest file aside and don’t include it in the archive so you have at least 1 for your file combine. Load this as a flat table and append after the file combine is done. So first quarter you have 1 file, second quarter you have 2, etc. just wait until it is slow again and then recreate the history file again.