schwiggiddy
New Member
- Joined
- Sep 5, 2017
- Messages
- 31
I am attempting to gather data from the web and append it together; however, the data are distributed on different pages. Fortunately, the pages' addresses are systemically defined and vary with just a single field.
I have created an Excel file that contains the variations of the field, since more can be added in the future (or some removed), and have that imported into the model.
For each field's data from the web table, I want to do transformations unique to that data set prior to appending it to the output of the rest. I have a query that fetches the data for one and performs the transformations, but am struggling setting it up to fetch the rest automatically.
I've done this extensively with combining files from within folders locally and online, but not from a web source and I'm stuck trying to reverse engineer the helper queries PQ/PBI now set up automatically.
Options explored and questions remain:
If I set up the field query as a list and use it as a parameter, will PBI online load all options for rapid recall if I have a visualization that toggles through them or will it have to refresh the data with each change?
If the field query is developed to insert the field value into the web address and have a column contain the web table data, I know I can convert that to the table and expand it, but this would run the index based transformations across different field values, which would only cause confusion; trying to avoid this method.
Ultimately, I know I need a function, but with it not being binary data, I'm a little stumped. Any help would be greatly appreciated.
I have created an Excel file that contains the variations of the field, since more can be added in the future (or some removed), and have that imported into the model.
For each field's data from the web table, I want to do transformations unique to that data set prior to appending it to the output of the rest. I have a query that fetches the data for one and performs the transformations, but am struggling setting it up to fetch the rest automatically.
I've done this extensively with combining files from within folders locally and online, but not from a web source and I'm stuck trying to reverse engineer the helper queries PQ/PBI now set up automatically.
Options explored and questions remain:
If I set up the field query as a list and use it as a parameter, will PBI online load all options for rapid recall if I have a visualization that toggles through them or will it have to refresh the data with each change?
If the field query is developed to insert the field value into the web address and have a column contain the web table data, I know I can convert that to the table and expand it, but this would run the index based transformations across different field values, which would only cause confusion; trying to avoid this method.
Ultimately, I know I need a function, but with it not being binary data, I'm a little stumped. Any help would be greatly appreciated.
