dispelthemyth
Well-known Member
- Joined
- Mar 16, 2006
- Messages
- 664
- Office Version
- 365
- Platform
- Windows
I am importing multiple small files (8 files ~ 800 rows total) and have 3 values
Values i want are in row 1, 2 and 4 of the Trial Balance column in all 8 files but obviously when combined will be in rows 1, 2, 4 then maybe 104, 105 and 107
I am currently doing 3 separate If statements via adding a column
As the 2nd one always starts "As at" i use that to find all 3 values i need
These If statements (any of them) make the query very slow to run, like 2+ minutes, which becomes unusable one i get 12 months of data (48 files)
Are there any better techniques for converting specific cells to a column label for multiple files?
Are If statements inherently slow in PQ or is mine just poorly written?
Values i want are in row 1, 2 and 4 of the Trial Balance column in all 8 files but obviously when combined will be in rows 1, 2, 4 then maybe 104, 105 and 107
I am currently doing 3 separate If statements via adding a column
As the 2nd one always starts "As at" i use that to find all 3 values i need
Power Query:
= Table.AddColumn(#"Added Index", "Division", each if Text.Start(#"Added Index" [Trial Balance] {[Index]}, 5) = "As at" then #"Added Index" [Trial Balance] {[Index] -1} else null)
Power Query:
= Table.AddColumn(#"Added Division Temp", "Date", each if Text.Start(#"Added Division Temp" [Trial Balance] {[Index]}, 5) = "As at" then #"Added Division Temp" [Trial Balance] {[Index]} else null)
Power Query:
= Table.AddColumn(#"Added Date Temp", "Sub Division", each if #"Added Date Temp" [Date]{[Index]} = null then null else #"Added Date Temp" [Trial Balance]{[Index] + 2})
These If statements (any of them) make the query very slow to run, like 2+ minutes, which becomes unusable one i get 12 months of data (48 files)
Are there any better techniques for converting specific cells to a column label for multiple files?
Are If statements inherently slow in PQ or is mine just poorly written?
Power Query:
let
Source = #"TBs Import location",
#"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File", each #"Transform File"([Content])),
Renamed_Columns = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
#"Removed Other Columns1" = Table.SelectColumns(Renamed_Columns, {"Source.Name", "Transform File"}),
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File", Table.ColumnNames(#"Transform File"(#"Sample File"))),
#"Changed Type" = Table.TransformColumnTypes(#"Expanded Table Column1",{{"Source.Name", type text}, {"Trial Balance", type text}, {"Column2", type text}, {"Column3", type any}, {"Column4", type any}}),
#"Replaced Value3" = Table.ReplaceValue(#"Changed Type","Company Name","Test Entity Name",Replacer.ReplaceText,{"Trial Balance"}),
#"Added Index" = Table.AddIndexColumn(#"Replaced Value3", "Index", 0, 1, Int64.Type),
#"Added Division Temp" = Table.AddColumn(#"Added Index", "Division", each if Text.Start(#"Added Index" [Trial Balance] {[Index]}, 5) = "As at" then #"Added Index" [Trial Balance] {[Index] -1} else null),
#"Added Date Temp" = Table.AddColumn(#"Added Division Temp", "Date", each if Text.Start(#"Added Division Temp" [Trial Balance] {[Index]}, 5) = "As at" then #"Added Division Temp" [Trial Balance] {[Index]} else null),
#"Added Custom" = Table.AddColumn(#"Added Date Temp", "Sub Division", each if #"Added Date Temp" [Date]{[Index]} = null then null else #"Added Date Temp" [Trial Balance]{[Index] + 2}),
#"Filled Down" = Table.FillDown(#"Added Custom",{"Division", "Date", "Sub Division"}),
#"Removed Columns" = Table.RemoveColumns(#"Filled Down",{"Source.Name"})
in
#"Removed Columns"
Attachments
Last edited: