Slow IF statements when converting a single cell to a column value for multiple files

dispelthemyth

Well-known Member
Joined
Mar 16, 2006
Messages
664
Office Version
  1. 365
Platform
  1. Windows
I am importing multiple small files (8 files ~ 800 rows total) and have 3 values

Values i want are in row 1, 2 and 4 of the Trial Balance column in all 8 files but obviously when combined will be in rows 1, 2, 4 then maybe 104, 105 and 107
1648118260974.png



I am currently doing 3 separate If statements via adding a column

As the 2nd one always starts "As at" i use that to find all 3 values i need

Power Query:
= Table.AddColumn(#"Added Index", "Division", each if Text.Start(#"Added Index" [Trial Balance] {[Index]}, 5) = "As at" then #"Added Index" [Trial Balance] {[Index] -1} else null)
Power Query:
= Table.AddColumn(#"Added Division Temp", "Date", each if Text.Start(#"Added Division Temp" [Trial Balance] {[Index]}, 5) = "As at" then #"Added Division Temp" [Trial Balance] {[Index]} else null)
Power Query:
= Table.AddColumn(#"Added Date Temp", "Sub Division", each if #"Added Date Temp" [Date]{[Index]} = null then null else #"Added Date Temp" [Trial Balance]{[Index] + 2})

These If statements (any of them) make the query very slow to run, like 2+ minutes, which becomes unusable one i get 12 months of data (48 files)

Are there any better techniques for converting specific cells to a column label for multiple files?
Are If statements inherently slow in PQ or is mine just poorly written?



Power Query:
let
    Source = #"TBs Import location",
    #"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true),
    #"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File", each #"Transform File"([Content])),
    Renamed_Columns = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
    #"Removed Other Columns1" = Table.SelectColumns(Renamed_Columns, {"Source.Name", "Transform File"}),
    #"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File", Table.ColumnNames(#"Transform File"(#"Sample File"))),
    #"Changed Type" = Table.TransformColumnTypes(#"Expanded Table Column1",{{"Source.Name", type text}, {"Trial Balance", type text}, {"Column2", type text}, {"Column3", type any}, {"Column4", type any}}),
    #"Replaced Value3" = Table.ReplaceValue(#"Changed Type","Company Name","Test Entity Name",Replacer.ReplaceText,{"Trial Balance"}),
    #"Added Index" = Table.AddIndexColumn(#"Replaced Value3", "Index", 0, 1, Int64.Type),
    #"Added Division Temp" = Table.AddColumn(#"Added Index", "Division", each if Text.Start(#"Added Index" [Trial Balance] {[Index]}, 5) = "As at" then #"Added Index" [Trial Balance] {[Index] -1} else null),
    #"Added Date Temp" = Table.AddColumn(#"Added Division Temp", "Date", each if Text.Start(#"Added Division Temp" [Trial Balance] {[Index]}, 5) = "As at" then #"Added Division Temp" [Trial Balance] {[Index]} else null),
    #"Added Custom" = Table.AddColumn(#"Added Date Temp", "Sub Division", each if #"Added Date Temp" [Date]{[Index]} = null then null else #"Added Date Temp" [Trial Balance]{[Index] + 2}),
    #"Filled Down" = Table.FillDown(#"Added Custom",{"Division", "Date", "Sub Division"}),
    #"Removed Columns" = Table.RemoveColumns(#"Filled Down",{"Source.Name"})
in
    #"Removed Columns"
 

Attachments

  • 1648117638322.png
    1648117638322.png
    18.1 KB · Views: 11
Last edited:

Excel Facts

What is the last column in Excel?
Excel columns run from A to Z, AA to AZ, AAA to XFD. The last column is XFD.
EDIT: TIL you can get a query called "Transform Sample File" created which you can amend to change all imported files before it hits your main query
 
Upvote 0
Solution

Forum statistics

Threads
1,224,823
Messages
6,181,178
Members
453,021
Latest member
Justyna P

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top