Speed up VBA code with VLOOKUP.

farmerscott

Well-known Member
Joined
Jan 26, 2013
Messages
824
Office Version
  1. 365
Platform
  1. Windows
Hi Everybody,

I have the following code that does a vlookup two ways. The first puts the formula into the worksheet and it then calculates it and the 2nd way VBA calculates it and then puts it into the worksheet (...or thats the way I understand it).

The first way of doing it is rather quick (covering 142K of rows) while the 2nd is very slow.

I would appreciate some suggestions to speed up the 2nd way.

Code:
Sub Get_and_organise_data()
Dim x As Long
lr = Worksheets("Sheet1").Cells(Rows.Count, "F").End(xlUp).Row

'1st way....
Range("G1:G" & lr).FormulaR1C1 = "=VLOOKUP(RC[-1],R1C1:R1762C2,2,False)"
 

'2nd way.......
For x = 1 To lr
On Error Resume Next
Value = Cells(x, 6).Value
Rng = Range("A1:C1762")
Answer = Application.WorksheetFunction.VLookup(Value, Rng, 3, False)
Cells(x, 8).Value = Answer
Next x

End Sub

Appreciate the help.

FarmerScott
 

Excel Facts

Using Function Arguments with nested formulas
If writing INDEX in Func. Arguments, type MATCH(. Use the mouse to click inside MATCH in the formula bar. Dialog switches to MATCH.
Hi FarmerScott, Out of curiosity, is there any reason you don't want to just use the 1st method for both lookups?

The 2nd code is slow because it's doing a lot of separate writes to the worksheet. Writing to the worksheet is a relatively slow process, so faster approaches tend to minimize the number of write operations. If you modify your code to store the results in an array as each is calculated, you can transfer the values from the array to the worksheet in one write operation.

There's other things you can do like temporarily turning off some Application settings; however those become relatively insignificant if you modify your code to do only one write.

As an aside, it's a best practice to declare the data types of all your variables and enforce that practice by placing Option Explicit at the top of your code module.

In your code example, since Rng is not declared, VBA uses a Variant data type for this variable.
If it were a Range object you would need to use a Set statement like..
Code:
Set Rng = Range("A1:C1762")

For the purpose of your procedure using a Variant is better than using a Range object; however naming the variable Rng can cause confusion.
 
Upvote 0
Hi JS411,

At this stage I am just learning what difference in speed there is between the approaches. At the end of the day the first approach is fine but I always thought that getting VBA to calculate the formula was slightly faster.

How would you modify the code to use an array?

thanks

FarmerScott
 
Upvote 0
Scott, Some code examples are posted below that you can compare.

Since lookup is such a frequently used function I did some time tests to compare the relative speed of a few methods of doing a lookup of the scale you describe.

Most results were as expected, but there were a couple of surprises.

Here's a comparison of the times:
Excel Workbook
BCDEF
1Processing time in seconds
2Test #Method Description10K lookups on 2K record table142K lookups on 2K record table2K lookups on 142K record table
3Test1R1C1 Formula in Range 0.3 4.4 4.4
4Test2Loop VBA App.Vlookup() slow read-slow write 28.0--
5Test3Loop VBA App.Vlookup() fast read-slow write 17.6--
6Test4Loop VBA App.Vlookup() fast read-fast write 17.1--
7Test5Scripting Dictionary 0.04 0.4 3.3
Sheet



Below is the code used for this time testing.

For all tests, several Application settings that affect performance like ScreenUpdating were turned off prior to timing the Procedure.

Code:
Sub Test1() '~1st way from OP....
'enters R1C1 formula into results range then
'  converts formulas to values

Dim x As Long, lr As Long

lr = Worksheets("Sheet1").Cells(Rows.Count, "F").End(xlUp).Row

With Range("G2:G" & lr)
   .FormulaR1C1 = "=VLOOKUP(RC[-1],R2C1:R1762C3,3,False)"
   .Value = .Value
End With

End Sub

Sub Test2() '~2nd way from OP....
'--Reads lookup values from range then
'  writes results of individual vlookups
'  directly to cells

Dim x As Long, lr As Long
Dim sValue As String
Dim vLookupTable As Variant
Dim vAnswer As Variant

lr = Worksheets("Sheet1").Cells(Rows.Count, "F").End(xlUp).Row

On Error Resume Next
For x = 2 To lr
   sValue = Cells(x, "F").Value
   vLookupTable = Range("A2:C1762")
   vAnswer = Application.VLookup(sValue, vLookupTable, 3, False)
   Cells(x, "G").Value = vAnswer
Next x
On Error GoTo 0

End Sub

Sub Test3()
'--Reads lookup values into array then
'  writes results of individual vlookups
'  directly to cells

Dim i As Long
Dim sValue As String
Dim vLookupValues As Variant
Dim vLookupTable As Variant
   
With Sheets("Sheet1")
   vLookupTable = .Range("A2:C1762").Value
   vLookupValues = .Range("F2:F" & _
      .Cells(.Rows.Count, "F").End(xlUp).Row).Value
   
   For i = LBound(vLookupValues) To UBound(vLookupValues)
       .Cells(i + 1, "G") = Application.VLookup( _
          vLookupValues(i, 1), vLookupTable, 3, False)
   Next i
End With
End Sub


Sub Test4()
'--Reads lookup values into array then
'  stores results of individual vlookups in array
'  then transfers array values to cells in one write

Dim i As Long
Dim sValue As String
Dim vLookupValues As Variant
Dim vLookupTable As Variant
   
With Sheets("Sheet1")
   vLookupTable = .Range("A2:C1762").Value
   vLookupValues = .Range("F2:F" & _
      .Cells(.Rows.Count, "F").End(xlUp).Row).Value
   
   For i = LBound(vLookupValues) To UBound(vLookupValues)
      vLookupValues(i, 1) = Application.VLookup( _
          vLookupValues(i, 1), vLookupTable, 3, False)
   Next i
   .Range("G2").Resize(UBound(vLookupValues), 1) = vLookupValues
End With

End Sub

Sub Test5()
'--Reads lookup table into dictionary
'  Reads lookup values into array then
'  stores results of dictionary lookups in array
'  then transfers array values to cells in one write

'--requires library reference to MS Scripting Runtime
Dim dicLookupTable As Scripting.Dictionary
Dim i As Long
Dim sKey As String
Dim vLookupValues As Variant
Dim vLookupTable As Variant

Set dicLookupTable = New Scripting.Dictionary
dicLookupTable.CompareMode = vbTextCompare

With Sheets("Sheet1")
   vLookupTable = .Range("A2:C1762").Value
   For i = LBound(vLookupTable) To UBound(vLookupTable)
      sKey = vLookupTable(i, 1)
      If Not dicLookupTable.Exists(sKey) Then _
         dicLookupTable(sKey) = vLookupTable(i, 3)
   Next i
   
   vLookupValues = .Range("F2:F" & _
      .Cells(.Rows.Count, "F").End(xlUp).Row).Value
   
   For i = LBound(vLookupValues) To UBound(vLookupValues)
      sKey = vLookupValues(i, 1)

      If dicLookupTable.Exists(sKey) Then
         vLookupValues(i, 1) = dicLookupTable(sKey)
      Else
         vLookupValues(i, 1) = CVErr(xlErrNA)
      End If
   Next i
   
   .Range("G2").Resize(UBound(vLookupValues) - _
      LBound(vLookupValues) + 1, 1) = vLookupValues
End With
End Sub

Most surprising to me was how little relative difference there was in storing the results of the Vlookups in an array versus writing the cells. When isolated, that writing process was 5-10 times quicker by itself, but that difference was somewhat insignificant to the total time required for Test3 and Test4.

It's tempting to jump to the conclusion that the Dictionary method is always the best choice, but speed is just one of the factors to consider. Also note that the relative speed benefits for the Dictionary diminish as the lookup table gets much larger.
 
Last edited:
Upvote 0
hI Jerry,

thanks for the analysis. I am getting much longer times to run my looping-vlookup code (approx 5+ mins).

The code is not something I will run that often but is is nice to know the relative speed of the different approaches.

What are you using to record the speed of the code?

cheers

FarmerScott
 
Upvote 0
Scott,

Sorry that I didn't provide a better explanation of the differences between the tests reported in Columns D,E,F of the results table in Post #4.
Column D represents the processing times for only 10,000 rows of data in Column F (with your ~1761 row lookup range).

I didn't want to run all the tests on your ~142,000 rows scenario; and 10,000 rows gave enough differentiation to understand the relative differences.

If you ran those tests on 142,000 rows your results for Tests2-Test4 should be on the order of 4-7 minutes...which sounds like what you're getting.

Column E reports the results of increasing the number of lookups in Column F to 142K rows with the same ~2K rows lookup table in Columns A:C. I only ran that on the two fastest code examples.

Lastly, Column F shows what happens if the lookup table in Columns A:C were increased to 142K rows and the lookups in Column F reduced to ~2K rows. The key finding from that last test is that as the Lookup Table grows, the speed benefits of the Dictionary become less significant relative to Test1 (R1C1 formulas). There is probably a tipping point at which the Test1 approach becomes faster than using a Dictionary.

The microtimer example code provided by Charles Williams in this article was the basis of the code I used to time each test.

Excel 2010 Performance: Improving Calculation Performance
 
Upvote 0
how much faster is this

moved Rng assignment out of the loop


Code:
Sub Get_and_organise_data()
Dim x As Long
lr = Worksheets("Sheet1").Cells(Rows.Count, "F").End(xlUp).Row


'1st way....
Range("G1:G" & lr).FormulaR1C1 = "=VLOOKUP(RC[-1],R1C1:R1762C2,2,False)"
 


'2nd way.......
Rng = Range("A1:C1762")


For x = 1 To lr
On Error Resume Next
Cells(x, 8).Value = Application.WorksheetFunction.VLookup(Cells(x, 6).Value, Rng, 3, False)
Next x


End Sub
 
Upvote 0
how much faster is this

moved Rng assignment out of the loop
...
...

Hi jsotola,

I totally missed that "2nd way" was assigning the lookup table within the loop!
Using the same code as Test2 but moving the assignment out of the loop, the results are about 17.8 seconds.

Let's call that Test2A.. and add that to the results table and reword descriptions to better explain the differences between 2, 2A, 3 and 4
Excel Workbook
BCDEF
1Processing time in seconds
2Test #Method Description10K lookups on 2K record table142K lookups on 2K record table2K lookups on 142K record table
3Test1R1C1 Formula in Range 0.3 4.4 4.4
4Test2Loop VBA App.Vlookup() (lookup assigned inside loop) 28.0--
5Test2ALoop VBA App.Vlookup() read lookups from cells-write results to cells 17.8--
6Test3Loop VBA App.Vlookup() read lookups to array-write results to cells 17.6--
7Test4Loop VBA App.Vlookup() read lookups to array-write results array to cells 17.1--
8Test5Scripting Dictionary 0.04 0.4 3.3
Sheet


The problem with "2nd way" that you caught, accounts for almost all the difference between Test2 and Test3.

We could almost throw out Test2 in an analysis of "approaches to consider" because unlike some of the other differences that involve pros and cons, having the assignment inside the loop offers no benefits. It is instructive however to see the added cost of this mistake. Thanks jsotola! :)

Here's the code used for Test2A.

Code:
Sub Test2A() '~2nd way from OP....range assignment outside loop
'--Reads lookup values from range then
'  writes results of individual vlookups
'  directly to cells

Dim x As Long, lr As Long
Dim sValue As String
Dim vLookupTable As Variant
Dim vAnswer As Variant

lr = Worksheets("Sheet1").Cells(Rows.Count, "F").End(xlUp).Row

vLookupTable = Range("A2:C1762")

On Error Resume Next
For x = 2 To lr
   sValue = Cells(x, "F").Value
   vAnswer = Application.VLookup(sValue, vLookupTable, 3, False)
   Cells(x, "G").Value = vAnswer
Next x
On Error GoTo 0

End Sub
 
Last edited:
Upvote 0

Forum statistics

Threads
1,223,911
Messages
6,175,337
Members
452,637
Latest member
Ezio2866

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top