Help with database cross-referencing (index-match on ~500.000 row database)

Ha23

New Member
Joined
Dec 14, 2022
Messages
1
Office Version
  1. 365
Platform
  1. Windows
Hello,

First of all thanks for all the help you provide. I've been finding solutions here for years to become more productive with excel.

I'm currently facing a task where I have to complete some inventory data (adding some classification to item e.g. 'non-alcoholic beverages' to coke, pepsi, water, etc.) before starting my analysis. I receive the data in several excels (5-6 workbooks), totaling around 200.000 rows of info and I have to cross-reference the items with a ~500.000 database to obtain the classification (placed on another excel workbook).

I'm currently using Index/Match formulas limited to cells with data in both workbooks (exact match, data cannot be ordered as far as I understand). Once the index/match finishes on each workbook, I paste as values and copy onto a new workbook (which will become the aggregated and processed data, starting point for the analysis. I perform the match on a "description" column which contains ~50 characters.

The issue is that this process is really slow, crashes excel often and it can take me days to aggregate the data. Each of the excels has different structure and it can change monthly so programming a solution would not be ideal as it would need frecuent adaptation.

Any ideas on how to speed up the process?

Thanks!
 

Excel Facts

Whats the difference between CONCAT and CONCATENATE?
The newer CONCAT function can reference a range of cells. =CONCATENATE(A1,A2,A3,A4,A5) becomes =CONCAT(A1:A5)

Forum statistics

Threads
1,223,237
Messages
6,170,928
Members
452,366
Latest member
TePunaBloke

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top