Excel Fuzzy Lookup Add-in - Exception out of memory

[Ahmedmaher]

I have been using Fuzzy Lookup for a while to match 2 lists with no problems.

Just recently I have started to experience an error that return the following exception - Exception of type 'System.OutOfMemoryException' was thrown. (mscorlib)

I am not sure if it's caused bu the volume of the lists I am trying to match - List 1 = 58,000 rows & list 2 = 8,000 rows

I am sure that prior this starting, I had run a number of fuzzy lookups on this data with no issues so I am curious as to why it has suddenly started.

I have tried to reboot, uninstall & re-install, running on 2007 (currently using 2013) but after about running through about 3,000 rows of the 58k dataset, this exception gets returned.

I can get by by taking the data not checked and running again but it's a pain and it's billable time which I can't bill as it's not my clients fault (

Any help with resolving this would be appreciated.

Details from the exception report -

Exception of type 'System.OutOfMemoryException' was thrown. (mscorlib)

------------------------------
Program Location:

at System.StubHelpers.ValueClassMarshaler.ConvertToNative(IntPtr dst, IntPtr src, IntPtr pMT, CleanupWorkList& pCleanupWorkList)
at Microsoft.DataIntegration.FuzzyMatching.IWordBreaker.BreakText(TEXT_SOURCE& pTextSource, IWordSink pWordSink, IPhraseSink pPhraseSink)
at Microsoft.DataIntegration.FuzzyMatching.FtsRecordTokenizer.Tokenize(TokenizerContext tokenizerContext, IDataRecord record)
at Microsoft.DataIntegration.FuzzyMatching.LookupUpdateContext.TokenizeAndWeigh(ComparisonProviderInfo cpi, RecordContext rc, IDataRecord r, Boolean weigh)
at Microsoft.DataIntegration.FuzzyMatching.LookupUpdateContext.TokenizeAndRuleMatch(IDataRecord record)
at Microsoft.DataIntegration.FuzzyMatching.FuzzyComparer.Compare(ISession _session, IDataRecord leftRecord, IDataRecord rightRecord, ComparisonResult& result)
at Microsoft.DataIntegration.FuzzyMatching.FuzzyLookupBuilder.CustomComparer.Compare(Session session, FuzzyComparer comparer1, ISession comparerSession1, FuzzyComparer comparer2, ISession comparerSession2, IDataRecord leftRecord, IDataRecord rightRecord, Double threshold, ComparisonResult& result)
at Microsoft.DataIntegration.FuzzyMatching.FuzzyLookupBuilder.CustomComparer.Compare(ISession _session, IDataRecord leftRecord, IDataRecord rightRecord, ComparisonResult& result)
at Microsoft.DataIntegration.FuzzyMatching.FuzzyQuery.ProcessRid(Session session, IFuzzyLookupStateManager stateManager, RecordContext leftRecordContext, IDataRecord leftRecord, Int32 rid)
at Microsoft.DataIntegration.FuzzyMatching.FuzzyQuery.ProcessRidList(Session session, IFuzzyLookupStateManager stateManager, RecordContext leftRecordContext, IDataRecord leftRecord, RidList ridList)
at Microsoft.DataIntegration.FuzzyMatching.FuzzyQuery.Process_EarlyTermination(IDataRecord inputRecord, Int32 maxResults, Session session, Int32 numIndexes)
at Microsoft.DataIntegration.FuzzyMatching.FuzzyQuery.Match(ISession _session, IDataRecord inputRecord, Int32 maxResults)
at Microsoft.DataIntegration.FuzzyMatching.FuzzyQuery.Match(IDataRecord inputRecord)
at FuzzyLookupAddInForExcel.FuzzyJoinControl.FuzzyJoin(Worksheet worksheet, ProgressDialog progressDialog)

 

[ranman256]

if you are using Excel 32 bit, on a OS 64 PC, you WILL get outta memory errors. Sorry.

 

[Ahmedmaher]

if you are using Excel 32 bit, on a OS 64 PC, you WILL get outta memory errors. Sorry.

Thanks a lot for your post.
I agree with you, but both my Windows and Office 2013 version are both 64 bit.