deleting duplicate entries in a .txt document

RobbieC

Active Member
Joined
Dec 14, 2016
Messages
376
Office Version
  1. 2010
Platform
  1. Windows
Hi there, I have a .txt document which my spreadsheet writes a string of values to.

Each line is 5 strings long seperated by a comma (,) eg:

Birmingham, 124, aaaaaa, bbbbbb, cccccc
Birmingham, 125, aaaaaa, bbbbbb, cccccc
London, 176, aaaaaa, bbbbbb, cccccc
Newcastle, 132, aaaaaa, bbbbbb, cccccc
London, 177, aaaaaa, bbbbbb, cccccc

What I'm looking to do is remove duplicates based on the first column (city name). In this example Birmingham & London. The entries which are higher up in the list are the older ones which need to be deleted (Birmingham 124 & London 176)

This code will remove duplicates from the text file and create a new (temp output) file based on column 1 (j=0) but it will ONLY filter out column 1

Code:
Sub removeMapDuplicates()
    Const ForReading = 1, ForWriting = 2
    Dim i, j
    Set objFSO = CreateObject("Scripting.FileSystemObject")
    Set objInputFile = objFSO.OpenTextFile(ThisWorkbook.Path & "file_to_remove_duplicates.txt", ForReading)
    Set objOutputFile = objFSO.OpenTextFile(ThisWorkbook.Path & "output.txt", ForWriting, True)
    Set objDict = CreateObject("Scripting.Dictionary")
    j = 0
    'On Error Resume Next
    While Not objInputFile.AtEndOfStream
        arrinputRecord = Split(objInputFile.Readline, ",")
        strFirstField = arrinputRecord(0)
        If objDict.Exists(strFirstField) Then
            j = j + 1
        Else
            objDict.Add strFirstField, strFirstField
        End If
    Wend
    colKeys = objDict.Keys
    
    For Each strKey In colKeys
        objOutputFile.WriteLine objDict.Item(strKey)
    Next
    
    objInputFile.Close
    objOutputFile.Close
End Sub

Is there a way (without importing the entire file into the workbook) to remove duplicates from my original .txt file?

If you can point me in the right direction, I'd be very grateful

Thanks

Rob
 

Excel Facts

Quick Sum
Select a range of cells. The total appears in bottom right of Excel screen. Right-click total to add Max, Min, Count, Average.

Forum statistics

Threads
1,223,905
Messages
6,175,297
Members
452,633
Latest member
DougMo

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top