Merge Duplicates Rows + Keep Unique Cells intact

XcelGrub

New Member
Joined
Jun 10, 2015
Messages
8
I have been currently working on a problem for the last few days, using a number of resources throughout this fantastic site and others - mixing and merging code to try and achieve (what I think is a unique problem) but have not quite landed successfully on a solution.

I currently take output in the form of a .csv (varying length of rows) however the columns (and their order) remain constant - this .csv will then run through a macro (just been converting the .csv to .xlsm and adding a module in) to produce the desired result. Currently I have 50 columns, of which the a number of rows are duplicates with only the end columns changing - what I would like to do is merge the duplicate rows, but keep the unique cells (and where I am falling flat - is they keep their format and location).

Now that I've successfully confused you all haha - here's a visual aid :).

Current:

Zw5wdD9.jpg


Desired:

wVqmxOv.jpg



Hopefully the above makes sense - everything I've come across/found/tried has resulted in the duplicates being merged fine but the unique values all being merged with commas, spaces and so on - I need the unique values to keep their line format so they can be read correctly. If they are all merged into the same cell and separated with comma or alike, it becomes very hard to make sense.

Some code that may assist that I've tried piecing together:

Merges Rows based on specifics - separator for non-duplicates is vblf - this is quite close but doesn't keep formatting/lines (credit: Tony Dallimore)

Code:
Sub MergeRows()

  ' Merges adjacent rows for which all columns listed in ColMatch are equal
  ' by appending the contents of the other columns from the second row to
  ' the first row and then deleting the second row.


  Dim CheckOK As Boolean
  Dim ColCrnt As Long
  Dim ColLast As Long
  Dim ColMatch() As Variant
  Dim ColMerge() As Variant
  Dim InxMatch As Long
  Dim InxMerge As Long
  Dim RowCrnt As Long
  Dim RowLast As Long
  Dim RowsMatch As Boolean
  Dim TimeStart As Single


  ' Defines the first row to be considered for merging.  This avoids
  ' looking at header rows (not very important) and allows a restart
  ' from row 600 or whatever (might be important).
  Const rowDataFirst As Long = 2


  ' Defines the string to be placed between the value in the first row
  ' and the value from the second row.
  Const Separator As String = vbLf


 ' Speeds up processing
  Application.ScreenUpdating = False


  ' Stops the code from being interrupted by event routines
  Application.EnableEvents = False


  ' Use status bar as a progress indicator
  Application.DisplayStatusBar = True


  ' Record seconds since midnight at start of routine.
  TimeStart = Timer


  ' Defines the columns which must have the same values in two
  ' adjacent rows for the second row to be merged into the
  ' first row.  Column numbers must be in ascending order.
  ColMatch = Array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21)


  ' Defines the columns for which values from the second row
  ' are to be appended to the first row of a matching pair.
  ' Column numbers must be in ascending order.  ColMatch and
  ' ColMerge together must specify every used column.
  ColMerge = Array(22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47)


  ' Replace "Merge" with the name of your worksheet
  With Worksheets("Sheet1")


    ' Find last used column and last used row
    ColLast = .Cells.Find("*", .Range("A1"), xlFormulas, xlWhole, _
                                         xlByColumns, xlPrevious).Column
    RowLast = .Cells.Find("*", .Range("A1"), xlFormulas, xlWhole, _
                                         xlByRows, xlPrevious).Row


    ' Validate column parameters.  Every column must be specified once
    ' in either ColMatch or ColMerge.
    InxMatch = 0        ' 0 = lower bound of array
    InxMerge = 0
    For ColCrnt = 1 To ColLast
      CheckOK = False   ' Set true if check successful
      If InxMatch > UBound(ColMatch) Then
        ' ColMatch array exhausted
      Else
        If ColCrnt = ColMatch(InxMatch) Then
          CheckOK = True
          InxMatch = InxMatch + 1
        End If
      End If
      If Not CheckOK Then
        If InxMerge > UBound(ColMerge) Then
          ' ColMerge array exhausted
        Else
          If ColCrnt = ColMerge(InxMerge) Then
            CheckOK = True
            InxMerge = InxMerge + 1
          End If
        End If
      End If
      If Not CheckOK Then
        Call MsgBox("I was unable to find column " & ColCrnt & " in either" & _
                    " ColMatch or ColMerge.  Please correct and try again.", _
                                                                       vbOKOnly)
        Exit Sub
      End If
    Next


    RowCrnt = rowDataFirst
    Do While True


      If RowCrnt Mod 100 = 0 Then
        ' Use status bar to indicate progress
        Application.StatusBar = "Row " & RowCrnt & " of " & RowLast
      End If


      ' Attempt to match RowCrnt and RowCrnt+1
      RowsMatch = True    ' Assume match until find otherwise
      For InxMatch = 0 To UBound(ColMatch)
        ColCrnt = ColMatch(InxMatch)
        If .Cells(RowCrnt, ColCrnt).Value <> _
           .Cells(RowCrnt + 1, ColCrnt).Value Then
          ' Rows do not match
          RowsMatch = False
          Exit For
        End If
      Next


      If RowsMatch Then
        ' Rows match.  Merge second into first.
        For InxMerge = 0 To UBound(ColMerge)
          ColCrnt = ColMerge(InxMerge)
          .Cells(RowCrnt, ColCrnt).Value = .Cells(RowCrnt, ColCrnt).Value & _
                                           Separator & _
                                           .Cells(RowCrnt + 1, ColCrnt).Value
        Next
        ' Second row merged into first.  Discard second row.
        .Rows(RowCrnt + 1).EntireRow.Delete
        ' RowLast has moved up.
        RowLast = RowLast - 1
        ' Do not step RowCrnt because there may be another match for it
        If RowCrnt = RowLast Then
          ' All rows checked.
          Exit Do
        End If
      Else
        ' Rows do not match.  RowCrnt no longer of interest.
        RowCrnt = RowCrnt + 1
        If RowCrnt = RowLast Then
          ' All rows checked.
          Exit Do
        End If
      End If
    Loop
  End With


  ' Output duration of macro to Immediate window
  Debug.Print Format(Timer - TimeStart, "#,##0.00")


  Application.StatusBar = False
  Application.EnableEvents = True
  Application.ScreenUpdating = True


End Sub

Simply merges rows based on A2 range - I've tried fiddling with this to sort on A2:* range but no luck:

Code:
Sub test()    
    Dim a, i As Long, ii As Long, n As Long, z As String, x As Long
    a = Sheets("Sheet1").Range("a2").CurrentRegion.Value
    n = 1
    With CreateObject("Scripting.Dictionary")
        .CompareMode = 1
        For i = 2 To UBound(a, 1)
            For ii = 2 To 11
                z = z & Chr(2) & a(i, ii)
            Next
            If Not .exists(z) Then
                n = n + 1: .Item(z) = n
                For ii = 1 To UBound(a, 2)
                    a(n, ii) = a(i, ii)
                Next
            Else
                x = .Item(z)
                For ii = 12 To UBound(a, 2)
                    If a(i, ii) <> "" Then
                        a(x, ii) = a(x, ii) & IIf(a(x, ii) <> "", ",", "") & a(i, ii)
                    End If
                Next
            End If
            z = ""
        Next
    End With
    On Error Resume Next
    Application.DisplayAlerts = False
    Sheets("result").Delete
    On Error GoTo 0
    Sheets.Add().Name = "Result"
    With Sheets("result").Cells(1).Resize(n, UBound(a, 2))
        .Value = a
        .EntireColumn.AutoFit
    End With
End Sub

Thanks so much!

-XG
 

Excel Facts

Remove leading & trailing spaces
Save as CSV to remove all leading and trailing spaces. It is faster than using TRIM().
If I'm correct you want to merge the first 12 columns for mutiple unique values in columns 1 to 11.
Hopefully that what this code will do.
Code:
[COLOR="Navy"]Sub[/COLOR] MG11Jun49
[COLOR="Navy"]Dim[/COLOR] Rng [COLOR="Navy"]As[/COLOR] Range, Dn [COLOR="Navy"]As[/COLOR] Range
[COLOR="Navy"]Dim[/COLOR] Dic [COLOR="Navy"]As[/COLOR] Object, oTxt [COLOR="Navy"]As[/COLOR] [COLOR="Navy"]String,[/COLOR] k [COLOR="Navy"]As[/COLOR] Variant, Ac [COLOR="Navy"]As[/COLOR] [COLOR="Navy"]Long[/COLOR]


[COLOR="Navy"]Set[/COLOR] Rng = Range(Range("A1"), Range("A" & Rows.Count).End(xlUp))
[COLOR="Navy"]Set[/COLOR] Dic = CreateObject("scripting.dictionary")
Dic.CompareMode = vbTextCompare
[COLOR="Navy"]For[/COLOR] [COLOR="Navy"]Each[/COLOR] Dn [COLOR="Navy"]In[/COLOR] Rng
    [COLOR="Navy"]With[/COLOR] Application
        oTxt = Join(.Transpose(.Transpose(Dn.Resize(, 11))), ",")
    [COLOR="Navy"]End[/COLOR] With
    [COLOR="Navy"]If[/COLOR] Not Dic.Exists(oTxt) [COLOR="Navy"]Then[/COLOR]
        Dic.Add oTxt, Dn
    [COLOR="Navy"]Else[/COLOR]
        [COLOR="Navy"]Set[/COLOR] Dic(oTxt) = Union(Dic(oTxt), Dn)
    [COLOR="Navy"]End[/COLOR] If
[COLOR="Navy"]Next[/COLOR]


Application.ScreenUpdating = False
[COLOR="Navy"]For[/COLOR] [COLOR="Navy"]Each[/COLOR] k [COLOR="Navy"]In[/COLOR] Dic.keys
    [COLOR="Navy"]If[/COLOR] Dic(k).Count > 1 [COLOR="Navy"]Then[/COLOR]
        Application.DisplayAlerts = False
            [COLOR="Navy"]For[/COLOR] Ac = 12 To 0 [COLOR="Navy"]Step[/COLOR] -1
                [COLOR="Navy"]With[/COLOR] Dic(k).Offset(, Ac)
                     .HorizontalAlignment = xlCenter
                     .VerticalAlignment = xlCenter
                     .WrapText = True
                     .MergeCells = True
                [COLOR="Navy"]End[/COLOR] With
            [COLOR="Navy"]Next[/COLOR] Ac
        Application.DisplayAlerts = True
    [COLOR="Navy"]End[/COLOR] If
[COLOR="Navy"]Next[/COLOR] k
Application.ScreenUpdating = True
[COLOR="Navy"]End[/COLOR] [COLOR="Navy"]Sub[/COLOR]
Regards Mick
 
Upvote 0
If I'm correct you want to merge the first 12 columns for mutiple unique values in columns 1 to 11.
Hopefully that what this code will do.
Code:
[COLOR="Navy"]Sub[/COLOR] MG11Jun49
[COLOR="Navy"]Dim[/COLOR] Rng [COLOR="Navy"]As[/COLOR] Range, Dn [COLOR="Navy"]As[/COLOR] Range
[COLOR="Navy"]Dim[/COLOR] Dic [COLOR="Navy"]As[/COLOR] Object, oTxt [COLOR="Navy"]As[/COLOR] [COLOR="Navy"]String,[/COLOR] k [COLOR="Navy"]As[/COLOR] Variant, Ac [COLOR="Navy"]As[/COLOR] [COLOR="Navy"]Long[/COLOR]


[COLOR="Navy"]Set[/COLOR] Rng = Range(Range("A1"), Range("A" & Rows.Count).End(xlUp))
[COLOR="Navy"]Set[/COLOR] Dic = CreateObject("scripting.dictionary")
Dic.CompareMode = vbTextCompare
[COLOR="Navy"]For[/COLOR] [COLOR="Navy"]Each[/COLOR] Dn [COLOR="Navy"]In[/COLOR] Rng
    [COLOR="Navy"]With[/COLOR] Application
        oTxt = Join(.Transpose(.Transpose(Dn.Resize(, 11))), ",")
    [COLOR="Navy"]End[/COLOR] With
    [COLOR="Navy"]If[/COLOR] Not Dic.Exists(oTxt) [COLOR="Navy"]Then[/COLOR]
        Dic.Add oTxt, Dn
    [COLOR="Navy"]Else[/COLOR]
        [COLOR="Navy"]Set[/COLOR] Dic(oTxt) = Union(Dic(oTxt), Dn)
    [COLOR="Navy"]End[/COLOR] If
[COLOR="Navy"]Next[/COLOR]


Application.ScreenUpdating = False
[COLOR="Navy"]For[/COLOR] [COLOR="Navy"]Each[/COLOR] k [COLOR="Navy"]In[/COLOR] Dic.keys
    [COLOR="Navy"]If[/COLOR] Dic(k).Count > 1 [COLOR="Navy"]Then[/COLOR]
        Application.DisplayAlerts = False
            [COLOR="Navy"]For[/COLOR] Ac = 12 To 0 [COLOR="Navy"]Step[/COLOR] -1
                [COLOR="Navy"]With[/COLOR] Dic(k).Offset(, Ac)
                     .HorizontalAlignment = xlCenter
                     .VerticalAlignment = xlCenter
                     .WrapText = True
                     .MergeCells = True
                [COLOR="Navy"]End[/COLOR] With
            [COLOR="Navy"]Next[/COLOR] Ac
        Application.DisplayAlerts = True
    [COLOR="Navy"]End[/COLOR] If
[COLOR="Navy"]Next[/COLOR] k
Application.ScreenUpdating = True
[COLOR="Navy"]End[/COLOR] [COLOR="Navy"]Sub[/COLOR]
Regards Mick

Thanks for the prompt reply Mike - that is correct, not near a PC to test right now but will be this evening.

Basically I want to merge down for all duplicates in columns 1-11, columns 12 + 13 will always have blanks under them until next 'Job' so merge those down as well and then for columns 14-16 keep their 'in-line' (not sure if that is the correct term) so that staff can easily see that those events happened WITHIN that specific job.

Exactly like the 'Desired' picture I created basically. Does that make a little more sense?

Basically merge down columns 1-13 into one large cell and have columns 14-16 remain lines within that one cell. There will all be 'x' number of events DURING each job number (two jobs used in the example pictures).

Cheers!

-XG
 
Upvote 0
Just to update MickG that did the job! Only issue I have with that code snippet is that it has taken well over 25 minutes to condense approximately 15000 lines - is there anything I can do to improve the code now we know that it is working?

If it helps, the real .csv I am running the macro across is a total of 41 columns, the first 29 of which are being 'merged down' - the number of lines vary between a few hundred and the more extremely one I was testing on (15000+).

Regards,
XG
 
Upvote 0
The problem is ,Merging cells is generally not a good idea, and the actual act of manipulating the data on the sheet is what takes the time.
I ran the code on 1000 lines in groups of 5. The first bit of the code that just loads the range references in the code, and could possible be altered, only took 0.178 secs, but the to run the whole code took 6.3 secs, so you can see that the majority of the time is used in the manipulation of the sheet data.
 
Upvote 0
The problem is ,Merging cells is generally not a good idea, and the actual act of manipulating the data on the sheet is what takes the time.
I ran the code on 1000 lines in groups of 5. The first bit of the code that just loads the range references in the code, and could possible be altered, only took 0.178 secs, but the to run the whole code took 6.3 secs, so you can see that the majority of the time is used in the manipulation of the sheet data.

Understood completely, and thank you again.

I've heard a few people saying stay clear of merging cells - is there anyway to achieve the above with another method? I've just tried running again and its taking upwards of 15 minutes - it looks GREAT but the time (plus the spinning wheel/cursor I'm sure will worry staff haha) is the only negative.

-XG
 
Upvote 0
I've been thinking - is it possible to convert the 'merging' section of your above code to simply color the subsequent duplicate cells white?

I've been playing around with the script, but for some reason can't get it to keep the initial line - its marking any and all duplicates white.
 
Upvote 0
I would think colouring those cells in would take time
Why don't you just remove the duplicates from the first 11 columns,(leaving the actual duplicate rows blank) I would think it would still be obvious what lines belong to data past column 11 .
Probably the quickest way would be to put all the data into an array , remove the duplicates in first 11 columns and then paste back into sheet.
 
Upvote 0
Try this to colour delete dups in first 11 columns:-
It certainly quicker
Code:
[COLOR="Navy"]Sub[/COLOR] MG16Jun07
[COLOR="Navy"]Dim[/COLOR] Rng [COLOR="Navy"]As[/COLOR] Range, Dn [COLOR="Navy"]As[/COLOR] Range, t
[COLOR="Navy"]Dim[/COLOR] Dic [COLOR="Navy"]As[/COLOR] Object, oTxt [COLOR="Navy"]As[/COLOR] [COLOR="Navy"]String,[/COLOR] k [COLOR="Navy"]As[/COLOR] Variant, Ac [COLOR="Navy"]As[/COLOR] [COLOR="Navy"]Long[/COLOR]
t = Timer
[COLOR="Navy"]Set[/COLOR] Rng = Range(Range("A1"), Range("A" & Rows.Count).End(xlUp))
[COLOR="Navy"]Set[/COLOR] Dic = CreateObject("scripting.dictionary")
Dic.CompareMode = vbTextCompare
[COLOR="Navy"]For[/COLOR] [COLOR="Navy"]Each[/COLOR] Dn [COLOR="Navy"]In[/COLOR] Rng
    [COLOR="Navy"]With[/COLOR] Application
        oTxt = Join(.Transpose(.Transpose(Dn.Resize(, 11))), ",")
    [COLOR="Navy"]End[/COLOR] With
    [COLOR="Navy"]If[/COLOR] Not Dic.Exists(oTxt) [COLOR="Navy"]Then[/COLOR]
        Dic.Add oTxt, Dn
    [COLOR="Navy"]Else[/COLOR]
        [COLOR="Navy"]Set[/COLOR] Dic(oTxt) = Union(Dic(oTxt), Dn)
    [COLOR="Navy"]End[/COLOR] If
[COLOR="Navy"]Next[/COLOR]
'[COLOR="Green"][B][r1] = Timer - t[/B][/COLOR]
Application.ScreenUpdating = False
[COLOR="Navy"]For[/COLOR] [COLOR="Navy"]Each[/COLOR] k [COLOR="Navy"]In[/COLOR] Dic.keys
    [COLOR="Navy"]If[/COLOR] Dic(k).Count > 1 [COLOR="Navy"]Then[/COLOR]
      [COLOR="Navy"]With[/COLOR] Dic(k).Offset(1).Resize(Dic(k).Count - 1, 12)
        .ClearContents
        .Interior.ColorIndex = 19 '[COLOR="Green"][B] Remove this line to stop colouring[/B][/COLOR]
        [COLOR="Navy"]End[/COLOR] With
    [COLOR="Navy"]End[/COLOR] If
[COLOR="Navy"]Next[/COLOR] k
Application.ScreenUpdating = True
'[COLOR="Green"][B][s1] = Timer - t[/B][/COLOR]
[COLOR="Navy"]End[/COLOR] [COLOR="Navy"]Sub[/COLOR]
Regards Mick
 
Upvote 0
Try this to colour delete dups in first 11 columns:-
It certainly quicker
Code:
[COLOR=Navy]Sub[/COLOR] MG16Jun07
[COLOR=Navy]Dim[/COLOR] Rng [COLOR=Navy]As[/COLOR] Range, Dn [COLOR=Navy]As[/COLOR] Range, t
[COLOR=Navy]Dim[/COLOR] Dic [COLOR=Navy]As[/COLOR] Object, oTxt [COLOR=Navy]As[/COLOR] [COLOR=Navy]String,[/COLOR] k [COLOR=Navy]As[/COLOR] Variant, Ac [COLOR=Navy]As[/COLOR] [COLOR=Navy]Long[/COLOR]
t = Timer
[COLOR=Navy]Set[/COLOR] Rng = Range(Range("A1"), Range("A" & Rows.Count).End(xlUp))
[COLOR=Navy]Set[/COLOR] Dic = CreateObject("scripting.dictionary")
Dic.CompareMode = vbTextCompare
[COLOR=Navy]For[/COLOR] [COLOR=Navy]Each[/COLOR] Dn [COLOR=Navy]In[/COLOR] Rng
    [COLOR=Navy]With[/COLOR] Application
        oTxt = Join(.Transpose(.Transpose(Dn.Resize(, 11))), ",")
    [COLOR=Navy]End[/COLOR] With
    [COLOR=Navy]If[/COLOR] Not Dic.Exists(oTxt) [COLOR=Navy]Then[/COLOR]
        Dic.Add oTxt, Dn
    [COLOR=Navy]Else[/COLOR]
        [COLOR=Navy]Set[/COLOR] Dic(oTxt) = Union(Dic(oTxt), Dn)
    [COLOR=Navy]End[/COLOR] If
[COLOR=Navy]Next[/COLOR]
'[COLOR=Green][B][r1] = Timer - t[/B][/COLOR]
Application.ScreenUpdating = False
[COLOR=Navy]For[/COLOR] [COLOR=Navy]Each[/COLOR] k [COLOR=Navy]In[/COLOR] Dic.keys
    [COLOR=Navy]If[/COLOR] Dic(k).Count > 1 [COLOR=Navy]Then[/COLOR]
      [COLOR=Navy]With[/COLOR] Dic(k).Offset(1).Resize(Dic(k).Count - 1, 12)
        .ClearContents
        .Interior.ColorIndex = 19 '[COLOR=Green][B] Remove this line to stop colouring[/B][/COLOR]
        [COLOR=Navy]End[/COLOR] With
    [COLOR=Navy]End[/COLOR] If
[COLOR=Navy]Next[/COLOR] k
Application.ScreenUpdating = True
'[COLOR=Green][B][s1] = Timer - t[/B][/COLOR]
[COLOR=Navy]End[/COLOR] [COLOR=Navy]Sub[/COLOR]
Regards Mick

Amazing! I spent an entire day yesterday playing around and didn't come close haha - just tested against the same .CSV that was taking 30mins+ and its down to just over a minute!!!

One question I had that may make the whole identifying of 'deleted' cells before column 14 is 'row color striping' - me being my usual annoying self, this would be easy if it was one row per entry, but as we are using multiple rows per 'job' is there anyway to achieve alternative row stripping per 'Job #' (we may need to apply it on differing Job # before we delete cells?)?

You are a legend Mick!
 
Last edited:
Upvote 0

Forum statistics

Threads
1,223,236
Messages
6,170,915
Members
452,366
Latest member
TePunaBloke

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top