Ebay Product Scraping code issue, works on .com not .co.uk

Sharid

Well-known Member
Joined
Apr 22, 2007
Messages
1,066
Office Version
  1. 2016
Platform
  1. Windows
Hi

I am hoping someone can explain why my code is not running when I change it to ebay.co.uk

The code runs in two parts,

1) It opens IE - Ebay
2) It places the products Title, Price and URL in columns A,B,C,

It works fine for .com but not for .co.uk I have checked the classes and they all see to be the same. I have left both link in, .co.uk is in green and as a comment for now, as it only works on 1 url at a time

Code:
Public IE As New SHDocVw.InternetExplorer

Sub GetData()

    Dim HTMLdoc As MSHTml.HTMLDocument
    Dim othwb As Variant
    Dim objShellWindows As New SHDocVw.ShellWindows

    Set IE = CreateObject("internetexplorer.application")
    
        With IE
            .Visible = True
            [COLOR=#008000]'.Navigate "https://www.ebay.co.uk/sch/i.html?_from=R40&_trksid=m570.l1313&_nkw=jackets&_sacat=0"[/COLOR]
            .Navigate "https://www.ebay.com/sch/i.html_from=R40&_nkw=ralph+lauren&_sacat=1059&LH_TitleDesc=0&_dmd=1&rt=nc"
            While .Busy Or .ReadyState <> 4: DoEvents: Wend

Set HTMLdoc = IE.Document
                ProcessHTMLPage HTMLdoc
                
                
            .Quit
        End With


End Sub

[B][COLOR=#ff8c00]THIS is the sub procedure and is in the same module [/COLOR][/B]

Sub ProcessHTMLPage(HTMLPage As MSHTml.HTMLDocument)

    Dim HTMLItem As MSHTml.IHTMLElement
    Dim HTMLItems As MSHTml.IHTMLElementCollection
    Dim HTMLInput As MSHTml.IHTMLElement
    Dim rownum As Long
    
    rownum = 1

    Set HTMLItems = HTMLPage.getElementsByClassName("s-item__title") [COLOR=#008000]'Gets the TITLE and puts in column A[/COLOR]
    
    For Each HTMLItem In HTMLItems

            Cells(rownum, 1).Value = HTMLItem.innerText
            rownum = rownum + 1
        
    Next HTMLItem
    
    rownum = 1
    
    Set HTMLItems = HTMLPage.getElementsByClassName("s-item__price") [COLOR=#008000]'Gets the PRICE and puts in column B[/COLOR]
    
    For Each HTMLItem In HTMLItems

            Cells(rownum, 2).Value = HTMLItem.innerText
            rownum = rownum + 1
        
    Next HTMLItem

    rownum = 1
     
  Set HTMLItems = HTMLPage.getElementsByClassName("s-item__link") [COLOR=#008000]'Gets the URL and puts in column C[/COLOR]
    For Each HTMLItem In HTMLItems
             Cells(rownum, 3).Value = HTMLItem.href
            rownum = rownum + 1
        
    Next HTMLItem

[COLOR=#008000]'Converts each text hyperlink selected into a working hyperlink from C1 to 25000 rows[/COLOR]
  Range("C1:C25000").Select
    For Each xCell In Selection
        ActiveSheet.Hyperlinks.Add Anchor:=xCell, Address:=xCell.Formula
    Next xCell
Range("C1").Select
End Sub
 

Excel Facts

Highlight Duplicates
Home, Conditional Formatting, Highlight Cells, Duplicate records, OK to add pink formatting to any duplicates in selected range.
I still can not work this one out, if anyone could please take a look, I would be very happy. I works fine with .com but not with .co.uk

I'm not sure why
 
Upvote 0
Could someone please help.

This code is now working on both .com and .co.uk NOW, if I don't include the code in orange. I added the orange code so it would allow the search criteria to come from Sheet1 A1 This worked fine for Google - see this thread last 2 posts on PAGE 2 https://www.mrexcel.com/forum/excel...-object-doesnt-support-property-method-2.html

The Problem I am having is

1)
It is not pulling the seach criteria from Sheet1 A1 to Ebay so there is nothing to search. Since it is not searching I can not test if it will navigate the pages as well.

If I set the url in the code to

- https://www.ebay.co.uk/ = (Ebay Main Page) the ebay urls = https://www.ebay.co.uk/n/error

- https://www.ebay.co.uk/sch/i.html?_from= = (Ebay Items listed in search) https://www.ebay.co.uk/sch/i.html?_from=tablets

It should look like this
https://www.ebay.co.uk/sch/i.html?_...RC2.A0.H0.Xtablets.TRS0&_nkw=tablets&_sacat=0

[TD="width: 64"][/TD]
If you delete all parts in orrange the code works fine.



Code:
Option Explicit

Public Sub GetDataEbay()
    Dim htmlDoc As MSHTML.HTMLDocument, ie As SHDocVw.InternetExplorer, ws As Worksheet

    Set ie = New SHDocVw.InternetExplorer
    Set htmlDoc = New MSHTML.HTMLDocument
    Set ws = ThisWorkbook.Worksheets("Sheet1")
[COLOR=#ff8c00]
   Dim nextPageElement As Object
    Dim pageNumber As Long[/COLOR]
    
    
    ' Takes seach from A1 and places it into ebay
    With ie
        .Visible = True
      [COLOR=#ff8c00]  '.Navigate2 "https://www.ebay.co.uk/sch/i.html?_from=" & Replace(Worksheets("Sheet1").Range("A1").Value, " ", "+") [/COLOR][COLOR=#008000]'this is also now showing as a comment, this code chould take search criteria from Sheet1 Column A

[/COLOR]
    [COLOR=#ee82ee].Navigate2 "https://www.ebay.co.uk/sch/i.html?_from=R40&_trksid=m570.l1313&_nkw=jackets&_sacat=0" [/COLOR][COLOR=#008000] 'Original Code for search, but only searches 1 item[/COLOR][COLOR=#ee82ee][/COLOR]
        While .Busy Or .readyState <> 4: DoEvents: Wend

        Dim index As Long, HTMLItems As Object, rowNum As Long, xCell As Range
        Dim cssSelectors(), i As Long
[COLOR=#ff8c00]Do[/COLOR]
        Select Case True
        Case InStr(.document.url, "ebay.co.uk") > 0
            cssSelectors = Array(".gvtitle a", ".amt", ".gvtitle a")
        Case InStr(.document.url, "ebay.com") > 0
            cssSelectors = Array(".s-item__title", ".s-item__price", ".s-item__link")
           
        End Select

 [COLOR=#ff8c00]If pageNumber >= 5 Then Exit Do [/COLOR][COLOR=#008000]'the first 5 pages[/COLOR][COLOR=#ff8c00]
        Set nextPageElement = htmlDoc.getElementById("pnnext") 'pnnext
        If nextPageElement Is Nothing Then Exit Do
        
        ' Clicks web next page
        nextPageElement.Click 'next web page
        Do While ie.Busy Or ie.readyState <> 4
            DoEvents
        Loop
        Application.Wait Now + TimeSerial(0, 0, 5)
        Set htmlDoc = ie.document
        pageNumber = pageNumber + 1
    Loop[/COLOR]

        With ws
            For i = LBound(cssSelectors) To UBound(cssSelectors)
                rowNum = 1
                Set HTMLItems = ie.document.querySelectorAll(cssSelectors(i))

                For index = 0 To HTMLItems.Length - 1
                    .Cells(rowNum, i + 1).Value = IIf(i = 2, HTMLItems.Item(index).getAttribute("href"), HTMLItems.Item(index).innerText)
                    rowNum = rowNum + 1
                Next
            Next
            For Each xCell In .Range("C1:C25000") '<= all these really?
                .Hyperlinks.Add Anchor:=xCell, Address:=xCell.Formula
            Next xCell
        End With
        .Quit
    End With
End Sub

Thanks
 
Upvote 0
Sorry ignore most of the above thread, I kind of worked it out, still has a bug. by the time i had finished changing the above post I ran out of time and it would not let me change it.

if the url code is changed to this,
Code:
.Navigate2 "https://www.ebay.co.uk/sch/i.html?_from=R40&_trksid=m570.l1313&_nkw=" & Replace(Worksheets("Sheet1").Range("A1").Value, " ", "+")

then it will take the search from Sheet1 Column A1

However when the page opens nothing is extracted, nor does it navigate to the next page. i get no errors.

I looked at the URLS and if the code is used then I get this as a url

-https://www.ebay.co.uk/sch/i.html?_from=R40&_trksid=m570.l1313&_nkw=laptops

Without code, same url

-https://www.ebay.co.uk/sch/i.html?_from=R40&_trksid=p2047675.m570.l1313.TR12.TRC2.A0.H0.Xlaptops.TRS0&_nkw=laptops&_sacat=0

is it because i left off
Code:
_sacat=0
as it was giving me an error
 
Last edited:
Upvote 0

Forum statistics

Threads
1,223,889
Messages
6,175,223
Members
452,620
Latest member
dsubash

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top