IE.Document.getElementsByTagName("A")

[jamescooper]

So I have some code which pulls URLs from URLs by tagname if it contains the If Instr in the code below.

I'd like this to focus just on one section of the webpage, Div Class:

Div Class: product-list-container

and exclude:

Div Class: recommender__wrapper

The code is as follows:

Sub Getalllinks_tescos()


Dim IE As Object: Set IE = CreateObject("InternetExplorer.Application")
Dim cnt As Long: cnt = 1
Dim AR() As Variant: AR = Range("A1:A3").Value
Dim AL() As Variant
Dim url_name As String
Dim LR As Long


IE.Visible = True


For i = LBound(AR) To UBound(AR)
    url_name = AR(i, 1)


    If url_name = "" Then Exit For
    IE.navigate (url_name)


    Do
        DoEvents
    Loop Until IE.ReadyState = 4


    Set AllHyperlinks = IE.Document.getElementsByTagName("A")


    For Each hyper_link In AllHyperlinks
        If InStr(hyper_link.href, "https://www.tesco.com/groceries/en-GB/products/") Then
            ReDim Preserve AL(1 To cnt)
            AL(cnt) = hyper_link.href
            cnt = cnt + 1
        End If
    Next
 
Next i


IE.Quit


Range("B1").Resize(UBound(AL), 1).Value = Application.Transpose(AL)


End Sub

Many thanks in advance.

 

[jamescooper]

Any ideas thanks?

 

[xenou]

Possibly:

    Set AllHyperlinks = IE.Document.getElementsByClassName("product-list-container")(0).getElementsByTagName("A")

resources for web scraping (might help with the basics):
http://automatetheweb.net/common-vba-methods-properties-web-automation/