How to use selectNodes to get elements from web page?

[Fractalis]

Hi all,

I'm trying to get a elements from a web page with MSXML library using Xpath but I'm not able to make it work.

For the code below that I have so far, I'm getting "Run-time error 13" in the line Set elem = doc.SelectNodes("//a[contains(.,'Message Board')]")

May someon help me with this. Thanks

Sub Test()
Dim URL As String
Dim oHttp As New MSXML2.XMLHTTP60
Dim elem As MSXML2.IXMLDOMNode


    URL = "https://www.mrexcel.com/"
    oHttp.Open "GET", URL, False
    oHttp.send
    
    Dim html As New HTMLDocument
    html.body.innerHTML = oHttp.responseText
    
    Set doc = New MSXML2.DOMDocument60
    doc.SetProperty "SelectionLanguage", "XPath"
    doc.Load oHttp.responseText
    
    Set elem = doc.SelectNodes("//a[contains(.,'Message Board')]")
    
End Sub

 

[Domenic]

You only need MSXML2.XMLHTTP60 to make your http request, and MSHTML.HTMLDocument to access the elements with the html document...

Option Explicit

Sub test()

    'Set a reference (VBE > Tools > References) to the following libraries:
    '   1) Microsoft XML, v6.0
    '   2) Microsoft HTML Object Library

    Dim xmlReq As MSXML2.XMLHTTP60
    Dim htmlDoc As MSHTML.HTMLDocument
    Dim htmlLinks As MSHTML.IHTMLDOMChildrenCollection
    Dim url As String
    Dim resp As String
    Dim linkIndex As Long
    
    url = "https://www.mrexcel.com"
    
    Set xmlReq = New MSXML2.XMLHTTP60
    
    Set htmlDoc = New MSHTML.HTMLDocument
    
    With xmlReq
        .Open "GET", url, False
        .send
        Do While .readyState <> 4
            DoEvents
        Loop
        If .Status <> 200 Then
            MsgBox "Error " & .Status & ":  " & .statusText
            Exit Sub
        End If
        resp = .responseText
    End With
    
    htmlDoc.body.innerHTML = resp
    
    Set htmlLinks = htmlDoc.querySelectorAll("a.nav-link")  ' use the querySelector to search by tag and class
    
    With htmlLinks
        For linkIndex = 0 To .Length - 1
            If Trim$(.Item(linkIndex).innerText) = "Message Board" Then
                Debug.Print .Item(linkIndex).href
                Exit For
            End If
        Next linkIndex
    End With
    
    Set xmlReq = Nothing
    Set htmlDoc = Nothing
    Set htmlLinks = Nothing

End Sub

Hope this helps!

 

[Fractalis]

Thanks @Domenic for the clarification about the only libraries I need. Your code works, but I was wondering if there is a way to get elements by Xpath (without the need to install Selenium driver), since I need a way to search the elements based on text since classes, IDs change from time to time in the website.

 

[Domenic]

In Excel, XPath is used for XML documents. However, if you want to use XPath for HTML documents, I think you'll need to use a third party driver, such as Selenium.