VBA Web Scraping - FindElementBy - DownloadCSV Button ElementNotVisibleError Runtime error 11

johnd3099

New Member
Joined
Aug 13, 2021
Messages
8
Office Version
  1. 2019
Platform
  1. Windows
  2. Web
Rich (BB code):
Option Explicit

Sub Download()
    Dim ChromeLocation As String
    ChromeLocation = "C:\Program Files\Google\Chrome\Application\chrome.exe" 'Location of Chrome.exe in your PC
   
    Dim cd As Selenium.ChromeDriver
    Set cd = New Selenium.ChromeDriver
    cd.SetBinary ("C:\Program Files\Google\Chrome\Application\chrome.exe")
   
    cd.Wait (3000)
   
    Dim FindBy As New Selenium.By
    Dim ResultSections As Selenium.WebElements
    Dim ResultSection As Selenium.WebElement
    Dim MyElement   As Object
    Dim SearchInput As Selenium.WebElement
    Dim SearchButton As Selenium.WebElement

    Call ClearImmediateWindow
   
'Genuine Stockbroker Username & Password        The ony way I could think of for someone to help solve my download problem
    Dim url As String
    url = "https://www.cmcmarketsstockbroking.com.au/"
   
    Dim UserName As String
    Dim Password As String
    UserName = "*********.com.au"                         'I  can provide these if requested
    Password = ".........."
   
    Dim DownloadURL As String
    DownloadURL = "https://www.cmcmarketsstockbroking.com.au/Manage/Watchlist/6213838?t=Watchlist"
'Genuine

    cd.AddArgument "start-maximized"
   
    cd.Start baseURL:=url
    cd.get "/", False
   
'    cd.Window.SetSize 1850, 1000
'    cd.Window.SetPosition 0, 0

'<input type="text" placeholder="Enter your username" id="logonAccount" name="logonAccount" class="sc-iktFfs bkQKJy sc-jQbJRc guveaB" value> == $0
    cd.FindElementById("logonAccount").Click
    cd.SendKeys (UserName)
   
'<input placeholder="Enter your password" id="logonPassword" name="logonPassword" class="sc-iktFfs bkQKJy sc-jUEmfL fsdOLa" type="password"> == $0
    cd.FindElementById("logonPassword").Click
    cd.SendKeys (Password)
   
'<button label="Login" type="submit" id="loginButton" class="sc-iqHYmW imzyTg sc-kIeSZW uCDoN"><span class="sc-gKseQn dhQxyI">Login</span></button>
    cd.FindElementById("loginButton").Click
   
  Debug.Print "Logged in to CMC"
 
'*****************************   AT THIS POINT,   WE HAVE LOGGED IN    ******************************************

    cd.get DownloadURL, False   'so far,  so good,  we have MarketCap1 page open,  wanting to press the Download CSV button
   
    If 1 = 1 Then               'This code is here to see .ResponseText,   because I've tried everything to Download
                                        'Change 'if 1 <> 1'  to  'if 1 = 1" to see the ResponseText in Sheet1        No longer applicable
        Dim XMLReq As Object
        Dim XMLHTTP As Object
        Set XMLHTTP = CreateObject("MSXML2.serverXMLHTTP")
       
        '<input type="button" class="inputbutton long btn" ng-click="ctrl.downloadCsv()" value="Download CSV">
        Dim HTMLDoc As MSHTML.HTMLDocument
        Dim HTMLDoc2 As MSHTML.HTMLDocument
        Dim LinkS As MSHTML.IHTMLElementCollection
        Dim Link As MSHTML.IHTMLElement
        Dim DownloadCSV As MSHTML.IHTMLElement
        Dim FileLinkFound As Boolean
        Dim FileURL As String
        Dim ReqStatus As Long
        Dim ResponseText As String
        Dim Http As New MSXML2.XMLHTTP60
   
        Set Http = CreateObject("MSXML2.XMLHTTP")
        With Http
            .Open "GET", DownloadURL, False
            .send (DownloadURL)
            Debug.Print .ResponseText
        End With
   
    End If

'Element        <input type="button" class="inputbutton long btn" ng-click="ctrl.downloadCsv()" value="Download CSV">
'xpath          //*[@id="autorefreshswitch"]/div/input
'css            #autorefreshswitch > div > input
'Full xpath     /html/body/form/div[4]/div[3]/div/div/div/div[4]/div[2]/div/input
    If 1 = 1 Then
        If Not cd.IsElementPresent(FindBy.Class("inputbutton")) Then
            Debug.Print "Could not find 'inputbutton'"
            Exit Sub
        End If
        Set SearchInput = cd.FindElementByClass("inputbutton")
        cd.SendKeys ("inputbutton")
   
        If Not cd.IsElementPresent(FindBy.Class("long")) Then
            Debug.Print "Could not find 'long'"
            Exit Sub
        End If
        Set SearchInput = cd.FindElementByClass("long")
        cd.SendKeys ("long")
       
        If Not cd.IsElementPresent(FindBy.Class("btn")) Then
            Debug.Print "Could not find 'btn'"
            Exit Sub
        End If
        Set SearchInput = cd.FindElementByClass("btn")
        cd.SendKeys ("btn")                                     'GOOD up to here
   
    'Narrowing the scope
        Dim OtdListItems As Selenium.WebElements
        Dim OtdListItem As Selenium.WebElement
        Dim OtdDiv As Selenium.WebElement
        Dim OtdLists As Selenium.WebElements
        Dim OtdList1 As Selenium.WebElement

        Set OtdDiv = cd.FindElementByXPath("//*[@id=""autorefreshswitch""]/div/input")       'did not bomb,  but no download
        Debug.Print OtdDiv.Text
        'Finding ng-click
        'xpath="//*[@id=""autorefreshswitch""]/div/input")
       
        Set OtdLists = cd.FindElementsByXPath("//*[contains(text(),' ng-click')]")
        If OtdLists Is Nothing Then
            Debug.Print "' ng-click' not found"
        Else
            Debug.Print "' ng-click' FOUND,  trying to CLICK"
            Debug.Print OtdLists.Count                                  'OtdLists.Count is ZERO
        End If

        cd.FindElementsByXPath ("//*[contains(text(),' ng-click')]")    'Does not BOMB
       
        For Each OtdList1 In OtdLists
            Debug.Print OtdList1.Text           'Nothing
        Next OtdList1

    End If
       
    '<input type="button" class="inputbutton long btn" ng-click="ctrl.downloadCsv()" value="Download CSV">
    cd.FindElementByClass ("btn")
    cd.FindElementByClass("btn").Click      'BOMBS here     ElementNotVisibleError  Runtime error 11
   
   
    'Don't know where to go from here ???
   
   
'---------------------------------------------------------------------------------------------------------------------------------------------

    'The following CODE is all stuff that I've tried
    Dim myCount As Long
    myCount = 0
   
    If 1 <> 1 Then
    'Find elements by tag
        Dim H2Headers As Selenium.WebElements
        Dim H2Header As Selenium.WebElement
        Dim Headlines As Selenium.WebElements
        Dim Headline As Selenium.WebElement
   
        'Set H2Headers = cd.FindElementsByXPath("//*[@id='autorefreshswitch']/div/input")
        'Set H2Headers = cd.FindElementsByXPath("ng-click=""ctrl.downloadCsv()""")
       
        cd.FindElementsByCss("#autorefreshswitch > div > input").Click
   
        Debug.Print "H2Headers.Count = " & H2Headers.Count
        If H2Headers.Count = 0 Then
            Debug.Print "No H2 headers found ... exitting"
            Exit Sub
        End If
   
        For Each H2Header In H2Headers
        myCount = myCount + 1
            Debug.Print myCount & "  H2Header.tagname = " & H2Header.tagname & "H2Header.Text = " & H2Header.Text
            Debug.Print H2Header.tagname & "   " & H2Header.Text
        Next H2Header
       
        H2Header(1).Click
        'H2Headers(1).submit
    End If
   
'Element        <input type="button" class="inputbutton long btn" ng-click="ctrl.downloadCsv()" value="Download CSV">
'xpath          //*[@id="autorefreshswitch"]/div/input
'css            #autorefreshswitch > div > input
'Full xpath     /html/body/form/div[4]/div[3]/div/div/div/div[4]/div[2]/div/input
   
    If 1 <> 1 Then
        If 1 <> 1 Then

            If Not cd.IsElementPresent(FindBy.XPath("//*[@id='autorefreshswitch']/div/input")) Then
                Debug.Print "element not present"
            End If
           
            While cd.Busy Or cd.ReadyState <> 4: DoEvents: Wend

            'cd.FindElementByXPath("//*[@id=""autorefreshswitch""]/div/input").submit       'StaleElementReferenceError
            cd.FindElementByXPath("//*[@id=""autorefreshswitch""]/div/input").Click         'did not bomb,  but did not download
           
        End If
       
        'css    #autorefreshswitch > div > input
        If 1 <> 1 Then
            'Set OtdDiv = cd.FindElementByCss("#autorefreshswitch")
            cd.FindElementByXPath("//*[@id=""autorefreshswitch""]/div/input").Click       'did not bomb,  but did not download
            'Set OtdDiv = cd.FindElementByXPath("//*[@id=""autorefreshswitch""]/div/input")
            'Set OtdDiv = cd.FindElementByClass("inputbutton")
            Debug.Print "#autorefreshswitch = " & OtdDiv.Text
            If OtdDiv.Text = "" Then
                Debug.Print "No mp-otd headers found ... exitting"
                'Exit Sub
            End If
           
            Set OtdLists = cd.FindElementsByClass("long")
            Debug.Print "div.Count = " & OtdLists.Count
            If OtdLists.Count = 0 Then
                Debug.Print "No ul headers found ... exitting"
                'Exit Sub
            End If
           
            Set OtdList1 = cd.FindElementByClass("btn")
            cd.FindElementByClass("btn").Click
            Debug.Print "input.Count = " & OtdLists.Count
            If OtdLists.Count = 0 Then
                Debug.Print "No ul headers found ... exitting"
                'Exit Sub
            End If
           
            For Each OtdList1 In OtdLists
                Debug.Print OtdList1.Text
            Next OtdList1

        End If
    End If
End Sub 'Download


Sub ClearImmediateWindow()
    If 1 <> 1 Then      'Not always working
        Application.SendKeys ("^g")            'Gets into Immediate window
       
        Application.SendKeys "^g^a"               'Select ALL in Immediate window
        Application.SendKeys "{BACKSPACE}"      'Send the DEL key
       
        Application.SendKeys "+{F7}"        'Gets into Code window
    End If
End Sub
 
Last edited by a moderator:

Excel Facts

Excel Joke
Why can't spreadsheets drive cars? They crash too often!
Hello, I believe the reason you haven't gotten an answer is because, you didn't say anything about your challenges with the code and the help you want. I think they didn't understand what ou wanted to do.
 
Upvote 0
Thanks Abdmujib, I was getting a bit discouraged.

I am tring to click the Download Button on the Web page, using VBA Chrome Selenium

Inspect shows :
'Element <input type="button" class="inputbutton long btn" ng-click="ctrl.downloadCsv()" value="Download CSV">
'xpath //*[@id="autorefreshswitch"]/div/input
'css #autorefreshswitch > div > input
'Full xpath /html/body/form/div[4]/div[3]/div/div/div/div[4]/div[2]/div/input

This is where it bombs.

cd.FindElementByClass ("btn")
cd.FindElementByClass("btn").Click 'BOMBS here ElementNotVisibleError Runtime error 11

I have tried FindElementByXPath("//*[@id=""autorefreshswitch""]/div/input")
FindElementsByXPath("//*[contains(text(),' ng-click')]")
and others

I don't know where to go from here.

Regards John
 
Upvote 0
Thanks Abdmujib, I was getting a bit discouraged.

I am tring to click the Download Button on the Web page, using VBA Chrome Selenium

Inspect shows :
'Element <input type="button" class="inputbutton long btn" ng-click="ctrl.downloadCsv()" value="Download CSV">
'xpath //*[@id="autorefreshswitch"]/div/input
'css #autorefreshswitch > div > input
'Full xpath /html/body/form/div[4]/div[3]/div/div/div/div[4]/div[2]/div/input

This is where it bombs.

cd.FindElementByClass ("btn")
cd.FindElementByClass("btn").Click 'BOMBS here ElementNotVisibleError Runtime error 11

I have tried FindElementByXPath("//*[@id=""autorefreshswitch""]/div/input")
FindElementsByXPath("//*[contains(text(),' ng-click')]")
and others

I don't know where to go from here.

Regards John
I don't know know vba but I believe @Anthony47 would be able to help you. He's a resourcefuland kind man
 
Upvote 0
It's difficult in your message to divide the code from the comments; so I assume that you can correctly login to the site using the code before the line '****************** AT THIS POINT, WE HAVE LOGGED IN **********

Now I understand you need to click the download button, whose html description is
'Element <input type="button" class="inputbutton long btn" ng-click="ctrl.downloadCsv()" value="Download CSV">

Well, after having stressed that these automation are mostly a try-and-error jobs, my best guess is that you should be able to do that using this snippet:
Code:
Dim tObj As Object
For I = 1 To 20
    cd.Wait 500
    Set tObj = cd.FindElementsByCss("input[class='inputbutton long btn']")
    If tObj.Count > 0 Then Exit For
Next I
Debug.Print "I=" & I, "Count=" & tObj.Count
If I < 20 Then
    tObj(1).Click
   Debug.Print "Clicked"
Else
    MsgBox ("Button not catched")
    '
    Stop
    '
End If

Didn't understand that the button is ready just after login or you first need to excecute the command cd.get DownloadURL, False

It will wait up to 10 seconds for the button to be ready; if it is there it will be clicked, otherwise the message "Button not catched" will be rised and the macro will halt on the Stop line

If it stops I suggest that:
-you open Notepad
-you open the Chrome inspector (rightclick on any element of the web page, chose Inspect)
-on the inspector frame locate the "<Body " tag (this is in the initial part of the html code)
-select the line, rightclick, select Copy /Outer html
-paste the code to notepad; save the file; share the file using a file sharing service (such as googledrive, onedrive, dropbox, ...)
Note: I don't think this code whould contain any confidential info, but doublecheck that your name or login information or any confidential information is not into the file; I expect the file be quite difficult to read, but it is a pure text file, and the Modify /Find command will certainly help

We will try to use this information to search for a viable method for starting the command
 
Upvote 0
Hi Anthony, I hope you are well. What timezone are you in? I am in Sydney Australia which is GMT + 11

Yes I do execute do.get DonoghueURL, False

Your code went through the motions, did not bomb, but did not do the download.

Here is the link for the OuterHtml
Image removed as contains confidential info.

I am using CMC Market Investing, but I set up this additional Login/Password (which does contain personal info) for this exercise only
I can give you the UserName and Password by some kind of private link, or by email?
 
Last edited by a moderator:
Upvote 0
That was supposed to be
Yes I do execute "do.get DownloadURL, False"
 
Upvote 0
You have a private message
The htm code start with the warning
"Your older web browser is no longer supported and some features of the platform, including online trading, will not work. We recommend upgrading your browser to the latest version of Chrome, Firefox or Internet Explorer"
So maybe it would be wise if you upgrade your Chrome (that will require updating also the selenium driver)

The code generated some "debug.print" messages, could you read that information and put it in your next message?

EDIT:
I was not allowed to send a pm to you, so "you don't have a pm"
 
Last edited:
Upvote 0
Hi Anthony, I don't know how to set up a pm, can you point out how to.

I do have latest chrome & selenium. I created OuterHtml on my live account and it shows the same blurb "Your older web browser ...."
but I have never had a problem downloading files, or anything else.

You can send to "johnd3099@tpg.com.au" if that is agreeable with you.

debug.print:
i=1 Count=1
Clicked
 
Upvote 0
Hello, I believe the reason you haven't gotten an answer is because, you didn't say anything about your challenges with the code and the help you want. I think they didn't understand what ou wanted to do.
Hello, could you show me how to set up my Private Message
 
Upvote 0

Forum statistics

Threads
1,223,886
Messages
6,175,195
Members
452,616
Latest member
intern444

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top