Web scraping slow using chrome driver

[SQUIDD]

Hi

So my current code is very slow. I know the reason it is slow. But i just dont know how to go about changing it up.

I believe it is because i am parsing the browser for each and every element.

Is there a way i can copy the table and output it in a sheet quicker?

Thanks
Dave

Sub GET_DOGS()
Set mybrowser = New Selenium.ChromeDriver
mybrowser.AddArgument "--headless"
mybrowser.Get "https://greyhoundbet.racingpost.com/#search-dog/dog_id=556694"
ROW_COUNT = mybrowser.FindElementById("sortableTable").FindElementsByTag("tr").Count
For b = 2 To ROW_COUNT
    For a = 1 To 16
        Cells(b, a) = mybrowser.FindElementByXPath("/html/body/div[3]/div[2]/div[2]/div[2]/div[2]/div[1]/div/div/table[2]/tbody/tr[" & b & "]/td[" & a & "]").Text
    Next a
Next b
mybrowser.Quit
End Sub

 

[alansidman]

You might try using Power Query. Look at my signature for more info on PQ

 

[SQUIDD]

Hi.

Thanks for your suggestion.

I have looked at power query before but it does not seem to have any data I can use.

 

[SQUIDD]

I have managed a workaround for anyone else that may find this useful, code below.

utilising the clipboard

you have to set references to MS forms object library also.

The copy is near immidiate

Sub GET_DOGS()
Application.ScreenUpdating = False
Dim clipb As MSForms.DataObject
Set clipb = New MSForms.DataObject
Set mybrowser = New Selenium.ChromeDriver
    mybrowser.AddArgument "--headless"
    mybrowser.Get "https://greyhoundbet.racingpost.com/#search-dog/dog_id=" & Sheets("RACING IDS").Range("B5")
    a = mybrowser.FindElementById("sortableTable").Attribute("outerHTML")
    clipb.SetText a
    clipb.PutInClipboard
    Range("A1").Select
    ActiveSheet.PasteSpecial Format:="Unicode Text", link:=False, DisplayAsIcon:=False, NoHTMLFormatting:=True
    Application.ScreenUpdating = True
End Sub

 

[Dan_W]

I quite like solution you arrived at where you've effectively copied and pasted HTML code (importantly, with the HTML Formatting!). I will try that next, thank you.
But aren't you comparing apples and oranges? In addition to using the MS Forms Object Library, you've also used a very different method of getting the data. Originally you used mybrowser.FindElementByXPath which goes column by column, and row by row - I can believe would be quite time consuming!

 

[Dan_W]

Hi.

Thanks for your suggestion.

I have looked at power query before but it does not seem to have any data I can use.

Sorry, one more thing - I tried @alansidman proposed PQ solution and it worked perfectly for me - I don't it didn't for you. It came out with the same data as when I ran your code.

 

[SQUIDD]

Hi Dan

Thanks, i really am not familiar with power query, and since i couldnet get any dat i assumed perhaps it wasnt compatible with that website.

however, noy say you have, i think i need to explore this further.

Yes, origianlly i was using xpath, but that just took so long, thats why i opted to copy the html.