VBA website scraping - multiple link clicks to show full table

Offspring21

New Member
Joined
Jul 17, 2018
Messages
1
Hi,


I would like to request some help in relation to the coding below. I am trying to load up the full table of the website indicated in the code, by clicking multiple times (>20 X) the text link "Show more matches".
As soon as the full table is loaded in the IE i will "dump it to the worksheet.
My issue is to re-iterate the "Show more matches" link in the bottom of the page.


Thank you for your help in advance.


---------------------------------------------------------------------------------------------------------------------------------
Sub TestResults()


Dim IE As Object
Dim i As Long
Dim strText As String
Dim jj As Long
Dim hBody As Object
Dim hTR As Object
Dim hTD As Object
Dim tb As Object
Dim bb As Object
Dim Tr As Object
Dim Td As Object
Dim ii As Long
Dim doc As Object
Dim hTable As Object
Dim y As Long
Dim z As Long
Dim wb As Excel.Workbook
Dim ws As Excel.Worksheet
Dim hyper_link As Object



ActiveSheet.Range("A1:P3000").ClearContents

Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
Application.DisplayAlerts = False
Application.EnableEvents = False




Set wb = Excel.ActiveWorkbook
Set ws = wb.ActiveSheet

Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True

y = 1 'Column A in Excel
z = 1 'Row in Excel



'Navigate in Internet Explorer
'------------------------------------------------------------




IE.navigate "ht.........w.flashscore.com/baseball/usa/mlb/results/"


Do While IE.Busy: DoEvents: Loop
Do While IE.readyState <> 4: DoEvents: Loop




'Show all table elements
'------------------------------------------------------------
Do
Set AllHyperlinks = Nothing
Set AllHyperlinks = IE.document.getElementsByTagName("a")

For Each hyper_link In AllHyperlinks

If hyper_link.innerText = "Show more matches" Then


hyper_link.Click

Do While IE.Busy: DoEvents: Loop
'Do While IE.readyState <> 4: DoEvents: Loop


Application.Wait (Now + TimeValue("00:00:02"))

Exit For

End If

Next

Loop While IE.readyState <> 4 'hyper_link.innerText = "Show more matches"




'Collecting table from web and paste it to the excel sheet
'------------------------------------------------------------


Set doc = IE.document
Set hTable = doc.getElementsByTagName("table")


For Each tb In hTable
Set hBody = tb.getElementsByTagName("tbody")
For Each bb In hBody
Set hTR = bb.getElementsByTagName("tr")
For Each Tr In hTR
Set hTD = Tr.getElementsByTagName("td")
y = 1 ' Resets back to column A
For Each Td In hTD
ws.Cells(z, y).Value = Td.innerText
y = y + 1
Next Td
DoEvents
z = z + 1
Next Tr
Exit For
Next bb
Exit For
Next tb




IE.Quit
IE.Quit
IE.Quit
Set IE = Nothing
Set doc = Nothing
Set hTable = Nothing
Application.DisplayAlerts = True
Application.ScreenUpdating = True
Application.Calculation = xlCalculationAutomatic
Application.EnableEvents = True



End Sub
 

Excel Facts

Which lookup functions find a value equal or greater than the lookup value?
MATCH uses -1 to find larger value (lookup table must be sorted ZA). XLOOKUP uses 1 to find values greater and does not need to be sorted.

Forum statistics

Threads
1,223,227
Messages
6,170,849
Members
452,361
Latest member
d3ad3y3

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top