VBA Scraping from dynamic web page

[Bruzio]

Hi everybody. I'm trying to download soccer results from Football results 21 august 2020, I can't as I'd like.
My question was posted on a italian forum, but we were unable to resolve.
I trust in your knowledge.


What do i need?

6 columns:

LEAGUE - START - HOME - AWAY - SCORE- STATUS

What are the problems?

1) Macro doesn't download all scheduled events.
2) The content of the rows isn't correct
2) The web page has a more results button that is always visible and clickable, and i need to load all hidden pages.

I expect this, see screen:

This is my current code:

Sub fromweb()
    Dim IE As Object
    Dim Doc As Object
    Dim x As Object
    Dim i As Long
    Dim myArray As Variant
    Set IE = CreateObject("InternetExplorer.Application")
    Application.ScreenUpdating = False
    Cells.Clear
    Const myURL As String = "fctables.com/livescore/21_08_2020/"
    With IE
        .navigate myURL
        .Visible = True
        Do While .Busy: DoEvents: Loop
        Do While .readyState <> 4: DoEvents: Loop
        Application.Wait Now + TimeValue("0:00:10")
    End With
    myArray = Array("LEAGUE", "START", "HOME", "AWAY", "SCORE", "STATUS")
    With Range("A1:F1")
        .Value = myArray
    End With
    Set Doc = IE.Document
    For Each x In Doc.getElementsByClassName("league")
        i = i + 1: j = 0
        Cells(i + 1, 1) = Replace(Replace(x.innerText, "Table", ""), Chr(10), "")
        For Each y In Doc.getElementsByClassName("col-xs-8 col-sm-9 col-lg-10 truncate")
            Cells(i + 1, 1) = y.innerText
            Cells(i + 1, 2) = Doc.getElementsByClassName("date")(j).innerText
            Cells(i + 1, 3) = Doc.getElementsByClassName("home")(j).innerText
            Cells(i + 1, 4) = Doc.getElementsByClassName("away")(j).innerText
            Cells(i + 1, 5) = Doc.getElementsByClassName("score")(j).innerText
            Cells(i + 1, 6) = Doc.getElementsByClassName("status_name")(j).innerText
            j = j + 1: i = i + 1
        Next
    Next
    IE.Quit
    Set IE = Nothing
    Set Doc = Nothing
    Application.ScreenUpdating = True
End Sub

How to proceed correctly?
Thanks for the attention

 

[Peter_SSs]

Welcome to the MrExcel board!

My question was posted on a italian forum

Being on an Italian forum it may not help us but #13 of our Forum Rules asks that you post a link to other forum threads where you have asked the same question so it would be good if you did that.
Thanks.

 

[Bruzio]

Hi sorry, original post is here: Domanda - Scaricare dati da web
Not solved so I asked to try on other forums.

This is the reply, in italian language:

" L'utente ha chiesto di fare la domanda su altri siti non avendo trovato soluzioni e ai sensi della regola n. 3 del Regolamento viene autorizzato. "

In summary, authorized.

 

[Bruzio]

Hi, no reply, it's ok my previous post?

 

[Peter_SSs]

it's ok my previous post?

Yes it is ok. Thanks for posting the link.
Your question is not in an area that I am strong in but somebody else may come along and help. However, in a free public forum like this you never know.
Good luck!

 

[Bruzio]

Hi, thank you
I wait for at least someone tell me if it's possible or not. I prefer a negative reply than nothing

 

[John_w]

Try this to click the 'Show more matches' button and load more leagues and matches:

    Dim moreMatchesDiv As Object 'HTMLDivElement
    Set moreMatchesDiv = Doc.getElementById("download_more_livescore")
    moreMatchesDiv.Focus
    moreMatchesDiv.FireEvent "onclick"

 

[Bruzio]

Hi, thank you but i have this error:

Dim moreMatchesDiv As Object 'HTMLDivElement
Set moreMatchesDiv = Doc.getElementById("download_more_livescore")
moreMatchesDiv.Focus
moreMatchesDiv.FireEvent "onclick"

 

[John_w]

What is the error? Where have you put my code?

 

[Bruzio]

Hi
This code works but it doesn't download all scheduled events

    Set div = ie.document.getElementById("download_more_livescore")
        div.Focus
        div.FireEvent "onclick"