VBA to wait until IE webpage finishes loading before running the next step

[cdalgorta]

Hi,
So the below is supposed to OPEN internet explorer, WAIT UNTIL IT LOADS, add username & password and click login, THEN WAIT AGAIN UNTIL THE PAGE LOADS and then many other clicks on hyperlinks, but up to here is enough to show my issue.
So far I've been using "Application.Wait Now + TimeValue("00:00:05")", but honestly it is not very helpful, because sometimes the page takes 1 second to load and other times it just randomly takes like 15 seconds(so macro gives error). I'm talking about hundreds of clicks I have to do on this website. If I could make the macro do the next click exactly when the page stops loading it would be the best.

It's probably an obvious solution, but I just fail to see it. I'm guessing it is my variables(the IE one), but I'm not sure how to fix it.

Could you please let me know what exactly I should change on the 2nd "Do While IE.readyState <> READYSTATE_COMPLETE Loop" on the below macro so it actually waits before continuing with the next lines?
I know for a fact it is not waiting for the loading to finish because the "Debug.Print HTMLA.getAttribute("href")" is giving me the "href"s from the login page(first page) instead of waiting and giving me the "href"s of the 2nd page.

Thank you very much in advance!
P.S. For obvious reasons, I have changed the URL to a random one(Google) and the username and password to simply username and password.
P.P.S. Most of the below was from checking WiseOwlTutorials. I'm still a beginner with VBA.

Option Explicit

Sub GetData()

    Dim IE As New SHDocVw.InternetExplorer
    Dim HTMLDoc As MSHTML.HTMLDocument
    Dim HTMLInput As MSHTML.IHTMLElement
    Dim HTMLAs As MSHTML.IHTMLElementCollection
    Dim HTMLA As MSHTML.IHTMLElement
    
'Open Internet Explorer
    IE.Visible = True
    IE.navigate "Google.com"

     Do While IE.readyState <> READYSTATE_COMPLETE                          [B]' This one works[/B]
    Loop
    
'Add Username and Password
    IE.document.forms("loginForm").elements("UserName").Value = "username"
    IE.document.forms("loginForm").elements("Password").Value = "Password"

'Click on Login Button
    Set HTMLDoc = IE.document
    Set HTMLInput = HTMLDoc.getElementById("submitButton")
    HTMLInput.Click
    
    Do While IE.readyState <> READYSTATE_COMPLETE                  [B]'This one DOES NOT work. I've been using "Application.Wait Now + TimeValue("00:00:05")" in here as an alternative[/B]
    Loop


'Find carrier Portal href hyperlink

    Set HTMLAs = HTMLDoc.getElementsByTagName("a")
    
    For Each HTMLA In HTMLAs
        Debug.Print HTMLA.getAttribute("href")
  
    Next HTMLA


End Sub

 

[Anthony47]

IE.readyState is useful when there is a "navigation" to a url; but it's almost useless when the new page is created by a script, as in case of submitting a form.
So you have to identify in your source code which element can confirm the new page is fully created before going on with your next command.
For example, if after the login the page will show your account name you could wait for that element be available in the source code. I normally use a loop looking like this

Dim myAccount As Object

On Error Resume Next
For I = 1 To 20
    Set myAccount = IE.document.getElementById("cip")   '**
    If Not myAccount Is Nothing Then Exit For
    Application.Wait Now + TimeValue("00:00:01")                      'Thus there is a timeout of 20 secs
Next I
On Error Resume Next
If I > 20 Then
    'Unsuccessful
    'what to do?
End If
'The key element is there, you may continue

Note**: I took an easy element, catchabe "byID", but you know that in the real life things are more complicated

 

[cdalgorta]

IE.readyState is useful when there is a "navigation" to a url; but it's almost useless when the new page is created by a script, as in case of submitting a form.
So you have to identify in your source code which element can confirm the new page is fully created before going on with your next command.
For example, if after the login the page will show your account name you could wait for that element be available in the source code. I normally use a loop looking like this

Dim myAccount As Object

On Error Resume Next
For I = 1 To 20
    Set myAccount = IE.document.getElementById("cip")   '**
    If Not myAccount Is Nothing Then Exit For
    Application.Wait Now + TimeValue("00:00:01")                      'Thus there is a timeout of 20 secs
Next I
On Error Resume Next
If I > 20 Then
    'Unsuccessful
    'what to do?
End If
'The key element is there, you may continue

Note**: I took an easy element, catchabe "byID", but you know that in the real life things are more complicated

Hi Anthony,
I tried your code(changing it to 10 seconds), but it can't find the class(I've tried a few) and it takes the whole 10 seconds before it continues with the next code(even though this page loads in 2-3 seconds). I cannot use any byID because I cannot find any ID on this webpage. Any ideas what else I could try?

 

[cdalgorta]

IE.readyState is useful when there is a "navigation" to a url; but it's almost useless when the new page is created by a script, as in case of submitting a form.
So you have to identify in your source code which element can confirm the new page is fully created before going on with your next command.
For example, if after the login the page will show your account name you could wait for that element be available in the source code. I normally use a loop looking like this

Dim myAccount As Object

On Error Resume Next
For I = 1 To 20
    Set myAccount = IE.document.getElementById("cip")   '**
    If Not myAccount Is Nothing Then Exit For
    Application.Wait Now + TimeValue("00:00:01")                      'Thus there is a timeout of 20 secs
Next I
On Error Resume Next
If I > 20 Then
    'Unsuccessful
    'what to do?
End If
'The key element is there, you may continue

Note**: I took an easy element, catchabe "byID", but you know that in the real life things are more complicated

It's ok. I figured it out.
I changed "Dim IE As New SHDocVw.InternetExplorer" for "Dim IE As Object" and "Set IE = CreateObject("InternetExplorer.Application")"
and then for the waits I'm using:
Do While IE.readyState = 4: DoEvents: Loop
Do Until IE.readyState = 4: DoEvents: Loop
Application.Wait Now + TimeValue("00:00:01")

The 1 second at the end seems to do the trick. without it it does not really wait until it finishes loading for some reason. The state is clearly at 4, but still does not wait. Weird. It's fine though, now I just have to wait 1 second after every time it stops loading! I'm dancing of happiness right now hahah.

Thank you for taking some of your time to help me Anthony. Have a great day!