Need help regarding IE Automation using VBA

image_1

New Member
Joined
Oct 17, 2010
Messages
9
I have to download a csv file from a website like 50 times a day. I want to automate the process using vba so that i can process it later in excel.
the details are :
website: http://www.nrldc.org/WBS/injsch.aspx
csv file to download labelled by : (ncr)
here is what i have tried but it is not working. need help



Sub Update()
Dim IE As Object
On Error GoTo error_handler
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Navigate "http://www.nrldc.org/WBS/injsch.aspx"
Do While .Busy: DoEvents: Loop
Do While .ReadyState <> 4: DoEvents: Loop
.Visible = True
End With
For Each lnk In objIE.document.Links
If (InStr(lnk, "(ncr)")) Then
'Code to be executed in case it's a good link
objIEPage.Navigate lnk
While objIEPage.Busy
DoEvents
Wend
End If
Next lnk


objIE.Quit
Set IE = Nothing
Exit Sub
error_handler:
MsgBox ("Unexpected Error, I'm quitting.")
objIE.Quit
Set IE = Nothing
End Sub
 
Hi John,

I think you can help me on this. I've been looking for the code where I'll be able to save the downloaded data from a website.
I am not really familiar with HTML to VBA in excel coding.


This is the part where I am stucked. How to extract the data silently without clicking the button OPEN?

ieApp.Navigate "http://www.intexpress.com/BizDesk/excelExport.aspx?new=excel"

It will open the downloaded data in excel with options OPEN, SAVE and CANCEL.
There's no need for me to save the data because the target is to copy it into the excel workbook where my macro is placed.


Below is my code.
Dim ieApp As InternetExplorer
Dim ieDoc As Object

Sub GetINTdata()
Call IE_Sledgehammer ' to close all IE open

'create a new instance of ie
Set ieApp = New InternetExplorer

'you don’t need this, but it’s good for debugging
ieApp.Visible = True


'assume we’re logged in and need to logout first

ieApp.Navigate "http://www.intexpress.com/BizDesk/Logout.aspx"
On Error Resume Next
ieApp.Navigate "http://www.intexpress.com/BizDesk/Login.aspx"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop

Set ieDoc = ieApp.Document

'fill in the login form – View Source from your browser to get the control names
With ieDoc.forms(0)
.ctl00_ContentPlaceHolder1_txtUsername.Value = "test"
.ctl00_ContentPlaceHolder1_txtPassword.Value = "test"
.ctl00_ContentPlaceHolder1_btnSubmit.Click
End With
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop


'now that we’re in, go to the page we want
ieApp.Navigate "http://www.intexpress.com/BizDesk/AWBReport.aspx"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop

'fill in the parameters of the report and filter then export the data
With ieDoc.forms(0)
'Account
.ctl00_ContentPlaceHolder1_txtFilterAccount.Value = "" 'this will be assigned later in a range from the master worksheet

'AWB#
.ctl00_ContentPlaceHolder1_txtFilterAWBNumber.Value = "" ' this will be assigned later in a range from the master worksheet

'Sender State
.ctl00_ContentPlaceHolder1_drpFirstSenderState.Value = "CA" ' this will be assigned later in a range from the master worksheet

'Reciever State
.ctl00_ContentPlaceHolder1_drpFirstRecipientState.Value = "GA" ' this will be assigned later in a range from the master worksheet

'Start Date (From)
.ctl00_ContentPlaceHolder1_txtFilterStartDate.Value = "07/01/2015" ' this will be assigned later in a range from the master worksheet

'End Date (To)
.ctl00_ContentPlaceHolder1_txtFilterEndDate.Value = "07/08/2015" ' this will be assigned later in a range from the master worksheet

'Filter Button
.ctl00_ContentPlaceHolder1_btnFilter.Click
End With


ieApp.Navigate "http://www.intexpress.com/BizDesk/excelExport.aspx?new=excel"


'ieApp.Navigate "http://www.intexpress.com/BizDesk/Logout.aspx" will be set later

End Sub


Hope to receive a reply from you.

Thanks,
blackorchids






sskicker23, I don't have Windows 7 so can't test it on that system. Regarding the mismatch error, make sure you have Option Explicit at the top of every VBA module, compile the project and correct any resulting errors.

Here is updated code which handles the 'File already exists. Do you want to replace it?' window and clicks the 'Yes' button.

The code automates IE, navigating to View Injection Schedule and clicking the '(ncr)' link to download the .csv file. The IE automation is specific to View Injection Schedule, but the code which handles the IE download (the Download_File subroutine) should work with most sites from which you can download files.

Put the following code in a standard VBA module (named modMain):
Code:
Option Explicit

'Note - IE may block the download, displaying its Information Bar at the top of the tab, and preventing this program from
'automatically downloading the file.  To prevent this, add http://www.nrldc.org to IE's Trusted sites (Tools - Internet Options -
'Security - Trusted sites - Sites)


Public Sub Main_Download()
    
    Dim URL As String
    Dim startDate As Date
    Dim saveInFolder As String, saveAsFilename As String
    
    URL = "http://www.nrldc.org/WBS/injsch.aspx"
    
    saveInFolder = "C:\temp\Excel\"    
    saveAsFilename = ""     'use default filename
    startDate = Date
    
    IE_Navigate_and_Download URL, startDate, saveInFolder, saveAsFilename

End Sub


Private Sub IE_Navigate_and_Download(URL As String, startDate As Date, saveInFolder As String, saveAsFilename As String)

    Dim IE As Object
    Dim dateInput As Object
    
    'Use existing IE window or open a new IE window
        
    Set IE = Get_IE_Window()
    If IE Is Nothing Then
        Set IE = CreateObject("InternetExplorer.Application")
    End If
    
    With IE
        SetForegroundWindow .hWnd
        .Visible = True
        .Navigate URL
        While .Busy Or .ReadyState <> 4: DoEvents: Wend
    
        'Populate HTML text box with specified date and refresh page
        
        Set dateInput = .Document.forms(0).elements("txtStartDate")
        If Not dateInput Is Nothing Then
            dateInput.Value = Format(startDate, "dd-mm-yyyy")
            .Navigate "javascript:__doPostBack('txtStartDate','')"
            While .Busy Or .ReadyState <> READYSTATE_COMPLETE: DoEvents: Wend
        Else
            MsgBox "txtStartDate HTML element not found - unable to set specified date"
        End If
        
        'Click the download ncr link
        
        .Navigate "javascript:__doPostBack('downloadncr','')"
    
    End With
    
    Download_File saveInFolder, saveAsFilename
    
    SetForegroundWindow Application.hWnd

End Sub


Private Sub Download_File(saveInFolder As String, saveAsFilename As String)

    'Handles the whole IE download file sequence
    
    Dim fileExists As Boolean
    
    'File Download window, with Open, Save and Cancel buttons - click Save
    
    File_Download_Click_Save
       
    'Save As window - set full filename
    
    Save_As_Set_Filename saveInFolder, saveAsFilename
    
    'Save As window - click Save and replace file if the '<file> already exists' popup window appears
    
    fileExists = Save_As_Click_Save
    If fileExists Then
        File_Already_Exists Replace:=True
    End If
    
    'Optional Download complete window, with Open, Open Folder and Close buttons - click Close, or time out if window is not present
    'This window also has a checkbox 'Close this dialog box when download completes'.  This checkbox setting is controlled
    'by 'Notify when downloads complete' in IE advanced options.  If this checkbox is ticked, the following line is not required,
    'though it shouldn't matter if it is left in as the routine will time out if the window isn't found.
    
    Download_complete_Click_Close
    
    Debug.Print "Download finished"

End Sub


Private Sub File_Download_Click_Save()
    
    Dim hWnd As Long
    Dim timeout As Date
    
    Debug.Print "File_Download_Click_Save"
    
    'Find the File Download window, waiting a maximum of 30 seconds for it to appear
    
    timeout = Now + TimeValue("00:00:30")
    Do
        hWnd = FindWindow("#32770", "File Download")
        DoEvents
        Sleep 200
    Loop Until hWnd Or Now > timeout
    
    Debug.Print "   File Download window "; Hex(hWnd)
    
    If hWnd Then

        'Find the child Save button
        
        hWnd = FindWindowEx(hWnd, 0, "Button", "&Save")
        Debug.Print "   Save button "; Hex(hWnd)
    End If
    
    If hWnd Then
    
        'Click the Save button
        
        SetForegroundWindow hWnd
        Sleep 600  'this sleep is required and 600 milliseconds seems to be the minimum that works
        SendMessage hWnd, BM_CLICK, 0, 0
    End If

End Sub


Private Sub Save_As_Set_Filename(folder As String, filename As String)

    'Populate the 'File name:' edit window in the Save As dialogue with the specified folder and/or filename.
    'If folder = "" a folder path is not prepended to the filename and therefore the default save folder is used.
    'If filename = "" the default file name (already populated) is used.
    
    'The Save As window has the following child window hierarchy:
    
    '   "Save As", #32770 Dialog
    '       "FileName", ComboBoxEx32        (FileName is the default File name value in combobox)
    '           "", ComboBox
    '               "FileName", Edit        (FileName is the default File name value in combobox's edit box)
    
    Dim hWnd As Long
    Dim timeout As Date
    Dim fullFilename As String
    
    Debug.Print "Save_As_Set_Filename"
    Debug.Print "folder   = " & folder
    Debug.Print "filename = " & filename
    
    'Find the Save As window, waiting a maximum of 10 seconds for it to appear
    
    timeout = Now + TimeValue("00:00:10")
    Do
        hWnd = FindWindow("#32770", "Save As")
        DoEvents
        Sleep 200
    Loop Until hWnd Or Now > timeout

    If hWnd Then
    
        SetForegroundWindow hWnd
        
        'Find the child ComboBoxEx32 window
        
        hWnd = FindWindowEx(hWnd, 0, "ComboBoxEx32", vbNullString)
        Debug.Print "   ComboBoxEx32 "; Hex(hWnd)
    End If
    
    If hWnd Then
    
        'Find the child ComboBox window
        
        hWnd = FindWindowEx(hWnd, 0, "ComboBox", "")
        Debug.Print "   ComboBox "; Hex(hWnd)
    End If
                 
    If hWnd Then
        
        SetForegroundWindow hWnd

        'Find the child Edit window
        
        hWnd = FindWindowEx(hWnd, 0, "Edit", "")
        Debug.Print "   Edit "; Hex(hWnd)
    End If
    
    If hWnd Then
            
        If filename = "" Then
            'Get default filename (already populated in Edit window)
            filename = Get_Window_Text(hWnd)
        End If
       
        'If specified, ensure folder ends with \
        If folder <> "" And Right(folder, 1) <> "\" Then folder = folder & "\"
        
        fullFilename = folder & filename
        Debug.Print "Full filename " & fullFilename
        
        'Populate the Edit window with the full file name
        
        Sleep 200
        SetForegroundWindow hWnd
        SendMessageByString hWnd, WM_SETTEXT, Len(fullFilename), fullFilename
    End If
    
End Sub


Private Function Get_Window_Text(hWnd As Long) As String

    'Returns the text in the specified window
    
    Dim Buffer As String
    Dim length As Long
    Dim result As Long
    
    SetForegroundWindow hWnd
    length = SendMessage(hWnd, WM_GETTEXTLENGTH, 0, 0)
    Buffer = Space(length + 1)  '+1 for the null terminator
    result = SendMessage(hWnd, WM_GETTEXT, Len(Buffer), ByVal Buffer)
    Debug.Print "Edit File name = " & Left(Buffer, length)
    Debug.Print "        length = " & length
    
    Get_Window_Text = Left(Buffer, length)
    
End Function


Private Function Save_As_Click_Save() As Boolean

    'Click the Save button in the Save As dialogue, returning True if the '<file> already exists'
    'window appears, otherwise False
    
    Dim hWnd As Long, hWndSaveAs As Long
    Dim timeout As Date
    
    Debug.Print "Save_As_Click_Save"

    'Find the Save As window, waiting a maximum of 10 seconds for it to appear
    
    timeout = Now + TimeValue("00:00:10")
    Do
        hWnd = FindWindow(vbNullString, "Save As")
        hWndSaveAs = hWnd
        DoEvents
        Sleep 200
    Loop Until hWnd Or Now > timeout
    Debug.Print "   Save As window "; Hex(hWnd)

    If hWnd Then
    
        SetForegroundWindow hWnd
            
        'Get the child Save button
        
        hWnd = FindWindowEx(hWnd, 0, "Button", "&Save")
        Debug.Print "   Save button "; Hex(hWnd)
    End If
    
    If hWnd Then
        
        'Click the Save button
        
        'This can cause the '<file> already exists' popup window to be displayed.  The button click is sent with PostMessage
        'because SendMessage doesn't return until the window is closed (by clicking Yes or No)
        
        'PostMessage - http://msdn.microsoft.com/en-us/library/ms644944%28v=VS.85%29.aspx
        'Places (posts) a message in the message queue associated with the thread that created the specified window and returns
        'without waiting for the thread to process the message.
        
        'SendMessage - http://msdn.microsoft.com/en-us/library/ms644950%28VS.85%29.aspx
        'The SendMessage function calls the window procedure for the specified window and does not return until the
        'window procedure has processed the message.
        
        'SendMessage hWnd, BM_CLICK, 0, 0
        Sleep 100
        SetForegroundWindow hWnd
        PostMessage hWnd, BM_CLICK, 0, 0
        Debug.Print "   Clicked Save button"
    End If
        
    If hWnd Then
    
        'Set function return value depending on whether or not the '<file> already exists' popup window exists
        
        'Note - the popup window is a modal dialogue box and a child of the main Save As window.  Both windows have the
        'same class (#32770) and caption (Save As).  This may present a problem in finding the popup window, however in all tests
        'FindWindow has correctly found the popup window.  Therefore the additional FindWindowEx, which looks for the Yes button
        'as a child of the popup window is not necessary
        
        Sleep 500
        hWnd = FindWindow("#32770", "Save As")
        
        'This alternative FindWindowEx call, which looks for the popup window as a child of the main Save As window, doesn't find it,
        'returning 0 for hWnd
        'hWnd = FindWindowEx(hWndSaveAs, 0, "#32770", "Save As")
        
        Debug.Print "   Save As - <file> already exists popup window "; Hex(hWnd)
        'hWnd = FindWindowEx(hWnd, 0, "Button", "&Yes")
        
        If hWnd Then
            Save_As_Click_Save = True
        Else
            Save_As_Click_Save = False
        End If
        
    End If
        
End Function


Private Sub File_Already_Exists(Replace As Boolean)

    'Click Yes or No in the '<file> already exists. Do you want to replace it?' window
    
    Dim hWnd As Long
    
    Debug.Print "File_Already_Exists("; Replace; ")"

    hWnd = FindWindow("#32770", "Save As")
    Debug.Print "   Save As popup window "; Hex(hWnd)
    
    If hWnd Then
        If Replace Then
            hWnd = FindWindowEx(hWnd, 0, "Button", "&Yes")
            Debug.Print "   Yes button "; Hex(hWnd)
        Else
            hWnd = FindWindowEx(hWnd, 0, "Button", "&No")
            Debug.Print "   No button "; Hex(hWnd)
        End If
    End If
    
    If hWnd Then
        SendMessage hWnd, BM_CLICK, 0, 0
    End If

End Sub


Private Sub Download_complete_Click_Close()
    
    Dim hWnd As Long
    Dim timeout As Date
    
    Debug.Print "Download_complete_Click_Close"
        
    'Find the Download complete window, waiting a maximum of 30 seconds for it to appear.  Timeout value is
    'dependent on the size of the download, so make it longer for larger files.
    
    timeout = Now + TimeValue("00:00:30")
    Do
        hWnd = FindWindow("#32770", "Download complete")
        DoEvents
        Sleep 200
    Loop Until hWnd Or Now > timeout

    Debug.Print "   Download complete window "; Hex(hWnd)
    
    If hWnd Then

        'Find the child Close button
        
        hWnd = FindWindowEx(hWnd, 0, "Button", "Close")
        Debug.Print "   Close button "; Hex(hWnd)
    End If
    
    If hWnd Then
    
        'Click the Close button
        
        SetForegroundWindow hWnd
        Sleep 600  'this sleep is required and 600 milliseconds seems to be the minimum that works
        SendMessage hWnd, BM_CLICK, 0, 0
    End If

End Sub


Private Function Get_IE_Window() As Object

    'Look for an IE browser window and, if found, return that browser as an InternetExplorer object.  Otherwise return Nothing

    Dim Shell As Object
    Dim IE As Object
    Dim i As Variant 'Integer
    
    Set Shell = CreateObject("Shell.Application")
    
    i = 0
    Set Get_IE_Window = Nothing
    While i < Shell.Windows.Count And Get_IE_Window Is Nothing
        Set IE = Shell.Windows.Item(i)
        If Not IE Is Nothing Then
            If TypeName(IE.Document) = "HTMLDocument" Then
                Set Get_IE_Window = IE
            
                'Activate this tab (the first tab) by pressing Ctrl+1
                
                SetForegroundWindow Get_IE_Window.hWnd
                PressKeys VK_CONTROL, Asc("1")
            
            End If
        End If
        i = i + 1
    Wend
    
End Function


Private Sub PressKeys(vKey1 As Long, vKey2 As Long)
    KeyEvent vKey1, True, False     'key1 down
    Sleep 10
    KeyEvent vKey2, True, True      'key2 down and up
    Sleep 10
    KeyEvent vKey1, False, True     'key1 up
End Sub


Private Sub KeyEvent(ByVal vKey As Long, ByVal keyDown As Boolean, ByVal keyUp As Boolean)
    If keyDown Then keybd_event vKey, 0, VK_KEYDOWN, -11
    If keyUp Then keybd_event vKey, 0, VK_KEYUP, -11
End Sub


'---------- Testing and debugging ---------

Private Sub Test_Save_As_Set_Filename()
    
    'Set the folder and filename in the Save As window.  The Save As window must be already displayed
    
    Dim folder As String, filename As String
    
    folder = ""     'use IE's default download folder
    filename = "test " & Format(Now, "hh_mm_ss") & ".xls"
    Save_As_Set_Filename folder, filename
End Sub


Private Sub Test_Save_As_Click_Save()

    'Click Save in the Save As window and replace file if it already exists.  The Save As window must be already displayed
    
    Dim fileExists As Boolean
    fileExists = Save_As_Click_Save
    If fileExists Then
        File_Already_Exists True
    End If
    Download_complete_Click_Close
End Sub
Put the following code in another standard VBA module (named modWindowsAPI):
Code:
Option Explicit

Public Declare Sub Sleep Lib "kernel32" _
    (ByVal dwMilliseconds As Long)

Public Declare Function FindWindow Lib "user32" Alias "FindWindowA" _
    (ByVal lpClassName As String, ByVal lpWindowName As String) As Long

Public Declare Function FindWindowEx Lib "user32" Alias "FindWindowExA" _
    (ByVal hWnd1 As Long, ByVal hWnd2 As Long, ByVal lpsz1 As String, ByVal lpsz2 As String) As Long

Public Declare Function SetForegroundWindow Lib "user32" _
    (ByVal hWnd As Long) As Long

Public Declare Function SendMessage Lib "user32" Alias "SendMessageA" _
    (ByVal hWnd As Long, ByVal wMsg As Long, ByVal wParam As Long, lParam As Any) As Long

Public Declare Function SendMessageByString Lib "user32" Alias "SendMessageA" _
    (ByVal hWnd As Long, ByVal wMsg As Long, ByVal wParam As Long, ByVal lParam As String) As Long

Public Declare Function PostMessage Lib "user32" Alias "PostMessageA" _
    (ByVal hWnd As Long, ByVal wMsg As Long, ByVal wParam As Long, lParam As Any) As Long

Public Declare Sub keybd_event Lib "user32" _
    (ByVal bVk As Byte, ByVal bScan As Byte, ByVal dwFlags As Long, ByVal dwExtraInfo As Long)

Public Const BM_CLICK = &HF5
Public Const WM_SETTEXT = &HC
Public Const WM_GETTEXT = &HD
Public Const WM_GETTEXTLENGTH = &HE

Public Const VK_KEYDOWN = &H0
Public Const VK_KEYUP = &H2
Public Const VK_CONTROL = &H11


</file></file></file></file></file></file>
 
Upvote 0

Excel Facts

How to change case of text in Excel?
Use =UPPER() for upper case, =LOWER() for lower case, and =PROPER() for proper case. PROPER won't capitalize second c in Mccartney
This is the part where I am stucked. How to extract the data silently without clicking the button OPEN?

ieApp.Navigate "http://www.intexpress.com/BizDesk/excelExport.aspx?new=excel"

It will open the downloaded data in excel with options OPEN, SAVE and CANCEL.
There's no need for me to save the data because the target is to copy it into the excel workbook where my macro is placed.
Immediately after the point in your code where it causes the IE dialogue window to appear with Open, Save and Cancel buttons, call the Download_File routine as posted in this thread, along with the supporting modules and procedures. Something like this:
Code:
    Dim saveInFolder As String, saveAsFilename As String
    Dim dlWb As Workbook
    
    saveInFolder = "C:\folder\path\"
    saveAsFilename = "Downloaded Excel data.xls"
    Download_File saveInFolder, saveAsFilename
        
    Set dlWb = Workbooks.Open(saveInFolder & saveAsFilename)
    'Here, copy data from downloaded workbook to destination workbook
    dlWb.Close False
You say you don't want to save the downloaded data, but it is easier to download and save it and then open the downloaded workbook and copy the data from it, as shown in the above code. If necessary, you could delete the downloaded file using the Kill statement.

Alternatively, instead of using IE automation and Windows API functions, you could use XMLhttp GET and/or POST requests to download the file. The code for this technique is shown in http://www.mrexcel.com/forum/excel-...n-internet-explorer-web-site.html#post3404965, however you'll have to customise it for your website.

PS - please put VBA code inside CODE tags.
 
Upvote 0
Thanks John for the reply. Unfortunately, HTML to VBA is not really my thing. I just have a beginners knowledge in VBA.
The reason to have created this VBA project is for me to free up time in preparing a lot of shipment manifests.
But sad to say, I really don't know how to add the codes you had given.

Anyway, thanks for the time and reply.


Immediately after the point in your code where it causes the IE dialogue window to appear with Open, Save and Cancel buttons, call the Download_File routine as posted in this thread, along with the supporting modules and procedures. Something like this:
Code:
    Dim saveInFolder As String, saveAsFilename As String
    Dim dlWb As Workbook
    
    saveInFolder = "C:\folder\path\"
    saveAsFilename = "Downloaded Excel data.xls"
    Download_File saveInFolder, saveAsFilename
        
    Set dlWb = Workbooks.Open(saveInFolder & saveAsFilename)
    'Here, copy data from downloaded workbook to destination workbook
    dlWb.Close False
You say you don't want to save the downloaded data, but it is easier to download and save it and then open the downloaded workbook and copy the data from it, as shown in the above code. If necessary, you could delete the downloaded file using the Kill statement.

Alternatively, instead of using IE automation and Windows API functions, you could use XMLhttp GET and/or POST requests to download the file. The code for this technique is shown in http://www.mrexcel.com/forum/excel-...n-internet-explorer-web-site.html#post3404965, however you'll have to customise it for your website.

PS - please put VBA code inside CODE tags.
 
Upvote 0

Forum statistics

Threads
1,223,888
Messages
6,175,208
Members
452,618
Latest member
Tam84

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top