Nested Web Scraping Breaks My Do Loop

znoxide

New Member
Joined
Oct 9, 2016
Messages
3
I am having trouble with a nested web scraping section breaking my Do Until Loop. I have a list that I am creating a unique file with two tabs. One contains some basic header information which I want to use to get the most relevant website and add the link. Then save, close and move on to the next item on the list. If I comment out the web portion it works great, but when I attempt to run that section it acts like I have entered and exited improperly. Any help would be appreciated and an explanation would be fantastic.

Code:
Sub FilePrep()

    Dim y As Integer
    Dim ws As Worksheet
    Dim wb As Workbook 
    Dim objIE As InternetExplorer 
    Dim divEle As HTMLDivElement
    Dim divRes As String
    Dim aEle As HTMLLinkElement 
    Dim result As String 
    Dim sID As String 
    Dim sName As String
    Dim sSearch As String
    Dim folderPath As String
    
    y = 2
    folderPath = Application.ActiveWorkbook.Path
    
    
    Do Until Workbooks("Schools.xlsm").Worksheets("trial").Cells(y, 1).Value = ""
        sID = Cells(y, 1).Value
        sName = Cells(y, 2).Value
        sSearch = Cells(y, 3).Value
            
            
            Set wb = Workbooks.Add
                With wb
                .Worksheets("Sheet1").Name = "School"
                .Worksheets.Add
                .Worksheets("Sheet2").Name = "Directory"
                .Worksheets("School").Range("A1").Value = sID
                .Worksheets("School").Range("B1").Value = sName
                .Worksheets("School").Range("C1").Value = sSearch
                
                    Set objIE = New InternetExplorer
                        objIE.Visible = True
                        objIE.navigate "http:\\duckduckgo.com"
                        Application.Wait Now + #12:00:03 AM#
                        objIE.document.getElementById("search_form_input_homepage").Value = sSearch
                        objIE.document.getElementById("search_button_homepage").Click
                        Application.Wait Now + #12:00:03 AM#
                        aEle = objIE.document.getElementById("r1-0").getElementsByClassName("results__a")
                        result = aEle


                .Sheets("School").Range("D1").Value = result
                .Sheets("School").Range("E1").Value = aEle.innerText
                .SaveAs (sID)
                .Close
                End With
                
                y = y + 1
    Loop


End Sub
 

Excel Facts

Save Often
If you start asking yourself if now is a good time to save your Excel workbook, the answer is Yes

Forum statistics

Threads
1,223,896
Messages
6,175,260
Members
452,627
Latest member
KitkatToby

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top