Import data form website into Excel

excel_1317

Board Regular
Joined
Jun 28, 2010
Messages
212
Doing all 47 pages at once may cause their site to block your IP.
Though usually only temporarily - a few minutes to a few days.
You could do it in sessions by changing "For i = 1 To 47" to the appropriate page numbers.
After each session be sure to rename the "Stats" sheet as the code starts by deleting and re-creating it.
After all sessions combine the various "Stat" sheets.

Doing all 47 at once will take a few minutes to complete and you won't see anything happening.
The cursor will only spin occasionally.
You wil get "Query complete" when it finishes.

Code:
Sub QueryWeb()
    Dim i As Integer
    Dim firstRow As Integer
    Dim lastRow As Integer
    Dim nextRow As Integer
    Dim URLstart As String
    Dim URLend As String
    Dim shStats As Worksheet
    Dim shQuery As Worksheet
    Dim rgQuery As Range
    Dim found As Range
    Dim TimeOutWebQuery
    Dim TimeOutTime
    Dim objIE As Object
    Application.ScreenUpdating = False
    URLstart = "http://stats.espncricinfo.com/ci/engine/stats/index.html?class=2;filter=advanced;orderby=start;page="
    URLend = ";size=200;spanmax1=12+Jul+2012;spanmin1=13+Jul+2009;spanval1=span;template=results;type=batting;view=innings;wrappertype=print"
    Application.DisplayAlerts = False
    On Error Resume Next
    Sheets("Stats").Delete
    On Error GoTo 0
    Application.DisplayAlerts = True
    Sheets.Add after:=Sheets(Sheets.Count)
    ActiveSheet.Name = "Stats"
    Set shStats = Sheets("Stats")
    For i = 1 To 47
        Sheets.Add after:=Sheets(Sheets.Count)
        Set shQuery = ActiveSheet
        Set objIE = CreateObject("InternetExplorer.Application")
        With objIE
            .Visible = False
            .Navigate CStr(URLstart & i & URLend)
        End With
        TimeOutWebQuery = 10
        TimeOutTime = DateAdd("s", TimeOutWebQuery, Now)
        Do Until objIE.ReadyState = 4
            DoEvents
            If Now > TimeOutTime Then
                objIE.stop
                GoTo ErrorTimeOut
            End If
        Loop
        objIE.ExecWB 17, 2
        objIE.ExecWB 12, 2
        shQuery.Range("A1").Select
        shQuery.PasteSpecial NoHTMLFormatting:=True
        objIE.Quit
        Set objIE = Nothing
        Set found = shQuery.Columns(1).Find("Player", , , xlWhole)
        If Not found Is Nothing Then
            firstRow = found.Row
            If i > 1 Then firstRow = firstRow + 1
        Else
            GoTo FormatError
        End If
        Set found = shQuery.Columns(1).Find("Page ", found, , xlPart)
        If Not found Is Nothing Then
            lastRow = found.Row - 1
        Else
            GoTo FormatError
        End If
        Set rgQuery = shQuery.Rows(firstRow & ":" & lastRow)
        nextRow = shStats.Cells(Rows.Count, "A").End(xlUp).Row
        If nextRow > 1 Then nextRow = nextRow + 1
        rgQuery.Copy shStats.Cells(nextRow, 1)
        Application.DisplayAlerts = False
        shQuery.Delete
        Application.DisplayAlerts = True
    Next i
    shStats.Columns.AutoFit
    MsgBox "Query complete"
    Exit Sub
FormatError:
    MsgBox "Format Error"
    Exit Sub
ErrorTimeOut:
    objIE.Quit
    Set objIE = Nothing
    MsgBox "WebSite Error"
End Sub

This code works like charm. Is is possible to develop a code like this which open each each name from this page- http://www.fitzpatrickcella.com/?p=2541&FirstName=&LastName=&LPA=&I=&L=&Rank=

AND

copy details like Attorney Name, email, phone and fax nos. and paste the same into excel rows.

Thanks in advance.
 

Excel Facts

How to calculate loan payments in Excel?
Use the PMT function: =PMT(5%/12,60,-25000) is for a $25,000 loan, 5% annual interest, 60 month loan.
Re: Import data from multiple pages of a website into a single Excel sheet

please in your VBE editor
go to tools, references and add references for

Microsoft, XML v6.0 (or any previous version) choose your latest

and
Microsoft HTML Object Library


PHP:
Option Explicit
Sub GTOOPQRSTETY()
DoEvents
Dim k As Integer
Dim w As Integer
Dim cnt As Integer: cnt = 0
Dim mxml As MSXML2.XMLHTTP: Set mxml = New MSXML2.XMLHTTP
Dim HTM As MSHTML.HTMLDocument: Set HTM = New MSHTML.HTMLDocument
Dim newHTM As MSHTML.HTMLDocument: Set newHTM = New MSHTML.HTMLDocument
Dim T As MSHTML.IHTMLElementCollection
Dim Col As MSHTML.IHTMLElementCollection
Dim EL As IHTMLElement
With mxml
    .Open "GET", "http://www.fitzpatrickcella.com/?p=2541&FirstName=&LastName=&LPA=&I=&L=&Rank=", False
    .send
    HTM.body.innerHTML = .responseText
End With
Dim vArr() As String
Range("A1:D1").Value = Array("Attorney Name", "eMail", "Phone", "Fax #")
Set T = HTM.getElementsByTagName("a")
For k = 0 To T.Length - 1
    If T(k).href Like "about:blank?t=3&A=*" Then
        ReDim Preserve vArr(cnt)
        vArr(cnt) = "http://www.fitzpatrickcella.com/" & Mid(T(k).href, 12)
            With mxml
                .Open "GET", vArr(cnt), False
                .send
                newHTM.body.innerHTML = .responseText
            End With
        For Each EL In newHTM.all
            If EL.className = "attorneyname" Then
                Range("A2").Offset(cnt, 0) = EL.innerText
                Exit For
            End If
        Next
            Set Col = newHTM.getElementsByTagName("dd")
                For w = 0 To Col.Length - 1
                    If InStrRev(Col(w).innerText, "@") > 0 Then
                        Range("A2").Offset(cnt, 1).Value = Mid(Col(w).innerText, 3)
                    End If
                    
                    If Col(w).innerText Like "T (###)*" Then
                        Range("A2").Offset(cnt, 2).Value = Col(w).innerText
                    End If
                    
                    If Col(w).innerText Like "F (###)*" Then
                        Range("A2").Offset(cnt, 3).Value = Col(w).innerText
                        Exit For
                    End If
                Next
      cnt = cnt + 1
    End If
Next
    Set mxml = Nothing
    Set HTM = Nothing
    Set newHTM = Nothing
    Set Col = Nothing
    Set EL = Nothing
End Sub
 
Last edited:
Upvote 0
Re: Import data from multiple pages of a website into a single Excel sheet

please in your VBE editor
go to tools, references and add references for

Microsoft, XML v6.0 (or any previous version) choose your latest

and
Microsoft HTML Object Library


PHP:
Option Explicit
Sub GTOOPQRSTETY()
DoEvents
Dim k As Integer
Dim w As Integer
Dim cnt As Integer: cnt = 0
Dim mxml As MSXML2.XMLHTTP: Set mxml = New MSXML2.XMLHTTP
Dim HTM As MSHTML.HTMLDocument: Set HTM = New MSHTML.HTMLDocument
Dim newHTM As MSHTML.HTMLDocument: Set newHTM = New MSHTML.HTMLDocument
Dim T As MSHTML.IHTMLElementCollection
Dim Col As MSHTML.IHTMLElementCollection
Dim EL As IHTMLElement
With mxml
    .Open "GET", "http://www.fitzpatrickcella.com/?p=2541&FirstName=&LastName=&LPA=&I=&L=&Rank=", False
    .send
    HTM.body.innerHTML = .responseText
End With
Dim vArr() As String
Range("A1:D1").Value = Array("Attorney Name", "eMail", "Phone", "Fax #")
Set T = HTM.getElementsByTagName("a")
For k = 0 To T.Length - 1
    If T(k).href Like "about:blank?t=3&A=*" Then
        ReDim Preserve vArr(cnt)
        vArr(cnt) = "http://www.fitzpatrickcella.com/" & Mid(T(k).href, 12)
            With mxml
                .Open "GET", vArr(cnt), False
                .send
                newHTM.body.innerHTML = .responseText
            End With
        For Each EL In newHTM.all
            If EL.className = "attorneyname" Then
                Range("A2").Offset(cnt, 0) = EL.innerText
                Exit For
            End If
        Next
            Set Col = newHTM.getElementsByTagName("dd")
                For w = 0 To Col.Length - 1
                    If InStrRev(Col(w).innerText, "@") > 0 Then
                        Range("A2").Offset(cnt, 1).Value = Mid(Col(w).innerText, 3)
                    End If
                    
                    If Col(w).innerText Like "T (###)*" Then
                        Range("A2").Offset(cnt, 2).Value = Col(w).innerText
                    End If
                    
                    If Col(w).innerText Like "F (###)*" Then
                        Range("A2").Offset(cnt, 3).Value = Col(w).innerText
                        Exit For
                    End If
                Next
      cnt = cnt + 1
    End If
Next
    Set mxml = Nothing
    Set HTM = Nothing
    Set newHTM = Nothing
    Set Col = Nothing
    Set EL = Nothing
End Sub

Only one word for this...."AWESOME"
 
Upvote 0
Re: Import data from multiple pages of a website into a single Excel sheet

Can you pleaseee do the same for one more website. Attorney Finder | Knobbe Martens Intellectual Property Law

Above url has 24 pages with 12 attorney names on each page. When we click a attorney name, his profile is opened. So we need to copy his name, jobtitle, email and phone number(So the information in yellow bar)

Thank you so much!
 
Upvote 0
Re: Import data from multiple pages of a website into a single Excel sheet

Can you pleaseee do the same for one more website. Attorney Finder | Knobbe Martens Intellectual Property Law

Above url has 24 pages with 12 attorney names on each page. When we click a attorney name, his profile is opened. So we need to copy his name, jobtitle, email and phone number(So the information in yellow bar)

Thank you so much!

Please help...
 
Upvote 0
Re: Import data from multiple pages of a website into a single Excel sheet

ill try to have a look by end of week, this requires a bit more time because you need to go through the html of the website
 
Upvote 0
Re: Import data from multiple pages of a website into a single Excel sheet

ill try to have a look by end of week, this requires a bit more time because you need to go through the html of the website

Thank you!!

I have posted the same question on another forum in case I get any help before weekend.(Extract information from website using VBA)
I will keep you posted in case of any luck
 
Upvote 0
Re: Import data from multiple pages of a website into a single Excel sheet

Thank you!!

I have posted the same question on another forum in case I get any help before weekend.(Extract information from website using VBA)
I will keep you posted in case of any luck

Friend, no luck on other forum. Counting on you otherwise i will be left with no option other than to do it manually.

Thanks for your help
 
Upvote 0
Re: Import data from multiple pages of a website into a single Excel sheet

maybe if you had done it manually, you would be done after 3 days :)
 
Upvote 0

Forum statistics

Threads
1,223,238
Messages
6,170,939
Members
452,368
Latest member
jayp2104

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top