Importing data from web pages

mimox

New Member
Joined
Jul 19, 2015
Messages
2
Hi,
I wanted to extract some information from a web page. I had a few questions about it.
My query is similar to this.

I read one of the answers and could understand the code upto the querying part:
Code:
Sub QueryWeb()
    Dim i As Integer
    Dim firstRow As Integer
    Dim lastRow As Integer
    Dim nextRow As Integer
    Dim URLstart As String
    Dim URLend As String
    Dim shStats As Worksheet
    Dim shQuery As Worksheet
    Dim rgQuery As Range
    Dim found As Range
    Dim TimeOutWebQuery
    Dim TimeOutTime
    Dim objIE As Object
    Application.ScreenUpdating = False
    URLstart = "http://stats.espncricinfo.com/ci/engine/stats/index.html?class=2;filter=advanced;orderby=start;page="
    URLend = ";size=200;spanmax1=12+Jul+2012;spanmin1=13+Jul+2009;spanval1=span;template=results;type=batting;view=innings;wrappertype=print"
    Application.DisplayAlerts = False
    On Error Resume Next
    Sheets("Stats").Delete
    On Error GoTo 0
    Application.DisplayAlerts = True
    Sheets.Add after:=Sheets(Sheets.Count)
    ActiveSheet.Name = "Stats"
    Set shStats = Sheets("Stats")
    For i = 1 To 47
        Sheets.Add after:=Sheets(Sheets.Count)
        Set shQuery = ActiveSheet
        Set objIE = CreateObject("InternetExplorer.Application")
        With objIE
            .Visible = False
            .Navigate CStr(URLstart & i & URLend)
        End With
        TimeOutWebQuery = 10
        TimeOutTime = DateAdd("s", TimeOutWebQuery, Now)
        Do Until objIE.ReadyState = 4
            DoEvents
            If Now > TimeOutTime Then
                objIE.stop
                GoTo ErrorTimeOut
            End If
        Loop

However, I couldn't understand the actual scraping part. So my questions:
1) Do I write code to scrap data from parts of the website? Can I specifiy the HTML tags so it searches for those everytime it goes through a site?
2) The site that I want to import data from might have a lot of blank pages too but with the similar lay out as every other site (i.e. same site with tags but no data). How I make the code skip those pages?
3) The website requires the user to log-in to access the data. Can I somehow login once manually and let the macro browse and import all the similar websites?

I am pretty new to VBA, hoping to get some help.

Thanks!
 
Last edited:

Excel Facts

Do you hate GETPIVOTDATA?
Prevent GETPIVOTDATA. Select inside a PivotTable. In the Analyze tab of the ribbon, open the dropown next to Options and turn it off

Forum statistics

Threads
1,223,238
Messages
6,170,939
Members
452,368
Latest member
jayp2104

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top