Import data from multiple pages of a website into a single Excel sheet

V S S Sarma

New Member
Joined
Jul 13, 2012
Messages
11
I am a cricket buff and like to see statistics. But they run to 47 pages running to 9,394 rows in a website. Copy and paste is not a solution. I look forward to your help in either developing an Excel VBA macro or any other way. Please see the first two pages here:

Page 1
http://stats.espncricinfo.com/ci/en...s;type=batting;view=innings;wrappertype=print
Page 2
http://stats.espncricinfo.com/ci/en...s;type=batting;view=innings;wrappertype=print

I will be grateful to receive an immediate response.
 
Hi All,

I am new here and I have similar query, if someone can help on below matter:

I need to extract historical indices data from money control URL :

[TABLE="width: 73"]
<tbody>[TR]
[TD="class: xl65, width: 73"]Historic prices of stocks and indices[/TD]
[/TR]
</tbody>[/TABLE]

Here My URL is having dynamic fields and its values are given in Sheet 1 as below: Indices Code (given in cell B1), Frequency (given in cell B2), From Date (given in cell B3), To Date (given in cell B4) and Page Number (this value dependent on number of records).

I want to download web query data tables in A1 cell of Sheet 2 till end of records. It would be great help if someone can provide VBA Macro Code for these.

If this macro takes page number data from 1 to till end of data than its excellent else I can put a formulae in cell B5 to calculate end page number based on difference of number of days between from date and to date and number of records on a page...

Need urgent help as I am stuck while preparing an automated calculator... Thanks in Advance.....
 
Upvote 0

Excel Facts

What is the fastest way to copy a formula?
If A2:A50000 contain data. Enter a formula in B2. Select B2. Double-click the Fill Handle and Excel will shoot the formula down to B50000.
Hi,

I am new to this forum and have registered today itself.

I also have similar query in excel ...
Read the solution provided by many experts but still not able to adjust that solution for my problem...

Experts please help me also.

Problem is described as under:

Need to extract all 55,099 tables from different html pages...

Link of page 1 is https://www.zaubacorp.com/company-list/p-1-company.html

Then have to click on next page for another table.....

Need this data for analyzing companies trend...

Can anyone help me urgently.. .Need to submit this analysis by Monday...

Regards
Akash Garg
 
Upvote 0
I have a similar question regarding web query tables. I would like to scrape the information from the table in the link below. However, I am unable to create the code to automatically move to the next table as there are no page numbers. Any help on this matter is greatly appreciated.

Player Game Finder | Basketball-Reference.com

Regards,
Dustin
 
Upvote 0
I've tried this on a particular link but no luck. the sites aren't technically in data table format, but if I do it manually it dumps into excel in a recognisable syntax, which is enough for me to work with the data myself once it's in excel.

https://dubai.storat.com/motors/cars-for-sale?page=3

the only issue is a couple of errors I get when I run your script

It shows up many errors with above mention link like format errors or check data

Could anyone please me in this query

Many thanks
 
Upvote 0
Hi All

I am trying to do a web query to obtain data from a table split across multiple web pages and then insert the data onto one sheet in Excel.

I have been unable to get this to work.

I get the following error on the Refresh.BackgroundQuery=False line - Run-time error '1004': Application-defined or object-defined error

The connection seems to be established in Excel and when I click on the query and then on edit I can see the data but it's not loaded into the sheet and the loop does not run

Below is my current effort

Your help will be highly appreciated!

Thanks


Sub WebQuery1()
'
' WebQuery1 Macro
'


'
Dim p As Integer
Dim r As Integer
Dim l As Integer
Dim t As Integer
Dim WebString As String
Dim WebString2 As String
Dim WebSource As String


r = 1
l = 1
t = 2




For p = 1 To 352


WebString = "https://www.website.ashx?v=111&r=" & r
WebString2 = Chr(34) & WebString & Chr(34)
WebSource = "Source = Web.Page(Web.Contents(" & WebString2 & ")),"



ActiveWorkbook.Queries.Add Name:="Table" & t, Formula:= _
"let" & Chr(13) & "" & Chr(10) & WebSource & Chr(13) & "" & Chr(10) & " Data2 = Source{2}[Data]," & Chr(13) & "" & Chr(10) & " #""Promoted Headers"" = Table.PromoteHeaders(Data2, [PromoteAllScalars=true])," & Chr(13) & "" & Chr(10) & " #""Changed Type"" = Table.TransformColumnTypes(#""Promoted Headers"",{{""No."", Int64.Type}, {""Ticker"", type text}, {""Company"", type text}, {""Sector" & _
""", type text}, {""Industry"", type text}, {""Country"", type text}, {""Market Cap"", type text}, {""P/E"", type text}, {""Price"", type number}, {""Change"", Percentage.Type}, {""Volume"", Int64.Type}})" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & " #""Changed Type"""


With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=""Table "" & t;Extended Properties=""""" _
, Destination:=Range("$A$" & l)).QueryTable
.CommandType = xlCmdSql
.CommandText = Array("SELECT * FROM [Table" & t & "]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = False
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = "Table_" & t
.Refresh BackgroundQuery:=False

End With

r = r + 20
t = t + 1
l = l + 21


Next p
End Sub
 
Upvote 0
Hello all,

I'm trying to do something similar, but I think a little more complicated as the data that I want to save doesn't have exactly the shape of a table.

The website is: https://motos.coches.net/ocasion/barcelona/

The data I have to save from each line is: tittle, km, year, cc, location and date

And I need this to be repeated as much pages as there are in the website. Sorry if you have already answered something like this, I tried with the examples on this forum but wasn't successful. Thanks a lot if somebody can help!
 
Upvote 0

Forum statistics

Threads
1,223,238
Messages
6,170,939
Members
452,368
Latest member
jayp2104

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top