VBA Macro Unable to scrape all Tables from webpage

ppnar007

New Member
Joined
Nov 22, 2022
Messages
10
Office Version
  1. 2016
Platform
  1. Windows
I am trying to fetch all web tables but I am not able to fetch them. I have tried same program for other website and its working but for this particular website its not working at all.

VBA Code:
Sub Export_HTML_Table_To_Excel()
    Dim htm As Object
    Dim Tr As Object
    Dim Td As Object
    Dim Tab1 As Object

    'Replace the URL of the webpage that you want to download
     Web_URL = "https://www.indiaratings.co.in/pressrelease/60901"
   
    'Create HTMLFile Object
    Set HTML_Content = CreateObject("htmlfile")

    'Get the WebPage Content to HTMLFile Object
    With CreateObject("msxml2.xmlhttp")
        .Open "GET", Web_URL, False
        .send
        HTML_Content.body.innerHTML = .responseText
    End With

    Column_Num_To_Start = 1
    iRow = 2
    iCol = Column_Num_To_Start
    iTable = 0

    'Loop Through Each Table and Download it to Excel in Proper Format
    For Each Tab1 In HTML_Content.getElementsByTagName("table")
        With HTML_Content.getElementsByTagName("table")(iTable)
            For Each Tr In .Rows
                For Each Td In Tr.Cells
                    Worksheets("ABC").Cells(iRow, iCol).Select
                    Worksheets("ABC").Cells(iRow, iCol) = Td.innerText
                    iCol = iCol + 1
                Next Td
                iCol = Column_Num_To_Start
                iRow = iRow + 1
            Next Tr
        End With
        iTable = iTable + 1
        iCol = Column_Num_To_Start
        iRow = iRow + 1
    Next Tab1

    MsgBox "Process Completed"
End Sub
 

Excel Facts

Why does 9 mean SUM in SUBTOTAL?
It is because Sum is the 9th alphabetically in Average, Count, CountA, Max, Min, Product, StDev.S, StDev.P, Sum, VAR.S, VAR.P.
If Tab1 is already an HTML object, no need for With HTML_Content.getElementsByTagName("table")(iTable)
on:
VBA Code:
    For Each Tab1 In HTML_Content.getElementsByTagName("table")
        With HTML_Content.getElementsByTagName("table")(iTable)

would be:
VBA Code:
    For Each Tab1 In HTML_Content.getElementsByTagName("table")
        With Tab1
 
Upvote 0
If Tab1 is already an HTML object, no need for With HTML_Content.getElementsByTagName("table")(iTable)
on:
VBA Code:
    For Each Tab1 In HTML_Content.getElementsByTagName("table")
        With HTML_Content.getElementsByTagName("table")(iTable)

would be:
VBA Code:
    For Each Tab1 In HTML_Content.getElementsByTagName("table")
        With Tab1

Not working at all, still I am unable to fetch tables
 
Upvote 0
Not working at all, still I am unable to fetch tables
If you print to file the downloaded HTML_Content.body.innerhtml, you will get only one element in <body>, this element: "<app-root>Loading...</app-root>"... which means it's loading something else. No matter if you force a wait, the element never gets loaded on the CreateObject("msxml2.xmlhttp").

If you go through the browser instead (overriding the VBA), there is a status=400 response in the download (which means Error), but you get what else it's already downloading. One of the downloads is the target you are after.

So the rigth URL should be "https://www.indiaratings.co.in/pres...0901&uniqueIdentifier=##.###.###.###-20230210". Where the ##.###.###.### stands for your IP address (although you can fake it with other numbers), followed by the date of the query. With this URL you get a JSON response with all the tables embedded. Without the &uniqueIdentifier= you will get nothing, as it will be stalled on the status=400 error, with some error when validating.

Done.
 
Upvote 0

Attachments

  • Capture1.JPG
    Capture1.JPG
    114.1 KB · Views: 13
Upvote 0
As returns comes in JSON, you should parse content with VBA-JSON by Tim Hall, a JSON converter for VBA, or something similar.
 
Upvote 0
As returns comes in JSON, you should parse content with VBA-JSON by Tim Hall, a JSON converter for VBA, or something similar.
I donno how to do it ...

Original Link is this :-

All Tables to be taken from this link:

I have tried but no success , can you tell me how to fetch all tables ?
 
Upvote 0

Forum statistics

Threads
1,224,823
Messages
6,181,177
Members
453,021
Latest member
Justyna P

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top