Web Scraping With VBA Using Selenium Reference Library

EnginerdUNH

New Member
Joined
Aug 14, 2019
Messages
2
Office Version
  1. 365
Platform
  1. Windows
I have been working VBA to scrape website HTML code using the Microsoft HTML Object Library and the Microsoft Internet Controls Library. Now that support for Internet Explorer is being phased out, I am trying to switch my code over to scrape Google Chrome using the Selenium Type Library which is an open source download. The website I am trying to pull data from into excel has the following line of HTML code that I am interested in assigning to a variable:
HTML:
<div jsname="iXWWee" class="enWFYd KDN9Hf" style="left: 39px; display: block; transform: translate3d(85px, 0px, 0px);">Feb 4, 2021</div>
I have been able to successfully assign a value to a variable using the following lines of code:
VBA Code:
Private ch As Selenium.ChromeDriver
Sub Test()
 Set ch = New Selenium.ChromeDriver
 
 ch.Start , "https://www.google.com"
 ch.Get "https://www.google.com/search?q=us+vaccine+tracker&oq=us+&aqs=edge.0.69i59l2j69i57j69i60l4.2010j0j1&sourceid=chrome&ie=UTF-8"
 
 Dim Element As Selenium.WebElement
 Set Element = ch.FindElementByCss("div[class='enWFYd KDN9Hf']")
 Debug.Print Element.Attribute("innerHTML")
End Sub
But what you will notice is that the the HTML code I provided has innerHTML which changes based on where your cursor is on the screen. The code above works but it only pulls the last instance where the innerHTML value is "Aug 3, 2021". What I want is something that will pull all of the data and not just the last instance. I tried to test it by changing the second to last line of code to Set Element = ch.FindElementByCss("div[class='enWFYd KDN9Hf'] [innerHTML='Feb 4, 2021']") but when I run the subroutine, I get an error saying that this element can't be found. I have also tried changing the last three lines as follows but it still is only pulling the last instance of this div tag.
VBA Code:
[LIST=1]
[*]Dim Elements As Selenium.WebElements
[*]Set Elements = ch.FindElementsByCss("div[class='enWFYd KDN9Hf']")
[*]For Each Element in Elements
[*]Debug.Print Element.Attribute("innerHTML")
[*]Next Element
[/LIST]
 

Excel Facts

Do you hate GETPIVOTDATA?
Prevent GETPIVOTDATA. Select inside a PivotTable. In the Analyze tab of the ribbon, open the dropown next to Options and turn it off
A gentle reminder:

Cross-posting (posting the same question in more than one forum) is not against our rules, but the method of doing so is covered by #13 of the Forum Rules.

Be sure to follow & read the link at the end of the rule too!

Cross posted at: Web Scraping With VBA Using Selenium Reference Library - OzGrid Free Excel/VBA Help Forum
If you have posted the question at more places, please provide links to those as well.

If you do cross-post in the future and also provide links, then there shouldn’t be a problem.
 
Upvote 0

Forum statistics

Threads
1,224,816
Messages
6,181,141
Members
453,021
Latest member
Justyna P

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top