web scraping help

dwatters

New Member
Joined
Jan 16, 2024
Messages
1
Office Version
  1. 2016
Platform
  1. Windows
i have an old macro that i use alot to pull information off of multiple sites. it was originally setup to use IE but now that site no longer loads correctly in IE. im not really sure how to go about moving it to another browser due to what i have been reading about other browsers not supporting the DOM. im pretty new to writing macros and the original maker is long gone.

i have tried to setup winhttp and have that atleast trying to access the url but the site needs my security token that i have to use once a day to login

this is a part of the code

'Gets the info from AM Console
Function ImportAmConsoleData(url) As String()
'Open and hide Internet Explorer
Dim appIE As InternetExplorerMedium
Dim objElement As HTMLDocument
Set appIE = New InternetExplorerMedium
With appIE
.Navigate url
.Visible = True
End With

Do
DoEvents
'If there was an error, the website was not done loading. return here on error and try again
On Error Resume Next

Set objElement = appIE.Document
Dim bodyText As String
bodyText = objElement.body.innerText

'Give AM console a bit of time before checking again
Sleep 50

'check to see if document as loaded completly with all required info.
'cannot check ready state, I believe this is because AM Console was written with Angular and will
'set the ready state to complete even though it is still waiting for info from the server.
hasName = InStr(bodyText, FIRST_NAME_SEARH) > 0
hasID = InStr(bodyText, ID_SEARCH) > 0
hasPhone = InStr(bodyText, PHONE_SEARCH) > 0
hasEmail = InStr(bodyText, EMAIL_SEARCH) > 0
hasDTAndBGC = InStr(bodyText, DRUG_TEST_SEARCH) > 0 And InStr(bodyText, BGC_COMPLETE_SEARCH) > 0
hasEverything = hasName And hasID And hasPhone And hasEmail And hasDTAndBGC


'add a dot to the elipsis to show the program is running
UpdateRunning

On Error GoTo 0
Loop Until hasEverything

'reset and close IE
appIE.Quit
Set appIE = Nothing

Dim dataArray(0 To 6) As String
dataArray(0) = findInfo(bodyText, FIRST_NAME_SEARH)
dataArray(1) = findInfo(bodyText, LAST_NAME_SEARCH)
dataArray(2) = findInfo(bodyText, ID_SEARCH)
dataArray(4) = findInfo(bodyText, PHONE_SEARCH)
dataArray(5) = findInfo(bodyText, EMAIL_SEARCH)

'start looking where onboarding tasks are located as to not pull from the dictionary
OnboardingStart = InStr(bodyText, ONBOARDING_SEARCH)
dataArray(6) = findInfo(bodyText, BGC_COMPLETE_SEARCH, OnboardingStart, 2) & "/" & findInfo(bodyText, DRUG_TEST_SEARCH, OnboardingStart, 2)

ImportAmConsoleData = dataArray
End Function
 

Excel Facts

Which came first: VisiCalc or Lotus 1-2-3?
Dan Bricklin and Bob Frankston debuted VisiCalc in 1979 as a Visible Calculator. Lotus 1-2-3 debuted in the early 1980's, from Mitch Kapor.

Forum statistics

Threads
1,223,893
Messages
6,175,240
Members
452,621
Latest member
Laura_PinksBTHFT

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top