Deconding nested ASPX website data into Excel (plus a little javascript)

[gordonpsmith]

I'm trying to pull some data from KY education websites (publicly available, starting at https://applications.education.ky.gov/SRC/Default.aspx). I've done programs before parsing website data, but at an HTML level; ASPX is a new beast for me, and so I'm in learning mode.

What I'm wanting to do is (admittedly inefficiently) copy various response website data that I can then use my standard tricks to parse out what's necessary. As if dealing with ASPX files aren't new enough to me, I'm dealing with nested ASPX files which opens this up to new levels of hell.

My desired automated navigation is:

• Start with the main page, and define the Year and District (iteratively, I want it all). Programmatically click "View Card" for the District Report Card(.aspx-based).
• Within the resulting District/Year page, select the Assessment Tab (.aspx-based), which defaults to a K-PREP subtab
• Select all the data and paste it into a worksheet (this gets it into a form where I can pull the parts I need)
• Select the K-PREP End-of-Year subtab (.aspx-based)
• Select all the data, and paste into another worksheet (or the same, after I've parsed the first pass and am done)
• Select the "Learning Environment" Tab (.aspx-based), which defaults to a Students subtab
• Select all the data and paste
• Select the "Teachers" subtab, select all the data and copy/paste
• Select the "Finance" tab, which defaults to "Financial Summary"
• Select the "Revenues and Expenditures" subtab
• Expand all the category data (which might be javascript?) so all the hidden data is visible
• Select all the data and copy/paste

Clearly, this is a lot, and my ultimate goal is to create a spreadsheet with each row being a particular school, and the columns being select subsections of the data on each of these page. It's 170ish districts, and several years worth of data, and that's for one state alone - I really need to automate this. I'm not asking for the whole code that does this (though I certainly wouldn't turn it down ), but I need some guidance on how to interact with .aspx files, and nested ones at that...

Thanks!