How can I send keyboard commands via Mathematica to a given URL?

Questions
1

I am trying to extract job listings from a website that presents many hundreds of jobs available; this listing is limited to 10 per page and it is possible to step through the long list with keyboard commands such as ‘Alt+Shift+P’. I have tried the Import[url,“Data”] which only returns the 10 listing on the currently supplied webpage. Would like to be able to send commands under Mathematica control to the website to step through this longer list and retrieve the entire listing of jobs. The attachment is a relevant snippet of the page HTML. I apologize if this is too off-topic for this discussion group; not really sure which group is the ‘correct’ one.

2

Unfortunately, you can’t send keyboard commands because Mathematica is only able to fetch the raw contents of the page. Would you be able to post a link to the website? It’s possible there’s some backend XMLHttpRequest API that’s fetching data every time you hit Alt+Shift+P.

And don’t worry, it’s not off-topic at all!

3

@Jessie Friedman – Thanks for your reply; sorry to take so long to get back to you.
Here is the link to the web page which exhibits the behaviour I described. In the page source you can find the references to the key commands:

Again thanks for your response! K
4

No problem about the delay. It looks like you didn’t post the right text, though.

5

Right!
Here is the url to the webpage in question:
https://re11.ultipro.com/KIM1000/JobBoard/ListJobs.aspx?__VT=ExtCan

6

We can append the URL parameter __RecordsPerPage to the URL to get all the jobs on one page: https://re11.ultipro.com/KIM1000/JobBoard/ListJobs.aspx?__VT=ExtCan&__RecordsPerPage=1000

And here we can make a Dataset of all the jobs:

maxrecords = 1000;
pagedata = 
  Import["https://re11.ultipro.com/KIM1000/JobBoard/ListJobs.aspx?__VT=ExtCan&__RecordsPerPage=" <>
    ToString[maxrecords], "Data"][[1, 2]];
jobs = Dataset[AssociationThread[pagedata[[1]], #] & /@ pagedata[[2 ;;]]]

jobsdataset.png
jobsdataset.png811×456 61.1 KB

If a given page has more than 1000 records, just change maxrecords to something like 10000.

7

Thank you very much Jesse.
Back to work on this after a prolonged hiatus.
I really appreciate your time and expertise on this effort.
coach