Message Boards Message Boards

GROUPS:

Use WebUnit with ParallelTable?

Posted 1 year ago
2283 Views
|
2 Replies
|
0 Total Likes
|

I have already asked this question on MMA.SE and was suggested to re-post it here. Also, that's my first post here so please feel free to edit/suggest anything such as more relevant tags/groups.

Is it possible to use OpenWebPage from WebUnit in ParallelTable? For example such as in:

Needs["WebUnit`"]
InstallWebUnit[]
StartWebSession[];

pages = {"www.wolfram.com", ... }
ParallelTable[
  OpenWebPage[pages[[i]]];
  html = GetPageHtml[];
  , {i, Length@pages}
  ]

I can start multiple browsing windows by running several StartWebSession[] but then how to dedicate a single window to each core?

2 Replies

You probably need to put the first three line inside a ParallelEvaluate (so every kernel has its own WebUnit loaded).

(I haven't tried this though)

Posted 1 year ago

I tried and it does work. However (i) it creates as many windows as iterations, which could be large, (ii) it is time consuming and thus not advantageous if the time spent on each webpage is small. Apart from that, thank you for this wonderful tool!

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract