Thank you, Rohit for your reply and I truly appreciate it! I have a few follow-up questions if you wouldn't mind.

I see that your function "data = AssociationMap[importer, cleanproxy[[1;;3]]]" sets the URLs equal to the importers of the URLs. That is, it seems that "data" is now a string containing URL -> Importer[URL]. However, when I run "data" alone, I was expecting to see a text because each URL eventually takes me to a website with texts. WM did not import the URLs in the text format.

For example, when I write: process[url, links] := (y2019 = Length[Position[data, "2019"]];) where I don't touch "url" or "links", the function after := doesn't seem to apply to the texts for each of the hyperlinks but to the output for "data" which is URL ->importer[URL]

I hope I am making sense, but I just wanted to find a way for WM to click on each URL and import the text of each URL and run the functions inside the process.

Please let me know I am unclear on any of my questions. Thank you once again for your help! Below is the code I have:

importfile = Drop[Import["proxy2020sample.xlsx"][[1, All, 4]], 1];
importfile;
cleanproxy = Flatten@importfile;



Cases[Import[cleanproxy[[1]], "Hyperlinks"], 
  StringContainsQ["https://www.sec.gov/"]];
importURL[url_] := 
  Select[Import[url, "Hyperlinks"], 
   StringContainsQ["https://www.sec.gov/"]];
data = AssociationMap[importer, cleanproxy];
data = AssociationMap[importer, cleanproxy[[1 ;; 3]]];


process[Hyperlinks_, importer_] := (data1 = Import[data, "Plaintext"]);

KeyValueMap[process, data]

Hi Young,

Separating data gathering from data processing is a good practice to follow. Gather the data, save it, then feed it to your processing code. That way if you discover an issue with the processing code or want to change the processing algorithm you do not have to re-download the data.

If cleanproxy is a List of URL's then

importURL[url_] := Select[Import[url, "Hyperlinks"], StringContainsQ["https://www.sec.gov/"]]

data = AssociationMap[importer, cleanproxy]

The result is an Association from each URL to a list of the filtered hyperlinks. Try it on a small sample first

data = AssociationMap[importer, cleanproxy[[1;;3]]]

You may want to save the data at this point. Take a look at DumpSave in the documentation.

Then write a function containing the processing code. The function takes two arguments, the URL and the list of matching hyperlinks. Do whatever you need to do in the function. The last statement in the function should evaluate to the result.

process[url_, links_] := (* do something *)

Then

KeyValueMap[process, data]