Message Boards Message Boards

How can I get the list of files in a remote directory?

Posted 9 years ago

Hi everyone,

Is there a Mathematica function for listing the files in a remote, public directory just like an FTP app does? I tried FileNames as in

FileNames[ftp://ftp.host.stuff/folder1/folder2/]

but it does not work.

POSTED BY: Gregory Lypny
2 Replies
Posted 9 years ago

Hi Jesse,

Thanks for the tip! I'm going to give this a whirl. It will be a good opportunity for me to learn about shell processes.

Regards,

Gregory

POSTED BY: Gregory Lypny

Unfortunately, I don't believe there's a built-in Wolfram Language function that can list the contents of a remote directory. But if you have a Unix-based machine, (Linux, BSD, Mac, etc.) you can use this function I wrote:

ftpFileNames[host_, path_: "/", type_: All] := Block[{matrix},
  matrix = 
   StringSplit[
    StringSplit[RunProcess[{"ftp", "-n"}, "StandardOutput",
      "open " <> host "
        user anonymous anonymous
        ls " <> path <> "
        bye"], "\n"], " " ..];
  Select[matrix, 
    StringMatchQ[StringTake[#[[1]], 1], 
      Switch[type, All, "d" | "-", "File", "-", "Directory", 
       "d"]] &][[All, -1]]
  ]

In[1]:= ftpFileNames["ftp.ncbi.nih.gov","/"]
Out[1]= {"1000genomes", "100GB", "10GB", "1GB", "50GB", "5GB", "README.ftp",
"bigwig", "bioproject", "biosample", "blast", "cgap", "clemson",
"cn3d", "dbgap", "entrez", "epigenomics", "eqtl", "fa2htgs",
"favicon.ico", "genbank", "gene", "genomes", "geo", "giab", "hapmap",
"mmdb", "ncbi-asn1", "pathogen", "pub", "pubchem", "pubmed",
"refseq", "repository", "robots.txt", "sequin", "sky-cgh", "snp",
"sra", "tech-reports", "toolbox", "tpa", "variation"}

By default it lists both files and directories, but you can pass "File" or "Directory" as a third argument to limit the results. I hope this helps!

POSTED BY: Jesse Friedman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract