Message Boards Message Boards

0
|
2691 Views
|
2 Replies
|
0 Total Likes
View groups...
Share
Share this post:

"FileNames" fails with large files number in Dropbox Service Connection

Posted 3 years ago

I'm building a workflow that needs to process lots of files that I would like to keep on Dropbox. Fortunately, Wolfram/Mathematica appear to support Dropbox, but I've run into a problem (I have submitted a bug report). If there are too many files in your Dropbox folder, using "FileNames" to get the list of file names fails. To determine how many is too many, I wrote a reproducible (at least for me) test case:

dropbox = ServiceConnect["Dropbox"];
testFolder = "/testfilenames";
maxTries = 1000;

Quiet[ServiceExecute[dropbox, 
   "DeleteDirectory", {"Path" -> testFolder}]];
ServiceExecute[dropbox, "CreateDirectory", {"Path" -> testFolder}];

Module[
  {
   test,
   successes
   },

  test[i_Integer] := Module[
    {
     filename,
     path,
     files
     },
    Print[i];
    filename = "test " <> ToString[i];
    path = FileNameJoin[{testFolder, filename}];
    ServiceExecute[dropbox, 
     "Put", {"Path" -> path, "Expression" -> filename}];
    files = 
     ServiceExecute[dropbox, "FileNames", {"Path" -> testFolder}];
    i == Length[files]
    ];

  successes = Catch[

    Scan[
     If[! test[#], Throw[#]] &,
     Range[1, maxTries]
     ];

    Throw[maxTries];
    ];
  ];

ServiceDisconnect[dropbox];

For me, the above code dies after 500 files.

Before I invest too much more time with Dropbox, I would like to know if it is working for others. Or, if I can expect even more frustration.

Thanks, and have a great and safe week.

POSTED BY: Mike Besso
2 Replies

Something is wrong with FileNames - I’m getting problem listing large number of files even for local folders: https://community.wolfram.com/groups/-/m/t/3140952

POSTED BY: Victor K
Posted 3 years ago

Just to follow up. Tech Support has confirmed that this is a bug that they are able to reproduce.

They are considering a fix for future versions of Mathematica. In the meantime, I will have to implement a costly work-around.

If this impacts you as well, please let Tech Support know so that they can properly prioritize the fix.

POSTED BY: Mike Besso
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract