Message Boards Message Boards

0
|
6828 Views
|
4 Replies
|
2 Total Likes
View groups...
Share
Share this post:

Invisible files in Cloud Access Folders

Posted 3 years ago

I uploaded jpeg files Kx1, Kx2 and Kx3 to the Test folder in Cloud Access. I defined the Directory.

SetDirectory[NotebookDirectory[]]

Then I exported some test files from the Notebook: x1.txt, x2.jpg and Xx2.jpg into the Notebook Directory .

Only the uploaded- and Notebook files where visible in the directory of interest! View of directory-test

But al the data files are present when I use FileNames[]

In[14]:= FileNames[]

Out[14]= {FaceInterpreter.nb,.FaceInterpreter.nb.lock,Kx1.jpg,Kx2.jpg,Kx3.jpg,Resources,x1.txt,x2.jpg,Xx2.jpg}

Is there some explanation for this difference?

POSTED BY: Chris Van Damme
4 Replies

Some extra information. The Kx3 file is quite big, 2.5MB. I can open this file in the Cloud explorer (3456x3456 pix).

In[20]:= FileByteCount["Kx3.jpg"]    
Out[20]= 2554372

The information of the session limits and "successful" import:

In[21]:= KeyTake[CloudAccountData[], {"SessionEvaluationTimeLimit","SessionMemoryLimit"}]  
Out[21]= \[LeftAssociation]SessionEvaluationTimeLimit->TemplateBox[{"300", "\"s\"", "seconds", "\"Seconds\""}, "Quantity", SyntaxForm -> Mod],SessionMemoryLimit->TemplateBox[{"1024", "\"MB\"", "megabytes", "\"Megabytes\""}, "Quantity", SyntaxForm -> Mod]\[RightAssociation]

In[22]:= kx3=CloudImport["Kx3.jpg"]; 

But then I tried to open the image file, again session fail:

enter image description here

I reduced one of the files (640x640) on the Pc and uploaded it into Cloud explorer (148350B). This could be successful processed by CloudImport. So is there a possibility to know where this limit is?

POSTED BY: Chris Van Damme
Posted 3 years ago

If a CloudImport is making the cloud notebook environment unresponsive like that, the main thing I think is that the kernel died, and it's possibly due to exceeding either time or memory constraints.

I'm trying to replicate your steps with a 3MB jpg and it's taking a while. Have a look at the output of KeyTake[CloudAccountData[], {"SessionEvaluationTimeLimit","SessionMemoryLimit"}] to see what those limits are for your cloud notebook session.

If this is what's killing your kernel (and if that's what's making it a strange situation requiring restart session), you could try the CloudImport but suppressing the output:

result = CloudImport["Kx3.jpg"];

If you needed to import the jpg to do some computation on it, and not to view it in the notebook, this might help.

POSTED BY: Joel Klein

Hello Joel, Thanks for putting me on the right way. It's clear for me now that there is a distinction between CloudObjects and Files.

Still working in Cloud Access: I can do a CloudExport of an image and find the information in the cloud file explorer, in CloudObjects and FileNames. I can do a CloudImport of this file with success.

But I can't do a CloudImport of "Kx3.jpg" , for exemple, (see previous post) who is uploaded from cloud file explorer and visible in the CloudObjects.

{CloudObject[
 "https://www.wolframcloud.com/obj/christiaan.van.damme/Test/\
FaceInterpreter(Copy).nb"], CloudObject[
 "https://www.wolframcloud.com/obj/christiaan.van.damme/Test/\
FaceInterpreter.nb"], CloudObject[
 "https://www.wolframcloud.com/obj/christiaan.van.damme/Test/Kx1.jpg"]\
, CloudObject[
 "https://www.wolframcloud.com/obj/christiaan.van.damme/Test/Kx2.jpg"]\
, CloudObject[
 "https://www.wolframcloud.com/obj/christiaan.van.damme/Test/Kx3.jpg"]
 } 

The CloudImport stopped without result, no error. Strange situation (see In(*) ) who need a Restart Session!

result

The same for the other jpg's. In fact this problem was the start for my question about the visibility of files.

POSTED BY: Chris Van Damme
Posted 3 years ago

This is just how cloud works -- there are cloud objects, and there are plain files on the cloud file system.

The ways to create cloud objects are with the WL functions like CloudDeploy, CloudExport, CloudPut, or through the cloud web UI, like creating a new notebook or uploading files.

The ways to view cloud objects are with the WL functions like CloudObjects[] or through the cloud file explorer in the cloud web UI.

If you use WL functions to create ordinary files on the cloud file system, they are only visible through WL functions like FileNames.

What makes a cloud object a cloud object is it is registered somewhere with information like its MIME type, the thing that tells the cloud what kind of thing it is so we know what to do when that object is requested (notebooks are handle differently from image files are handled differently from APIFunctions, etc.). While most cloud objects do have a file on the file system, it's that "registered somewhere" that makes them visible to people using the cloud as a web server.

In your case, if you used CloudExport instead of Export, you'd see those exported files in your cloud file browser.

This dichotomy of cloud objects vs plain files is confusing, but I hope this explanation helps a bit. Please let me know if you have further questions.

POSTED BY: Joel Klein
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract