Community RSS Feed
https://community.wolfram.com
RSS Feed for Wolfram Community showing any discussions in tag Import and Export sorted by activeAutomate converting an imported excel spreadsheet to a dataset?
https://community.wolfram.com/groups/-/m/t/1566323
I would like to import a large excel spreadsheet with many columns into Mathematica. The first row of the spreadsheet contains the column headings and the remaining rows contain the data. After importing I first separate the column headings and data into separate variables and would like to convert the data to a dataset. Here is an example of what I have tried.
These steps simulate of importing the spreadsheet and separating into columns and data
cols = {"col1", "col2", "col3", "col4", "col5"};
dat = RandomReal[{0, 1}, {10, 5}];
I now need to create an association to use to map the data to the columns but here is where I am having trouble. The best I am able to come up with is to generate
assoc = Table[cols[[k]] -> #[[k]], {k, 1, 5}]
This results in some error messages, but produces
{"col1" -> 1, "col2" -> #1[[2]], "col3" -> #1[[3]], "col4" -> #1[[4]],
"col5" -> #1[[5]]}
which is nearly the result desired. I can then copy that result, paste it into a new cell, and edit that cell by hand, changing the first element of the list, applying an association function, and applying a function to get the desired function to apply to the data
assoc1 = Association[{"col1" -> #[[1]], "col2" -> #1[[2]],
"col3" -> #1[[3]], "col4" -> #1[[4]], "col5" -> #1[[5]]}] &
Map[assoc1,dat]
With this association I can then generate the dataset. Is there any way of generating this function without the necessity of the hand editing step?Mike Luntz2018-12-05T14:55:47ZImport large data files for Machine Learning?
https://community.wolfram.com/groups/-/m/t/1599142
I want to run a machine learning task on my Win 10 PC, 16GB RAM, Mathematica 11.3.0, but I am facing the following problems: training set size 10GB CSV file, with 700,000,000 x 2 datasets. Mathematica simply stops during import via Import or ReadList function. My idea is to split the input file into several smaller files that could be imported and to load the smaller files in a batch to feed the Predict function or perhabs a neural network. Any idea how to make it happen? Do you have a better idea?
Many thanks in advance for support!Jürgen Kanz2019-01-26T11:41:17ZImport .dae 3D format files without ignoring nodes?
https://community.wolfram.com/groups/-/m/t/1633798
Collada .dae 3D format is in fact xml document which contains <library_geometries> section with some "meshes", where the coordinates and normals of surface meshes points are provided.
But that coordinates are not refereed to basic coordinate system. DAE file contains an other, <library_visual_scenes> section, which holds a tree of nested lists of "nodes" - transformation matrices describing coordinate transformation of the nested node from the parent one. And only the last, leaf node contains reference to some mesh name, so the mesh coordinate system is given by set of coordinate transformation. A lot of 3D modelling soft exports a set of model "parts" in very such way.
Looks like Mathematica Import[] function do not take into account nodes section, so imported complex models have details teared apart and grouped near the center of coordinates. Import[SomeModel, "CoordinateTransform"] provides a single and unit transformation matrix not changes at all on change of "nodes" matrices
1. Have tried to import some .dae models. Import[SomeModel, "MeshRegion"] and Import[SomeModel, "GraphicsComplex"] both give the same teared apart result.
2. Coded some handmade parser, which extracts meshes from <library_geometries> and provides the resulting set of 3D surfaces without additional transformations - completely the same "teared" result as in stage 1.
3. Coded additional parser, which extracts a set of transformation matrices for every mesh, and applied a result to it - valid 3D model with the parts on proper places.
ps: it would be very convenient if Import[] did not "melts" all meshes in a single one, but provides them as a set of objects together with their transformation martices, or have a option for this.Snegirev Maksym2019-03-16T09:40:04Z[Numberphile] - The Square-Sum Problem
https://community.wolfram.com/groups/-/m/t/1264240
As part of my Numberphile series of posts:
- [\[Numberphile\] - Frog Jumping - Solving the puzzle][1]
- [\[Numberphile\] - The Illumination Problem][2]
- [\[Numberphile\] - Sandpiles - Done in the Wolfram Language][3]
here is another one about a recent video called [The Square-Sum Problem][4]
[![enter image description here][5]][6]
The question is: if you have the integers 1 through n, can you arrange that list in such a way that every two adjacent ones sum to a square number. As seen in the video and the [extra footage][7].
We can easily check this in the Wolfram Language:
Let's see which number can pair up to a pair:
SquareEdges[n_Integer?Positive]:=Reap[Do[If[IntegerQ[Sqrt[i+j]],Sow[{i,j}]],{i,n-1},{j,i+1,n}]][[2,1]]
Now let's try for 15, as in the main video:
n = 15;
poss = SquareEdges[n];
gr = Graph[TwoWayRule @@@ poss, VertexLabels -> Automatic];
path = FindHamiltonianPath[gr, PerformanceGoal :> "Speed"]
HighlightGraph[gr, BlockMap[Rule @@ # &, path, 2, 1]]
giving:
{9, 7, 2, 14, 11, 5, 4, 12, 13, 3, 6, 10, 15, 1, 8}
![enter image description here][8]
In the extra footage, it is revealed that they found the solution for up to n=299. Can we do better? Yes we can! Changing n to 300 in the above code and rerunning gives us the solution in 0.28 sec on my laptop.
{289,35,65,259,30,294,67,257,32,292,69,100,44,125,71,154,135,189,211,113,248,8,281,119,205,195,166,158,283,6,250,191,133,156,285,4,252,277,12,244,117,207,193,168,273,16,240,160,164,236,20,269,131,94,230,59,197,92,232,57,199,90,234,22,267,217,224,137,152,73,123,46,150,75,121,48,148,77,179,110,214,270,19,237,163,161,239,17,272,128,41,103,297,27,262,62,227,97,99,190,210,114,175,50,146,79,177,112,212,188,253,3,286,155,134,266,23,233,91,198,58,231,93,196,60,229,95,130,159,165,276,13,243,118,206,194,167,274,15,241,288,1,255,186,138,223,218,143,181,108,88,201,55,170,86,203,53,172,84,37,107,182,142,299,25,264,220,221,140,184,216,225,64,36,85,171,54,202,87,169,56,200,89,235,21,268,132,157,284,5,251,278,11,245,116,208,192,249,7,282,247,9,280,204,120,136,153,72,124,45,151,74,122,47,149,76,180,109,215,185,139,222,219,265,24,300,141,183,106,38,83,173,52,144,81,40,104,296,28,261,63,226,98,127,42,102,298,26,263,61,228,96,129,271,18,238,162,279,10,246,115,209,275,14,242,287,2,254,187,213,111,178,78,147,49,176,80,145,51,174,82,39,105,295,29,260,101,43,126,70,291,33,256,68,293,31,258,66,34,290}
and a completely mess of a graph:
![enter image description here][9]
Can we go beyond? Let's optimize a code a bit, and let's find the solution for larger n:
Let's store our intermediate results in the association **db**:
SetDirectory[NotebookDirectory[]];
$HistoryLength=1;
db=If[FileExistsQ["squaresumdb.mx"],
Import["squaresumdb.mx"]
,
<||>
];
And now our main code:
ClearAll[SquareEdges,SquareEdges2,CheckSol,TryToFind]
SquareEdges[n_Integer?Positive]:=Reap[Do[If[IntegerQ[Sqrt[i+j]],Sow[{i,j}]],{i,n-1},{j,i+1,n}]][[2,1]]
SquareEdges2[n_Integer]:=Module[{tmp},
tmp=Table[
{i,#}&/@(Range[Ceiling[Sqrt[2 i]],Floor[Sqrt[i+n]]]^2-i)
,
{i,1,n-1}
];
tmp=Join@@tmp;
Select[tmp,Less@@#&]
]
CheckSol[l_List]:=Sort[l]===Range[Length[l]]\[And](And@@BlockMap[IntegerQ@*Sqrt@*Total,l,2,1])
TryToFind[n_Integer?Positive]:=Module[{edges,out},
If[!KeyExistsQ[db,n],
edges=SquareEdges2[n];
If[Union[Flatten[edges]]===Range[n],
edges=TwoWayRule@@@edges;
edges=RandomSample[edges];
Do[
out=TimeConstrained[FindHamiltonianPath[Graph[edges],PerformanceGoal:>"Speed"],5+i,$Failed];
If[out=!=$Failed,
If[Length[out]==0,
Print[Style["No solution for ",Red],n];
,
status=Row[{"Found solution for ",n,":",i}];
];
AssociateTo[db,n->out];
Break[]
];
Print["Failed ",n,":",i];
edges=RandomSample[edges];
,
{i,5}
]
,
Print["Edges are not connected for ",n];
AssociateTo[db,n->{}]
]
]
]
Let's scan the first 1000:
Dynamic[status]
status = "";
Do[TryToFind[k], {k, 3, 1000}]
Export["squaresumdb.mx", db];
Note that if finding the Hamiltonian path takes too long I mix the edges and try again, sometimes, seemingly random, it then finds the solution quickly.
I can tell you now that all of them have a solution. In fact I went up to larger numbers and found that all up to 2667 have a solution, and possibly beyond. I attached the notebook and the solutions in form of a mx file.
[1]: http://community.wolfram.com/groups/-/m/t/1055504
[2]: http://community.wolfram.com/groups/-/m/t/1048489
[3]: http://community.wolfram.com/groups/-/m/t/1058615
[4]: https://www.youtube.com/watch?v=G1m7goLCJDY
[5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2018-01-11at23.36.51.png&userId=73716
[6]: https://www.youtube.com/watch?v=G1m7goLCJDY
[7]: https://www.youtube.com/watch?v=7_ph5djCCnM
[8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2018-01-11at23.43.52.png&userId=73716
[9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2018-01-11at23.46.51.png&userId=73716Sander Huisman2018-01-11T23:29:05ZGet two columns of numerical data as output data format?
https://community.wolfram.com/groups/-/m/t/1631879
Hi
I am solving a system of two equations with two variables z and x repeatedly in a cycle. At each step the solution is output into a file using **Write** function. I've pasted a part of the file below. Apart from the numerical data, it contains also other things, like curly brackets, symbols like **z->** . But I need to make a file that contains only two columns of numerical data, nothing else. How can I do that?
{z -> -2.1337292689248377, x -> -1.0028011752012573}
{z -> -1.5494511566425633, x -> -1.0015118850013995}
{z -> 0.14651664469911463, x -> -622.9327284292021}
{z -> 2.3907571924477877, x -> -1.0037755798274968}
{z -> 3.928880270839741, x -> -52.74880957186027}
{z -> 5.2048915646949965, x -> -1.017772661581148}
{z -> 9.062107349506263, x -> -1.0515309552374674}
{z -> 10.152134293099394, x -> -1.0641971180448309}
{z -> 14.312077199555635, x -> -395.73358957571725}Lev Dorosinskiy2019-03-14T08:31:22ZGather and analyse data from internet using WebSearch and WebAccess?
https://community.wolfram.com/groups/-/m/t/1630225
Hi everyone,
I'm trying to use Mathematica to automatically gather and analyze data from the internet. It's a pet project but one that I hope to become quite useful some day. I've made decent progress but then ran into some issues. At the moment, I'm trying to solve two issues, and would appreciate any help or suggestions how to resolve them:
1. WebSearch stopped working. I've been using WebSearch to find websites of solar energy related topics. It worked great at the beginning, but after two days of using it, the command stopped working. Now it executes as if it was an unknown command (i.e. just returns the written text WebSearch["topic"]. Does anyone have an idea what happened there? Neither restarting the computer nor quitting the kernel helped.
2. This might be a stupid question, but I have banged my head against it for too long. How do I reset the saved ServiceConnect connection to GoogleCustomSearch? I want the program to prompt the API again, but have so far been unsuccessful (even after updating from 11.2 to 11.3 it didn't prompt the API again). ServiceDisconnect doesn't do the trick, as it doesn't delete saved connection. Anyone any ideas?
Thanks so much!Ian Marius Peters2019-03-12T15:04:55ZAvoid extra comma while importing a text file as a vector?
https://community.wolfram.com/groups/-/m/t/1630177
Hello everyone. I am currently having some issues using Import on a txt file which is a Python output.
When I import said file as a vector, a comma appears before one of the elements, making it impossible to work with said vector. I will adjoint images of my problem in case someone can aid me.
Thanks in advance.![enter image description here][1]
![enter image description here][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=comma_issue.png&userId=1570098
[2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=file_in_question.png&userId=1570098Jaime de la Mota2019-03-12T14:17:00Z