https://community.wolfram.com RSS Feed for Wolfram Community showing any discussions tagged with Packages sorted by most viewed. https://community.wolfram.com/groups/-/m/t/1108337 By steps Insert/FilePath, I got a file path: "D:\\Mathematica\\Mathematica Programer II\\MATHPROG\\LOGICPROGRAMMING.M", but Needs["D:\\Mathematica\\Mathematica Programer \ II\\MATHPROG`LOGICPROGRAMMING.m`"] Mathematica always says Needs::cxt: Invalid context specified at position 1 in Needs[D:\Mathematica\Mathematica Programer II\MATHPROG`LOGICPROGRAMMING.m` ]. A context must consist of valid symbol names separated ` In fact I put `, What's wrong? Math Logic 2017-05-26T03:00:05Z https://community.wolfram.com/groups/-/m/t/2141352 Hello all :) CUDA is not working on 12.2. Look at this: What I am supposed to do? I also tried by downloading the CUDA packlets: change nothing. I have an NVIDIA RTX 3090 and the drivers are ok. Thank you foe helping me. Regards to all, Jean-Michel Jean-Michel Collard 2020-12-18T02:04:15Z https://community.wolfram.com/groups/-/m/t/787142 ## General information and download links ## If you're interested in crystal structures, you can now download the Crystallica application from the Wolfram Library Archive, and then you can do things like this: Needs["Crystallica`"]; CrystalPlot[ {{5.4,0,0},{0,5.4,0},{0,0,5.4}}, {{0,0,0},{0,0,.5},{0,.5,0},{.5,0,0},{.24,.24,.24},{.24,.76,.76},{.76,.24,.76},{.76,.76,.24}}, {1,2,2,2,3,3,3,3}, AtomCol->{"Firebrick","YellowGreen",White},AtomRad->.4, BondStyle->2,BondDist->3, CellLineStyle->False,AddQ->True,Lighting->{{"Directional",White,ImageScaled[{0,0,1}]}},Background->Black] ![enter image description here][1] Here are the download links for Crystallica and two other packages you may need: [Crystallica][2] - contains the functions `CrystalPlot` and `CrystalChange` [CifImport][3] - contains an import function for CIF files [VaspImport][4] - contains an import function for files related to [VASP][5] Once you've installed Crystallica (by saving the entire Crystallica folder - not the zip archive - to `$USerBaseDirectory/Applications` and re-starting the Kernel), you can enter Crystallica into the Documentation Center and you'll find lots of useful examples. Most of the examples in this post are taken from the Documentation. For the other two packages, just install them and evaluate this: ?CifImport ?VaspImport I'll first show you a few things the `CrystalPlot` function can do when you already have crystal structure data inside Mathematica, wherever it may have come from. Then we'll take a look at how to get the data into Mathematica in the first place, which is where `CifImport` and `VaspImport` will come into play - but we'll get data from other sources as well. I'll cover the different import solutions in separate replies to this thread, because I have a feeling that I'll be rambling on and on and on... ## Simple plot ## Traditional ball-and-stick plots are usually just fine, so the simplest thing you can do is this: CrystalPlot[ {{4.5,0,0},{0,4.5,0},{0,0,3}}, {{0,0,0},{.5,.5,.5},{.2,.8,.5},{.3,.3,0},{.7,.7,0},{.8,.2,.5}}, {1,1,2,2,2,2}] ![enter image description here][6] As you can see, `CrystalPlot` expects three arguments. The first one contains the lattice vectors, which are simply the three vectors that create the parallelepiped that constitutes the cell. The second argument contains the atomic coordinates, but they're given in the basis of the lattice vectors (which is quite useful in crystallography). The third argument is a list of integers that gives the atom types, with one entry for each atom. If you want to plot a molecule instead, you can call `CrystalPlot` with just two arguments: A list of atom coordinates in cartesian space, and a list of atom types. Everything else you see in the plot - the atoms, bonds, colours, arrows etc. - represents the default settings of various layout options. ## Advanced atoms and bonds ## Let's take a look at some more advanced options just for fun. For instance, atoms and bonds can look any way you need them to, because you can specify your own functions for them. You can also fine-tune where to put bonds and what to do with their thickness and colour in a physically (or chemically) meaningful way, but I won't show that here. So here are some customized atoms and bonds: Row[Table[ CrystalPlot[{{4,0,0},{0,4,0},{0,0,4}},{{0,0,0},{.4,.4,.4},{.8,.8,.8}},{1,2,3}, AtomRad->{.4,1.2,.7},AtomFunction->style,ImageSize->400], {style,{ (Ball[#1,#2]&), (Scale[Sphere[#1,#2],{1,1,.5}]&), ({EdgeForm[Thick],Opacity[.7],Cuboid[#1-.5*#2,#1+.5*#2]}&) }}]] ![enter image description here][7] Row[Table[ CrystalPlot[{{0,0,0},{5,0,0},{2.5,4,0}},{1,2,3},BondDist->6,BondStyle->style,ImageSize->400], {style,{ 1, Function[{bonds,partcol},Table[{If[ii<.5,partcol[#,1],partcol[#,2]],Sphere[bonds[[#,1]]+ii*(bonds[[#,2]]-bonds[[#,1]]),.15]},{ii,0,1,1/9}]&/@Range[Length[bonds]]], Function[{bonds,partcol},Module[{spiral,points,rad=.05}, spiral[atoms_]:=Module[{scale=.5,dist=atoms[[2]]-atoms[[1]],curls=60,normal,rot,scaled}, normal=Table[{scale*Cos[ii],scale*Sin[ii],.1*ii},{ii,0,curls,\[Pi]/10}]; scaled={#[[1]],#[[2]],10*Norm[dist]/curls*#[[3]]}&/@normal; rot=scaled.Quiet[RotationMatrix[{dist,{0,0,1}}]]; Join[{atoms[[1]]},#+atoms[[1]]&/@(rot[[25;;-25]]),{atoms[[2]]}]]; points=spiral/@bonds; {partcol[#,1],Tube[BSplineCurve[points[[#,;;Round[Length[points[[#]]]/2]]],rad]],partcol[#,2],Tube[BSplineCurve[points[[#,Round[Length[points[[#]]]/2];;]],rad]]}&/@Range[Length[bonds]] ]] }}]] ![enter image description here][8] ## Lattice planes ## Crystallica can also add lattice planes to the plot. You can specify them using [h,k,l] Miller indices and distance to the origin. CrystalPlot[{{3,0,0},{0,3,0},{0,0,3}},{{0,0,0}},{1}, AddQ->True,AtomRad->.3,AtomCol->"CadmiumYellow",Sysdim->2,CellLineStyle->2, LatticePlanes->Table[{{1,1,1},dist},{dist,1,5}],ContourStyle->{"TerreVerte",Opacity[.7]},BoundaryStyle->Thick] ![enter image description here][9] ## Coordination polyhedra ## You can automatically search for and plot coordination polyhedra. This is not limited to the commonly occurring tetrahedra and octahedra - you can actually look for polyhedra with arbitrary numbers of corners. There are also options to fine-tune both the searching and the rendering. plot[corners_,mixed_]:=CrystalPlot[{{0,0,0},{0,0,1.8},{-.9,-1.5,-.6},{-.9,1.5,-.6},{1.7,0,-.6},{.8,.8,.8}},{1,2,2,2,2,3}, BondStyle->False,ImageSize->250, PolyMode[corners]->{"Show"->All,"AllowMixed"->mixed},PolyStyle[corners]->Directive[Opacity[.5],EdgeForm[Thick]]]; Grid[{{ "", "Search for polyhedra with \n4 corners", "Search for polyhedra with \n5 corners" },{ "Allow \nmixed corners", plot[4,True], plot[5,True] },{ "Don't allow \nmixed corners", plot[4,False], plot[5,False] }},Dividers->All] ![enter image description here][10] CrystalPlot[{{2.5,-4.3,0},{2.5,4.3,0},{0,0,5.5}}, {{.5,0,0},{0,.5,.7},{.5,.5,.3},{.2,.4,.5},{.6,.8,.2},{.2,.8,.8},{.8,.6,.5},{.4,.2,.2},{.8,.2,.8}},{1,1,1,2,2,2,2,2,2}, PolyMode[4]->True,PolyStyle[4]->EdgeForm[None],AddQ->True, Sysdim->2,AtomRad->0,CellLineStyle->False,AtomCol->{"SlateGray","Firebrick"}, ViewAngle->.4,ViewPoint->{3.2,0,1.1},ViewVertical->{.5,0,1.2}] ![enter image description here][11] ## Other things ## Visualization aside, you can also build supercells, change cell shapes, or add, remove and sort atoms... but that's a bit boring to read, so I'll refer you to the Documentation page of the `CrystalChange` function instead. If you're interested, we can use this thread to talk about any questions you may have, or you can share your use of the package (if you decide to use it). I'm not offering full support here, but I'll be floating around, and I'd like to hear your feedback. We don't have any intentions to be involved in further development. But if you have a good idea and some time, then by all means, work on it for yourself, or host it on your favourite code collaboration site. Bianca Eifert and Christian Heiliger [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=teaser.png&userId=69107 [2]: http://library.wolfram.com/infocenter/MathSource/9372/ [3]: http://library.wolfram.com/infocenter/MathSource/9373/ [4]: http://library.wolfram.com/infocenter/MathSource/9375/ [5]: http://vasp.at/ [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=9692simple.png&userId=69107 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=atoms.png&userId=69107 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=bonds.png&userId=69107 [9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=planes.png&userId=69107 [10]: http://community.wolfram.com//c/portal/getImageAttachment?filename=polys.png&userId=69107 [11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=polys2.png&userId=69107 [12]: http://rruff.geo.arizona.edu/AMS/CIF_text_files/13532_cif.txt [13]: http://cms.mpi.univie.ac.at/vasp/vasp/POSCAR_file.html [14]: http://wiki.jmol.org/index.php/File:Caffeine.mol Bianca Eifert 2016-02-05T18:43:18Z https://community.wolfram.com/groups/-/m/t/3118205 ![enter image description here][1] &[Wolfram Notebook][2] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=heroimage.png&userId=1539902 [2]: https://www.wolframcloud.com/obj/5b3b5265-f1e2-406c-aef4-45a5f9ec363b Mohammad Bahrami 2024-02-07T19:21:52Z https://community.wolfram.com/groups/-/m/t/2330900 Daily Study Groups are back, now providing expertise on practical programming! After a short break, we're starting up with a series that picks up where Wolfram Language Basics sessions ended. This 3-week series will take you from basic programming concepts to package development. Need tips on improving your code’s speed or functionality? Programming Tutorials offer a great opportunity to level-up your skills while interacting with software development professionals. See hands-on examples and get questions answered by Wolfram Language experts. Plus: participants who pass weekly quizzes are awarded a program completion certificate. Level 1 certification is available for those who pass manually graded exercises. Join us any time between August 2 and August 20! Check out our [registration page][1] for more details. [1]: https://wolfr.am/Study_Group15 Wolfram U 2021-07-30T15:24:02Z https://community.wolfram.com/groups/-/m/t/1321057 This is a cautionary tale for those who choose Mathematica as the main tool for their work. It is now clear to me that Wolfram has simply abandoned the [Graphs and Networks](http://reference.wolfram.com/language/guide/GraphsAndNetworks.html) functionality area and I am left high and dry. I have no recourse because Mathematica is closed source so there is only so much a user can do to fix or work around problems. Reporting bugs in this particular area has now clearly proven to be useless. Most simply do not get fixed, no matter how serious they are, or how great a hindrance they are to practical use. No new functionality has been added since version 10.0. My colleagues who use other tools (mostly Python and R packages) are more productive at this point, but I have a handicap with those systems because I made the mistake of investing most of my time into Mathematica, and stayed optimistic about it even in the face of the most obvious warning signs. I am writing this post because those people who have not heavily invested in Mathematica, and in particular this functionality area of Mathematica, are not in a position to see this and may fall in the same trap I did. What if the same thing happens to the functionality area that is critical to *your* work? Wolfram Research, of course, will not tell you that they gave up on `Graph`. Thus, after my experience, I think I owe it to the community to warn you about the situation. ---- Some might ask me what specifically is wrong. I have made many posts on this forum about `Graph`-bugs (you only have to search), and I reported many more to WRI. There is always a last straw—it would be pointless to show it. Those who know me will know that I am not writing this admittedly emotional post out of ill will towards WRI. I have betted on Mathematica more than most, and have been advocating for it throughout the years. I even have a network analysis package with ~250 functions. If I am forced to abandon Mathematica for this type of work, then the countless hours that went into this package will all have been in vain. I admit that I am writing this public post partly out of desperation to try to get WRI to either fix the many serious `Graph`-problems, or otherwise publicly state that `Graph` is now abandoned so those of us who have been using it can stop wasting our time. Szabolcs Horvát 2018-04-16T09:54:42Z https://community.wolfram.com/groups/-/m/t/214901 In a response to an earlier thread I posted a notebook containing a basic tutorial on creating packages in Mathematica. Since the link to the notebook was embedded down in the responses to that post I thought it might be useful to some folks to have access to it at the top level here since they might have missed it. I have updated that notebook a bit since the last time.  I hope it is useful to some people.  It was originally written as a tutorial for a consulting client of mine and he agreed to allow me to share with others.   Enjoy. It is attached to this posting David Reiss 2014-03-08T02:32:19Z https://community.wolfram.com/groups/-/m/t/1715017 Hi, How can i run Min GW in Mac, do I need to rearrange these codes to make it run in Mac? Can anyone please advise? I posted the codes below. Clear["Global`*"]; Needs["CCompilerDriver`GenericCCompiler`"]; $CCompiler = {"Compiler" -> CCompilerDriver`GenericCCompiler`GenericCCompiler, "CompilerInstallation" -> "C:\\MinGW", "CompilerName" -> "gcc.exe"}; CompileOptions = {CompilationTarget -> "C", RuntimeOptions -> {"CatchMachineOverflow" -> False, "CatchMachineUnderflow" -> False, "CatchMachineIntegerOverflow" -> False, "CompareWithTolerance" -> False, "EvaluateSymbolically" -> False, "RuntimeErrorHandler" -> Evaluate, "WarningMessages" -> False}, CompilationOptions -> {"InlineExternalDefinitions" -> True, "InlineCompiledFunctions" -> True, "ExpressionOptimization" -> True}}; Thanks. adrian leong 2019-06-29T10:27:32Z https://community.wolfram.com/groups/-/m/t/984003 I'm going to show you how you can use existing functionality to run a Mathematica script across a managed HPC cluster. Before I start, I must be upfront with you: though the individual commands are documented, this method, as a whole, is not. Thus, support for this procedure is outside the scope of Wolfram Technical Support. However, I'm hoping that once the ground-work has been laid, that Wolfram Community members can work together to fill in the missing details. My assumptions: 1. Mathematica is installed and properly licensed on the managed cluster 2. once your job has been given resources, that you can freely SSH between them (1) This is up to your local cluster's System Admin to figure out by talking with their organization and a Wolfram Sales Representative, and possibly Wolfram Technical Support ([support.wolfram.com][1]). (2) Again, this is up to your local SysAdmin to ensure. It's also known as a public/private key pair between nodes. In the following, I'm assuming the cluster uses Torque ([Torque SysAdmin Guide][2]), but in principle other managers can be used. A generic Mathematica script job submission may look like the following: #PBS -N Job_name #PBS -l walltime=10:30 #PBS -l nodes=4:ppn=6 #PBS -m be math -script hpc.wl In this example, - the job is called "Job_name" - the job will finish in 10 and a half minutes - it is requesting 4 nodes with 6 processors-per-node, for a total of 24 resources (CPU cores) - an email will be sent to the account associated with the username when the job (b)egins and when it (e)nds If you are not familiar with job submissions to a managed HPC cluster, then I suggest you read any guides provided by your organization The Wolfram Language script "hpc.wl" does the rest of the work. It generically follows this order: 1. gather the environment variables associated with the list of provided resources 2. launch remote subkernels for each CPU core 3. do the parallel computations 4. close the subkernels 5. end the job (*get association of resources, name of local host, and remove local host from available resources*) hosts = Counts[ReadList[Environment["PBS_NODEFILE"], "String"]]; local = First[StringSplit[Environment["HOSTNAME"],"."]]; hosts[local]--; (*launch subkernels and connect them to the controlling Wolfram Kernel*) Needs["SubKernels`RemoteKernels`"]; Map[If[hosts[#] > 0, LaunchKernels[RemoteMachine[#, hosts[#]]]]&, Keys[hosts]]; (* ===== regular Wolfram Language code goes here ===== *) Print[ {$MachineName, $KernelID} ] (* ===== end of Wolfram Language program ===== *) CloseKernels[]; Quit On Torque there is the environment variable "PBS\_NODEFILE" ([Torque environment variables][3]) that lists the different nodes that are provided to the job. It is my understanding that the name is repeated for each CPU core. That's why a simple Count of the node list tells us everything. The other piece of information, which is probably not necessary, is "HOSTNAME". This is where the Wolfram controlling kernel is running. In the above, we remove it from the list of available resources, but I don't believe this is necessary. According to the documentation ([3]), this may be known as "PBS\_O\_HOSTNAME". The Mathematica script should not need to change save for the code between the commented lines. I'm also assuming that `$RemoteCommand` (provided by Subkernels\`RemoteKernels\`) is the same on each node. This is usually the case as most clusters use a cloned file system. SLURM should be very similar except that the environment variables will be different. It is my understanding that headNode = Environment["SLURMD_NODENAME"]; nodes = ReadList["!scontrol show hostname $SLURM_NODELIST",String]; provides the headnode and list of resources. I encourage discussion. [1]: http://support.wolfram.com [2]: http://docs.adaptivecomputing.com/torque/4-0-2/help.htm [3]: http://docs.adaptivecomputing.com/torque/4-0-2/help.htm#topics/2-jobs/exportedBatchEnvVar.htm Kevin Daily 2016-12-19T21:29:53Z https://community.wolfram.com/groups/-/m/t/1807709 Hello everybody! I wrote a number of notebooks calling R packages. But using up-to-date versions (Mathematica 12 and R 3.6.1), nothing works any more! For instance, it seems the default distribution doesn't exist now: In[2]:= InstallR[] REvaluate["sample(1:100,10)"] During evaluation of In[2]:= InstallR::invldrhome: The specified path to R home directory does not point to a valid directory Out[2]= $Failed I tried to use my own distribution, but it is not recognized! In[4]:= InstallR["RHomeLocation" -> "C:/PROGRA~1/R/R-36~1.1", "RVersion" -> "3.6.1"] During evaluation of In[5]:= InstallR::fail: Failed to install R. The following error was encountered: crash in low-level RLink component or in R runtime Out[5]= $Failed It seems there is an issue with the R.dll file (dll: 3.6.1, expecting 3.5.0). What can I do? Install an older version of R? Which one? Best regards, Claude Claude Mante 2019-10-15T09:19:47Z https://community.wolfram.com/groups/-/m/t/1170226 One of the great things about Mathematica is that it makes it so easy to do so much in so little code. It is partly because of this that I am so frustrated with the amount of time I need to spend developing the *interface* of package functions compared to their *functionality*. Suppose you wrote a Wolfram Language function that does something useful and interesting. Now you want to wrap it up into a package, and make it usable by everyone. A polished package is expected to have functions that: - Will check their input for errors - Will report errors in an informative way - Will use messages appropriately (i.e. associate them with the correct symbol name) - Will adhere to the de-facto Mathematica interface conventions: proper use of optional arguments, options, option inheritance (as in Graphics -> Plot), default option value handling, use of `Automatic`, etc. - Have SyntaxInformation I find that not infrequently I spend more time on making the function user-friendly than developing its functionality. **How do people generally deal with this task? How do you implement error checking and reporting in your packages?** To give an example, take a function as simple a moving average calculator. It is really easy to implement: movingAverage[vec_, n_] := Mean /@ Partition[vec, n, 1] But to bring it to the quality of the built-in `MovingAverage`, it should at least: - check the number of arguments (precisely 2) - check the types of arguments (a list and an integer) - check the values of arguments for correctness (non-empty list and positive integer) - make sure that all these checks don't introduce severe performance degradation (such as array unpacking, which can even be triggered by an inefficient argument pattern) This also involves the introduction of multiple messages (for each type of error) associated to `movingAverage`. If we now want to add a `movingMedian`, we will find that we will mostly need to carry out the same checks and report the same messages. There will be small differences though, e.g. average calculations are feasible for symbolic lists like `{1,x}`, but not median calculations. So the checks won't quite be identical. The messages will be mostly identical, but each function must associate messages to its own symbol, which means a lot of duplication. So if we care about a high-quality interface and high-quality error reporting, we will end up writing considerably more code for this than for the function's core task. We will also end up with a lot of code duplication, which is frustrating and a maintenance burden. The whole thing ends up being a lot of work and not a lot of fun (which is not very Mathematica-like :-) ) Are there good ways to simplify these tasks? Option handling also used to be error-prone and frustrating, but the introduction of `OptionsPattern[]` and `OptionValue[]` made it much easier. ---- I can see that there are some built-in tool to ease these tasks, though they are mostly undocumented. One example is `ArgumentCountQ`, another is ``Developer`CheckArgumentCount``, or some tools in ``GeneralUtilities` ``. I would love to hear from others about how they deal with the tasks I described, which such internal functions they make use of, etc. Szabolcs Horvát 2017-08-25T11:15:29Z https://community.wolfram.com/groups/-/m/t/560469 *WOLFRAM MATERIALS for the ARTICLE:* > Szabolcs Horvát, Jakub Podkalicki, Gábor Csárdi, Tamás Nepusz, Vincent Traag, Fabio Zanini, Daniel Noom, (2022). > *IGraph/M: graph theory and network analysis for Mathematica*. > arXiv:2209.09145 **[physics.soc-ph]**. > https://doi.org/10.48550/arXiv.2209.09145 [](https://igraph.discourse.group) [](https://github.com/szhorvat/IGraphM/releases) [](https://github.com/szhorvat/IGraphM#contributions) [](https://doi.org/10.5281/zenodo.1134932) ---- ##Article abstract IGraph/M is an efficient general purpose graph theory and network analysis package for Mathematica. IGraph/M serves as the Wolfram Language interfaces to the igraph C library, and also provides several unique pieces of functionality not yet present in igraph, but made possible by combining its capabilities with Mathematica's. The package is designed to support both graph theoretical research as well as the analysis of large-scale empirical networks. ---- ##Introduction The post below was written for the original release of IGraph/M. The package has come a long way since then and now contains ~300 functions. See http://szhorvat.net/mathematica/IGraphM for more details on the current release. Compatibility: 64-it Windows/macOS/Linux or Raspberry Pi; Mathematica <del>10.0</del> 11.0 or later. <a href="http://szhorvat.net/mathematica/IGraphM"><img src="https://community.wolfram.com//c/portal/getImageAttachment?filename=IGraphM-ad-3.png&userId=38370" width="300"></a> ---- I would like to announce IGraph/M, a new igraph interface for Mathematica: http://szhorvat.net/mathematica/IGraphM [igraph](http://igraph.org/) is a graph manipulation and analysis package. IGraph/M makes its functionality available from Mathematica. This initial release, version 0.1, covers only some igraph functions, as I focused on the things that I need personally. However the main framework is complete, and new functions can be added quickly. If anyone would like to contribute, please contact me. Binary packages for OS X (10.9 or later) and Linux can be downloaded [from GitHub](https://github.com/szhorvat/IGraphM/releases). Unfortunately, I was unable to compile the development version of igraph for Windows, so I cannot provide a Windows version. If you can help with compiling igraph itself (not IGraph/M) on Windows, please let me know! Functionality in this release that is not built into Mathematica: * Vertex betweenness centrality for weighted graphs * Estimates of vertex betweenness, edge betweenness and closeness centrality; for large graphs * Minimum feedback arc set for weighted and unweighted graphs * Find all cliques (not just maximal ones) * Count 3- and 4-motifs * Rewire edges, keeping either the density or the degree sequence * Alternative algorithms for isomorphism testing: Bliss, VF2 * Subgraph isomorphism * Test if a degree sequence is graphical * Alternative algorithms for generating random graphs with given degree sequence * Layout algorithms that take weights into account Note that IGraph/M is *not a replacement* for Mathematica's graphs and networks functionality. It is meant to complement what is already available in Mathematica, thus it primarily focuses on adding functionality that is not already present. Why did I release the package before covering most of the igraph functionality? I do not have time to work on things I do not personally need or use, so I am unlikely to extend it further unless the need comes up. I do think that the functions that are included in v0.1 can already be useful to others too. I would also like to give the opportunity for people to contribute to the project if they wish to. The groundwork has been laid, so further extensions should be quick and relatively easy. Also check out a related project, [IGraphR](https://github.com/szhorvat/IGraphR), which makes igraph available for Mathematica users through RLink. I wrote IGraph/M because I needed higher performance and greater reliability (especially for parallel computing) than what RLink could provide. ---- **A request:** If any of you have used IGraphR in the past to access igraph from Mathematica, please post a response to this thread and let me know which specific functions you were using. &[Wolfram Notebook][1] [1]: https://www.wolframcloud.com/obj/94639221-60b4-47e3-8862-d996caa40388 Szabolcs Horvát 2015-09-06T12:55:14Z https://community.wolfram.com/groups/-/m/t/1484511 Originally [here](https://www.wolframcloud.com/objects/b3m2a1/home/playing-with-youtube-from-mathematica.html#main-content) (the embedded iframes actually work there) --- Playing with YouTube from Mathematica --- Today I'm gonna quickly show off something I made for a friend (and about which I'd been thinking for some time). We're gonna use the YouTube API in Mathematica to do a have some fun (but not too much fun, mind you!). ### Installing the YouTube Connection My connection to the YouTube API is built on Mathematica's [```ServiceConnect```](https://reference.wolfram.com/language/ref/ServiceConnect.html) framework. I've [talked](https://www.wolframcloud.com/objects/b3m2a1/home/serviceconnections---mathematica-on-the-web.html) [about](https://www.wolframcloud.com/objects/b3m2a1/home/questions-on-the-mathematica-stack-exchange.html#main-content) [these](https://www.wolframcloud.com/objects/b3m2a1/home/questions-on-the-mathematica-stack-exchange-part-2.html#main-content) [before](https://mathematica.stackexchange.com/questions/163638/how-can-i-access-nasa-images-and-apis) in [many](http://community.wolfram.com/groups/-/m/t/1261562) [different](https://mathematica.stackexchange.com/a/167702/38205) [contexts](https://mathematica.stackexchange.com/questions/163780/sophisticated-api-integrations-making-a-card-game-in-mathematica) but today we'll just use it. To install we pull the [paclet off my server](https://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer/serviceconnectionyoutube.html#main-content) : PacletInstall["ServiceConnection_YouTube", "Site"-> "http://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer/" ] (*Out:*)  ### Authenticating with YouTube One installed, we connect to the service as usual: $so = ServiceConnect["YouTube"] This will open a dialog:  We'll click the sign in button, pick and account (I'll use my [new personal account steve.wolfraum@gmail.com](https://www.wolframcloud.com/objects/b3m2a1/home/making-a-blog-in-30-minutes.html#main-content) ):  When we click "Allow" either we'll be taken to a nicely formatted local page with an access code running via ```HTTPHandling`StartWebServer``` or if that's being glitchy as it was right now, we'll copy the part of the URL starting with ```"code=4/..."``` :  Finally, we're connected: (*Out:*)  ### Getting Data The majority of the currently supported methods have to do with getting data about YouTube: $so["Requests"] (*Out:*) {"Authentication","DeleteVideo","ID","Information","LastRequest","ListCaptions","ListChannelActivity","ListChannels","ListChannelSections","ListComments","ListCommentThreads","ListPlaylistItems","ListPlaylists","ListSubscriptions","ListVideos","Name","RawRequests","RequestData","RequestParameters","Search","UpdateVideo","UploadVideo","VideoURL"} We'll start with a search, maybe. First let's see what parameters we have to work with: $so["RequestParameters", "Request"->"Search"] (*Out:*) <|"Parameters"->{"q","part","forContentOwner","forDeveloper","forMine","relatedToVideoId","channelId","channelType","eventType","location","locationRadius","maxResults","onBehalfOfContentOwner","order","pageToken","publishedAfter","publishedBefore","q","regionCode","relevanceLanguage","safeSearch","topicId","type","videoCaption","videoCategoryId","videoDefinition","videoDimension","videoDuration","videoEmbeddable","videoLicense","videoSyndicated","videoType"},"Required"->{"part"}|> Then we'll search for something bland, like "Steve Wolfraum". The YouTube API requires this ```"part"``` parameter to know what to return to use. I like to get the ```"id"``` and ```"snippet"``` parts, but you can [pick for yourself](https://developers.google.com/youtube/v3/getting-started#part) which parts you want. Just comma-separate your values and you can get multiple different ones: mySearch = $so["Search", "q"->"Steve Wolfraum", "part"->"id,snippet"] (*Out:*)  We see there are *tons* of results for "my" name but we only get them five at a time. We can use the ```"nextPageToken"``` to get the next page of results, though. We can bump up the number we get with the ```"maxResults"``` parameter, but I see no need for that right now. Let's look at the first result here. I'll drill straight into that snippet parameter, as that's the most interesting part: mySearch["items", 1, "snippet"] (*Out:*)  And we see, sadly, that instead of getting results for my name, Steve Wolfraum, we get them for some random dude name Stephen Wolfram. Not to be deterred, let's at least get something out of this and see a thumbnail for all our results. Tragically, we can't download videos with the YouTube API, but at least we get thumbnails...hooray? To make things more fun, we'll also add a filter to all these results, as the raw thumbnails really aren't all that interesting. ImageAdjust@LaplacianGaussianFilter[Import[#], 3]&/@ Normal@mySearch["items", All, "snippet", "thumbnails", "high", "url"] (*Out:*)  Cool. There's some data. We can also pull out statistics and things, say for different videos broadcast on the Wolfram Research channel. First we'll get their channel ID: wriChannel= $so["Search", "q"->"Wolfram Research", "type"->"channel", "part"->"id", "maxResults"->"1"]["items", 1, "id", "channelId"] (*Out:*) "UCJekgf6k62CQHdENWf2NgAQ" Now we'll use this to extract the 50 most popular videos: wriVids=$so["Search", "channelId"->wriChannel, "type"->"video", "order"->"viewCount", "part"->"id,snippet", "maxResults"->"50"] (*Out:*)  And now we'll get view counts: wriStats=$so["ListVideos", "id"->StringRiffle[Normal@wriVids["items", All, "id", "videoId"], ","], "part"->"statistics" ] (*Out:*)  And then we'll make a [```WordCloud```](https://reference.wolfram.com/language/ref/WordCloud.html) of titles vs viewcounts: WordCloud@ AssociationThread[ Normal@wriVids["items", All, "snippet", "title"], ToExpression@Normal@wriStats["items", All, "statistics", "viewCount"] ] (*Out:*)  And we see as far as YouTube cares, there's really only one video WRI has made. But dropping that one we might see some more interesting structure: WordCloud@ AssociationThread[ Rest@Normal@wriVids["items", All, "snippet", "title"], Rest@ToExpression@Normal@wriStats["items", All, "statistics", "viewCount"] ] (*Out:*)  And it seems that people really like the basics and...really like seismic waves? I'll embed the first of those so we can see how good the really are: $so["EmbedIFrame", "id"->wriVids["items", 3, "id", "videoId"]] (* Out *) <iframe id="ytplayer" type="text/html" width="640" height="360" src="https://www.youtube.com/embed/2rYjlVPU9U4?autoplay=0" frameborder="0"></iframe> ### Uploading Videos A YouTube API wouldn't be much of a YouTube API if it couldn't upload videos, of course, so I cooked that in as well. Here's a sample of that. We'll first pull an animation off the [```Manipulate```](https://reference.wolfram.com/language/ref/Manipulate.html) docs: anim = Export[ FileNameJoin@{$TemporaryDirectory, "plot3d.flv"}, Manipulate[ Plot3D[Sin[n x y], {x, 0, 3}, {y, 0, 3}, ViewPoint -> Dynamic[{2, v, 2}], SphericalRegion -> True, Ticks -> None, PerformanceGoal->"Quality" ], {n, 1, 4}, {v, -2, 2} ] ]; Then we'll upload the video we made: upload = $so["UploadVideo", "part"->"id", "BodyContent"->anim]; Then we attach a title and things in a second request: $so["UpdateVideo", "id" -> upload["id"], "part" -> "snippet", "Title" -> "Mathematica Examples: Plot 3D", "CategoryID" -> "22" ] (*Out:*)  And then now we'll embed that video we just made: $so["EmbedIFrame", "id"->upload["id"]] (* Out *) <iframe id="ytplayer" type="text/html" width="640" height="360" src="https://www.youtube.com/embed/Quzcr4Vuq_Q?autoplay=0" frameborder="0"></iframe> b3m2a1 ​  2018-09-29T04:42:36Z https://community.wolfram.com/groups/-/m/t/1646303 # UNET [](https://zenodo.org/badge/latestdoi/137186334) [](https://github.com/dwyl/esta/issues) ---------- ![Automated 3D muscle segmentation using UNET / RESNET using DIXON MRI data][17] A package to generate and train a UNET deep convolutional network for 2D and 3D image segmentation. Some code was based on [work][1] by [@Ali Hashmi][at0], which was also dicussed in [this post][2] The full version of the toolbox can be found on my [github page][3]. * [Information](#information) * [Install toolbox](#install-toolbox) * [Using the toolbox](#using-the-toolbox) * [Functionality](#functionality) * [Visualization](#visualization) * [Example](#example) ## Information UNET is developed for [Mathematica](https://www.wolfram.com/mathematica/). It contains the following toolboxes: - UnetCore - UnetSupport Documentation of all functions and their options is fully integrated in the Mathematica documentation. The toolbox always works within the latest version of Mathematica and does not support any backward compatibility. All code and documentation is maintained and uploaded to github using [Workbench](https://www.wolfram.com/workbench/). ## Install toolbox Install the toolbox in the Mathematica UserBaseDirectory > Applications. FileNameJoin[{$UserBaseDirectory, "Applications"}] ## Using the toolbox The toolbox can be loaded by using <<UNET` The notbook ``UNET.nb`` shows examples of how to use the toolbox on artificially generated 2D data. There are also examples how to visualize the layer of your trained network and how to visualize the training itself. ## Functionality The network supports multi channel inputs and multi class segmentation. * UNET generates a UNET convolutional network. * 2D UNET ![UNET 2D][4] * 3D UNET ![UNET 3D][5] * Loss Layers: Training the data is done using three loss layers: a SoftDiceLossLayer, BrierLossLayer and a CrossEntropyLossLayer. ![SoftDiceLossLayer, BrierLossLayer and a CrossEntropyLossLayer][6] * Convolution Blocks: The toobox contains five different convolution blocks that build up the network: [UNET][7], UResNet, [RestNet][8], UDenseNet, [DensNet][9]. ![Convolution blocks][10] * SplitTrainData splits the data and labels into training, validation and test data. ![split train Data][11] * TrainUNET trains the network. ![Train UNET][12] ## Visualization * Visualize the network and results. * Visualize the features of the layers. ![Visualize layer features][13] * Visualize the results. ![Visualize the results][14] * Animate the training process. ![UNET 2D animation][15] ![UNET 3D animation][16] ## Example * Example: 3D segmentation of lower legg muscles using MRI data. ![Automated 3D muscle segmentation using UNET / RESNET using DIXON MRI data][17] [1]: https://github.com/alihashmiii/UNet-Segmentation-Wolfram [2]: https://community.wolfram.com/groups/-/m/t/1341081?p_p_auth=w8PIeeiA [3]: https://github.com/mfroeling [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=UNET2D.PNG&userId=1332602 [5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=UNET3D.PNG&userId=1332602 [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Loss.PNG&userId=1332602 [7]: https://arxiv.org/abs/1505.04597 [8]: https://arxiv.org/abs/1512.03385 [9]: https://arxiv.org/abs/1608.06993 [10]: https://community.wolfram.com//c/portal/getImageAttachment?filename=convblocks.PNG&userId=1332602 [11]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Split.PNG&userId=1332602 [12]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Train.PNG&userId=1332602 [13]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Visualize1.PNG&userId=1332602 [14]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Visualize2.PNG&userId=1332602 [15]: https://community.wolfram.com//c/portal/getImageAttachment?filename=amin0-v2.gif&userId=1332602 [16]: https://community.wolfram.com//c/portal/getImageAttachment?filename=amin4-v2.gif&userId=1332602 [17]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Muscle_Segmentation.jpg&userId=1332602 [at0]: https://community.wolfram.com/web/alihashmi87 Martijn Froeling 2019-04-03T20:01:26Z https://community.wolfram.com/groups/-/m/t/1655203 This is a cross-post from [here](https://b3m2a1.github.io/easyide-an-ide-for-mathematica.html) --- Contrary to what would probably be best practice, I do all of my Mathematica development inside Mathematica itself. To support this I built out a suite of [application development tools](https://paclets.github.io/PacletServer/btools.html) , a [web site builder](https://paclets.github.io/PacletServer/ems.html) , a [bug tracker](https://github.com/b3m2a1/BugTracker) , and a [documentation writing system](https://paclets.github.io/PacletServer/simpledocs.html) . Each of these worked nicely for me separately, but each of these required a palette and each one ran on notebooks, which meant that my screen filled with too many notebooks to keep track of. And then for each of these palettes and systems I had to write new resource finding code based off the palette or some arbitrarily imposed root directory or else provide some other way to specify where things would be found. In short, it got messy. Then, in a very relaxing hiatus from Mathematica I did some python development, writing a package for linking [Mathematica to python](https://paclets.github.io/PacletServer/pjlink.html) as well as some stuff for coordinate transforms and finite differencing and other little utilities. In doing this I noticed that everything was just...better. Partly this is because python is much nicer to write significant amounts of code in, being a language that actually supports developers and with actual object orientation and modularity. But another significant part of it was in the tools available to me. In particular I had the python plugin to [IntelliJ](https://www.jetbrains.com/idea/) , which is also repackaged as [PyCharm](https://www.jetbrains.com/pycharm/) . The fact that I had tabbing, plugins (e.g. for [Git](https://git-scm.com/) ), a file browser inside my dev environment, etc. was at once so entirely normal (I used to be a python programmer before switching over to mostly using Mathematica) and at the same time so nice. I then tried to use the *very nice and well constructed* [IntelliJ plugin for Mathematica](https://wlplugin.halirutan.de/) but it was just too much of a hurdle to lose everything I was used to and liked about writing my code directly in Mathematica. And that long, unnecessary background is why today we're gonna look at a Mathematica IDE written and operating entirely within Mathematica. # EasyIDE Mostly for the rhyme, I called this thing [EasyIDE](https://paclets.github.io/PacletServer/easyide.html) but it is pretty easy to use, too. ## Basics ###Installation Install it off the [Paclet Server](https://paclets.github.io/PacletServer/) : << https://paclets.github.io/PacletServer/Install.wl PublicPacletInstall["EasyIDE"] (*Out:*)  ###Making a New IDE Notebook This IDE system is also basically just a package and a stylesheet, so it's pretty easy to get started. Simply go to `Format ? Stylesheet ? EasyIDE ? LightMode` . It'll prompt you for a directory to use as the root directory. Here's a video as an example: [](https://www.youtube.com/watch?v=TMHiN9Ov2fQ) You can play around with the file browser now or the plugin menus in the top right ###Notebooks, Packages, and Text Files As things currently stand, the IDE recognizes three types of files to handle in different ways. The first, of course, are plain notebooks. These can be manipulated like normal. Here's an example of making and editing a notebook file in the IDE: [](https://www.youtube.com/watch?v=I1MnB7duneA) Text and package files can be made in the same way--just assign the appropriate file extension. Each of these files will work basically as a regular file would, except their contents will be saved to their original file on the disk rather than the current `NotebookFileName[]` . ###The File Browser One of the most useful and intuitive features of this IDE is the file browser it has built in. This allows you to quickly find files inside the active directory. Here's a screen shot of what that can look like:  Each entry in this has a `ContextMenu` that allows for some file- or directory-specific actions. ###Stylesheet / Extension Based Behavior EasyIDE is built to be extensible. It provides a way to get different behavior depending on what would be useful for the specific type of notebook or file is being fed in. These are controlled in the EasyIDE settings, in particular at `EasyIDE ? Resources ? Settings ? Mappings` where there are many files that control how these should map. This directory may also be created in `$UserBaseDirectory/ApplicationData` and the settings there will take precedence over those in the paclet folder itself. These customizations include stylesheets, toolbars, and what to do when the file browser is active. ###Plugins and Toolbars Probably the best feature of having something like EasyIDE is the ability to hook external code into the IDE and have it give new, more powerful capabilities. To make this easy to work with I added both a plugin system and a toolbar system (although the latter is really just a special case of the first). Plugins appear as either menus--such as the `File` and `Project` menus which are themselves just plugins--or as commands under the plugins menu. Currently I already have a decent number of these:  All of these add new functionality to the IDE based on code I'd written before. In that screenshot you can also see a toolbar, which exists right below the tabs. This can be stylesheet specific and thus adds an even more targeted way to add functionality to the system. Here's an example of the four different toolbars I've implemented as well as the different stylesheets they go with: [](https://www.youtube.com/watch?v=fFiDiO-oM7w) In that you can also see the major downside of putting everything into an IDE: when the files get big (as is the one I'm using to write this post) things can get slower. On the other hand as long as one is only writing code, this is never an issue. And even with a ~12MB file like this things are still more than fast enough to not be frustrating to work with. ## Extensions ###Styles EasyIDE was built to be customizable. This holds first and foremost for the stylesheets it works with. Even though currently there is only a set of LightMode styles, as DarkMode style set could be constructed without too much more difficulty. To do this, one would merely have to take the existing LightMode stylesheet, copy it, and make the necessary cosmetic changes. These changes should then propagate reasonably naturally to the extension styles if the inheritance is changed. This is on the TODO list, but if there is a quality existing DarkMode stylesheet to work off that would also make life much easier. ###Plugins and Toolbars These may be hooked in by adding things to `EasyIDE ? Resources ? Settings ? Plugins` and `EasyIDE ? Resources ? Settings ? Toolbars` . There are a number of good examples there already. ###Miscellaneous Extensions I had already implemented stuff for creating nice docs, Markdown notebooks, websites, bug tracking, paclet creation, etc. and some of this has made it in as plugins already. More is forthcoming, but for now one can always play with what's in the Plugins menu. In particular the Git plugin is useful for me as I write and develop. ###The EasyIDE API EasyIDE is just a collection of functions wrapped into a single unit. These were designed to (hopefully) be modular and clean to work with. Eventually all core functionality will also make its way to being attached to a single object, the `IDENotebookObject` . The API for this is based off of my [InterfaceObjects](https://paclets.github.io/PacletServer/interfaceobjects.html) package and is object-oriented. This will be documented in due time, but as a taste here's what it can look like: ide = IDENotebookObject[] (*Out:*)  ide@"Methods" (*Out:*) {"Open","Save","Close","SwitchTab","Path","Data","SetData","ToggleFileViewer","AddToolbar","RemoveToolbar","AddStyles","RemoveStyles","GetStylesheet","SetStylesheet","SetProjectDirectory","CreateMessage","CreateDialog"} These `"Methods"` are all operations that the IDE notebook referenced to by `EvaluationNotebook[]` can perform. Here's an example of creating a message: ide@"CreateMessage"["Hello!"] (*Out:*)   As the IDE grows in sophistication so will the methods the API supports. For now, though, these provide the most direct control that is possible to get with the IDE. b3m2a1 ​  2019-04-09T10:47:07Z https://community.wolfram.com/groups/-/m/t/1527021 MATH-TWS is a new Mathematica package that connects Wolfram Mathematica to the Interactive Brokers TWS platform via the C++ API. It enables the user to retrieve information from TWS on accounts, portfolios and positions, as well as historical and real-time market data. MATH-TWS also enables the user to place and amend orders and obtain execution confirmations from Mathematica. I have wanted a way to connect Wolfram Mathematica to Interactive Brokers’ Trader Workstation for the longest time. Now that it is finally available with MATH-TWS I am excited by the possibilities for Mathematica users. The first release of MATH-TWS will be available within a couple of weeks. Anyone interested in licensing a copy should email algorithmicexecution@gmail.com with MATH-TWS in the subject line. ![enter image description here][1] ![enter image description here][2] ![enter image description here][3] ![enter image description here][4] ![enter image description here][5] ![enter image description here][6] ![enter image description here][7] ![enter image description here][8] ![enter image description here][9] ![enter image description here][10] ![enter image description here][11] ![enter image description here][12] ![enter image description here][13] [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=twsConnect.png&userId=773999 [2]: http://community.wolfram.com//c/portal/getImageAttachment?filename=twsAccounts.png&userId=773999 [3]: http://community.wolfram.com//c/portal/getImageAttachment?filename=twsAccounts2.png&userId=773999 [4]: http://community.wolfram.com//c/portal/getImageAttachment?filename=twsAccounts3.png&userId=773999 [5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=HistoricalData.png&userId=773999 [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=DataManipulation.png&userId=773999 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=TimeSeries.png&userId=773999 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=IntradayData.png&userId=773999 [9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Currentmarketdata.png&userId=773999 [10]: http://community.wolfram.com//c/portal/getImageAttachment?filename=News.png&userId=773999 [11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Ordermanagement.png&userId=773999 [12]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Modifyorder.png&userId=773999 [13]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Cancelorder.png&userId=773999 Jonathan Kinlay 2018-10-22T12:09:47Z https://community.wolfram.com/groups/-/m/t/1736686 Recently I updated my [Bayesian inference reporitory][1] with a new function called `BayesianLinearRegression` to provide a Bayesian alternative to Mathematica's `LinearModelFit`. I also submitted the code for this function to the [Wolfram function repository][2] to make it easier to access, so the function can also be used with `ResourceFunction["BayesianLinearRegression"]`. The example notebook in the repository provides several examples of how the function can be used, some of which I will reproduce below. Please refer to the GitHub README.md file (which is displayed on the homepage of the repository) for instructions about installing the `BayesianInference` package. The code is in [this file][3], in case you're interested in taking a look under the hood. ## Fitting polynomials ## First generate some test data: data = RandomVariate[ MultinormalDistribution[{{1, 0.7}, {0.7, 1}}], 20 ]; ListPlot[data] ![enter image description here][4] The usage of `BayesianLinearRegression` is similar to that of `LinearModelFit` with one significant difference: I decided that the `Rule`-based data specification used by `Predict` and `NetTrain` is often more convenient than the matrix input used by the `Fit` family, so I decided that both types of data specifications should work. The main reason for this is that `BayesianLinearRegression` also supports regression of vector outputs (an example of that is given in the repository example notebook), in which case the `Rule`-based format is often easier to understand. When you fit a model, `BayesianLinearRegression` returns an `Association` containing all relevant information about the fit. Fit a model of the form `y == a + b x + randomness` where `randomness` is distributed as `NormalDistribution[0, sigma]`. Here, `a`, `b` and `sigma` are unknowns that need to be fitted to the data: In[20]:= Clear[x]; model = BayesianLinearRegression[data, {1, x}, x]; Below is a more detailed description of the information return by `BayesianLinearRegression`, but let's first concentrate on the main fit result, which is the [posterior predictive distribution][5] : In[61]:= model["Posterior", "PredictiveDistribution"] Out[61]= StudentTDistribution[-0.267818 + 0.986121 x, 0.726105 Sqrt[1.05016 + 0.00692863 x + 0.0635094 x^2], 2001/100 ] As you can see, the result is returned is an `x`-dependent `StudentTDistribution`. Visualize the predictions: With[{ predictiveDist = model["Posterior", "PredictiveDistribution"], bands = {95, 50, 5} }, Show[ Plot[ Evaluate@InverseCDF[predictiveDist, bands/100], {x, -4, 4}, Filling -> {1 -> {2}, 3 -> {2}}, PlotLegends -> bands ], ListPlot[data], PlotRange -> All ] ] ![enter image description here][6] In the `BayesianInference` package on GitHub, I included a function called `regressionPlot1D` which makes it a bit easier to make this plot: Show[ regressionPlot1D[model["Posterior", "PredictiveDistribution"], {x, -4, 4}, {95, 50, 5}], ListPlot[data] ] (* same result*) ## Model comparison and model mixing ## Often you'll want to compare multiple models against a data set to see which one works best. For example, here is some data where it's a bit unclear if you should use a first or second order model: data = {{-1.5`,-1.375`},{-1.34375`,-2.375`},{1.5`,0.21875`},{1.03125`,0.6875`},{-0.5`,-0.59375`}, {-1.875`,-2.59375`},{1.625`,1.1875`},{-2.0625`,-1.875`},{1.0625`,0.5`},{-0.4375`,-0.28125`},{-0.75`,-0.75`},{2.125`,0.375`},{0.4375`,0.6875`},{-1.3125`,-0.75`},{-1.125`,-0.21875`}, {0.625`,0.40625`},{-0.25`,0.59375`},{-1.875`,-1.625`},{-1.`,-0.8125`},{0.4375`,-0.09375`}} ListPlot[data] ![enter image description here][7] Fit the data with polynomials up to fourth degree; rank the log-evidences and inspect them. As you can see, the first and second order fits are almost equally likely: In[152]:= models = AssociationMap[ BayesianLinearRegression[Rule @@@ data, x^Range[0, #], x] &, Range[0, 4] ]; ReverseSort@models[[All, "LogEvidence"]] Out[153]= <|1 -> -30.0072, 2 -> -30.1774, 3 -> -34.4292, 4 -> -38.7037, 0 -> -38.787|> Show the prediction bands: Show[ regressionPlot1D[ models[1, "Posterior", "PredictiveDistribution"], {x, -3, 3}], regressionPlot1D[models[2, "Posterior", "PredictiveDistribution"], {x, -3, 3}, PlotStyle -> Dashed, PlotLegends -> None], ListPlot[data] ] ![Comparison of the first (solid) and second order (dashed) prediction bands][8] Instead of picking just one of the models, we can define a mixture over all of them. First calculate the weights for each fit: In[94]:= weights = Normalize[ (* subtract the max to reduce rounding error *) Exp[models[[All, "LogEvidence"]] - Max[models[[All, "LogEvidence"]]]], Total ]; ReverseSort[weights] Out[95]= <|1 -> 0.516002, 2 -> 0.47702, 3 -> 0.0068122, 4 -> 0.0000982526, 0 -> 0.0000667854|> Define a mixture of posterior predictive distributions and show the predictions: In[96]:= mixDist = MixtureDistribution[ Values[weights], Values@models[[All, "Posterior", "PredictiveDistribution"]] ]; Show[ regressionPlot1D[mixDist, {x, -3, 3}], ListPlot[data] ] ![Prediction bands of a mixture of polynomial models up to degree 4][9] As you can see, if one model is not clearly the best, you can just split the difference between them. Another interesting thing you can do with `BayesianLinearRegression` is Bayesian updating. Basically, this means that if you've fitted some data in the past and are now finding new data, you can incorporate the new data by updating your fit without having to re-fit all of the data (old and new) in one go. An example of this is given in the example code notebook on GitHub (in the section about the `"PriorParameters"` options of the function). ## Detailed explanation about the returned values ## `BayesianLinearRegression` returns an `Association` with the following keys: Keys[model] Out[21]= {"LogEvidence", "PriorParameters", "PosteriorParameters", "Posterior", "Prior", "Basis", "IndependentVariables"} - `"LogEvidence"`: In a Bayesian setting, the evidence (also called [marginal likelihood][10]) measures how well the model fits the data (with a higher evidence indicating a better fit). The evidence has the virtue that it naturally penalizes models for their complexity and therefore does not suffer from over-fitting in the way that measures like the sum-of-squares or likelihood do. - `"Basis"`, `"IndependentVariables"`: Simply the basis functions and independent variable specified by the user. - `"Posterior"`, `"Prior"`: These two keys each hold an association with 4 distributions: 1. `"PredictiveDistribution"`: A distribution that depends on the independent variables (`x` in the example above). By filling in a value for `x`, you get a distribution that tells you where you could expect to find future `y` values. This distribution accounts for all relevant uncertainties in the model: model variance caused by the term `randomness`; uncertainty in the values of `a` and `b`; and uncertainty in `sigma`. 2. `"UnderlyingValueDistribution"`: Similar to `"PredictiveDistribution"`, but this distribution give the possible values of `a + b x` without the `randomness` error term. 3. `"RegressionCoefficientDistribution"`: The join distribution over `a` and `b`. 4. `"ErrorDistribution"`: The distribution of the variance `sigma^2`. - ` "PriorParameters"`, `"PosteriorParameters"`: These parameters are not immediately important most of the time, but they contain all of the relevant information about the fit. ## Sources ## The formulas underlying BayesianLinearRegression are based mainly on the following Wikipedia articles. The names of the hyper parameters returned by the function are based on the article about multivariate linear regression. - [Bayesian linear regression (Wikipedia)][11] - [Bayesian multivariate linear regression (Wikipedia)][12] [1]: https://github.com/ssmit1986/BayesianInference [2]: https://resources.wolframcloud.com/FunctionRepository/resources/BayesianLinearRegression [3]: https://github.com/ssmit1986/BayesianInference/blob/master/BayesianInference/BayesianLinearRegression.wl [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=testdata.png&userId=839741 [5]: https://en.wikipedia.org/wiki/Posterior_predictive_distribution [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=fit.png&userId=839741 [7]: https://community.wolfram.com//c/portal/getImageAttachment?filename=testdata2.png&userId=839741 [8]: https://community.wolfram.com//c/portal/getImageAttachment?filename=fit2.png&userId=839741 [9]: https://community.wolfram.com//c/portal/getImageAttachment?filename=fitmix.png&userId=839741 [10]: https://en.wikipedia.org/wiki/Marginal_likelihood [11]: https://en.wikipedia.org/wiki/Bayesian_linear_regression [12]: https://en.wikipedia.org/wiki/Bayesian_multivariate_linear_regression Sjoerd Smit 2019-07-14T12:30:33Z https://community.wolfram.com/groups/-/m/t/1039591 Do you ever wish you could save data directly into a notebook instead of having to export it? I sometimes do things like mydata = {1,2,3, ...}; just to be able to use this data next time I open the notebook. This could be data I pasted from the web, or something that just took several minutes to generate. But saving it like this is inconvenient, and often takes up a lot of space. Today I published a blog post about a better method that I have been using recently. I thought people here may be interested: * [Save data directly in Mathematica notebooks](http://szhorvat.net/pelican/save-data-in-notebooks.html) The idea is to use first `Compress` the data, then use `Interpretation` to create a compact display for it. Here's a small function that packages all of this up: ClearAll[SaveToCell] SaveToCell::usage = "SaveToCell[variable] creates an input cell that reassigns the current value of variable.\n" <> "SaveToCell[variables, display] shows 'display' on the right-hand-side of the assignment."; SetAttributes[SaveToCell, HoldFirst] SaveToCell[var_, name : Except[_?OptionQ] : "data", opt : OptionsPattern[]] := With[{data = Compress[var], panel = ToBoxes@Tooltip[Panel[name, FrameMargins -> Small], DateString[]]}, CellPrint@Cell[ BoxData@RowBox[{ MakeBoxes[var], "=", InterpretationBox[panel, Uncompress[data]], ";" }], "Input", (* prevent deletion by Cell > Delete All Output: *) GeneratedCell -> False, (* CellLabel is special: last occrrence takes precedence, so it comes before opt: *) CellLabel -> "(saved)", opt, CellLabelAutoDelete -> False ] ] If you have your data in the variable `var`, simply run `SaveToCell[var]`, which will create an input cell that re-assign the value of `var`. It looks like this: ![enter image description here][1] We can also customize the display: var = Range[1000]; SaveToCell[var, Short[var]] ![enter image description here][2] Hovering the display will show the date when the data was saved. `SaveToCell` also takes arbitrary `Cell` options, and passes them down to the generated cell. Something strange I observed while writing this function is that with some options, such as `CellLabel`, [it is not the first but the last occurrence of the option that takes precedence][3]. Does anyone know why? We can use this functionality to change the cell style, add a different label, or to protect the cell against accidental deletion: `SaveToCell[var, Deletable -> False]`. I have been using this little function for a while, and I hope that others will find it useful too. Do be careful though: notebooks are not designed for storing large data. I would avoid storing data as large as several tens of megabytes within notebooks. [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=in1.png&userId=38370 [2]: http://community.wolfram.com//c/portal/getImageAttachment?filename=in3.png&userId=38370 [3]: http://mathematica.stackexchange.com/q/140989/12 Szabolcs Horvát 2017-03-25T21:56:48Z https://community.wolfram.com/groups/-/m/t/963158 Recently John Fultz [announced][1] GitLink as open source on GitHub. He also mentioned the desire to offer more Wolfram related open source to the community. Draw was not mentioned by John but readers of his post followed the GitHub [link][2] and discovered the Draw repository. Naturally questions followed so here we are with an introduction of Draw, the latest open source contribution to the community. Draw is a vector drawing application. Many of the technical illustrations found in the Wolfram Language documentation were created using it. The release of Draw is to promote the advantages of using the Wolfram Language for software development. Draw was a personal project that grew from an idea to develop a simple tool to automate the drawing of flowcharts. This simple tool expanded in response to the need of functionality for solving specific challenges required by various illustrations. The Wolfram Language was very instrumental in helping Draw evolve during this process. Some of the more prominent features of Draw include: - Orthographic 3D drawing - Circuit, Mechanical and 3D library presets - Adjustable shapes such as gears or spirals - Shape arrays - Image autotrace The gallery of examples below demonstrate the capabilities of Draw: ![gallery][3] A look inside the application reveals an interface divided into four distinct sections: ![interface sections][4] An example of Draw being used to work on a flowchart: ![interface][5] The Circuit, Mechanical and 3D libraries: ![libraries][6] The adjustable gear shape: ![gear][7] The shape array: ![shape array][8] Hope this brief introduction is enough to inspire you to download the Draw notebook from [this GitHub link][9]. Start creating illustrations or explore the Wolfram Language code to discover how the application works. Support is available by clicking the blue help button for topics such as the basics, features, interaction, tools, examples and tips. Tooltips appear for all buttons detailing their behavior. Remember this is open source software and comes with no guarantee. That being said, feel free to kick the tires and take it for a spin. Draw works best when used in Mathematica 11 but most functionality is supported by Mathematica 10.4.1. [1]: http://community.wolfram.com/groups/-/m/t/960333 [2]: https://github.com/WolframResearch [3]: http://community.wolfram.com//c/portal/getImageAttachment?filename=examples.png&userId=28355 [4]: http://community.wolfram.com//c/portal/getImageAttachment?filename=interfaceSections.png&userId=28355 [5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=interface.png&userId=28355 [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=libraries.png&userId=28355 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=gear.png&userId=28355 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=shapearray.png&userId=28355 [9]: https://github.com/shdlbwr/draw Tim Shedelbower 2016-11-14T20:15:12Z https://community.wolfram.com/groups/-/m/t/1468475 Here's a cross-post of something I originally wrote [here](https://www.wolframcloud.com/objects/b3m2a1/home/pjlink-hooking-up-mathematica-and-python.html) about how to get python and Mathematica to work together like JLink. I thought this might have broad appeal here. PJLink: Hooking up Mathematica and Python --- Mathematica is an incredibly powerful platform with a fun and intellectually pleasing language, but is incredibly expensive and closed source. Python is a convenient, pretty powerful language with a lot of support from the developer community. For as long as the two have existed people have been trying to tie them together, but very little has been done to do so at the native level with efficient, convenient exchange between the two. That's why over the past few weeks I took the time to build a clean, convenient link between the two. This post will go into how the link was built and some of its features, but first I think a little demo is appropriate. ## A Quick Demo ### Installing PJLink The link is based off of the [J/Link](https://reference.wolfram.com/language/JLink/tutorial/Overview.html) interface built into Mathematica for hooking up Java and Mathematica. To wit, I called it [PJ/Link](https://github.com/b3m2a1/PJLink) . It lives on my paclet server as well as GitHub, so we can easily install it from there: PacletInstall["PJLink", "Site"->"http://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer"] (*Out:*)  ### Loading PJLink in Jupyter For this demo we'll need the path to this thing as well (note that the version might change in the future): %["Location"] (*Out:*) "~/Library/Mathematica/Paclets/Repository/PJLink-1.0.0" Now we'll leave Mathematica and open up a Jupyter notebook:  Next we'll get that path available so we can actually make use of the package. Then we'll load things from the subsidiary ```SubprocessKernel``` package which is included in the paclet and makes use of PJLink: <pre>import os, sys pjlink_path = "~/Library/Mathematica/Paclets/Repository/PJLink-1.0.0" #this is whatever path was extracted before sys.path.insert(0, os.path.expanduser (pjlink_path)) from SubprocessKernel import SubprocessKernel from SubprocessKernel import MathematicaBlock, LinkEnvironment ## these are helpers I' ll use in the demo</pre>  ### Bidirectional Communication Once we have this we can start a subprocess kernel which will open a Mathematica front-end to interact with. We'll also start and evaluator Mathematica can use to call back into python. You may see a long string of output from your C compiler as the setup.py file builds out the native library that PJLink uses. Don't worry, this should only happen once. If it fails, raise an issue on [GitHub](https://github.com/b3m2a1/PJLink/issues) so I can deal with it. Once Mathematica has loaded, we'll use the ```MathematicaBlock``` context manager so we can write something that looks a lot like Mathematica code and use the ```MEval``` function we'll define to run the code. That code for all this looks like: <pre>ker = SubprocessKernel() def MEval (expr, wait = True, kernel = ker) : "" "MEval evaluates a Mathematica expression in the Mathematica kernel " "" kernel.drain() # just to make sure things are clen return kernel.evaluate (expr, wait = wait) ker.start() ker.start_evaluator()</pre> After that we can simply call into Mathematica: <pre>with MathematicaBlock(): res = MEval (Set (M.hi, "Hello from python!")) res</pre>  We can see string ```"Hello from python!"``` was set to the symbol ```hi``` on the Mathematica side and was returned back by ```MEval``` . Symbols that aren't in the ```"System`"``` context need to be prefaced by an ```M.``` as that's a special class that can resolve symbol names like that. We can also get efficient data transfer of arrays from either side. Here we'll take some Mathematica data and get it back out on the python side. The first thing we need to do is go to the Mathematica notebook that opened and load the ```"PJLink`"``` context. Then we'll install the python runtime that the ```SubprocessKernel``` object configured. This looks like: <<PJLink` InstallPython[ LinkObject->SubprocessKernel`$PyEvaluateLink, ProcessObject->None]; Once it's installed, we'll use it directly via ```PyEvaluate``` : With[{arr= RandomReal[{-1, 1}, {50, 50, 50}]}, PyEvaluate[dat=arr] ] Calls into python are done in an environment held only by the link, so to access that we need to wrap the evaluator we started ( ```ker.evaluator``` ) in a ```LinkEnvironment``` context manager: <pre>with LinkEnvironment(ker.evaluator): res = dat.shape res</pre>  Arrays are held as NumPy arrays by default on the python side, although this may be disabled. If disabled, they're held as a data type called ```BufferedNDArray``` which holds the data as a single C-contiguous array and allows slicing and viewing into it (although no efficient math or manipulation of any sort). Finally, to close out the demo, we'll plot something on the Mathematica side and watch it come back on the python side. The code for this should be pretty self-explanatory by this point, but there is one cute feature to note: <pre>with MathematicaBlock(): res = MEval( Rasterize( Plot(Sin (M.x), List (M.x, 0, Times (2, Pi)), ImageSize = [250, 250], PlotLabel = "sin(x) as plotted in Mathematica" ) ) ) res</pre> Unfortunately it really does matter that we pass a ```List``` expression instead of a python list for the second argument to ```Plot``` as otherwise the system hangs for reasons that aren't totally clear. On the other hand, we can see how nice options passing is in the interface. We make use of the python ```**kwargs``` setup and that ```ImageSize= ...``` and ```PlotLabel= ...``` both get automatically converted into rules (albeit with a ```String``` key instead of a ```Symbol``` ). The ```Rasterize``` is, sadly, similarly necessary as there is currently no logic in the package to automatically convert ```Graphics``` expression into their rasterized forms.  I think we'll close out the demo here, though, and move onto a description of how this works. ## PJLink Native Library The heart of PJLink is the C library that connects a python runtime to MathLink. The source for this can be found [here](https://github.com/b3m2a1/PJLink/blob/master/PJLink/PJLinkNativeLibrary/src/PJLinkNativeLibrary.cpp) . This library, once compiled by the setup.py file packaged with it, implements the basic MathLink calls in a way that python can use them and attempts to do so with efficient memory usage and data transfer. ### Data Sharing in the Native Library The heart of the native library is the set of ```PutArray``` and ```GetArray``` functions it implements. Beyond anything else, it is the fast transfer of large arrays of data that makes a C-level connection so appealing. The way we handle this on the python side is via the python [buffer protocol](https://docs.python.org/3/c-api/buffer.html) . We enforce the condition that all data sent and received on the python side must be handled by an object that can work with a C-contiguous buffer of data. By default this is done with [NumPy](http://www.numpy.org/) if it is installed, but if not there is a custom object called ```BufferedNDArray``` in the [HelperClasses](https://github.com/b3m2a1/PJLink/blob/master/PJLink/HelperClasses.py) package that deals with this. ### Threading in the Native Library Python has something called the [Global Interpreter Lock (GIL)](https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock) which is a method for synchronizing python state. Unfortunately for us, the presence of the GIL means that standard C calls of the kind we'll be using will cause all threads to lock. To get around this, every call into the MathLink library in the native library is wrapped in the ```MLTHREADED``` macro which handles the releasing and reacquiring of the lock. This allows our threads to work once more. Any extensions to the library should keep this in mind. ## Class Structures PJLink provides a glut of classes that handle the details communication, so we will quickly detail what the important ones do. More information is always available upon request. ### The *Link classes PJLink is based off of JLink and so it makes use of the same kind of class structure that JLink does. This means that it has a ```MathLink``` class that provides a template for the kind of link we'll work with and a ```KernelLink``` class that works specifically with Mathematica kernels. In general, we will only really work with a subclass of a ```KernelLink``` called a ```WrappedKernelLink``` that implements the ```KernelLink``` interface by calling into a ```NativeLink``` which is the only class which actually touches the native library at all. If one is controlling a Mathematica kernel from python, it will be handled by a ```WrappedKernelLink``` . ### Reader class The ```Reader``` class handles the other half of the communication. It waits for calls from Mathematica and processes them via the ```KernelLink._handlePacket``` function. Most commonly these calls in turn call ```KernelLink.__callPython``` which builds a python call from the symbolic python packet that ```PyEvaluate``` sends over the link. A ```Reader``` does its best not to completely prevent its link from passing data *to* Mathematica, but in general it is best not to depend on this as the ```NativeLink``` interface allows only a single thread to access the library at once for reasons of safety and stability. ### MathLinkEnvironment and MathLinkException The ```MathLinkEnvironment``` is a standalone class that handles all of the various flags and state that the links need. It centralizes all information about what a given token or flag from MathLink means and provides utility functions for working with this. ```MathLinkException``` is a subclass of the standard python ```Exception``` class that handles the MathLink-specific exceptions that are returned. It in turn calls into ```MathLinkEnvironment``` to learn what various exceptions mean. ### MPackage, MLSym, and MLExpr The HelperClasses package provides a large number of (generally) smaller classes that serve to make code cleaner in its implementation. A big part of this is done by the ```MPackage``` , ```MLSym``` , and ```MLExpr``` classes, which allow for a way to create packets with a syntax that looks more like standard Mathematica code. ```MLSym``` and ```MLExpr``` are types that a ```KernelLink``` knows how to put onto a link and ```MPackage``` provides utilities and a custom ```__getattr__``` so that the packet code can look like Mathematica code. ### MathematicaBlock and LinkEnvironment Both ```MathematicaBlock``` and ```LinkEnvironment``` are also in the HelperClasses. They both edit the current evaluation state as [context managers](https://docs.python.org/3/reference/datamodel.html#context-managers) so that explicit references to ```MPackage``` can be dropped and variables held by a given link can be easily accessed. Being context managers, they are both used via ```with``` statements and change the execution environment of the enclosing block. ### BufferedNDArray, ImageData, and SparseArrayData These are all data classes that allow for more efficient and convenient data transfer. The ```ImageData``` and ```SparseArrayData``` classes hold data coming in from Mathematica as put using ```PJLink`Package`AddTypeHints``` . They have methods to efficiently transform to more standard formats like ```PIL.Image``` and ```scipy.sparse.csr_matrix``` . As more data types are handled by ```AddTypeHints``` it can be assumed that more classes like these will be written. ## Mathematica-side Package That was all to do with the python side of things, which is where most of the development work had to go. On the other hand, the Mathematica side of the equation still requires some explanation. The package itself is really quite simple, so please feel free to [peruse the source](https://github.com/b3m2a1/PJLink/blob/master/Mathematica/PJLink.wl) . ### InstallPython ```InstallPython``` is the most basic function in the package. It either finds or is given a python version or executable, attempts to open it via ```StartProcess``` , then links to it via ```LinkCreate``` and the ```start_kernel.py``` script provided in the PJLink python package. Notably, all it really requires is a ```LinkObject``` , so you can pass one directly via the ```LinkObject``` option. It will also by default try to make a python ```ProcessObject``` but you can pass that via the ```ProcessObject``` option or you can pass ```None``` in which case it won't attach to a Mathematica controlled process. ### ClosePython ```ClosePython``` is the counterpart to ```InstallPython``` . It closes an opened python runtime by version or executable. When a new kernel is installed it is added to ```PJLink`Package`$PythonKernels``` and this is what ```ClosePython``` looks for to close. ### PyEvaluate / PyEvaluateString This is the heart of the package. It takes Mathematica-esque code, converts it into a structure that can be processed by ```KernelLink.__callPython()``` and sends it over and waits for a response. The conversion is handled by ```PJLink`SymbolicPython`ToSymbolicPython``` which was originally written for the [PyTools package](https://github.com/b3m2a1/mathematica-PyTools) . This is the best way to move data to python as things like ```Image``` objects, packable arrays, and ```SparseArray``` objects will be moved over intelligently. ```PyEvaluateString``` is like ```PyEvaluate``` , but with the recognition that ```ToSymbolicPython``` will always be a little bit lacking. It allows one to simply call a string of python code on the link and get the results back. ### PyWrite / PyWriteString / PyRead / PyReadErr These are all functions that make use of the fact that when the ```Reader``` object started it allowed an interactive session to keep running and reading / writing on stdin, stdout, and stderr. The ```Read``` functions read from stdout and stderr and the write functions write to stdin. The former takes Mathematica code and auto-converts it into a string. The latter simply passes in the given string. ## Future Work PJLink 1.0.0, beefy as it already is, should only really be seen as the beginning. My hope is that much more can be done to allow for more native data type transfer and for intelligent communication between the two systems. In my demo I tried to show some of the things that make the interoperation of the two so nice, but I obviously don't have the breadth of knowledge to know all of the many applications this can be put to. Applications built off of PJLink are always welcome and I'm happy to provide any requisite information and extensions to PJLink to get them built. Alongside that, I think better integration on the Mathematica side is necessary. There is a partial interface for allowing a ```PythonObject``` structure to hide the details of ```PyEvaluate``` on the Mathematica side, but this needs work from both ends, first hooking up the ```Language`MutatationHandler``` interface and then extending the same on the python side. After that, a ```JavaBlock``` -like setup that allows a link to be opened, used, and cleaned up would be highly useful for sandboxing. Finally, I'm sure there are numerous bugs hiding in the package as it stands. Please find them and let me know about them so they can be worked out. In the meantime, I hope you enjoy PJLink and being able to use my two favorite languages symbiotically. b3m2a1 ​  2018-09-20T09:18:51Z