https://community.wolfram.com RSS Feed for Wolfram Community showing ideas tagged with Packages sorted by most viewed. https://community.wolfram.com/groups/-/m/t/787142 ## General information and download links ## If you're interested in crystal structures, you can now download the Crystallica application from the Wolfram Library Archive, and then you can do things like this: Needs["Crystallica`"]; CrystalPlot[ {{5.4,0,0},{0,5.4,0},{0,0,5.4}}, {{0,0,0},{0,0,.5},{0,.5,0},{.5,0,0},{.24,.24,.24},{.24,.76,.76},{.76,.24,.76},{.76,.76,.24}}, {1,2,2,2,3,3,3,3}, AtomCol->{"Firebrick","YellowGreen",White},AtomRad->.4, BondStyle->2,BondDist->3, CellLineStyle->False,AddQ->True,Lighting->{{"Directional",White,ImageScaled[{0,0,1}]}},Background->Black] ![enter image description here][1] Here are the download links for Crystallica and two other packages you may need: [Crystallica][2] - contains the functions `CrystalPlot` and `CrystalChange` [CifImport][3] - contains an import function for CIF files [VaspImport][4] - contains an import function for files related to [VASP][5] Once you've installed Crystallica (by saving the entire Crystallica folder - not the zip archive - to `$USerBaseDirectory/Applications` and re-starting the Kernel), you can enter Crystallica into the Documentation Center and you'll find lots of useful examples. Most of the examples in this post are taken from the Documentation. For the other two packages, just install them and evaluate this: ?CifImport ?VaspImport I'll first show you a few things the `CrystalPlot` function can do when you already have crystal structure data inside Mathematica, wherever it may have come from. Then we'll take a look at how to get the data into Mathematica in the first place, which is where `CifImport` and `VaspImport` will come into play - but we'll get data from other sources as well. I'll cover the different import solutions in separate replies to this thread, because I have a feeling that I'll be rambling on and on and on... ## Simple plot ## Traditional ball-and-stick plots are usually just fine, so the simplest thing you can do is this: CrystalPlot[ {{4.5,0,0},{0,4.5,0},{0,0,3}}, {{0,0,0},{.5,.5,.5},{.2,.8,.5},{.3,.3,0},{.7,.7,0},{.8,.2,.5}}, {1,1,2,2,2,2}] ![enter image description here][6] As you can see, `CrystalPlot` expects three arguments. The first one contains the lattice vectors, which are simply the three vectors that create the parallelepiped that constitutes the cell. The second argument contains the atomic coordinates, but they're given in the basis of the lattice vectors (which is quite useful in crystallography). The third argument is a list of integers that gives the atom types, with one entry for each atom. If you want to plot a molecule instead, you can call `CrystalPlot` with just two arguments: A list of atom coordinates in cartesian space, and a list of atom types. Everything else you see in the plot - the atoms, bonds, colours, arrows etc. - represents the default settings of various layout options. ## Advanced atoms and bonds ## Let's take a look at some more advanced options just for fun. For instance, atoms and bonds can look any way you need them to, because you can specify your own functions for them. You can also fine-tune where to put bonds and what to do with their thickness and colour in a physically (or chemically) meaningful way, but I won't show that here. So here are some customized atoms and bonds: Row[Table[ CrystalPlot[{{4,0,0},{0,4,0},{0,0,4}},{{0,0,0},{.4,.4,.4},{.8,.8,.8}},{1,2,3}, AtomRad->{.4,1.2,.7},AtomFunction->style,ImageSize->400], {style,{ (Ball[#1,#2]&), (Scale[Sphere[#1,#2],{1,1,.5}]&), ({EdgeForm[Thick],Opacity[.7],Cuboid[#1-.5*#2,#1+.5*#2]}&) }}]] ![enter image description here][7] Row[Table[ CrystalPlot[{{0,0,0},{5,0,0},{2.5,4,0}},{1,2,3},BondDist->6,BondStyle->style,ImageSize->400], {style,{ 1, Function[{bonds,partcol},Table[{If[ii<.5,partcol[#,1],partcol[#,2]],Sphere[bonds[[#,1]]+ii*(bonds[[#,2]]-bonds[[#,1]]),.15]},{ii,0,1,1/9}]&/@Range[Length[bonds]]], Function[{bonds,partcol},Module[{spiral,points,rad=.05}, spiral[atoms_]:=Module[{scale=.5,dist=atoms[[2]]-atoms[[1]],curls=60,normal,rot,scaled}, normal=Table[{scale*Cos[ii],scale*Sin[ii],.1*ii},{ii,0,curls,\[Pi]/10}]; scaled={#[[1]],#[[2]],10*Norm[dist]/curls*#[[3]]}&/@normal; rot=scaled.Quiet[RotationMatrix[{dist,{0,0,1}}]]; Join[{atoms[[1]]},#+atoms[[1]]&/@(rot[[25;;-25]]),{atoms[[2]]}]]; points=spiral/@bonds; {partcol[#,1],Tube[BSplineCurve[points[[#,;;Round[Length[points[[#]]]/2]]],rad]],partcol[#,2],Tube[BSplineCurve[points[[#,Round[Length[points[[#]]]/2];;]],rad]]}&/@Range[Length[bonds]] ]] }}]] ![enter image description here][8] ## Lattice planes ## Crystallica can also add lattice planes to the plot. You can specify them using [h,k,l] Miller indices and distance to the origin. CrystalPlot[{{3,0,0},{0,3,0},{0,0,3}},{{0,0,0}},{1}, AddQ->True,AtomRad->.3,AtomCol->"CadmiumYellow",Sysdim->2,CellLineStyle->2, LatticePlanes->Table[{{1,1,1},dist},{dist,1,5}],ContourStyle->{"TerreVerte",Opacity[.7]},BoundaryStyle->Thick] ![enter image description here][9] ## Coordination polyhedra ## You can automatically search for and plot coordination polyhedra. This is not limited to the commonly occurring tetrahedra and octahedra - you can actually look for polyhedra with arbitrary numbers of corners. There are also options to fine-tune both the searching and the rendering. plot[corners_,mixed_]:=CrystalPlot[{{0,0,0},{0,0,1.8},{-.9,-1.5,-.6},{-.9,1.5,-.6},{1.7,0,-.6},{.8,.8,.8}},{1,2,2,2,2,3}, BondStyle->False,ImageSize->250, PolyMode[corners]->{"Show"->All,"AllowMixed"->mixed},PolyStyle[corners]->Directive[Opacity[.5],EdgeForm[Thick]]]; Grid[{{ "", "Search for polyhedra with \n4 corners", "Search for polyhedra with \n5 corners" },{ "Allow \nmixed corners", plot[4,True], plot[5,True] },{ "Don't allow \nmixed corners", plot[4,False], plot[5,False] }},Dividers->All] ![enter image description here][10] CrystalPlot[{{2.5,-4.3,0},{2.5,4.3,0},{0,0,5.5}}, {{.5,0,0},{0,.5,.7},{.5,.5,.3},{.2,.4,.5},{.6,.8,.2},{.2,.8,.8},{.8,.6,.5},{.4,.2,.2},{.8,.2,.8}},{1,1,1,2,2,2,2,2,2}, PolyMode[4]->True,PolyStyle[4]->EdgeForm[None],AddQ->True, Sysdim->2,AtomRad->0,CellLineStyle->False,AtomCol->{"SlateGray","Firebrick"}, ViewAngle->.4,ViewPoint->{3.2,0,1.1},ViewVertical->{.5,0,1.2}] ![enter image description here][11] ## Other things ## Visualization aside, you can also build supercells, change cell shapes, or add, remove and sort atoms... but that's a bit boring to read, so I'll refer you to the Documentation page of the `CrystalChange` function instead. If you're interested, we can use this thread to talk about any questions you may have, or you can share your use of the package (if you decide to use it). I'm not offering full support here, but I'll be floating around, and I'd like to hear your feedback. We don't have any intentions to be involved in further development. But if you have a good idea and some time, then by all means, work on it for yourself, or host it on your favourite code collaboration site. Bianca Eifert and Christian Heiliger [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=teaser.png&userId=69107 [2]: http://library.wolfram.com/infocenter/MathSource/9372/ [3]: http://library.wolfram.com/infocenter/MathSource/9373/ [4]: http://library.wolfram.com/infocenter/MathSource/9375/ [5]: http://vasp.at/ [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=9692simple.png&userId=69107 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=atoms.png&userId=69107 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=bonds.png&userId=69107 [9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=planes.png&userId=69107 [10]: http://community.wolfram.com//c/portal/getImageAttachment?filename=polys.png&userId=69107 [11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=polys2.png&userId=69107 [12]: http://rruff.geo.arizona.edu/AMS/CIF_text_files/13532_cif.txt [13]: http://cms.mpi.univie.ac.at/vasp/vasp/POSCAR_file.html [14]: http://wiki.jmol.org/index.php/File:Caffeine.mol Bianca Eifert 2016-02-05T18:43:18Z https://community.wolfram.com/groups/-/m/t/3118205 ![enter image description here][1] &[Wolfram Notebook][2] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=heroimage.png&userId=1539902 [2]: https://www.wolframcloud.com/obj/5b3b5265-f1e2-406c-aef4-45a5f9ec363b Mohammad Bahrami 2024-02-07T19:21:52Z https://community.wolfram.com/groups/-/m/t/2330900 Daily Study Groups are back, now providing expertise on practical programming! After a short break, we're starting up with a series that picks up where Wolfram Language Basics sessions ended. This 3-week series will take you from basic programming concepts to package development. Need tips on improving your code’s speed or functionality? Programming Tutorials offer a great opportunity to level-up your skills while interacting with software development professionals. See hands-on examples and get questions answered by Wolfram Language experts. Plus: participants who pass weekly quizzes are awarded a program completion certificate. Level 1 certification is available for those who pass manually graded exercises. Join us any time between August 2 and August 20! Check out our [registration page][1] for more details. [1]: https://wolfr.am/Study_Group15 Wolfram U 2021-07-30T15:24:02Z https://community.wolfram.com/groups/-/m/t/214901 In a response to an earlier thread I posted a notebook containing a basic tutorial on creating packages in Mathematica. Since the link to the notebook was embedded down in the responses to that post I thought it might be useful to some folks to have access to it at the top level here since they might have missed it. I have updated that notebook a bit since the last time.  I hope it is useful to some people.  It was originally written as a tutorial for a consulting client of mine and he agreed to allow me to share with others.   Enjoy. It is attached to this posting David Reiss 2014-03-08T02:32:19Z https://community.wolfram.com/groups/-/m/t/984003 I'm going to show you how you can use existing functionality to run a Mathematica script across a managed HPC cluster. Before I start, I must be upfront with you: though the individual commands are documented, this method, as a whole, is not. Thus, support for this procedure is outside the scope of Wolfram Technical Support. However, I'm hoping that once the ground-work has been laid, that Wolfram Community members can work together to fill in the missing details. My assumptions: 1. Mathematica is installed and properly licensed on the managed cluster 2. once your job has been given resources, that you can freely SSH between them (1) This is up to your local cluster's System Admin to figure out by talking with their organization and a Wolfram Sales Representative, and possibly Wolfram Technical Support ([support.wolfram.com][1]). (2) Again, this is up to your local SysAdmin to ensure. It's also known as a public/private key pair between nodes. In the following, I'm assuming the cluster uses Torque ([Torque SysAdmin Guide][2]), but in principle other managers can be used. A generic Mathematica script job submission may look like the following: #PBS -N Job_name #PBS -l walltime=10:30 #PBS -l nodes=4:ppn=6 #PBS -m be math -script hpc.wl In this example, - the job is called "Job_name" - the job will finish in 10 and a half minutes - it is requesting 4 nodes with 6 processors-per-node, for a total of 24 resources (CPU cores) - an email will be sent to the account associated with the username when the job (b)egins and when it (e)nds If you are not familiar with job submissions to a managed HPC cluster, then I suggest you read any guides provided by your organization The Wolfram Language script "hpc.wl" does the rest of the work. It generically follows this order: 1. gather the environment variables associated with the list of provided resources 2. launch remote subkernels for each CPU core 3. do the parallel computations 4. close the subkernels 5. end the job (*get association of resources, name of local host, and remove local host from available resources*) hosts = Counts[ReadList[Environment["PBS_NODEFILE"], "String"]]; local = First[StringSplit[Environment["HOSTNAME"],"."]]; hosts[local]--; (*launch subkernels and connect them to the controlling Wolfram Kernel*) Needs["SubKernels`RemoteKernels`"]; Map[If[hosts[#] > 0, LaunchKernels[RemoteMachine[#, hosts[#]]]]&, Keys[hosts]]; (* ===== regular Wolfram Language code goes here ===== *) Print[ {$MachineName, $KernelID} ] (* ===== end of Wolfram Language program ===== *) CloseKernels[]; Quit On Torque there is the environment variable "PBS\_NODEFILE" ([Torque environment variables][3]) that lists the different nodes that are provided to the job. It is my understanding that the name is repeated for each CPU core. That's why a simple Count of the node list tells us everything. The other piece of information, which is probably not necessary, is "HOSTNAME". This is where the Wolfram controlling kernel is running. In the above, we remove it from the list of available resources, but I don't believe this is necessary. According to the documentation ([3]), this may be known as "PBS\_O\_HOSTNAME". The Mathematica script should not need to change save for the code between the commented lines. I'm also assuming that `$RemoteCommand` (provided by Subkernels\`RemoteKernels\`) is the same on each node. This is usually the case as most clusters use a cloned file system. SLURM should be very similar except that the environment variables will be different. It is my understanding that headNode = Environment["SLURMD_NODENAME"]; nodes = ReadList["!scontrol show hostname $SLURM_NODELIST",String]; provides the headnode and list of resources. I encourage discussion. [1]: http://support.wolfram.com [2]: http://docs.adaptivecomputing.com/torque/4-0-2/help.htm [3]: http://docs.adaptivecomputing.com/torque/4-0-2/help.htm#topics/2-jobs/exportedBatchEnvVar.htm Kevin Daily 2016-12-19T21:29:53Z https://community.wolfram.com/groups/-/m/t/1170226 One of the great things about Mathematica is that it makes it so easy to do so much in so little code. It is partly because of this that I am so frustrated with the amount of time I need to spend developing the *interface* of package functions compared to their *functionality*. Suppose you wrote a Wolfram Language function that does something useful and interesting. Now you want to wrap it up into a package, and make it usable by everyone. A polished package is expected to have functions that: - Will check their input for errors - Will report errors in an informative way - Will use messages appropriately (i.e. associate them with the correct symbol name) - Will adhere to the de-facto Mathematica interface conventions: proper use of optional arguments, options, option inheritance (as in Graphics -> Plot), default option value handling, use of `Automatic`, etc. - Have SyntaxInformation I find that not infrequently I spend more time on making the function user-friendly than developing its functionality. **How do people generally deal with this task? How do you implement error checking and reporting in your packages?** To give an example, take a function as simple a moving average calculator. It is really easy to implement: movingAverage[vec_, n_] := Mean /@ Partition[vec, n, 1] But to bring it to the quality of the built-in `MovingAverage`, it should at least: - check the number of arguments (precisely 2) - check the types of arguments (a list and an integer) - check the values of arguments for correctness (non-empty list and positive integer) - make sure that all these checks don't introduce severe performance degradation (such as array unpacking, which can even be triggered by an inefficient argument pattern) This also involves the introduction of multiple messages (for each type of error) associated to `movingAverage`. If we now want to add a `movingMedian`, we will find that we will mostly need to carry out the same checks and report the same messages. There will be small differences though, e.g. average calculations are feasible for symbolic lists like `{1,x}`, but not median calculations. So the checks won't quite be identical. The messages will be mostly identical, but each function must associate messages to its own symbol, which means a lot of duplication. So if we care about a high-quality interface and high-quality error reporting, we will end up writing considerably more code for this than for the function's core task. We will also end up with a lot of code duplication, which is frustrating and a maintenance burden. The whole thing ends up being a lot of work and not a lot of fun (which is not very Mathematica-like :-) ) Are there good ways to simplify these tasks? Option handling also used to be error-prone and frustrating, but the introduction of `OptionsPattern[]` and `OptionValue[]` made it much easier. ---- I can see that there are some built-in tool to ease these tasks, though they are mostly undocumented. One example is `ArgumentCountQ`, another is ``Developer`CheckArgumentCount``, or some tools in ``GeneralUtilities` ``. I would love to hear from others about how they deal with the tasks I described, which such internal functions they make use of, etc. Szabolcs Horvát 2017-08-25T11:15:29Z https://community.wolfram.com/groups/-/m/t/560469 *WOLFRAM MATERIALS for the ARTICLE:* > Szabolcs Horvát, Jakub Podkalicki, Gábor Csárdi, Tamás Nepusz, Vincent Traag, Fabio Zanini, Daniel Noom, (2022). > *IGraph/M: graph theory and network analysis for Mathematica*. > arXiv:2209.09145 **[physics.soc-ph]**. > https://doi.org/10.48550/arXiv.2209.09145 [](https://igraph.discourse.group) [](https://github.com/szhorvat/IGraphM/releases) [](https://github.com/szhorvat/IGraphM#contributions) [](https://doi.org/10.5281/zenodo.1134932) ---- ##Article abstract IGraph/M is an efficient general purpose graph theory and network analysis package for Mathematica. IGraph/M serves as the Wolfram Language interfaces to the igraph C library, and also provides several unique pieces of functionality not yet present in igraph, but made possible by combining its capabilities with Mathematica's. The package is designed to support both graph theoretical research as well as the analysis of large-scale empirical networks. ---- ##Introduction The post below was written for the original release of IGraph/M. The package has come a long way since then and now contains ~300 functions. See http://szhorvat.net/mathematica/IGraphM for more details on the current release. Compatibility: 64-it Windows/macOS/Linux or Raspberry Pi; Mathematica <del>10.0</del> 11.0 or later. <a href="http://szhorvat.net/mathematica/IGraphM"><img src="https://community.wolfram.com//c/portal/getImageAttachment?filename=IGraphM-ad-3.png&userId=38370" width="300"></a> ---- I would like to announce IGraph/M, a new igraph interface for Mathematica: http://szhorvat.net/mathematica/IGraphM [igraph](http://igraph.org/) is a graph manipulation and analysis package. IGraph/M makes its functionality available from Mathematica. This initial release, version 0.1, covers only some igraph functions, as I focused on the things that I need personally. However the main framework is complete, and new functions can be added quickly. If anyone would like to contribute, please contact me. Binary packages for OS X (10.9 or later) and Linux can be downloaded [from GitHub](https://github.com/szhorvat/IGraphM/releases). Unfortunately, I was unable to compile the development version of igraph for Windows, so I cannot provide a Windows version. If you can help with compiling igraph itself (not IGraph/M) on Windows, please let me know! Functionality in this release that is not built into Mathematica: * Vertex betweenness centrality for weighted graphs * Estimates of vertex betweenness, edge betweenness and closeness centrality; for large graphs * Minimum feedback arc set for weighted and unweighted graphs * Find all cliques (not just maximal ones) * Count 3- and 4-motifs * Rewire edges, keeping either the density or the degree sequence * Alternative algorithms for isomorphism testing: Bliss, VF2 * Subgraph isomorphism * Test if a degree sequence is graphical * Alternative algorithms for generating random graphs with given degree sequence * Layout algorithms that take weights into account Note that IGraph/M is *not a replacement* for Mathematica's graphs and networks functionality. It is meant to complement what is already available in Mathematica, thus it primarily focuses on adding functionality that is not already present. Why did I release the package before covering most of the igraph functionality? I do not have time to work on things I do not personally need or use, so I am unlikely to extend it further unless the need comes up. I do think that the functions that are included in v0.1 can already be useful to others too. I would also like to give the opportunity for people to contribute to the project if they wish to. The groundwork has been laid, so further extensions should be quick and relatively easy. Also check out a related project, [IGraphR](https://github.com/szhorvat/IGraphR), which makes igraph available for Mathematica users through RLink. I wrote IGraph/M because I needed higher performance and greater reliability (especially for parallel computing) than what RLink could provide. ---- **A request:** If any of you have used IGraphR in the past to access igraph from Mathematica, please post a response to this thread and let me know which specific functions you were using. &[Wolfram Notebook][1] [1]: https://www.wolframcloud.com/obj/94639221-60b4-47e3-8862-d996caa40388 Szabolcs Horvát 2015-09-06T12:55:14Z https://community.wolfram.com/groups/-/m/t/1484511 Originally [here](https://www.wolframcloud.com/objects/b3m2a1/home/playing-with-youtube-from-mathematica.html#main-content) (the embedded iframes actually work there) --- Playing with YouTube from Mathematica --- Today I'm gonna quickly show off something I made for a friend (and about which I'd been thinking for some time). We're gonna use the YouTube API in Mathematica to do a have some fun (but not too much fun, mind you!). ### Installing the YouTube Connection My connection to the YouTube API is built on Mathematica's [```ServiceConnect```](https://reference.wolfram.com/language/ref/ServiceConnect.html) framework. I've [talked](https://www.wolframcloud.com/objects/b3m2a1/home/serviceconnections---mathematica-on-the-web.html) [about](https://www.wolframcloud.com/objects/b3m2a1/home/questions-on-the-mathematica-stack-exchange.html#main-content) [these](https://www.wolframcloud.com/objects/b3m2a1/home/questions-on-the-mathematica-stack-exchange-part-2.html#main-content) [before](https://mathematica.stackexchange.com/questions/163638/how-can-i-access-nasa-images-and-apis) in [many](http://community.wolfram.com/groups/-/m/t/1261562) [different](https://mathematica.stackexchange.com/a/167702/38205) [contexts](https://mathematica.stackexchange.com/questions/163780/sophisticated-api-integrations-making-a-card-game-in-mathematica) but today we'll just use it. To install we pull the [paclet off my server](https://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer/serviceconnectionyoutube.html#main-content) : PacletInstall["ServiceConnection_YouTube", "Site"-> "http://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer/" ] (*Out:*)  ### Authenticating with YouTube One installed, we connect to the service as usual: $so = ServiceConnect["YouTube"] This will open a dialog:  We'll click the sign in button, pick and account (I'll use my [new personal account steve.wolfraum@gmail.com](https://www.wolframcloud.com/objects/b3m2a1/home/making-a-blog-in-30-minutes.html#main-content) ):  When we click "Allow" either we'll be taken to a nicely formatted local page with an access code running via ```HTTPHandling`StartWebServer``` or if that's being glitchy as it was right now, we'll copy the part of the URL starting with ```"code=4/..."``` :  Finally, we're connected: (*Out:*)  ### Getting Data The majority of the currently supported methods have to do with getting data about YouTube: $so["Requests"] (*Out:*) {"Authentication","DeleteVideo","ID","Information","LastRequest","ListCaptions","ListChannelActivity","ListChannels","ListChannelSections","ListComments","ListCommentThreads","ListPlaylistItems","ListPlaylists","ListSubscriptions","ListVideos","Name","RawRequests","RequestData","RequestParameters","Search","UpdateVideo","UploadVideo","VideoURL"} We'll start with a search, maybe. First let's see what parameters we have to work with: $so["RequestParameters", "Request"->"Search"] (*Out:*) <|"Parameters"->{"q","part","forContentOwner","forDeveloper","forMine","relatedToVideoId","channelId","channelType","eventType","location","locationRadius","maxResults","onBehalfOfContentOwner","order","pageToken","publishedAfter","publishedBefore","q","regionCode","relevanceLanguage","safeSearch","topicId","type","videoCaption","videoCategoryId","videoDefinition","videoDimension","videoDuration","videoEmbeddable","videoLicense","videoSyndicated","videoType"},"Required"->{"part"}|> Then we'll search for something bland, like "Steve Wolfraum". The YouTube API requires this ```"part"``` parameter to know what to return to use. I like to get the ```"id"``` and ```"snippet"``` parts, but you can [pick for yourself](https://developers.google.com/youtube/v3/getting-started#part) which parts you want. Just comma-separate your values and you can get multiple different ones: mySearch = $so["Search", "q"->"Steve Wolfraum", "part"->"id,snippet"] (*Out:*)  We see there are *tons* of results for "my" name but we only get them five at a time. We can use the ```"nextPageToken"``` to get the next page of results, though. We can bump up the number we get with the ```"maxResults"``` parameter, but I see no need for that right now. Let's look at the first result here. I'll drill straight into that snippet parameter, as that's the most interesting part: mySearch["items", 1, "snippet"] (*Out:*)  And we see, sadly, that instead of getting results for my name, Steve Wolfraum, we get them for some random dude name Stephen Wolfram. Not to be deterred, let's at least get something out of this and see a thumbnail for all our results. Tragically, we can't download videos with the YouTube API, but at least we get thumbnails...hooray? To make things more fun, we'll also add a filter to all these results, as the raw thumbnails really aren't all that interesting. ImageAdjust@LaplacianGaussianFilter[Import[#], 3]&/@ Normal@mySearch["items", All, "snippet", "thumbnails", "high", "url"] (*Out:*)  Cool. There's some data. We can also pull out statistics and things, say for different videos broadcast on the Wolfram Research channel. First we'll get their channel ID: wriChannel= $so["Search", "q"->"Wolfram Research", "type"->"channel", "part"->"id", "maxResults"->"1"]["items", 1, "id", "channelId"] (*Out:*) "UCJekgf6k62CQHdENWf2NgAQ" Now we'll use this to extract the 50 most popular videos: wriVids=$so["Search", "channelId"->wriChannel, "type"->"video", "order"->"viewCount", "part"->"id,snippet", "maxResults"->"50"] (*Out:*)  And now we'll get view counts: wriStats=$so["ListVideos", "id"->StringRiffle[Normal@wriVids["items", All, "id", "videoId"], ","], "part"->"statistics" ] (*Out:*)  And then we'll make a [```WordCloud```](https://reference.wolfram.com/language/ref/WordCloud.html) of titles vs viewcounts: WordCloud@ AssociationThread[ Normal@wriVids["items", All, "snippet", "title"], ToExpression@Normal@wriStats["items", All, "statistics", "viewCount"] ] (*Out:*)  And we see as far as YouTube cares, there's really only one video WRI has made. But dropping that one we might see some more interesting structure: WordCloud@ AssociationThread[ Rest@Normal@wriVids["items", All, "snippet", "title"], Rest@ToExpression@Normal@wriStats["items", All, "statistics", "viewCount"] ] (*Out:*)  And it seems that people really like the basics and...really like seismic waves? I'll embed the first of those so we can see how good the really are: $so["EmbedIFrame", "id"->wriVids["items", 3, "id", "videoId"]] (* Out *) <iframe id="ytplayer" type="text/html" width="640" height="360" src="https://www.youtube.com/embed/2rYjlVPU9U4?autoplay=0" frameborder="0"></iframe> ### Uploading Videos A YouTube API wouldn't be much of a YouTube API if it couldn't upload videos, of course, so I cooked that in as well. Here's a sample of that. We'll first pull an animation off the [```Manipulate```](https://reference.wolfram.com/language/ref/Manipulate.html) docs: anim = Export[ FileNameJoin@{$TemporaryDirectory, "plot3d.flv"}, Manipulate[ Plot3D[Sin[n x y], {x, 0, 3}, {y, 0, 3}, ViewPoint -> Dynamic[{2, v, 2}], SphericalRegion -> True, Ticks -> None, PerformanceGoal->"Quality" ], {n, 1, 4}, {v, -2, 2} ] ]; Then we'll upload the video we made: upload = $so["UploadVideo", "part"->"id", "BodyContent"->anim]; Then we attach a title and things in a second request: $so["UpdateVideo", "id" -> upload["id"], "part" -> "snippet", "Title" -> "Mathematica Examples: Plot 3D", "CategoryID" -> "22" ] (*Out:*)  And then now we'll embed that video we just made: $so["EmbedIFrame", "id"->upload["id"]] (* Out *) <iframe id="ytplayer" type="text/html" width="640" height="360" src="https://www.youtube.com/embed/Quzcr4Vuq_Q?autoplay=0" frameborder="0"></iframe> b3m2a1 ​  2018-09-29T04:42:36Z https://community.wolfram.com/groups/-/m/t/1646303 # UNET [](https://zenodo.org/badge/latestdoi/137186334) [](https://github.com/dwyl/esta/issues) ---------- ![Automated 3D muscle segmentation using UNET / RESNET using DIXON MRI data][17] A package to generate and train a UNET deep convolutional network for 2D and 3D image segmentation. Some code was based on [work][1] by [@Ali Hashmi][at0], which was also dicussed in [this post][2] The full version of the toolbox can be found on my [github page][3]. * [Information](#information) * [Install toolbox](#install-toolbox) * [Using the toolbox](#using-the-toolbox) * [Functionality](#functionality) * [Visualization](#visualization) * [Example](#example) ## Information UNET is developed for [Mathematica](https://www.wolfram.com/mathematica/). It contains the following toolboxes: - UnetCore - UnetSupport Documentation of all functions and their options is fully integrated in the Mathematica documentation. The toolbox always works within the latest version of Mathematica and does not support any backward compatibility. All code and documentation is maintained and uploaded to github using [Workbench](https://www.wolfram.com/workbench/). ## Install toolbox Install the toolbox in the Mathematica UserBaseDirectory > Applications. FileNameJoin[{$UserBaseDirectory, "Applications"}] ## Using the toolbox The toolbox can be loaded by using <<UNET` The notbook ``UNET.nb`` shows examples of how to use the toolbox on artificially generated 2D data. There are also examples how to visualize the layer of your trained network and how to visualize the training itself. ## Functionality The network supports multi channel inputs and multi class segmentation. * UNET generates a UNET convolutional network. * 2D UNET ![UNET 2D][4] * 3D UNET ![UNET 3D][5] * Loss Layers: Training the data is done using three loss layers: a SoftDiceLossLayer, BrierLossLayer and a CrossEntropyLossLayer. ![SoftDiceLossLayer, BrierLossLayer and a CrossEntropyLossLayer][6] * Convolution Blocks: The toobox contains five different convolution blocks that build up the network: [UNET][7], UResNet, [RestNet][8], UDenseNet, [DensNet][9]. ![Convolution blocks][10] * SplitTrainData splits the data and labels into training, validation and test data. ![split train Data][11] * TrainUNET trains the network. ![Train UNET][12] ## Visualization * Visualize the network and results. * Visualize the features of the layers. ![Visualize layer features][13] * Visualize the results. ![Visualize the results][14] * Animate the training process. ![UNET 2D animation][15] ![UNET 3D animation][16] ## Example * Example: 3D segmentation of lower legg muscles using MRI data. ![Automated 3D muscle segmentation using UNET / RESNET using DIXON MRI data][17] [1]: https://github.com/alihashmiii/UNet-Segmentation-Wolfram [2]: https://community.wolfram.com/groups/-/m/t/1341081?p_p_auth=w8PIeeiA [3]: https://github.com/mfroeling [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=UNET2D.PNG&userId=1332602 [5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=UNET3D.PNG&userId=1332602 [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Loss.PNG&userId=1332602 [7]: https://arxiv.org/abs/1505.04597 [8]: https://arxiv.org/abs/1512.03385 [9]: https://arxiv.org/abs/1608.06993 [10]: https://community.wolfram.com//c/portal/getImageAttachment?filename=convblocks.PNG&userId=1332602 [11]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Split.PNG&userId=1332602 [12]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Train.PNG&userId=1332602 [13]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Visualize1.PNG&userId=1332602 [14]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Visualize2.PNG&userId=1332602 [15]: https://community.wolfram.com//c/portal/getImageAttachment?filename=amin0-v2.gif&userId=1332602 [16]: https://community.wolfram.com//c/portal/getImageAttachment?filename=amin4-v2.gif&userId=1332602 [17]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Muscle_Segmentation.jpg&userId=1332602 [at0]: https://community.wolfram.com/web/alihashmi87 Martijn Froeling 2019-04-03T20:01:26Z https://community.wolfram.com/groups/-/m/t/1655203 This is a cross-post from [here](https://b3m2a1.github.io/easyide-an-ide-for-mathematica.html) --- Contrary to what would probably be best practice, I do all of my Mathematica development inside Mathematica itself. To support this I built out a suite of [application development tools](https://paclets.github.io/PacletServer/btools.html) , a [web site builder](https://paclets.github.io/PacletServer/ems.html) , a [bug tracker](https://github.com/b3m2a1/BugTracker) , and a [documentation writing system](https://paclets.github.io/PacletServer/simpledocs.html) . Each of these worked nicely for me separately, but each of these required a palette and each one ran on notebooks, which meant that my screen filled with too many notebooks to keep track of. And then for each of these palettes and systems I had to write new resource finding code based off the palette or some arbitrarily imposed root directory or else provide some other way to specify where things would be found. In short, it got messy. Then, in a very relaxing hiatus from Mathematica I did some python development, writing a package for linking [Mathematica to python](https://paclets.github.io/PacletServer/pjlink.html) as well as some stuff for coordinate transforms and finite differencing and other little utilities. In doing this I noticed that everything was just...better. Partly this is because python is much nicer to write significant amounts of code in, being a language that actually supports developers and with actual object orientation and modularity. But another significant part of it was in the tools available to me. In particular I had the python plugin to [IntelliJ](https://www.jetbrains.com/idea/) , which is also repackaged as [PyCharm](https://www.jetbrains.com/pycharm/) . The fact that I had tabbing, plugins (e.g. for [Git](https://git-scm.com/) ), a file browser inside my dev environment, etc. was at once so entirely normal (I used to be a python programmer before switching over to mostly using Mathematica) and at the same time so nice. I then tried to use the *very nice and well constructed* [IntelliJ plugin for Mathematica](https://wlplugin.halirutan.de/) but it was just too much of a hurdle to lose everything I was used to and liked about writing my code directly in Mathematica. And that long, unnecessary background is why today we're gonna look at a Mathematica IDE written and operating entirely within Mathematica. # EasyIDE Mostly for the rhyme, I called this thing [EasyIDE](https://paclets.github.io/PacletServer/easyide.html) but it is pretty easy to use, too. ## Basics ###Installation Install it off the [Paclet Server](https://paclets.github.io/PacletServer/) : << https://paclets.github.io/PacletServer/Install.wl PublicPacletInstall["EasyIDE"] (*Out:*)  ###Making a New IDE Notebook This IDE system is also basically just a package and a stylesheet, so it's pretty easy to get started. Simply go to `Format ? Stylesheet ? EasyIDE ? LightMode` . It'll prompt you for a directory to use as the root directory. Here's a video as an example: [](https://www.youtube.com/watch?v=TMHiN9Ov2fQ) You can play around with the file browser now or the plugin menus in the top right ###Notebooks, Packages, and Text Files As things currently stand, the IDE recognizes three types of files to handle in different ways. The first, of course, are plain notebooks. These can be manipulated like normal. Here's an example of making and editing a notebook file in the IDE: [](https://www.youtube.com/watch?v=I1MnB7duneA) Text and package files can be made in the same way--just assign the appropriate file extension. Each of these files will work basically as a regular file would, except their contents will be saved to their original file on the disk rather than the current `NotebookFileName[]` . ###The File Browser One of the most useful and intuitive features of this IDE is the file browser it has built in. This allows you to quickly find files inside the active directory. Here's a screen shot of what that can look like:  Each entry in this has a `ContextMenu` that allows for some file- or directory-specific actions. ###Stylesheet / Extension Based Behavior EasyIDE is built to be extensible. It provides a way to get different behavior depending on what would be useful for the specific type of notebook or file is being fed in. These are controlled in the EasyIDE settings, in particular at `EasyIDE ? Resources ? Settings ? Mappings` where there are many files that control how these should map. This directory may also be created in `$UserBaseDirectory/ApplicationData` and the settings there will take precedence over those in the paclet folder itself. These customizations include stylesheets, toolbars, and what to do when the file browser is active. ###Plugins and Toolbars Probably the best feature of having something like EasyIDE is the ability to hook external code into the IDE and have it give new, more powerful capabilities. To make this easy to work with I added both a plugin system and a toolbar system (although the latter is really just a special case of the first). Plugins appear as either menus--such as the `File` and `Project` menus which are themselves just plugins--or as commands under the plugins menu. Currently I already have a decent number of these:  All of these add new functionality to the IDE based on code I'd written before. In that screenshot you can also see a toolbar, which exists right below the tabs. This can be stylesheet specific and thus adds an even more targeted way to add functionality to the system. Here's an example of the four different toolbars I've implemented as well as the different stylesheets they go with: [](https://www.youtube.com/watch?v=fFiDiO-oM7w) In that you can also see the major downside of putting everything into an IDE: when the files get big (as is the one I'm using to write this post) things can get slower. On the other hand as long as one is only writing code, this is never an issue. And even with a ~12MB file like this things are still more than fast enough to not be frustrating to work with. ## Extensions ###Styles EasyIDE was built to be customizable. This holds first and foremost for the stylesheets it works with. Even though currently there is only a set of LightMode styles, as DarkMode style set could be constructed without too much more difficulty. To do this, one would merely have to take the existing LightMode stylesheet, copy it, and make the necessary cosmetic changes. These changes should then propagate reasonably naturally to the extension styles if the inheritance is changed. This is on the TODO list, but if there is a quality existing DarkMode stylesheet to work off that would also make life much easier. ###Plugins and Toolbars These may be hooked in by adding things to `EasyIDE ? Resources ? Settings ? Plugins` and `EasyIDE ? Resources ? Settings ? Toolbars` . There are a number of good examples there already. ###Miscellaneous Extensions I had already implemented stuff for creating nice docs, Markdown notebooks, websites, bug tracking, paclet creation, etc. and some of this has made it in as plugins already. More is forthcoming, but for now one can always play with what's in the Plugins menu. In particular the Git plugin is useful for me as I write and develop. ###The EasyIDE API EasyIDE is just a collection of functions wrapped into a single unit. These were designed to (hopefully) be modular and clean to work with. Eventually all core functionality will also make its way to being attached to a single object, the `IDENotebookObject` . The API for this is based off of my [InterfaceObjects](https://paclets.github.io/PacletServer/interfaceobjects.html) package and is object-oriented. This will be documented in due time, but as a taste here's what it can look like: ide = IDENotebookObject[] (*Out:*)  ide@"Methods" (*Out:*) {"Open","Save","Close","SwitchTab","Path","Data","SetData","ToggleFileViewer","AddToolbar","RemoveToolbar","AddStyles","RemoveStyles","GetStylesheet","SetStylesheet","SetProjectDirectory","CreateMessage","CreateDialog"} These `"Methods"` are all operations that the IDE notebook referenced to by `EvaluationNotebook[]` can perform. Here's an example of creating a message: ide@"CreateMessage"["Hello!"] (*Out:*)   As the IDE grows in sophistication so will the methods the API supports. For now, though, these provide the most direct control that is possible to get with the IDE. b3m2a1 ​  2019-04-09T10:47:07Z https://community.wolfram.com/groups/-/m/t/1527021 MATH-TWS is a new Mathematica package that connects Wolfram Mathematica to the Interactive Brokers TWS platform via the C++ API. It enables the user to retrieve information from TWS on accounts, portfolios and positions, as well as historical and real-time market data. MATH-TWS also enables the user to place and amend orders and obtain execution confirmations from Mathematica. I have wanted a way to connect Wolfram Mathematica to Interactive Brokers’ Trader Workstation for the longest time. Now that it is finally available with MATH-TWS I am excited by the possibilities for Mathematica users. The first release of MATH-TWS will be available within a couple of weeks. Anyone interested in licensing a copy should email algorithmicexecution@gmail.com with MATH-TWS in the subject line. ![enter image description here][1] ![enter image description here][2] ![enter image description here][3] ![enter image description here][4] ![enter image description here][5] ![enter image description here][6] ![enter image description here][7] ![enter image description here][8] ![enter image description here][9] ![enter image description here][10] ![enter image description here][11] ![enter image description here][12] ![enter image description here][13] [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=twsConnect.png&userId=773999 [2]: http://community.wolfram.com//c/portal/getImageAttachment?filename=twsAccounts.png&userId=773999 [3]: http://community.wolfram.com//c/portal/getImageAttachment?filename=twsAccounts2.png&userId=773999 [4]: http://community.wolfram.com//c/portal/getImageAttachment?filename=twsAccounts3.png&userId=773999 [5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=HistoricalData.png&userId=773999 [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=DataManipulation.png&userId=773999 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=TimeSeries.png&userId=773999 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=IntradayData.png&userId=773999 [9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Currentmarketdata.png&userId=773999 [10]: http://community.wolfram.com//c/portal/getImageAttachment?filename=News.png&userId=773999 [11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Ordermanagement.png&userId=773999 [12]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Modifyorder.png&userId=773999 [13]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Cancelorder.png&userId=773999 Jonathan Kinlay 2018-10-22T12:09:47Z https://community.wolfram.com/groups/-/m/t/1736686 Recently I updated my [Bayesian inference reporitory][1] with a new function called `BayesianLinearRegression` to provide a Bayesian alternative to Mathematica's `LinearModelFit`. I also submitted the code for this function to the [Wolfram function repository][2] to make it easier to access, so the function can also be used with `ResourceFunction["BayesianLinearRegression"]`. The example notebook in the repository provides several examples of how the function can be used, some of which I will reproduce below. Please refer to the GitHub README.md file (which is displayed on the homepage of the repository) for instructions about installing the `BayesianInference` package. The code is in [this file][3], in case you're interested in taking a look under the hood. ## Fitting polynomials ## First generate some test data: data = RandomVariate[ MultinormalDistribution[{{1, 0.7}, {0.7, 1}}], 20 ]; ListPlot[data] ![enter image description here][4] The usage of `BayesianLinearRegression` is similar to that of `LinearModelFit` with one significant difference: I decided that the `Rule`-based data specification used by `Predict` and `NetTrain` is often more convenient than the matrix input used by the `Fit` family, so I decided that both types of data specifications should work. The main reason for this is that `BayesianLinearRegression` also supports regression of vector outputs (an example of that is given in the repository example notebook), in which case the `Rule`-based format is often easier to understand. When you fit a model, `BayesianLinearRegression` returns an `Association` containing all relevant information about the fit. Fit a model of the form `y == a + b x + randomness` where `randomness` is distributed as `NormalDistribution[0, sigma]`. Here, `a`, `b` and `sigma` are unknowns that need to be fitted to the data: In[20]:= Clear[x]; model = BayesianLinearRegression[data, {1, x}, x]; Below is a more detailed description of the information return by `BayesianLinearRegression`, but let's first concentrate on the main fit result, which is the [posterior predictive distribution][5] : In[61]:= model["Posterior", "PredictiveDistribution"] Out[61]= StudentTDistribution[-0.267818 + 0.986121 x, 0.726105 Sqrt[1.05016 + 0.00692863 x + 0.0635094 x^2], 2001/100 ] As you can see, the result is returned is an `x`-dependent `StudentTDistribution`. Visualize the predictions: With[{ predictiveDist = model["Posterior", "PredictiveDistribution"], bands = {95, 50, 5} }, Show[ Plot[ Evaluate@InverseCDF[predictiveDist, bands/100], {x, -4, 4}, Filling -> {1 -> {2}, 3 -> {2}}, PlotLegends -> bands ], ListPlot[data], PlotRange -> All ] ] ![enter image description here][6] In the `BayesianInference` package on GitHub, I included a function called `regressionPlot1D` which makes it a bit easier to make this plot: Show[ regressionPlot1D[model["Posterior", "PredictiveDistribution"], {x, -4, 4}, {95, 50, 5}], ListPlot[data] ] (* same result*) ## Model comparison and model mixing ## Often you'll want to compare multiple models against a data set to see which one works best. For example, here is some data where it's a bit unclear if you should use a first or second order model: data = {{-1.5`,-1.375`},{-1.34375`,-2.375`},{1.5`,0.21875`},{1.03125`,0.6875`},{-0.5`,-0.59375`}, {-1.875`,-2.59375`},{1.625`,1.1875`},{-2.0625`,-1.875`},{1.0625`,0.5`},{-0.4375`,-0.28125`},{-0.75`,-0.75`},{2.125`,0.375`},{0.4375`,0.6875`},{-1.3125`,-0.75`},{-1.125`,-0.21875`}, {0.625`,0.40625`},{-0.25`,0.59375`},{-1.875`,-1.625`},{-1.`,-0.8125`},{0.4375`,-0.09375`}} ListPlot[data] ![enter image description here][7] Fit the data with polynomials up to fourth degree; rank the log-evidences and inspect them. As you can see, the first and second order fits are almost equally likely: In[152]:= models = AssociationMap[ BayesianLinearRegression[Rule @@@ data, x^Range[0, #], x] &, Range[0, 4] ]; ReverseSort@models[[All, "LogEvidence"]] Out[153]= <|1 -> -30.0072, 2 -> -30.1774, 3 -> -34.4292, 4 -> -38.7037, 0 -> -38.787|> Show the prediction bands: Show[ regressionPlot1D[ models[1, "Posterior", "PredictiveDistribution"], {x, -3, 3}], regressionPlot1D[models[2, "Posterior", "PredictiveDistribution"], {x, -3, 3}, PlotStyle -> Dashed, PlotLegends -> None], ListPlot[data] ] ![Comparison of the first (solid) and second order (dashed) prediction bands][8] Instead of picking just one of the models, we can define a mixture over all of them. First calculate the weights for each fit: In[94]:= weights = Normalize[ (* subtract the max to reduce rounding error *) Exp[models[[All, "LogEvidence"]] - Max[models[[All, "LogEvidence"]]]], Total ]; ReverseSort[weights] Out[95]= <|1 -> 0.516002, 2 -> 0.47702, 3 -> 0.0068122, 4 -> 0.0000982526, 0 -> 0.0000667854|> Define a mixture of posterior predictive distributions and show the predictions: In[96]:= mixDist = MixtureDistribution[ Values[weights], Values@models[[All, "Posterior", "PredictiveDistribution"]] ]; Show[ regressionPlot1D[mixDist, {x, -3, 3}], ListPlot[data] ] ![Prediction bands of a mixture of polynomial models up to degree 4][9] As you can see, if one model is not clearly the best, you can just split the difference between them. Another interesting thing you can do with `BayesianLinearRegression` is Bayesian updating. Basically, this means that if you've fitted some data in the past and are now finding new data, you can incorporate the new data by updating your fit without having to re-fit all of the data (old and new) in one go. An example of this is given in the example code notebook on GitHub (in the section about the `"PriorParameters"` options of the function). ## Detailed explanation about the returned values ## `BayesianLinearRegression` returns an `Association` with the following keys: Keys[model] Out[21]= {"LogEvidence", "PriorParameters", "PosteriorParameters", "Posterior", "Prior", "Basis", "IndependentVariables"} - `"LogEvidence"`: In a Bayesian setting, the evidence (also called [marginal likelihood][10]) measures how well the model fits the data (with a higher evidence indicating a better fit). The evidence has the virtue that it naturally penalizes models for their complexity and therefore does not suffer from over-fitting in the way that measures like the sum-of-squares or likelihood do. - `"Basis"`, `"IndependentVariables"`: Simply the basis functions and independent variable specified by the user. - `"Posterior"`, `"Prior"`: These two keys each hold an association with 4 distributions: 1. `"PredictiveDistribution"`: A distribution that depends on the independent variables (`x` in the example above). By filling in a value for `x`, you get a distribution that tells you where you could expect to find future `y` values. This distribution accounts for all relevant uncertainties in the model: model variance caused by the term `randomness`; uncertainty in the values of `a` and `b`; and uncertainty in `sigma`. 2. `"UnderlyingValueDistribution"`: Similar to `"PredictiveDistribution"`, but this distribution give the possible values of `a + b x` without the `randomness` error term. 3. `"RegressionCoefficientDistribution"`: The join distribution over `a` and `b`. 4. `"ErrorDistribution"`: The distribution of the variance `sigma^2`. - ` "PriorParameters"`, `"PosteriorParameters"`: These parameters are not immediately important most of the time, but they contain all of the relevant information about the fit. ## Sources ## The formulas underlying BayesianLinearRegression are based mainly on the following Wikipedia articles. The names of the hyper parameters returned by the function are based on the article about multivariate linear regression. - [Bayesian linear regression (Wikipedia)][11] - [Bayesian multivariate linear regression (Wikipedia)][12] [1]: https://github.com/ssmit1986/BayesianInference [2]: https://resources.wolframcloud.com/FunctionRepository/resources/BayesianLinearRegression [3]: https://github.com/ssmit1986/BayesianInference/blob/master/BayesianInference/BayesianLinearRegression.wl [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=testdata.png&userId=839741 [5]: https://en.wikipedia.org/wiki/Posterior_predictive_distribution [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=fit.png&userId=839741 [7]: https://community.wolfram.com//c/portal/getImageAttachment?filename=testdata2.png&userId=839741 [8]: https://community.wolfram.com//c/portal/getImageAttachment?filename=fit2.png&userId=839741 [9]: https://community.wolfram.com//c/portal/getImageAttachment?filename=fitmix.png&userId=839741 [10]: https://en.wikipedia.org/wiki/Marginal_likelihood [11]: https://en.wikipedia.org/wiki/Bayesian_linear_regression [12]: https://en.wikipedia.org/wiki/Bayesian_multivariate_linear_regression Sjoerd Smit 2019-07-14T12:30:33Z https://community.wolfram.com/groups/-/m/t/963158 Recently John Fultz [announced][1] GitLink as open source on GitHub. He also mentioned the desire to offer more Wolfram related open source to the community. Draw was not mentioned by John but readers of his post followed the GitHub [link][2] and discovered the Draw repository. Naturally questions followed so here we are with an introduction of Draw, the latest open source contribution to the community. Draw is a vector drawing application. Many of the technical illustrations found in the Wolfram Language documentation were created using it. The release of Draw is to promote the advantages of using the Wolfram Language for software development. Draw was a personal project that grew from an idea to develop a simple tool to automate the drawing of flowcharts. This simple tool expanded in response to the need of functionality for solving specific challenges required by various illustrations. The Wolfram Language was very instrumental in helping Draw evolve during this process. Some of the more prominent features of Draw include: - Orthographic 3D drawing - Circuit, Mechanical and 3D library presets - Adjustable shapes such as gears or spirals - Shape arrays - Image autotrace The gallery of examples below demonstrate the capabilities of Draw: ![gallery][3] A look inside the application reveals an interface divided into four distinct sections: ![interface sections][4] An example of Draw being used to work on a flowchart: ![interface][5] The Circuit, Mechanical and 3D libraries: ![libraries][6] The adjustable gear shape: ![gear][7] The shape array: ![shape array][8] Hope this brief introduction is enough to inspire you to download the Draw notebook from [this GitHub link][9]. Start creating illustrations or explore the Wolfram Language code to discover how the application works. Support is available by clicking the blue help button for topics such as the basics, features, interaction, tools, examples and tips. Tooltips appear for all buttons detailing their behavior. Remember this is open source software and comes with no guarantee. That being said, feel free to kick the tires and take it for a spin. Draw works best when used in Mathematica 11 but most functionality is supported by Mathematica 10.4.1. [1]: http://community.wolfram.com/groups/-/m/t/960333 [2]: https://github.com/WolframResearch [3]: http://community.wolfram.com//c/portal/getImageAttachment?filename=examples.png&userId=28355 [4]: http://community.wolfram.com//c/portal/getImageAttachment?filename=interfaceSections.png&userId=28355 [5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=interface.png&userId=28355 [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=libraries.png&userId=28355 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=gear.png&userId=28355 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=shapearray.png&userId=28355 [9]: https://github.com/shdlbwr/draw Tim Shedelbower 2016-11-14T20:15:12Z https://community.wolfram.com/groups/-/m/t/1039591 Do you ever wish you could save data directly into a notebook instead of having to export it? I sometimes do things like mydata = {1,2,3, ...}; just to be able to use this data next time I open the notebook. This could be data I pasted from the web, or something that just took several minutes to generate. But saving it like this is inconvenient, and often takes up a lot of space. Today I published a blog post about a better method that I have been using recently. I thought people here may be interested: * [Save data directly in Mathematica notebooks](http://szhorvat.net/pelican/save-data-in-notebooks.html) The idea is to use first `Compress` the data, then use `Interpretation` to create a compact display for it. Here's a small function that packages all of this up: ClearAll[SaveToCell] SaveToCell::usage = "SaveToCell[variable] creates an input cell that reassigns the current value of variable.\n" <> "SaveToCell[variables, display] shows 'display' on the right-hand-side of the assignment."; SetAttributes[SaveToCell, HoldFirst] SaveToCell[var_, name : Except[_?OptionQ] : "data", opt : OptionsPattern[]] := With[{data = Compress[var], panel = ToBoxes@Tooltip[Panel[name, FrameMargins -> Small], DateString[]]}, CellPrint@Cell[ BoxData@RowBox[{ MakeBoxes[var], "=", InterpretationBox[panel, Uncompress[data]], ";" }], "Input", (* prevent deletion by Cell > Delete All Output: *) GeneratedCell -> False, (* CellLabel is special: last occrrence takes precedence, so it comes before opt: *) CellLabel -> "(saved)", opt, CellLabelAutoDelete -> False ] ] If you have your data in the variable `var`, simply run `SaveToCell[var]`, which will create an input cell that re-assign the value of `var`. It looks like this: ![enter image description here][1] We can also customize the display: var = Range[1000]; SaveToCell[var, Short[var]] ![enter image description here][2] Hovering the display will show the date when the data was saved. `SaveToCell` also takes arbitrary `Cell` options, and passes them down to the generated cell. Something strange I observed while writing this function is that with some options, such as `CellLabel`, [it is not the first but the last occurrence of the option that takes precedence][3]. Does anyone know why? We can use this functionality to change the cell style, add a different label, or to protect the cell against accidental deletion: `SaveToCell[var, Deletable -> False]`. I have been using this little function for a while, and I hope that others will find it useful too. Do be careful though: notebooks are not designed for storing large data. I would avoid storing data as large as several tens of megabytes within notebooks. [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=in1.png&userId=38370 [2]: http://community.wolfram.com//c/portal/getImageAttachment?filename=in3.png&userId=38370 [3]: http://mathematica.stackexchange.com/q/140989/12 Szabolcs Horvát 2017-03-25T21:56:48Z https://community.wolfram.com/groups/-/m/t/1468475 Here's a cross-post of something I originally wrote [here](https://www.wolframcloud.com/objects/b3m2a1/home/pjlink-hooking-up-mathematica-and-python.html) about how to get python and Mathematica to work together like JLink. I thought this might have broad appeal here. PJLink: Hooking up Mathematica and Python --- Mathematica is an incredibly powerful platform with a fun and intellectually pleasing language, but is incredibly expensive and closed source. Python is a convenient, pretty powerful language with a lot of support from the developer community. For as long as the two have existed people have been trying to tie them together, but very little has been done to do so at the native level with efficient, convenient exchange between the two. That's why over the past few weeks I took the time to build a clean, convenient link between the two. This post will go into how the link was built and some of its features, but first I think a little demo is appropriate. ## A Quick Demo ### Installing PJLink The link is based off of the [J/Link](https://reference.wolfram.com/language/JLink/tutorial/Overview.html) interface built into Mathematica for hooking up Java and Mathematica. To wit, I called it [PJ/Link](https://github.com/b3m2a1/PJLink) . It lives on my paclet server as well as GitHub, so we can easily install it from there: PacletInstall["PJLink", "Site"->"http://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer"] (*Out:*)  ### Loading PJLink in Jupyter For this demo we'll need the path to this thing as well (note that the version might change in the future): %["Location"] (*Out:*) "~/Library/Mathematica/Paclets/Repository/PJLink-1.0.0" Now we'll leave Mathematica and open up a Jupyter notebook:  Next we'll get that path available so we can actually make use of the package. Then we'll load things from the subsidiary ```SubprocessKernel``` package which is included in the paclet and makes use of PJLink: <pre>import os, sys pjlink_path = "~/Library/Mathematica/Paclets/Repository/PJLink-1.0.0" #this is whatever path was extracted before sys.path.insert(0, os.path.expanduser (pjlink_path)) from SubprocessKernel import SubprocessKernel from SubprocessKernel import MathematicaBlock, LinkEnvironment ## these are helpers I' ll use in the demo</pre>  ### Bidirectional Communication Once we have this we can start a subprocess kernel which will open a Mathematica front-end to interact with. We'll also start and evaluator Mathematica can use to call back into python. You may see a long string of output from your C compiler as the setup.py file builds out the native library that PJLink uses. Don't worry, this should only happen once. If it fails, raise an issue on [GitHub](https://github.com/b3m2a1/PJLink/issues) so I can deal with it. Once Mathematica has loaded, we'll use the ```MathematicaBlock``` context manager so we can write something that looks a lot like Mathematica code and use the ```MEval``` function we'll define to run the code. That code for all this looks like: <pre>ker = SubprocessKernel() def MEval (expr, wait = True, kernel = ker) : "" "MEval evaluates a Mathematica expression in the Mathematica kernel " "" kernel.drain() # just to make sure things are clen return kernel.evaluate (expr, wait = wait) ker.start() ker.start_evaluator()</pre> After that we can simply call into Mathematica: <pre>with MathematicaBlock(): res = MEval (Set (M.hi, "Hello from python!")) res</pre>  We can see string ```"Hello from python!"``` was set to the symbol ```hi``` on the Mathematica side and was returned back by ```MEval``` . Symbols that aren't in the ```"System`"``` context need to be prefaced by an ```M.``` as that's a special class that can resolve symbol names like that. We can also get efficient data transfer of arrays from either side. Here we'll take some Mathematica data and get it back out on the python side. The first thing we need to do is go to the Mathematica notebook that opened and load the ```"PJLink`"``` context. Then we'll install the python runtime that the ```SubprocessKernel``` object configured. This looks like: <<PJLink` InstallPython[ LinkObject->SubprocessKernel`$PyEvaluateLink, ProcessObject->None]; Once it's installed, we'll use it directly via ```PyEvaluate``` : With[{arr= RandomReal[{-1, 1}, {50, 50, 50}]}, PyEvaluate[dat=arr] ] Calls into python are done in an environment held only by the link, so to access that we need to wrap the evaluator we started ( ```ker.evaluator``` ) in a ```LinkEnvironment``` context manager: <pre>with LinkEnvironment(ker.evaluator): res = dat.shape res</pre>  Arrays are held as NumPy arrays by default on the python side, although this may be disabled. If disabled, they're held as a data type called ```BufferedNDArray``` which holds the data as a single C-contiguous array and allows slicing and viewing into it (although no efficient math or manipulation of any sort). Finally, to close out the demo, we'll plot something on the Mathematica side and watch it come back on the python side. The code for this should be pretty self-explanatory by this point, but there is one cute feature to note: <pre>with MathematicaBlock(): res = MEval( Rasterize( Plot(Sin (M.x), List (M.x, 0, Times (2, Pi)), ImageSize = [250, 250], PlotLabel = "sin(x) as plotted in Mathematica" ) ) ) res</pre> Unfortunately it really does matter that we pass a ```List``` expression instead of a python list for the second argument to ```Plot``` as otherwise the system hangs for reasons that aren't totally clear. On the other hand, we can see how nice options passing is in the interface. We make use of the python ```**kwargs``` setup and that ```ImageSize= ...``` and ```PlotLabel= ...``` both get automatically converted into rules (albeit with a ```String``` key instead of a ```Symbol``` ). The ```Rasterize``` is, sadly, similarly necessary as there is currently no logic in the package to automatically convert ```Graphics``` expression into their rasterized forms.  I think we'll close out the demo here, though, and move onto a description of how this works. ## PJLink Native Library The heart of PJLink is the C library that connects a python runtime to MathLink. The source for this can be found [here](https://github.com/b3m2a1/PJLink/blob/master/PJLink/PJLinkNativeLibrary/src/PJLinkNativeLibrary.cpp) . This library, once compiled by the setup.py file packaged with it, implements the basic MathLink calls in a way that python can use them and attempts to do so with efficient memory usage and data transfer. ### Data Sharing in the Native Library The heart of the native library is the set of ```PutArray``` and ```GetArray``` functions it implements. Beyond anything else, it is the fast transfer of large arrays of data that makes a C-level connection so appealing. The way we handle this on the python side is via the python [buffer protocol](https://docs.python.org/3/c-api/buffer.html) . We enforce the condition that all data sent and received on the python side must be handled by an object that can work with a C-contiguous buffer of data. By default this is done with [NumPy](http://www.numpy.org/) if it is installed, but if not there is a custom object called ```BufferedNDArray``` in the [HelperClasses](https://github.com/b3m2a1/PJLink/blob/master/PJLink/HelperClasses.py) package that deals with this. ### Threading in the Native Library Python has something called the [Global Interpreter Lock (GIL)](https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock) which is a method for synchronizing python state. Unfortunately for us, the presence of the GIL means that standard C calls of the kind we'll be using will cause all threads to lock. To get around this, every call into the MathLink library in the native library is wrapped in the ```MLTHREADED``` macro which handles the releasing and reacquiring of the lock. This allows our threads to work once more. Any extensions to the library should keep this in mind. ## Class Structures PJLink provides a glut of classes that handle the details communication, so we will quickly detail what the important ones do. More information is always available upon request. ### The *Link classes PJLink is based off of JLink and so it makes use of the same kind of class structure that JLink does. This means that it has a ```MathLink``` class that provides a template for the kind of link we'll work with and a ```KernelLink``` class that works specifically with Mathematica kernels. In general, we will only really work with a subclass of a ```KernelLink``` called a ```WrappedKernelLink``` that implements the ```KernelLink``` interface by calling into a ```NativeLink``` which is the only class which actually touches the native library at all. If one is controlling a Mathematica kernel from python, it will be handled by a ```WrappedKernelLink``` . ### Reader class The ```Reader``` class handles the other half of the communication. It waits for calls from Mathematica and processes them via the ```KernelLink._handlePacket``` function. Most commonly these calls in turn call ```KernelLink.__callPython``` which builds a python call from the symbolic python packet that ```PyEvaluate``` sends over the link. A ```Reader``` does its best not to completely prevent its link from passing data *to* Mathematica, but in general it is best not to depend on this as the ```NativeLink``` interface allows only a single thread to access the library at once for reasons of safety and stability. ### MathLinkEnvironment and MathLinkException The ```MathLinkEnvironment``` is a standalone class that handles all of the various flags and state that the links need. It centralizes all information about what a given token or flag from MathLink means and provides utility functions for working with this. ```MathLinkException``` is a subclass of the standard python ```Exception``` class that handles the MathLink-specific exceptions that are returned. It in turn calls into ```MathLinkEnvironment``` to learn what various exceptions mean. ### MPackage, MLSym, and MLExpr The HelperClasses package provides a large number of (generally) smaller classes that serve to make code cleaner in its implementation. A big part of this is done by the ```MPackage``` , ```MLSym``` , and ```MLExpr``` classes, which allow for a way to create packets with a syntax that looks more like standard Mathematica code. ```MLSym``` and ```MLExpr``` are types that a ```KernelLink``` knows how to put onto a link and ```MPackage``` provides utilities and a custom ```__getattr__``` so that the packet code can look like Mathematica code. ### MathematicaBlock and LinkEnvironment Both ```MathematicaBlock``` and ```LinkEnvironment``` are also in the HelperClasses. They both edit the current evaluation state as [context managers](https://docs.python.org/3/reference/datamodel.html#context-managers) so that explicit references to ```MPackage``` can be dropped and variables held by a given link can be easily accessed. Being context managers, they are both used via ```with``` statements and change the execution environment of the enclosing block. ### BufferedNDArray, ImageData, and SparseArrayData These are all data classes that allow for more efficient and convenient data transfer. The ```ImageData``` and ```SparseArrayData``` classes hold data coming in from Mathematica as put using ```PJLink`Package`AddTypeHints``` . They have methods to efficiently transform to more standard formats like ```PIL.Image``` and ```scipy.sparse.csr_matrix``` . As more data types are handled by ```AddTypeHints``` it can be assumed that more classes like these will be written. ## Mathematica-side Package That was all to do with the python side of things, which is where most of the development work had to go. On the other hand, the Mathematica side of the equation still requires some explanation. The package itself is really quite simple, so please feel free to [peruse the source](https://github.com/b3m2a1/PJLink/blob/master/Mathematica/PJLink.wl) . ### InstallPython ```InstallPython``` is the most basic function in the package. It either finds or is given a python version or executable, attempts to open it via ```StartProcess``` , then links to it via ```LinkCreate``` and the ```start_kernel.py``` script provided in the PJLink python package. Notably, all it really requires is a ```LinkObject``` , so you can pass one directly via the ```LinkObject``` option. It will also by default try to make a python ```ProcessObject``` but you can pass that via the ```ProcessObject``` option or you can pass ```None``` in which case it won't attach to a Mathematica controlled process. ### ClosePython ```ClosePython``` is the counterpart to ```InstallPython``` . It closes an opened python runtime by version or executable. When a new kernel is installed it is added to ```PJLink`Package`$PythonKernels``` and this is what ```ClosePython``` looks for to close. ### PyEvaluate / PyEvaluateString This is the heart of the package. It takes Mathematica-esque code, converts it into a structure that can be processed by ```KernelLink.__callPython()``` and sends it over and waits for a response. The conversion is handled by ```PJLink`SymbolicPython`ToSymbolicPython``` which was originally written for the [PyTools package](https://github.com/b3m2a1/mathematica-PyTools) . This is the best way to move data to python as things like ```Image``` objects, packable arrays, and ```SparseArray``` objects will be moved over intelligently. ```PyEvaluateString``` is like ```PyEvaluate``` , but with the recognition that ```ToSymbolicPython``` will always be a little bit lacking. It allows one to simply call a string of python code on the link and get the results back. ### PyWrite / PyWriteString / PyRead / PyReadErr These are all functions that make use of the fact that when the ```Reader``` object started it allowed an interactive session to keep running and reading / writing on stdin, stdout, and stderr. The ```Read``` functions read from stdout and stderr and the write functions write to stdin. The former takes Mathematica code and auto-converts it into a string. The latter simply passes in the given string. ## Future Work PJLink 1.0.0, beefy as it already is, should only really be seen as the beginning. My hope is that much more can be done to allow for more native data type transfer and for intelligent communication between the two systems. In my demo I tried to show some of the things that make the interoperation of the two so nice, but I obviously don't have the breadth of knowledge to know all of the many applications this can be put to. Applications built off of PJLink are always welcome and I'm happy to provide any requisite information and extensions to PJLink to get them built. Alongside that, I think better integration on the Mathematica side is necessary. There is a partial interface for allowing a ```PythonObject``` structure to hide the details of ```PyEvaluate``` on the Mathematica side, but this needs work from both ends, first hooking up the ```Language`MutatationHandler``` interface and then extending the same on the python side. After that, a ```JavaBlock``` -like setup that allows a link to be opened, used, and cleaned up would be highly useful for sandboxing. Finally, I'm sure there are numerous bugs hiding in the package as it stands. Please find them and let me know about them so they can be worked out. In the meantime, I hope you enjoy PJLink and being able to use my two favorite languages symbiotically. b3m2a1 ​  2018-09-20T09:18:51Z https://community.wolfram.com/groups/-/m/t/866748 ## Introduction This post describes the use of face-like diagrams to visualize multidimensional data introduced by Herman Chernoff in 1973, see [1]. The idea to use human faces in order to understand, evaluate, or easily discern (the records of) multidimensional data is very creative and inspirational. As Chernoff says in [1], the object of the idea is to "represent multivariate data, subject to strong but possibly complex relationships, in such a way that an investigator can quickly comprehend relevant information and then apply appropriate statistical analysis." It is an interesting question how useful this approach is and it seems that there at least several articles discussing that; see for example [2]. I personally find the use of Chernoff faces useful in a small number of cases, but that is probably true for many "creative" data visualization methods. Below are given both simple and more advanced examples of constructing Chernoff faces for data records using the *Mathematica* package [3]. The considered data is categorized as: 1. a small number of records, each with small number of elements; 2. a large number of records, each with less elements than Chernoff face parts; 3. a list of long records, each record with much more elements than Chernoff face parts; 4. a list of nearest neighbors or recommendations. For several of the visualizing scenarios the records of two "real life" data sets are used: Fisher Iris flower dataset [7], and "Vinho Verde" wine quality dataset [8]. For the rest of the scenarios the data is generated. A fundamental restriction of using Chernoff faces is the necessity to properly transform the data variables into the ranges of the Chernoff face diagram parameters. Therefore, proper data transformation (standadizing and rescaling) is an inherent part of the application of Chernoff faces, and this document describes such data transformation procedures (also using [3]). ### Package load The packages [3,4] are used to produce the diagrams in this post. The following two commands load them. Import["https://raw.githubusercontent.com/antononcube/\ MathematicaForPrediction/master/ChernoffFaces.m"] Import["https://raw.githubusercontent.com/antononcube/\ MathematicaForPrediction/master/MathematicaForPredictionUtilities.m"] ### Mirror This Community post mirrors the blog post ["Making Chernoff faces for data visualization"](https://mathematicaforprediction.wordpress.com/2016/06/03/making-chernoff-faces-for-data-visualization/). ## Making faces ### Just a face Here is a face produced by the function `ChernoffFace` of [3] with its simplest signature: SeedRandom[152092] ChernoffFace[] [](http://i.imgur.com/PIcrIcK.gif) Here is a list of faces: SeedRandom[152092] Table[ChernoffFace[ImageSize -> Tiny], {7}] [](http://i.imgur.com/SPWgX0r.gif) ### Proper face making The "proper" way to call ChernoffFace is to use an association for the facial parts placement, size, rotation, and color. The options are passed to Graphics. SeedRandom[2331]; ChernoffFace[ AssociationThread[ Keys[ChernoffFace["FacePartsProperties"]] -> RandomReal[1, Length[ChernoffFace["FacePartsProperties"]]]], ImageSize -> Small , Background -> GrayLevel[0.85]] [](http://i.imgur.com/wY3bgKC.gif) The Chernoff face drawn with the function `ChernoffFace` can be parameterized to be asymmetric. The parameters argument mixes (1) face parts placement, sizes, and rotation, with (2) face parts colors and (3) a parameter should it be attempted to make the face symmetric. All facial parts parameters have the range [0,1] Here is the facial parameter list: Keys[ChernoffFace["FacePartsProperties"]] (* {"FaceLength", "ForheadShape", "EyesVerticalPosition", "EyeSize", \ "EyeSlant", "LeftEyebrowSlant", "LeftIris", "NoseLength", \ "MouthSmile", "LeftEyebrowTrim", "LeftEyebrowRaising", "MouthTwist", \ "MouthWidth", "RightEyebrowTrim", "RightEyebrowRaising", \ "RightEyebrowSlant", "RightIris"} *) The order of the parameters is chosen to favor making symmetric faces when a list of random numbers is given as an argument, and to make it easier to discern the faces when multiple records are visualized. For experiments and discussion about which facial features bring better discern-ability see [2]. One of the conclusions of [2] is that eye size and eye brow slant are most decisive, followed by face size and shape. Here are the rest of the parameters (colors and symmetricity): Complement[Keys[ChernoffFace["Properties"]], Keys[ChernoffFace["FacePartsProperties"]]] (* {"EyeBallColor", "FaceColor", "IrisColor", "MakeSymmetric", \ "MouthColor", "NoseColor"} *) ### Face coloring The following code make a row of faces by generating seven sequences of random numbers in $[0,1]$, each sequence with length the number of facial parameters. The face color is assigned randomly and the face color or a darker version of it is used as a nose color. If the nose color is the same as the face color the nose is going to be shown "in profile", otherwise as a filled polygon. The colors of the irises are random blend between light brown and light blue. The color of the mouth is randomly selected to be black or red. SeedRandom[201894]; Block[{pars = Keys[ChernoffFace["FacePartsProperties"]]}, Grid[{#}] &@ Table[ChernoffFace[Join[ AssociationThread[pars -> RandomReal[1, Length[pars]]], <|"FaceColor" -> (rc = ColorData["BeachColors"][RandomReal[1]]), "NoseColor" -> RandomChoice[{Identity, Darker}][rc], "IrisColor" -> Lighter[Blend[{Brown, Blue}, RandomReal[1]]], "MouthColor" -> RandomChoice[{Black, Red}]|>], ImageSize -> 100], {7}] ] [](http://i.imgur.com/QN5Rhvf.png) ### Symmetric faces The parameter "MakeSymmetric" is by default `True`. Setting "MakeSymmetric" to true turns an incomplete face specification into a complete specification with the missing paired parameters filled in. In other words, the symmetricity is not enforced on the specified paired parameters, only on the ones for which specifications are missing. The following faces are made symmetric by removing the facial parts parameters that start with "R" (for "Right") and the parameter "MouthTwist". SeedRandom[201894]; Block[{pars = Keys[ChernoffFace["FacePartsProperties"]]}, Grid[{#}] &@Table[( asc = Join[AssociationThread[ pars -> RandomReal[1, Length[pars]]], <|"FaceColor" -> (rc = ColorData["BeachColors"][RandomReal[1]]), "NoseColor" -> RandomChoice[{Identity, Darker}][rc], "IrisColor" -> Lighter[Blend[{Brown, Blue}, RandomReal[1]]], "MouthColor" -> RandomChoice[{Black, Red}]|>]; asc = Pick[asc, StringMatchQ[Keys[asc], x : (StartOfString ~~ Except["R"] ~~ __) /; x != "MouthTwist"]]; ChernoffFace[asc, ImageSize -> 100]), {7}]] [](http://i.imgur.com/7J7vl12.png) Note that for the irises we have two possibilities of synchronization: pars = <|"LeftIris" -> 0.8, "IrisColor" -> Green|>; {ChernoffFace[Join[pars, <|"RightIris" -> pars["LeftIris"]|>], ImageSize -> 100], ChernoffFace[ Join[pars, <|"RightIris" -> 1 - pars["LeftIris"]|>], ImageSize -> 100]} [](http://i.imgur.com/8i16NUg.gif) ## Visualizing records (first round) The conceptually straightforward application of Chernoff faces is to visualize ("give a face" to) each record in a dataset. Because the parameters of the faces have the same ranges for the different records, proper rescaling of the records have to be done first. Of course standardizing the data can be done before rescaling. First let us generate some random data using different distributions: SeedRandom[3424] {dists, data} = Transpose@Table[( rdist = RandomChoice[{NormalDistribution[RandomReal[10], RandomReal[10]], PoissonDistribution[RandomReal[4]], GammaDistribution[RandomReal[{2, 6}], 2]}]; {rdist, RandomVariate[rdist, 12]}), {10}]; data = Transpose[data]; The data is generated in such a way that each column comes from a certain probability distribution. Hence, each record can be seen as an observation of the variables corresponding to the columns. This is how the **columns** of the generated data look like using `DistributionChart`: DistributionChart[Transpose[data], ChartLabels -> Placed[MapIndexed[ Grid[List /@ {Style[#2[[1]], Bold, Red, Larger]}] &, dists], Above], ChartElementFunction -> "PointDensity", ChartStyle -> "SandyTerrain", ChartLegends -> dists, BarOrigin -> Bottom, GridLines -> Automatic, ImageSize -> 900] [](http://i.imgur.com/iXQfzZG.gif) At this point we can make a face for each **record** of the rescaled data: faces = Map[ChernoffFace, Transpose[Rescale /@ Transpose[data]]]; and visualize the obtained faces in a grid. Row[{Grid[ Partition[#, 4] &@Map[Append[#, ImageSize -> 100] &, faces]], " ", Magnify[#, 0.85] &@ GridTableForm[ List /@ Take[Keys[ChernoffFace["FacePartsProperties"]], Dimensions[data][[2]]], TableHeadings -> {"Face part"}] }] [](http://i.imgur.com/LMTsbCf.gif) (The table on the right shows which facial parts are used for which data columns.) ## Some questions to consider Several questions and observations arise from the example in the previous section. #### 1. What should we do if the data records have more elements than facial parts parameters of the Chernoff face diagram? This is another fundamental restriction of Chernoff faces -- the number of data columns is limiter by the number of facial features. One way to resolve this is to select important variables (columns) of the data; another is to represent the records with a vector of statistics. The latter is shown in the section "Chernoff faces for lists of long lists". #### 2. Are there Chernoff face parts that are easier to perceive or judge than others and provide better discern-ability for large collections of records? Research of the [pre-attentiveness](https://en.wikipedia.org/wiki/Pre-attentive_processing) and effectiveness with Chernoff faces, [2], shows that eye size and eyebrow slant are the features that provide best discern-ability. Below this is used to select some of the variable-to-face-part correspondences. #### 3. How should we deal with outliers? Since we cannot just remove the outliers from a record -- we have to have complete records -- we can simply replace the outliers with the minimum or maximum values allowed for the corresponding Chernoff face feature. (All facial features of `ChernoffFace` have the range $[0,1]$.) See the next section for an example. ## Data standardizing and rescaling Given a full array of records, we most likely have to standardize and rescale the columns in order to use the function `ChernoffFace`. To help with that the package [3] provides the function `VariablesRescale` which has the options "StandardizingFunction" and "RescaleRangeFunction". Consider the following example of `VariableRescale` invocation in which: 1. each column is centered around its median and then divided by the inter-quartile half-distance (quartile deviation), 2. followed by clipping of the outliers that are outside of the disk with radius 3 times the quartile deviation, and 3. rescaling to the unit interval. rdata = VariablesRescale[N@data, "StandardizingFunction" -> (Standardize[#, Median, QuartileDeviation] &), "RescaleRangeFunction" -> ({-3, 3} QuartileDeviation[#] &)]; TableForm[rdata /. {0 -> Style[0, Bold, Red], 1 -> Style[1, Bold, Red]}] [](http://i.imgur.com/LdhvQJM.png) **Remark:** The bottom outliers are replaced with 0 and the top outliers with 1 using `Clip`. ## Chernoff faces for a small number of short records In this section we are going use the Fisher Iris flower data set [7]. By "small number of records" we mean few hundred or less. ### Getting the data These commands get the Fisher Iris flower data set shipped with *Mathematica*: irisDataSet = Map[Flatten, List @@@ ExampleData[{"MachineLearning", "FisherIris"}, "Data"]]; irisColumnNames = Most@Flatten[ List @@ ExampleData[{"MachineLearning", "FisherIris"}, "VariableDescriptions"]]; Dimensions[irisDataSet] (* {150, 5} *) Here is a summary of the data: Grid[{RecordsSummary[irisDataSet, irisColumnNames]}, Dividers -> All, Alignment -> Top] [](http://i.imgur.com/upxeRHv.png) ### Simple variable dependency analysis Using the function `VariableDependenceGrid` of the package [4] we can plot a grid of variable cross-dependencies. We can see from the last row and column that "Petal length" and "Petal width" separate setosa from versicolor and virginica with a pretty large gap. Magnify[#, 1] &@ VariableDependenceGrid[irisDataSet, irisColumnNames, "IgnoreCategoricalVariables" -> False] [](http://i.imgur.com/vJA6kth.png) ### Chernoff faces for Iris flower records Since we want to evaluate the usefulness of Chernoff faces for discerning data records groups or clusters, we are going to do the following steps. 1. Data transformation. This includes standardizing and rescaling and selection of colors. 2. Make a Chernoff face for each record **without** the label class "Species of iris". 3. Plot shuffled Chernoff faces and attempt to visually cluster them or find patterns. 4. Make a Chernoff face for each record using the label class "Specie of iris" to color the faces. (Records of the same class get faces of the same color.) 5. Compare the plots and conclusions of step 2 and 4. #### 1. Data transformation First we standardize and rescale the data: chernoffData = VariablesRescale[irisDataSet[[All, 1 ;; 4]]]; These are the colors used for the different species of iris: faceColorRules = Thread[Union[ irisDataSet[[All, -1]]] \ -> Map[Lighter[#, 0.5] &, {Purple, Blue, Green}]] (* {"setosa" -> RGBColor[0.75, 0.5, 0.75], "versicolor" -> RGBColor[0.5, 0.5, 1.], "virginica" -> RGBColor[0.5, 1., 0.5]} *) Add the colors to the data for the faces: chernoffData = MapThread[ Append, {chernoffData, irisDataSet[[All, -1]] /. faceColorRules}]; Plot the distributions of the rescaled variables: DistributionChart[ Transpose@chernoffData[[All, 1 ;; 4]], GridLines -> Automatic, ChartElementFunction -> "PointDensity", ChartStyle -> "SandyTerrain", ChartLegends -> irisColumnNames, ImageSize -> Large] [](http://i.imgur.com/rwm71tn.gif) #### 2. Black-and-white Chernoff faces Make a black-and-white Chernoff face for each record without using the species class: chfacesBW = ChernoffFace[ AssociationThread[{"NoseLength", "LeftEyebrowTrim", "EyeSize", "LeftEyebrowSlant"} -> Most[#]], ImageSize -> 100] & /@ chernoffData; Since "Petal length" and "Petal width" separate the classes well for those columns we have selected the parameters "EyeSize" and "LeftEyebrowSlant" based on [2]. #### 3. Finding patterns in a collection of faces Combine the faces into a image collage: ImageCollage[RandomSample[chfacesBW], Background -> White] [](http://i.imgur.com/zmBSQZR.gif) We can see that faces with small eyes tend have middle-lowered eyebrows, and that faces with large eyes tend to have middle raised eyebrows and large noses. #### 4. Chernoff faces colored by the species Make a Chernoff face for each record using the colors added to the rescaled data: chfaces = ChernoffFace[ AssociationThread[{"NoseLength", "LeftEyebrowTrim", "EyeSize", "LeftEyebrowSlant", "FaceColor"} -> #], ImageSize -> 100] & /@ chernoffData; Make an image collage with the obtained faces: ImageCollage[chfaces, Background -> White] [](http://i.imgur.com/uPBZJuf.gif) #### 5. Comparison We can see that the collage with colored faces completely explains the patterns found in the black-and-white faces: setosa have smaller petals (both length and width), and virginica have larger petals. ## Browsing a large number of records with Chernoff faces If we have a large number of records each comprised of a relative small number of numerical values we can use Chernoff faces to browse the data by taking small subsets of records. Here is an example using "Vinho Verde" wine quality dataset [8]. [](http://i.imgur.com/jDwXgfT.png) [](http://i.imgur.com/ZOqwuFT.png) ## Chernoff faces for lists of long lists In this section we consider data that is a list of lists. Each of the lists (or rows) is fairly long and represents values of the same variable or process. If the data is a full array, then we can say that in this section we deal with transposed versions of the data in the previous sections. Since each row is a list of many elements visualizing the rows directly with Chernoff faces would mean using a small fraction of the data. A natural approach in those situations is to summarize each row with a set of descriptive statistics and use Chernoff faces for the row summaries. The process is fairly straightforward; the rest of the section gives concrete code steps of executing it. ### Data generation Here we create 12 rows of 200 elements by selecting a probability distribution for each row. SeedRandom[1425] {dists, data} = Transpose@Table[( rdist = RandomChoice[{NormalDistribution[RandomReal[10], RandomReal[10]], PoissonDistribution[RandomReal[4]], GammaDistribution[RandomReal[{2, 6}], 2]}]; {rdist, RandomVariate[rdist, 200]}), {12}]; Dimensions[data] (* {12, 200} *) We have the following 12 "records" each with 200 "fields": DistributionChart[data, ChartLabels -> MapIndexed[ Row[{Style[#2[[1]], Red, Larger], " ", Style[#1, Larger]}] &, dists], ChartElementFunction -> ChartElementData["PointDensity", "ColorScheme" -> "SouthwestColors"], BarOrigin -> Left, GridLines -> Automatic, ImageSize -> 1000] [](http://i.imgur.com/HJWZo6T.gif) Here is the summary of the records: Grid[ArrayReshape[RecordsSummary[Transpose@N@data], {3, 4}], Dividers -> All, Alignment -> {Left}] [](http://i.imgur.com/HmIUX0d.png) ### Data transformation Here we "transform" each row into a vector of descriptive statistics: statFuncs = {Mean, StandardDeviation, Kurtosis, Median, QuartileDeviation, PearsonChiSquareTest}; sdata = Map[Through[statFuncs[#]] &, data]; Dimensions[sdata] (* {12, 6} *) Next we rescale the descriptive statistics data: sdata = VariablesRescale[sdata, "StandardizingFunction" -> (Standardize[#, Median, QuartileDeviation] &)]; For kurtosis we have to do special rescaling if we want to utilize the property that Gaussian processes have kurtosis 3: sdata[[All, 3]] = Rescale[#, {3, Max[#]}, {0.5, 1}] &@Map[Kurtosis, N[data]]; Here is the summary of the columns of the rescaled descriptive statistics array: Grid[{RecordsSummary[sdata, ToString /@ statFuncs]}, Dividers -> All] [](http://i.imgur.com/SHbLThq.png) ### Visualization First we define a function that computes and tabulates (descriptive) statistics over a record. Clear[TipTable] TipTable[vec_, statFuncs_, faceParts_] := Block[{}, GridTableForm[ Transpose@{faceParts, statFuncs, NumberForm[Chop[#], 2] & /@ Through[statFuncs[vec]]}, TableHeadings -> {"FacePart", "Statistic", "Value"}]] /; Length[statFuncs] == Length[faceParts]; To visualize the descriptive statistics of the records using Chernoff faces we have to select appropriate facial features. faceParts = {"NoseLength", "EyeSize", "EyeSlant", "EyesVerticalPosition", "FaceLength", "MouthSmile"}; TipTable[First@sdata, statFuncs, faceParts] [](http://i.imgur.com/bEOFgXH.png) One possible visualization of all records is with the following commands. Note the addition of the parameter "FaceColor" to also represent how close a standardized row is to a sample from Normal Distribution. {odFaceColor, ndFaceColor} = {White, ColorData[7, "ColorList"][[8]]}; Grid[ArrayReshape[Flatten@#, {4, 3}, ""], Dividers -> All, Alignment -> {Left, Top}] &@ MapThread[ (asc = AssociationThread[faceParts -> #2]; chFace = ChernoffFace[ Join[asc, <| "FaceColor" -> Blend[{odFaceColor, ndFaceColor}, #2[[-1]]], "IrisColor" -> GrayLevel[0.8], "NoseColor" -> ndFaceColor|>], ImageSize -> 120, AspectRatio -> Automatic]; tt = TipTable[N@#3, Join[statFuncs, {Last@statFuncs}], Join[faceParts, {"FaceColor"}]]; Column[{Style[#1, Red], Grid[{{Magnify[#4, 0.8], Tooltip[chFace, tt]}, {Magnify[tt, 0.7], SpanFromAbove}}, Alignment -> {Left, Top}]}]) & , {Range[Length[sdata]], sdata, data, dists}] [](http://i.imgur.com/CGVbZck.png) ## Visualizing similarity with nearest neighbors or recommendations ### General idea Assume the following scenario: 1. we have a set of items (movies, flowers, etc.), 2. we have picked one item, 3. we have computed the Nearest Neighbors (NNs) of that item, and 4. we want to visualize how much of a good fit the NNs are to the picked item. Conceptually we can translate the phrase "how good the found NNs (or recommendations) are" to: * "how similar the NNs are to the selected item", or * "how different the NNs are to the selected item." If we consider the picked item as the prototype of the most normal or central item then we can use Chernoff faces to visualize item's NNs deviations. **Remark:** Note that Chernoff faces provide similarity visualization most linked to Euclidean distance that to other distances. ### Concrete example The code in this section demonstrates how to visualize nearest neighbors by Chernoff faces variations. First we create a nearest neighbors finding function over the Fisher Iris data set (without the species class label): irisNNFunc = Nearest[irisDataSet[[All, 1 ;; -2]] -> Automatic, DistanceFunction -> EuclideanDistance] Here are nearest neighbors of some random row from the data. itemInd = 67; nnInds = irisNNFunc[irisDataSet[[itemInd, 1 ;; -2]], 20]; We can visualize the distances with of the obtained NNs with the prototype: ListPlot[Map[ EuclideanDistance[#, irisDataSet[[itemInd, 1 ;; -2]]] &, irisDataSet[[nnInds, 1 ;; -2]]]] [](http://i.imgur.com/dnUlWSD.gif) Next we subtract the prototype row from the NNs data rows, we standardize, and we rescale the interval $[ 0, 3 \sigma ]$ to $[ 0.5, 1 ]$: snns = Transpose@Map[ Clip[Rescale[ Standardize[#, 0 &, StandardDeviation], {0, 3}, {0.5, 1}], {0, 1}] &, Transpose@ Map[# - irisDataSet[[itemInd, 1 ;; -2]] &, irisDataSet[[nnInds, 1 ;; -2]]]]; Here is how the original NNs data row look like: GridTableForm[ Take[irisDataSet[[nnInds]], 12], TableHeadings -> irisColumnNames] [](http://i.imgur.com/7r6habk.png) And here is how the rescaled NNs data rows look like: GridTableForm[Take[snns, 12], TableHeadings -> Most[irisColumnNames]] [](http://i.imgur.com/P7xnwR5.png) Next we make Chernoff faces for the rescaled rows and present them in a easier to grasp way. We use the face parts: Take[Keys[ChernoffFace["FacePartsProperties"]], 4] (* {"FaceLength", "ForheadShape", "EyesVerticalPosition", "EyeSize"} *) To make the face comparison easier, the first face is the one of the prototype, each Chernoff face is drawn within the same rectangular frame, and the NNs indices are added on top of the faces. chfaces = ChernoffFace[#, Frame -> True, PlotRange -> {{-1, 1}, {-2, 1.5}}, FrameTicks -> False, ImageSize -> 100] & /@ snns; chfaces = MapThread[ ReplacePart[#1, 1 -> Append[#1[[1]], Text[Style[#2, Bold, Red], {0, 1.4}]]] &, {chfaces, nnInds}]; ImageCollage[chfaces, Background -> GrayLevel[0.95]] [](http://i.imgur.com/EKpJ7mj.gif) We can see that the first few - i.e. closest -- NNs have fairly normal looking faces. Note that using a large number of NNs would change the rescaled values and in that way the first NNs would appear more similar. ## References \[1\] Herman Chernoff (1973). "The Use of Faces to Represent Points in K-Dimensional Space Graphically" (PDF). Journal of the American Statistical Association (American Statistical Association) 68 (342): 361-368. doi:10.2307/2284077. JSTOR 2284077. URL: [http://lya.fciencias.unam.mx/rfuentes/faces-chernoff.pdf](http://lya.fciencias.unam.mx/rfuentes/faces-chernoff.pdf) . \[2\] Christopher J. Morris; David S. Ebert; Penny L. Rheingans, "Experimental analysis of the effectiveness of features in Chernoff faces", Proc. SPIE 3905, 28th AIPR Workshop: 3D Visualization for Data Exploration and Decision Making, (5 May 2000); doi: 10.1117/12.384865. URL: [http://www.research.ibm.com/people/c/cjmorris/publications/Chernoff_990402.pdf](http://www.research.ibm.com/people/c/cjmorris/publications/Chernoff_990402.pdf) . \[3\] Anton Antonov, [Chernoff Faces implementation in Mathematica](https://github.com/antononcube/MathematicaForPrediction/blob/master/ChernoffFaces.m), (2016), source code at [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), package [ChernofFacess.m](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/ChernoffFaces.m) . \[4\] Anton Antonov, [MathematicaForPrediction utilities](https://github.com/antononcube/MathematicaForPrediction/blob/master/MathematicaForPredictionUtilities.m), (2014), source code [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), package [MathematicaForPredictionUtilities.m](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/MathematicaForPredictionUtilities.m). \[5\] Anton Antonov, [Variable importance determination by classifiers implementation in Mathematica](https://github.com/antononcube/MathematicaForPrediction/blob/master/VariableImportanceByClassifiers.m)[, ](https://github.com/antononcube/MathematicaForPrediction/blob/master/IndependentComponentAnalysis.m)(2015), source code at [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), package [VariableImportanceByClassifiers.m](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/VariableImportanceByClassifiers.m). \[6\] Anton Antonov, ["Importance of variables investigation guide"](https://github.com/antononcube/MathematicaForPrediction/blob/master/Documentation/Importance-of-variables-investigation-guide.pdf), (2016), [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), [https://github.com/antononcube/MathematicaForPrediction](https://github.com/antononcube/MathematicaForPrediction), folder [Documentation](https://github.com/antononcube/MathematicaForPrediction/tree/master/Documentation). \[7\] Wikipedia entry, Iris flower data set, [https://en.wikipedia.org/wiki/Iris_flower_data _set](https://en.wikipedia.org/wiki/Iris_flower_data_set) . \[8\] P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis. Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4):547-553, 2009. URL [https://archive.ics.uci.edu/ml/datasets/Wine+Quality](https://archive.ics.uci.edu/ml/datasets/Wine+Quality) . Anton Antonov 2016-06-03T02:17:45Z https://community.wolfram.com/groups/-/m/t/824280 Dear all, I have been working on an open source deep learning library in Mathematica which I would like to share. The code is hosted on github under an MIT license: [https://github.com/jfrancis71/Machine-Vision][1] I have succesfully used it to train a face recognition system, and also automatically drive my lego EV3 robot. Face Recognition: [https://www.youtube.com/watch?v=TdRtUnSppB0][2] EV3 Car: [https://www.youtube.com/watch?v=DCad82UdDFA][3] [1]: https://github.com/jfrancis71/Machine-Vision [2]: https://www.youtube.com/watch?v=TdRtUnSppB0 [3]: https://www.youtube.com/watch?v=DCad82UdDFA This has arisen from a homegrown project, so the code is probably a little scruffy (and not well documented) at the moment. Having said that it does work and I would welcome constructive feedback and help turning this into an active open source project. It's not really aimed at someone who just wants a good face recogniser, the new built in Mathematica functions are probably more suitable for that. it's more suitable for someone interested in the internals or developing their own neural nets. The distinction with the current Mathematica machine learning functionality, is this library is designed to support multiple layer networks including convolutional layers (much used in computer vision presently). There is more information available on the github site in the link above. Thanks, Julian Francis. Julian Francis 2016-03-16T16:19:18Z https://community.wolfram.com/groups/-/m/t/2445356 ![enter image description here][1] The popular game Wordle can take up a lot of your time. The author designed it so that it you can only play it once a day, thus saving us from ourselves :-). [Wordle][2] [NYTimes article on Wordle][3] But I couldn't resist the challenge to create a version of it in Mathematica, just for fun and because I was bored this past weekend. See the attached notebook and enjoy. Alas, since you can run it any number of times you are only to blame for yourself it you spend too much time on it. After executing the notebook just execute MWordle[Deploy] to bring up the game. A few additional comments. The notebook MWordle.nb has the option AutoGeneratedPackage -> Automatic which causes it, when saved, to create an MWordle.m package file in its same directory. The code in MWordle.nb is set up as a package with the context MWordle`Mwordle` If you want to set things up so that the package gets loaded and the MWordle game is automatically launched, do the following. Create a directory MWordleGame on your disk (The name MWordleGame can actually be whatever you wish.) And in the MWordleGame directory create a new directory called MWordle. (This name must be exactly that so that the MWordle`Mwordle` Context is property respected.) Put the MWordle.nb notebook in the MWordle dierectory, open it in Mathematica and save it so that the MWordle.m file is created in the MWordle directory. Then you can close the MWordle.nb notebook. Now in your MWordleGame directory save a new notebook -- you can call it whatever you wish, but something like LaunchMwordle.nb is a sensible choice. In that notebook create a button with the following command: CellPrint[TextCell[Button["Launch MWordle", Monitor[ If[! MemberQ[$Path, NotebookDirectory[]], AppendTo[$Path, NotebookDirectory[]]]; Needs["MWordle`MWordle`"]; MWordle`MWordle`MWordle[Deploy], Row[{ProgressIndicator[Appearance -> "Necklace", ImageSize -> Small], Spacer[5], Style["Launching MWordle...", 12, Blue, FontFamily -> "Arial"]}]], Method -> "Queued"], "Text", GeneratedCell -> False, CellAutoOverwrite -> False]] You now have a button in your LaunchMwordle.nb notebook which you can use any time you want to launch MWordle without having to execute the cells in the MWordle.nb notebook. Download the actual notebook from the link at the end of this post. The following is a version here to read. &[Wolfram Notebook][4] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Wordle.gif&userId=20103 [2]: https://www.powerlanguage.co.uk/wordle/ [3]: https://www.nytimes.com/2022/01/03/technology/wordle-word-game-creator.html [4]: https://www.wolframcloud.com/obj/08c015e2-0d65-4634-bf54-4b73e518f6d5 David Reiss 2022-01-13T21:47:44Z https://community.wolfram.com/groups/-/m/t/762950 **RadarChart Mathematica Package** The *RadarChart* mathematica package incorporates Radar Charts (also known as Spider Charts, Web Charts and Star Plots) as a new set of visualization tools into the mathematica environment. A detailed explanation on the use of radar charts can be found in the following [wikipedia page](https://en.wikipedia.org/wiki/Radar_chart). The project is listed at the mathematica package repository ([http://packagedata.net][1]), or can be accessed directly at [github][2]. Thanks to [halirutan](https://github.com/halirutan) for his guidance on setting up this project. Installation Instructions are heavily based on his existing projects. --- **Navigation** - [Detailed Usage](#-detailed-usage) - [Examples](#examples) - [Neat Examples](#neat-examples) - [Installation](#installation) - [Automatic Installation for *Mathematica* 9 and above](#automatic-installation-for-mathematica-9-and-above) - [Manual Installation](#manual-installation) - [Contact](#-contact) --- ## Detailed Usage RadarChart[{y1,y2,y3,...}] generates a radar plot (also known as web, star, spider, cobweb or kiviat diagram) corresponding to a list of values. This type of chart is suitable for showing commonality and outliers across different variables. RadarChart[{list1,list2,...}] generates a radar plot to compare several series. RadarChart[association] generates a radar plot to view the variable values in an association. RadarChart[{association1,association2,...}] generates a radar plot to compare several series. RadarChart[dataset] generates a radar plot to compare several series. Options are listed in the help file, and also in the readme document @ the github site. ##Examples Needs["RadarChart`"] RadarChart[{1, 2, 4, 5, 3}, ChartLegends -> {"Private Label Strawberry Juice"}, AxesLabel -> {"Ripe", "Green", "Candy", "Juicy", "Sulphur"}, PlotLabel -> Style["Sensory Map Strawberry Juice", Bold, Large], ImageSize -> Medium]  Multiple Series can be charted. Needs["RadarChart`"] RadarChart[{{1, 4, 3, 5, 2}, {2, 4, 3, 2, 1}}, Filling -> Axis, AxesLabel -> {"Sweet", "Sour", "Salty", "Bitter", "Umami"}, PlotStyle -> {Red, Blue}, ChartLegends -> {"Fernet-Cola", "Jugo"}]  Example of a start plot. Needs["RadarChart`"]; RadarChart[{{1, 4, 3, 5, 2}, {2, 4, 3, 2, 1}}, AxesType -> "Star"]  Compare survey results. Needs["RadarChart`"]; RadarChart[{{3, 4, 3, 4, 2}, {4, 5, 4, 5, 3}}, AxesLabel -> {"Assets", "Reliability", "Cost Control", "Abstenteeism", "Revenue"}, ChartLegends -> {"Past Year", "Current Year"}, Filling -> Axis, PlotStyle -> {{Gray}, {Black}}, ImageSize -> Medium]  ##Neat Examples Analyze and compare crime statistics across states. Needs["RadarChart`"]; states = EntityClass["AdministrativeDivision", "AllUSStatesPlusDC"]; stateNames = First@StringSplit[#, ","] & /@ states["Name"]; crimeProps = {"AggravatedAssaultRate", "BurglaryRate", "ForcibleRapeRate", "LarcenyTheftRate", "MurderNonnegligentManslaughterRate", "PropertyCrimeRate", "RobberyRate", "ViolentCrimeRate"}; crimeData = EntityValue[states, EntityProperty["AdministrativeDivision", #] & /@ crimeProps]; crimeRanking = Transpose[ Ordering[crimeData[[All, #]]] & /@ Range@Length@crimeProps]; ds = Dataset[ AssociationThread[stateNames, AssociationThread[crimeProps, #] & /@ crimeRanking]]; GraphicsRow[{RadarChart[ds["Georgia"], AxesType -> "Star", PlotLabel -> Style["Georgia", Bold, Large], Epilog -> {Dashed, Circle[{0, 0}, 25.5]}, ImageSize -> Large], GraphicsGrid[ Partition[ RadarChart[ds[#], PlotLabel -> #, AxesLabel -> None, PlotRange -> {0, 50}, AxesType -> "Star", PlotRangePadding -> Full, FrameTicks -> None, Epilog -> {Dashed, Circle[{0, 0}, 25.5]}] & /@ stateNames, UpTo[6]], ImageSize -> 600]}]]  #Installation This package should work with Mathematica >8 if not using associations or datasets. The package was developed using mathematica 10. The installation is simple: Copy the `RadarChart` package directory into a location where *Mathematica* can find it. Usually this is the `Applications` directory in your `$UserBaseDirectory`. Just evaluate FileNameJoin[{$UserBaseDirectory, "Applications"}] to see it. If there is an old installation of the `RadarChart`, remove it. Please find detailed steps below. ###Automatic Installation for *Mathematica* 9 and above We have set up [an installation script](https://raw.githubusercontent.com/catrasca/RadarChart/master/RadarChart/installer.m) that does all the steps, except deleting old installations, for you. If it finds an old installation, it will prompt you with the location and quit, so that you can remove the old installation. After removing the old files, just start it again and it will proceed through all the steps pointed out in the manual installation section. To start the installation script, simply call Import["http://tinyurl.com/ntmhkca"] After this, the package should be available in your mathematica instance. ###Manual Installation ####Removing old Installations Old installation packages can be found by simply searching directories in your `$Path`. FileNames["RadarChart", $Path] Please remove old installation directories that appear after evaluating the commands above. You can use DeleteDirectory[dir, DeleteContents -> True] for that, but note that on Windows this might fail, because there, some files are locked when *Mathematica* is running. In this case, close *Mathematica* and do it manually using an explorer. ###Downloading, Extracting and Copying the New Version The easiest way is, to download the whole repository as zip file. Use [this master.zip](https://github.com/catrasca/RadarChart/archive/master.zip) or click the *Download ZIP* on the right side on this page. After you have downloaded the file extract it. If you have no tool for this on Windows, you could use the [free 7-Zip](http://7-zip.org/). Under Mac OSX and Linux this should work out of the box. Inside the extracted directory, you will find a subdirectory `RadarChart` which has the following structure RadarChart/ ??? Documentation ? ??? English ? ??? ReferencePages ? ??? Symbols ? ??? RadarChart.nb ??? Kernel ? ??? init.m ??? Installer.m ??? pacletInfo.m ??? RadarChart.m Copy the whole `RadarChart` directory with all its content to your `Applications` folder under your `$UserBaseDiretory`. If everything is in place proceed to the next step. ###Finishing the Installation To make the package works, you can simply restart *Mathematica*. ## Contact If you find bugs or have any other questions, please [create a new issue](https://github.com/catrasca/RadarChart/issues) in the bug-tracker. [1]: http://packagedata.net/ [2]: http://github.com/catrasca/RadarChart Diego Zviovich 2015-12-22T20:02:41Z https://community.wolfram.com/groups/-/m/t/1421180 Two months ago, Albert Rich posted ["What's the hardest integral Mathematica running Rubi can find?"](http://community.wolfram.com/groups/-/m/t/1343015) here on the Wolfram Community. You might have also seen that I responded in detail, and pointed out a few things that could help improve Rubi (Rule-based integrator). While it appears nothing really happened afterward, this is far from reality. Since then, Albert and I have worked closely together to make Rubi more accessible and user-friendly. If you would like to learn how our productive collaboration evolved, let me invite you to read [my latest blog-post](http://halirutan.de/programming/Rubi/). However, here, we want to share an update that should serve as an overview of what we have done to improve Rubi. First of all, Rubi has got a new home under [rulebasedintegration.org](https://rulebasedintegration.org/), and its old website will no longer be updated. On the new website, you will find information, installation instructions, and links to the source-code and test-suites. Secondly, we created a [Rubi Organization](https://github.com/RuleBasedIntegration) on GitHub that serves as the headquarters for all things Rubi. It contains all Rubi's code, notebooks, and test-suites nicely structured into several repositories. At the moment, we provide repositories for the * loadable package files and notebook source files defining over 6700 integration rules, * PDF files displaying the rules in human-readable mathematical notation alongside the Mathematica code, and * test-suite files containing over 71000 integration problems and their solutions. The integration test files are available in the syntax used by 4 popular computer algebra systems (Mathematica, Maple, Maxima, and Axiom). The test-suite can be used to compare Rubi's results with other symbolic integrators, including Mathematica's `Integrate` function. In addition to the transition to GitHub, we recently released version 4.16.0.3 of Rubi which significantly expands the class of expressions the system can integrate. But the most noticeable change for users is the completely reworked display of the rules and intermediate steps Rubi uses to integrate expressions. Although installation, usage, and examples are given on [Rubi's website](https://rulebasedintegration.org/), let me show you how easy it is to install and run Rubi 4.16.0.3 using Mathematica 11.3: The command PacletInstall["https://github.com/RuleBasedIntegration/Rubi/releases/download/4.16.0.3/Rubi-4.16.0.3.paclet"]; installs the Rubi-4.16.0.3 paclet on your computer. After that, to load Rubi into Mathematica all you have to do is issue the `Get` command << Rubi` Then to integrate an expression with respect to a variable, use Rubi's `Int` command similar to Mathematica's `Integrate` command. For example, evaluating Int[(Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]), x] returns the antiderivative (a.k.a. the indefinite integral) ![enter image description here][1] Rubi's unique ability to display the steps it uses to integrate expressions is a great feature of the system. For example, the `Steps` command Steps@Int[(Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]), x] displays ![enter image description here][2] Here, in red are the rules used to integrate the expression, and in blue are the intermediate results. Each rule can be expanded to show the rule number, which directly corresponds to the index of the rule in `Int`'s list of DownValues. More importantly, you can see the conditions that have to be satisfied so the rule can be applied. ![enter image description here][3] Rubi's `Stats` command provides statistics about the integration. For example, Stats[Int[(Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]), x]] displays ![enter image description here][4] The leaf-count size of an antiderivative is a crude measure of its complexity. As you can see, Rubi's antiderivative for this integral has a leaf-count of 25. Now compare Rubi's antiderivative with that produced by Mathematica 11.3 for the same integral: Integrate[(Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]), x] ![enter image description here][5] LeafCount[%] (* 290 *) Note that not only is Mathematica's result more than 11 times the size of Rubi's, it unnecessarily involves elliptic integral functions *and* the imaginary unit. Skeptics might be inclined to ask if Rubi's dramatically simpler result is actually a valid antiderivative. Since symbolic differentiation is much easier than integration, antiderivatives can be verified correct by seeing if its derivative equals the original integrand as follows: expr = (Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]); FullSimplify[D[Int[expr, x], x] == expr] FullSimplify[D[Integrate[expr, x], x] == expr] ![enter image description here][6] As you can see Mathematica easily verifies Rubi's antiderivative correct, but has a hard time verifying its own antiderivative correct... Albert and I are working on publishing the program used to thoroughly test each new version of Rubi before being released. The test program ensures Rubi's result equals the optimal antiderivative for the over 71000 problems in the test-suite. And yes, the optimal antiderivatives have all been verified correct by differentiation. Of course, the optimal antiderivatives stored in the test-suite are actually just the simplest ones found so far. If you should find a substantially simpler antiderivative than the one in the test-suite, please report it so the test-suite can be made even harder on Rubi! If all that has got you interested in joining Rubi's community of users, check out its website or talk to us in our [Gitter chatroom](https://gitter.im/Rule-Based-Integration/Lobby). [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=10564sSs5M.png&userId=11733 [2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=qDtIH.png&userId=11733 [3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=vQ4GR.png&userId=11733 [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=43t8e.png&userId=11733 [5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=xjKTJ.png&userId=11733 [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=eCoWB.png&userId=11733 Patrick Scheibe 2018-08-24T02:50:37Z