Community RSS Feed
https://community.wolfram.com
RSS Feed for Wolfram Community showing any discussions in tag sorted by activeThe Not-So-Secret Fourth Parameter of Quantum Codes
https://community.wolfram.com/groups/-/m/t/3073281
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=7931heroimage.png&userId=20103
[2]: https://www.wolframcloud.com/obj/efdd0ac6-8bff-4684-91a0-07f8a26847e0Eric Kubischta2023-11-28T23:11:04Z[BOOK] Explore Algebra with Wolfram Language: New High School Textbook Available!
https://community.wolfram.com/groups/-/m/t/3067312
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=10328hero.png&userId=20103
[2]: https://www.wolframcloud.com/obj/4ecce92c-d212-4e82-8ec6-4ee75be60facAthanasios Paraskevopoulos2023-11-18T17:06:36ZThe Isomorphism of $H_4$ and $E_8$
https://community.wolfram.com/groups/-/m/t/3073391
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=2503HeroImage.png&userId=20103
[2]: https://www.wolframcloud.com/obj/4ad9f13b-cd72-47b3-a597-1c68151d0ef4J Gregory Moxness2023-11-28T20:40:36ZCertain .stl files couldn't be Imported
https://community.wolfram.com/groups/-/m/t/3072329
I tried to import a 3d object into Masematica.
However, it stopped advancing from the running screen.
Other 3d object files could be imported.
If the file size is too large, can it not be imported?
Attached files that could not be imported.Megumi Kitano2023-11-27T06:04:35ZHow to remove noise in the plot generated from ListContourPlot?
https://community.wolfram.com/groups/-/m/t/3073060
I am trying to do ListContourPlot a list of three-dimensional data. The notebook file and the dependent file are attached here.
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/886cabfd-bc5d-4b21-bedf-b882bb08dc1c
I think the portion marked in the red-colored box is noise generated from the Mathematica module. Is there any way to remove these gittery plots from the original plot?Bibekananda Patra2023-11-28T07:24:30Z[WSG23] Daily Study Group: Introduction to Finite Mathematics
https://community.wolfram.com/groups/-/m/t/3053258
![enter image description here][1]
A Wolfram U Daily Study Group previewing the upcoming Introduction to Finite Mathematics course begins on Monday **November 6, 2023**.
Join me and your fellow learners for an introduction to various topics with important real-world application to economics, business, sociology, psychology, and other sciences. A curriculum concept to emerge from Dartmouth as an alternative to calculus-based courses of study, finite mathematics has become a common course offering at universities around the world.
After reviewing a few concepts from algebra, apply them to the study of linear optimization, matrices, annuities, probability, Markov chains, and two-player games.
The Introduction to Finite Mathematics course will soon be added to the Wolfram U catalog, so get a sneak peak of the lessons as part of this live study group. Participate in live Q&A, check your understanding with in-session polls, and more.
This Study Group runs weekdays **Nov 6-17**, with the exception of Nov 10.
> [**REGISTER HERE**][2]
Please use this thread to collaborate and share ideas, materials and links to other resources with fellow learners.
Creating the full course has been a lot of fun, so I hope to see you all in the live study group!
![enter image description here][3]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=WolframUbanner.png&userId=2643831
[2]: https://www.bigmarker.com/series/daily-study-group-introduction-finite-mathematics-wsg47/series_details?utm_bmcr_source=community
[3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=10522simplex.png&userId=2643831John McNally2023-10-20T22:06:16Z[WSG21] New Intro to Wolfram Language Daily Study Group
https://community.wolfram.com/groups/-/m/t/2494009
We're starting a new [Daily Study Group][1] for new users of Wolfram Language! If you'd like to learn to program in Wolfram Language with a combination of short lessons, poll questions to review key concepts, practice problems and Q&A, this is the group for you! We'll be working through the [Elementary Introduction to Wolfram Language][2] book by Stephen Wolfram with lots of interactive examples.
A certificate of program completion will be awarded to participants who attend online sessions and pass quizzes.
A list of daily topics can be found on the [Daily Study Groups][3] page. The study group sessions include videos and reading materials for the course and time for discussion and Q&A.
> **REGISTER HERE:** https://www.bigmarker.com/series/daily-study-group-an-elementary-introduction-to-the-wolfram-language/series_details
> **About This Study Group:** Learn the Wolfram Language and modern computational thinking from Stephen Wolfram's book with veteran Wolfram Language instructor and developer David Withoff and Wolfram instructional designer and technologist Rory Foulger. No prior programming knowledge is required.
> Sessions include short lessons, poll questions to review key concepts, practice problems and Q&A. A certificate of program completion will be awarded to participants who attend online sessions and pass quizzes.
> Study Group sessions run daily, Monday through Friday. This Study Group begins Monday, March 21, 2022.
![enter image description here][4]
[1]: https://www.bigmarker.com/series/daily-study-group-an-elementary-introduction-to-the-wolfram-language/series_details?utm_bmcr_source=commmunity
[2]: https://www.wolfram.com/language/elementary-introduction/2nd-ed/what-is-the-wolfram-language.html
[3]: https://www.bigmarker.com/series/daily-study-group-an-elementary-introduction-to-the-wolfram-language/series_details?utm_bmcr_source=commmunity
[4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=WolframUbanner.jpeg&userId=1835305Rory Foulger2022-03-21T17:10:40ZAI vision via Wolfram Language
https://community.wolfram.com/groups/-/m/t/3072318
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/c298ab8f-16de-4178-bb27-268c24bd8485Anton Antonov2023-11-27T03:45:43ZDirect API access to new features of GPT-4 (including vision, DALL-E, and TTS)
https://community.wolfram.com/groups/-/m/t/3062403
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/34ac42c4-10de-4201-a158-6af47eda2bd8Marco Thiel2023-11-08T21:48:25ZA primer on Association and Dataset
https://community.wolfram.com/groups/-/m/t/1167544
*NOTE: all Wolfram Language code and data are available in the attached notebook at the end of the post.*
----------
For my class this fall, I developed a little primer on Association and Dataset that I think might be useful for many people. So, I'm sharing the attached notebook. It's somewhat about the concepts embedded inside these features. It's intended for people at a beginner-intermediate level of Mathematica/Wolfram Language programming, but might be of value even to some more advanced users who have not poked about the Dataset functionality.
The sections of the notebook are:
1. The world before Associations and Datasets
2. Datasets without Associations
3. Enter the Association
4. Creating a Dataset from a List of Associations
5. Nice queries with Dataset
6. Query
7. Some Recipes
#The world before Associations and Datasets#
Here' s an array of data. The data happens to represent the cabin class, age, gender, and survival of some of the passengers on the Titanic.
t = {{"1st", 29, "female", True}, {"1st", 30, "male", False}, {"1st",
58, "female", True}, {"1st", 52, "female", True}, {"1st", 21,
"female", True}, {"2nd", 54, "male", False}, {"2nd", 29, "female",
False}, {"3rd", 42, "male", False}};
As it stands, our data is a List of Lists.
Head[t]
> List
Head /@ t
> {List, List, List, List, List, List, List, List}
Suppose I wanted to get the second and fifth rows of the data. This is how I could do it.
t[[{2, 5}]]
> {{"1st", 30, "male", False}, {"1st", 21, "female", True}}
Suppose we want to group the passengers by gender and then compute the mean age. We could do this with the following pretty confusing code.
Use and enjoy. Constructive feedback appreciated.
grouped = GatherBy[t, #[[3]] &];
justTheAges = grouped[[All, All, 2]];
Mean /@ justTheAges
> {189/5, 42}
Or I could write it as a one liner this way.
Map[Mean, GatherBy[t, #[[3]] &][[All, All, 2]]]
> {189/5, 42}
But either way, realize that I have to remember that gender is the third column and that age is the second column. When there is a lot of data, this can get hard to remember.
#Datasets without Associations#
I could, if I wanted, convert this data into a Dataset. I do this below simply by wrapping Dataset about t. You see there is now some formatting about the data. But there are no column headers (because no one has told Dataset what to use). And there are no row headers, again because no one has told Dataset what to use.
t2 = Dataset[t]
![enter image description here][1]
The head of the expression has changed.
Head[t2]
> Dataset
Now, I can now access the data in a different way.
Query[{2, 5}][t2]
![enter image description here][2]
Or, I can do this. Mathematica basically converts this expression into Query[{2,5}][t2]. The expression t2[{2,5}] is basically syntactic sugar.
t2[{2, 5}]
![enter image description here][3]
##Digression : Using Query explicitly or using syntactic sugar##
Why, by the way would anyone use the longer form if Mathematica does the work for you? Suppose you want to store a Dataset operation -- perhaps a complex series of Dataset operations -- but you want it to work not just on a particular Dataset but on any Dataset (that is compatible). Here's how you could do it.
q = Query[{2, 5}]
> Query[{2, 5}]
q[t2]
![enter image description here][4]
Now, let' s create a permutation of the t2 Dataset so that the rows are scrambled up.
t2Scrambled = t2[{1, 4, 8, 3, 2, 7, 5}]
![enter image description here][5]
We can now run the q operation on t2Scrambled. Notice that the output has changed even though the query has stayed the same.
q[t2Scrambled]
![enter image description here][6]
We can also generate Query objects with functions. Here's a trivial example. There are very few languages of which I am aware that have the ability to generate queries by using a function. The one other example is Julia.
makeASimpleQuery[n_] := Query[n]
makeASimpleQuery[{3, 4, 7}][t2]
![enter image description here][7]
##MapReduce operations on Dataset objects##
Now, if I want to know the mean ages of the genders I can use this code. This kind of grouping of data and then performing some sort of aggregation operation on the groups is sometimes known as a MapReduce. (I'm not a fan of the name, but it is widely used). It's also sometimes known as a rollup or an aggregation.
Query[GroupBy[#[[3]] &], Mean, #[[2]] &][t2]
![enter image description here][8]
Or this shorthand form in which the Query is constructed.
t2g = t2[GroupBy[#[[3]] &], Mean, #[[2]] &]
![enter image description here][9]
I think this is a little cleaner. But we still have to remember the numbers of the columns, which can be challenging.
By the way, just to emphasize how we can make this all functional, here's a function that creates a query that can run any operation (not just computing the mean) on the Dataset grouped by gender and then working on age.
genderOp[f_] := Query[GroupBy[#[[3]] &], f, #[[2]] &]
genderOp[Max][t2]
![enter image description here][10]
To test your understanding, see if you can find the minimum age for each class of passenger on the Titanic in our Dataset **t2**.
Query[GroupBy[#[[1]] &], Min, #[[2]] &][t2]
![enter image description here][11]
#Enter the Association#
##Review of Association##
If you feel comfortable with Associations, you can skip this section; otherwise read it carefully. Basically the key to understanding most Dataset operations is understanding Associations.
###Construction of Associations###
Now let' s alter the data so that we don't have to remember those facts. To do this we will create an **Association**. Here's an example called **assoc1**. Notice that we do so by creating a sequence of rules and then wrapping it in an Association head. Notice that the standard output does not preserve the word "Association" as the head but, just as List is outputted as stuff inside curly braces, Association is outputted as stuff inside these funky "<|" and "|>" glyphs.
assoc1 = Association["class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True|>
I could equivalently have created a list of rules rather than a sequence. Mathematica would basically unwrap the **List** and create a sequence.
assoc1L = Association[{"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True}]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True|>
We can use **AssociationThread** to create Associations in a different way. The first argument is the list of things that go on the left hand side of the Rules -- the "keys" -- and the second argument is the list of things that go on the right hand side of the Rules -- the "values".
assoc1T = AssociationThread[{"class", "age", "gender", "survived"}, {"1st", 29, "female", True}]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True|>
Now let's use **AssociationThread** function to create a list of Associations similar to our original data.
convertListToAssociation =
list \[Function]
AssociationThread[{"class", "age", "gender", "survived"}, list]
> Function[list, AssociationThread[{"class", "age", "gender", "survived"}, list]]
I start with t and Map the **convertListToAssociation** function over the rows of the data. I end up with a list of Associations.
t3 = Map[convertListToAssociation, t]
![enter image description here][12]
###Keys and Values###
Associations have keys and values. These data structures are used in other computer languages but known by different names: *Python* and *Julia* call them dictionaries. *Go* and *Scala* call them maps. *Perl* and *Ruby* call them hashes. *Java* calls it a *HashMap*. And *Javascript* calls it an object. But they all work pretty similarly. Anyway, the keys of an **Association** are the things on the left hand side of the Rules.
Keys[assoc1]
> {"class", "age", "gender", "survived"}
And the values of an Association are the things on the right hand side of the Rules.
Values[assoc1]
> {"1st", 29, "female", True}
That' s about all there is too it. Except for one thing. Take a look at the input and output that follows.
assoc2 = Association["a" -> 3, "b" -> 4, "a" -> 5]
> <|"a" -> 5, "b" -> 4|>
You can' t have duplicate keys in an Association. So, when Mathematica confronts duplicate keys, it uses the last key it saw. You might think this is a minor point, but it is actually very important in coding. We will see why soon.
###Nested Associations###
A funny thing happens if you nest an **Association** inside another **Association**.
Association[assoc1, assoc2]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True, "a" -> 5, "b" -> 4|>
You end up with a single un - nested (flat) association. That's a little unusual for Mathematica, but we can exploit this flattening as a way of adding elements to an Association.
Association[Association["dances" -> False], assoc1]
> <|"dances" -> False, "class" -> "1st", "age" -> 29, "gender" ->
> "female", "survived" -> True|>
Or, here' s a function that exploits the flattening to add elements to an **Association**.
addstuff = Association[#, "dances" -> False, "sings" -> True] &
> Association[#1, "dances" -> False, "sings" -> True] &
addstuff[assoc1]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True, "dances" -> False, "sings" -> True|>
###Extracting Values from Associations###
Just as the values contained in a **List** can be accessed by using the **Part** function, the values contained in an **Association** can likewise be accessed. Suppose, for example that I wanted to compute double the age of the person in **assoc1**.
It turns out there are a lot of ways of doing this. The first is to treat the Association as a list except that the indices, instead of being integers, are the "keys" that are on the left hand side of the rules.
2*Part[assoc1, "age"]
> 58
2*assoc1[["age"]]
> 58
A second way is to use Query. We can wrap the "key" in the head **Key** just to make sure Mathematica understands that the thing is a Key.
2*Query[Key["age"]][assoc1]
> 58
Usually we can omit the Key and everything works fine.
2*Query["age"][assoc1]
> 58
A third way is to write a function that has an association as its argument.
af = Function[Slot["age"]]
> "#age &"
Now look what we can do.
2*Query[af][assoc1]
> 58
We can shorten this approach by using a simpler syntax for a function.
2*Query[#age &][assoc1]
> 58
Note, though that this still will not work. Basically, Mathematica is confused. It thinks the function itself is the key.
2*assoc1[af]
> 2 Missing["KeyAbsent", #age &]
But here' s a simple workaround. For very simple functions, I can just use the name of the key.
2*assoc1["age"]
> 58
##A Note on Slot Arguments##
And please pay attention to this : sometimes the Mathematica parser gets confused when it confronts a "slot argument" written as #something. If you see this happening, write it as Slot["something"].
Slot["iamaslot"] === #iamaslot
> True
Here' s another problem. What if the key in the association has spaces or non-standard characters in it. Any of these, for example, are perfectly fine keys: the string "I have a lot of spaces in me", the string "I_have_underscores", the symbol True, the integer 43. But if we try to denote those keys by putting a hash in front of them, it will lead to confusion and problems.
problemAssociation = Association["I have a lot of spaces in me" -> 1, "I_have_underscores" -> 2, True -> 3, 43 -> 4]
> <|"I have a lot of spaces in me" -> 1, "I_have_underscores" -> 2,
True -> 3, 43 -> 4|>
{Query[#I have a lot of spaces in me &][problemAssociation],
Query[#I _have _underescores &][problemAssociation]}
![enter image description here][13]
Here' s a solution.
{Query[Slot["I have a lot of spaces in me"] &][problemAssociation],
Query[Slot["I_have_underscores"] &][problemAssociation]}
> {1, 2}
Here' s how we solve the use of True and an integer as keys. We preface them with **Key**.
{Query[#True &][problemAssociation], Query[#43 &][problemAssociation]}
![enter image description here][14]
{Query[Key[True]][problemAssociation],
Query[Key[43]][problemAssociation]}
> {3, 4}
##Working with Associations and Lists of Associations##
Here' s something we can do with the data in the form of an Association. I could ask for the gender of the person in the third row as follows. Notice I did not have to remember that "gender" was generally in the third position.
t3[[3]][["gender"]]
> "female"
So, even if I scramble the rows, I can still use the same code.
t3Scrambled = Map[convertListToAssociation, t[[All, {4, 1, 3, 2}]]]
![enter image description here][15]
t3Scrambled[[3]][["gender"]]
> female
I could also group the people according to their cabin class. Here I use Query on a list of Associations.
Query[GroupBy[#class &]][t3]
![enter image description here][16]
Again, the following code, which does not explicitly use **Query**, won' t work. Basically, nothing has told Mathematica to translate t3[stuff___] \[RightArrow]Query[stuff][t3]. If t3 had a head of Dataset, Mathematica would know to make the translation.
t3[GroupBy[#class &]]
![enter image description here][17]
I can also get certain values for all the Associations in a list of Associations.
Query[All, #age &][t3]
> {29, 30, 58, 52, 21, 54, 29, 42}
I can also map a function onto the result. I don't have to go outside the Query form to do so.
Query[f, #age &][t3]
> f[{29, 30, 58, 52, 21, 54, 29, 42}]
Or, without exiting the Query form, I can map a function onto each element of the result.
Query[Map[f], #age &][t3]
> {f[29], f[30], f[58], f[52], f[21], f[54], f[29], f[42]}
I could also do the same thing as follows.
Query[All, #age &, f][t3]
> {f[29], f[30], f[58], f[52], f[21], f[54], f[29], f[42]}
#Creating a Dataset from a List of Associations#
To get full use out of Query and to permit syntactic shorthands, we need for Mathematica to understand that the list of Associations is in fact a Dataset. Here' s all it takes.
d3 = Dataset[t3]
![enter image description here][18]
We can recover our original list of associations by use of the **Normal** command.
t3 === Normal[d3]
> True
With the data inside a Dataset object we now have pretty formatting. But we have more.
We can still do this. We get the same result but in a more attractive form.
d3g = Query[GroupBy[#class &]][d3]
![enter image description here][19]
But now this shorthand works too.
d3g = d3[GroupBy[#class &]]
![enter image description here][20]
And compare these two elements of code. When the data is in the form of a dataset, Mathematica understands that the stuff in the brackets is not intended as a key but rather is intended to be transformed into a Query.
{Query[#age &][t3[[1]]], d3[[1]][#age &]}
> {29, 29}
##A Dataset that is an Association of Associations##
Let' s look under the hood of **d3g**.
d3gn = Normal[d3g]
![enter image description here][21]
Note : if you *really* want to look under the hood of a **Dataset** ask to see the **Dataset** in **FullForm**. You can also get more information by running the undocumented package Dataset`, but this is definitely NOT recommended for the non-advanced user.
What we see is an Association in which each of the values is itself a list of Associations.
We can map a function over d3gn.
Map[f, d3gn]
![enter image description here][22]
I can of course do the mapping within the **Query** construct.
Query[All, f][d3gn]
![enter image description here][23]
If I try synactic sugar, it doesn' t work because d3gn is not a Dataset.
d3gn[All, f]
> Missing["KeyAbsent", All]
But, if I use the Dataset version, it does work. (The first line may be an ellipsis depending on your operating system and display, but if you look under the hood it looks just like the values for 2nd and 3rd. I have no idea why an ellipsis is being inserted.
d3g[All, f]
![enter image description here][24]
##A Dataset that just has a single Association inside.##
We can also have a Dataset that just has a single Association inside. Mathematica presents the information with the keys and values displayed vertically.
Dataset[d3[[1]]]
![enter image description here][25]
In theory, we could have a Dataset that just had a single number inside it.
Dataset[6]
![enter image description here][26]
#Nice queries with Dataset#
Now I can construct a query that takes a dataset and groups it by the gender column. It then takes each grouping and applies the Mean function to at least part of it. What part? The "age" column part. Notice that I no longer have to remember that gender is the third column and age is the second column.
qd = Query[GroupBy[#gender &], Mean, #age &]
> Query[GroupBy[#gender &], Mean, #age &]
Now I can run this query on t3.
qd[d3]
![enter image description here][27]
We can now learn a lot about Query. So long as our data is in the form of a Dataset we can write the query as either a formal Query or use syntactic sugar.
#Query#
A major part of working with data is to understand **Query**. Let's start with a completely abstract **Query**, that we will call **q1**.
q1 = Query[f];
Now let' s run q1 on t3.
q1[t3]
![enter image description here][28]
We end up with a list of Associations that has f wrapped around it at the highest level. It's the same as if I wrote the following code.
f[t3] === q1[t3]
> True
Now, let' s write a **Query** that applies the function g at the top level of the list of associations and the function **f** at the second level, i.e. to each of the rows. Why does it work at the second level? Because it's the second argument to **Query**.
q2 = Query[g, f];
q2[t3]
![enter image description here][29]
The result is the same as if I mapped **f** onto t3 at its first level and then wrapped **g** around it.
g[Map[f, t3, {1}]] === q2[d3]
Query[All, MapAt[StringTake[#, 1] &, #, {{"class"}, {"gender"}}] &][d3]
Here' s a function **firstchar** that takes the first character in a string.
firstchar = StringTake[#, 1] &
> StringTake[#1, 1] &
Now, let' s construct a query **cg1** that applies **firstchar** to the class and gender keys in each row.
cg1 = Query[All,
a \[Function] MapAt[firstchar, a, {{"class"}, {"gender"}}]]
> Query[All, Function[a, MapAt[firstchar, a, {{"class"}, {"gender"}}]]]
We apply **cg1** to our little dataset **d3**.
cg1[d3]
![enter image description here][30]
What if we want to apply the same function to every element of the Dataset. We just apply it at the lowest level. Here's one way.
Query[Map[f, #, {-1}] &][d3]
![enter image description here][31]
We can also combine it with column wise and entirety wise operations. For reasons that are not clear, Mathematica can't understand this as a Dataset and returns the Normal form.
Query[(Map[f, #, {-1}] &) /* entiretywise, columnwise][d3]
![enter image description here][32]
Here' s how we could actually a multilevel **Query**.
Suppose we want to write a function that computes the fraction of the people in this little dataset that survived. The first step is simply going to be to extract the survival value and convert it to 1 if True and 0 otherwise. There's a built in function Boole that does this.
{Boole[True], Boole[False]}
> {1, 0}
q3 = Query[something,
assoc \[Function] assoc["survived"] /. {True -> 1, _ -> 0}]
> Query[something, Function[assoc, assoc["survived"] /. {True -> 1, _
> -> 0}]]
q3[t3]
> something[{1, 0, 1, 1, 1, 0, 0, 0}]
So, now we have something wrapping a list of 1 s and 0 s. By making **something** the **Mean** function, we can achieve our result.
q4 = Query[Mean, Boole[#survived] &]
> Query[Mean, Boole[#survived] &]
q4[d3]
> 1/2
We can also examine survival by gender. Notice that **Query** is a little like **Association**: it gets automatically flattened.
Query[GroupBy[#gender &], q4][t3]
> <|"female" -> 4/5, "male" -> 0|>
If the data is held in a **Dataset**, we can also write the final step as follows.
d3[GroupBy[#gender &], q4]
![enter image description here][33]
Notice that even if we omit the "Query", this code works. Mathematica just figures out that you meant Query.
The code immediately above is in the form we typically see and often use.
#Some Recipes#
titanic = ExampleData[{"Dataset", "Titanic"}]
![enter image description here][34]
How to add a value to the Dataset based on values external to the existing columns.
Here' s some additional data. Notice that the data is the same length as the titanic dataset.
stuffToBeAdded =
Table[Association["id" -> i,
"weight" -> RandomInteger[{80, 200}]], {i, Length[titanic]}]
![enter image description here][35]
We use **Join** at level 2.
augmentedTitanic = Join[titanic, stuffToBeAdded, 2]
![enter image description here][36]
##How to add a column to a Dataset based on values in the existing columns and to do so row-wise##
Notice that the query below does NOT change the value of the titanic dataset. To change the value of the titanic dataset, one would need to set titanic to the result of the computation. Remember, Mathematica generally does not have side effects or do modifications in place.
Query[All, Association[#, "classsex" -> {#class, #sex}] &][titanic]
![enter image description here][37]
We can add multiple columns this way.
Query[All,
Association[#, "classsex" -> {#class, #sex},
"agesqrt" -> Sqrt[#age]] &][titanic]
![enter image description here][38]
##How to change the value of an existing column : row - wise##
Age everyone one year.
Query[All, Association[#, "age" -> #age + 1] &][titanic]
![enter image description here][39]
How to change the value of columns selectively.
Query[All,
Association[#,
"age" -> If[#sex === "male", #age + 1, #age]] &][titanic]
![enter image description here][40]
How to create a new column based on some aggregate operator applied to another column.
With[{meanAge = Query[Mean, #age &][titanic]},
Query[All,
Association[#, "ageDeviation" -> #age - meanAge] &]][titanic]
![enter image description here][41]
Can you develop your own recipes?
[1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=17751.png&userId=20103
[2]: http://community.wolfram.com//c/portal/getImageAttachment?filename=47222.png&userId=20103
[3]: http://community.wolfram.com//c/portal/getImageAttachment?filename=60813.png&userId=20103
[4]: http://community.wolfram.com//c/portal/getImageAttachment?filename=83664.png&userId=20103
[5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=44435.png&userId=20103
[6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=105416.png&userId=20103
[7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=49777.png&userId=20103
[8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=16898.png&userId=20103
[9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=28239.png&userId=20103
[10]: http://community.wolfram.com//c/portal/getImageAttachment?filename=262710.png&userId=20103
[11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=749611.png&userId=20103
[12]: http://community.wolfram.com//c/portal/getImageAttachment?filename=209912.png&userId=20103
[13]: http://community.wolfram.com//c/portal/getImageAttachment?filename=925313.png&userId=20103
[14]: http://community.wolfram.com//c/portal/getImageAttachment?filename=851614.png&userId=20103
[15]: http://community.wolfram.com//c/portal/getImageAttachment?filename=627315.png&userId=20103
[16]: http://community.wolfram.com//c/portal/getImageAttachment?filename=679516.png&userId=20103
[17]: http://community.wolfram.com//c/portal/getImageAttachment?filename=674717.png&userId=20103
[18]: http://community.wolfram.com//c/portal/getImageAttachment?filename=707518.png&userId=20103
[19]: http://community.wolfram.com//c/portal/getImageAttachment?filename=1053319.png&userId=20103
[20]: http://community.wolfram.com//c/portal/getImageAttachment?filename=579820.png&userId=20103
[21]: http://community.wolfram.com//c/portal/getImageAttachment?filename=980221.png&userId=20103
[22]: http://community.wolfram.com//c/portal/getImageAttachment?filename=450322.png&userId=20103
[23]: http://community.wolfram.com//c/portal/getImageAttachment?filename=113723.png&userId=20103
[24]: http://community.wolfram.com//c/portal/getImageAttachment?filename=932624.png&userId=20103
[25]: http://community.wolfram.com//c/portal/getImageAttachment?filename=754825.png&userId=20103
[26]: http://community.wolfram.com//c/portal/getImageAttachment?filename=836826.png&userId=20103
[27]: http://community.wolfram.com//c/portal/getImageAttachment?filename=393227.png&userId=20103
[28]: http://community.wolfram.com//c/portal/getImageAttachment?filename=577928.png&userId=20103
[29]: http://community.wolfram.com//c/portal/getImageAttachment?filename=158329.png&userId=20103
[30]: http://community.wolfram.com//c/portal/getImageAttachment?filename=984930.png&userId=20103
[31]: http://community.wolfram.com//c/portal/getImageAttachment?filename=664831.png&userId=20103
[32]: http://community.wolfram.com//c/portal/getImageAttachment?filename=1090332.png&userId=20103
[33]: http://community.wolfram.com//c/portal/getImageAttachment?filename=450733.png&userId=20103
[34]: http://community.wolfram.com//c/portal/getImageAttachment?filename=740234.png&userId=20103
[35]: http://community.wolfram.com//c/portal/getImageAttachment?filename=143835.png&userId=20103
[36]: http://community.wolfram.com//c/portal/getImageAttachment?filename=36.png&userId=20103
[37]: http://community.wolfram.com//c/portal/getImageAttachment?filename=37.png&userId=20103
[38]: http://community.wolfram.com//c/portal/getImageAttachment?filename=38.png&userId=20103
[39]: http://community.wolfram.com//c/portal/getImageAttachment?filename=39.png&userId=20103
[40]: http://community.wolfram.com//c/portal/getImageAttachment?filename=40.png&userId=20103
[41]: http://community.wolfram.com//c/portal/getImageAttachment?filename=41.png&userId=20103Seth Chandler2017-08-20T19:26:30ZThe normal form for complex projective plane cubic curves
https://community.wolfram.com/groups/-/m/t/3072983
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/b3e3f177-abb7-4bee-8152-2493730a60b0Barry H Dayton2023-11-28T15:17:14ZVisualizing the Fibonacci subproblem dependency graph
https://community.wolfram.com/groups/-/m/t/3071742
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/7756ffe4-7a43-4c3b-94d5-1b90a3fb58d3Chase Marangu2023-11-26T07:33:55ZHow to handle UM using Wolfram Language?
https://community.wolfram.com/groups/-/m/t/3072905
Hi,
I'm trying to understand how WM handles UM.
That is I expect that a calculation should give some UM, but I don't succeed.
Let's take as an example
(314.159 m Sqrt[((7.34102*10^-6 - 0.0000178801 I) s^2)/m^2])/s
The results should be a scalar that I can calculate using coth[] but I'm unable to "convince" WM to give me that,
Thanks for any helpTeodoro Marinucci2023-11-27T20:44:38ZRunning multiple kernels, another attempt
https://community.wolfram.com/groups/-/m/t/3072409
Previous question was removed by moderation team. I read the docs, but still don't know how to make this work. Here is a pic of the message I get after I follow the steps in this suggested link: https://reference.wolfram.com/language/workflow/UseASeparateKernelForEvaluation.html ![the message I get in new notebook][1]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshot2023-11-27080133.png&userId=166789Iuval Clejan2023-11-27T14:05:58ZExploring Real-World Problems: Simplifying complexity of nonlinear PDE models
https://community.wolfram.com/groups/-/m/t/3072567
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=3310hero.jpg&userId=20103
[2]: https://www.wolframcloud.com/obj/0d314114-0ddb-44d2-bcad-974bcc113b52Vedat Senol2023-11-27T19:20:06Z[BOOK] Demystifying Epidemics: A Beginner's Guide to Infectious Disease Modeling
https://community.wolfram.com/groups/-/m/t/3070525
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=9515Cover.png&userId=20103
[2]: https://www.wolframcloud.com/obj/97559e36-c684-41f1-a543-00317cdb6709Athanasios Paraskevopoulos2023-11-23T22:22:06ZFibonacci Day 2023: Computing and visualizing the first 100 digits
https://community.wolfram.com/groups/-/m/t/3070628
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/bf11b939-1545-4981-80b1-f215587558e0Daniel Carvalho2023-11-24T00:26:28ZMore efficient way to extract info about Wolfram symbols and their docs?
https://community.wolfram.com/groups/-/m/t/3071114
Hello! I am working on the Clojure -> Wolfram bridge lib, https://github.com/scicloj/wolframite . One of the things we want to do is to "import" information about available Wolfram symbols (functions, entities, etc.) and their docs. What we do now is asking Wolfram to evaluate
Map[Function[{e},{e["Name"],e["PlaintextUsage"]}],WolframLanguageData[]]
but it is really slow, taking something between 30 and 60s on my Mac. I wonder if there is some faster way to get this information?
Thank you! Cheers, JakubJakub Holý2023-11-24T12:33:24ZAI vision via Raku
https://community.wolfram.com/groups/-/m/t/3071989
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/4bde8026-2a53-4648-9c91-e8db8db10ed1Anton Antonov2023-11-26T21:59:43ZHow to efficiently compute the integral of a piecewise demand function?
https://community.wolfram.com/groups/-/m/t/3071035
Dear all,
This is a question about efficient coding. In the code below, I write demands as integrals of piecewise functions (three pieces). I then need to find the price for which the derivative is null. In spite of my numerous trials, I didn't manage to make the code efficient enough for Mathematica to find the answer.
Do you see a way to make it more efficient, or do you think the problem in itself is too intricate to be solved?
Code:
Clear["Global`*"]
a = 1/2;
t = 4/10;
da = Integrate[Piecewise[{{1, pa + t*x < 1/4},
{((3/4) - (pa + t*x))/((3/4) - (1/4)) + ((pa + t*x - 1/4) (t*
x))/(2*((3/4) - (1/4))) + ((pa + t*x - 1/4) (1 - pb -
t*x))/((3/4) - (1/4)), 1/4 < pa + t*x && pb + t*x < 1},
{((3/4) - (pa + t*x))/((3/4) - (1/4)) + (1 - pb)^2/(
2*((3/4) - (1/4))), 1/4 < pa + t*x && pb + t*x > 1}}], {x, 0,
1}];
db = Integrate[Piecewise[{{(1 - pb), pa + t*x < 1/4},
{((1 - pb) ((3/4) - (pa + t*x)))/((3/4) - (1/4)) + ((pa + t*x -
1/4) (t*x))/(
2*((3/4) - (1/4))) + ((pa + t*x - 1/4) (1 - pb - t*x))/((3/
4) - (1/4)), 1/4 < pa + t*x && pb + t*x < 1},
{((1 - pb) ((3/4) - (pa + t*x)))/((3/4) - (1/4)) + (1 - pb)^2/(
2*((3/4) - (1/4))), 1/4 < pa + t*x && pb + t*x > 1}}], {x, 0,
1}];
pi = da*pa + pb*db;
dbdev = Integrate[Piecewise[{{1 - pbdev, pa + t*x < 1/4},
{((1 - pbdev) ((3/4) - (pa + pb + t*x - pbdev)))/((3/4) - (1/
4)) + ((pb + t*x - pbdev) (pa + pb + t*x - pbdev - 1/4))/(
2*((3/4) - (1/4))) + ((pa + pb + t*x - pbdev - 1/4) (1 - pb -
t*x))/((3/4) - (1/4)), 1/4 < pa + t*x && pb + t*x < 1},
{((3/4 - (pa + pb + t*x - pbdev)) (1 - pbdev))/((3/4) - (1/
4)) + (1 - pbdev)^2/(2*((3/4) - (1/4))),
1/4 < pa + t*x && pb + t*x > 1}}], {x, 0, 1}];
pidev = da*pa + pbdev*dbdev;
Solve[D[pidev, pbdev] == 0 && D[D[pidev, pbdev], pbdev] < 0, pbdev, Reals]
Thank you so much for your help !Sébastien Cerles2023-11-24T14:53:22Z[WSRP23] On the mergers of numerical multiway systems
https://community.wolfram.com/groups/-/m/t/2965206
![Multiway graph][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Untitled.png&userId=2964459
[2]: https://www.wolframcloud.com/obj/b6ad5a15-e8d2-4359-ae89-b72d3e3d06d4Eric Archerman2023-07-13T23:11:59ZSeeking techniques for detailed step-by-step solution of integral equation transformation
https://community.wolfram.com/groups/-/m/t/3071960
Hello Mathematica Community,
I am working on a mathematical problem where I need to transform an integral equation into a simpler form using Euler's Formula and the Error Function. My current Mathematica code successfully simplifies the equation, but I'm looking for ways to extract more detailed intermediate steps within Mathematica.
The original equation is:
![enter image description here][1]
And it is transformed into:
![enter image description here][2]
I have used Euler's formula for the sine function and Mathematica provides the simplified result directly as:
![enter image description here][3]
However, I'm interested in understanding the intermediate steps that Mathematica goes through to arrive at this result. Specifically, I am looking for:
1. Mathematica techniques or functions that can help me break down the transformation process into more granular steps.
2. Insights into how Mathematica handles integral transformations, especially involving complex functions and the error function.
3. Any special Mathematica tricks or commands that could make these intermediate steps more explicit.
I would greatly appreciate any advice or guidance on how to use Mathematica more effectively for this purpose. Understanding the intermediate steps is crucial for my learning process.
Thank you for your help!
&[Wolfram Notebook][4]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshotfrom2023-11-2616-36-29.png&userId=3071927
[2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshotfrom2023-11-2616-39-32.png&userId=3071927
[3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshotfrom2023-11-2616-39-45.png&userId=3071927
[4]: https://www.wolframcloud.com/obj/dda2fad8-6402-412f-bf94-a2eabbc753f8Rob Leny2023-11-26T14:48:20ZPrime numbers on Fibonacci spirals
https://community.wolfram.com/groups/-/m/t/3070895
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/c0cd6f12-2753-423c-a2b6-bb6839a488d4Sandor Kabai2023-11-25T20:22:18ZGet the citation management package in Mathematica for MacOS?
https://community.wolfram.com/groups/-/m/t/1194058
Neither citation management package nor functionality seem to be available in Mathematica for MacOS.
[Mathematica Documentation: Citation Management][1] says that "Currently this functionality is only available on Windows." I read from Mathematica Meta Stack Exchange [post][2] that Presentation package by David Park contains a sub-package for citation management. Unfortunately this Presentation package is no longer available or sold.
I basically need two functionalities: 1. Create a bibliography from BibTeX or any similar format; 2.Create references to a bibliography.
I have looked through many forums like Mathematica Stack Exchange however there seem to be no simple solution.
For Mac users, how do you circumvent citation management issue?
[1]: http://reference.wolfram.com/language/tutorial/CitationManagement.html
[2]: https://mathematica.meta.stackexchange.com/questions/1770/where-is-david-parks-mathematica-siteTadashi Kikuno2017-09-29T04:54:21ZBuilding multi-qubit quantum channels
https://community.wolfram.com/groups/-/m/t/3064068
Does anyone have experience building multiqubit quantum channels? I can make a simple one with a single Krauss operator, but am having trouble defining a multiqubit channel with multiple Krauss operators. In this example, I'm applying a stochastic XX operator on qubits 1&2 and the identity on qubits 3&4.
This works:
channel =
QuantumChannel[
Sqrt[0.5] QuantumTensorProduct[QuantumOperator["X", {1, 2}],
QuantumOperator["I", {3, 4}]]]
Now if I want to apply XX or YY with equal probability on qubits 1&2, I fail.
This doesn't work:
channel =
QuantumChannel[{Sqrt[0.5]
QuantumTensorProduct[QuantumOperator["X", {1, 2}],
QuantumOperator["I", {3, 4}]],
Sqrt[0.5]
QuantumTensorProduct[QuantumOperator["Y", {1, 2}],
QuantumOperator["I", {3, 4}]]}]
I get this error message:
Thread::tdlen: Objects of unequal length in {4}->{1,2,3,4} cannot be combined.
Thanks in advance for any help.David Hayes2023-11-12T23:30:14ZPrime numbers on radian spirals
https://community.wolfram.com/groups/-/m/t/3070945
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/b4a52602-a572-4072-a63f-5e31811e8092Sandor Kabai2023-11-24T15:22:45ZIntegrating C++ and the Wolfram Language with LibraryLink Utilities
https://community.wolfram.com/groups/-/m/t/2133603
If you are a Wolfram Language user with C/C++ background, at some point you might find yourself thinking:
> *I have this great C++ library, I wish I could use it in my Wolfram Language code.*
or
> *I love programming in the WL, but some things are easier for me to implement in C++. If only there was a way to combine these two languages.*
A quick google search reveals two main options: [WSTP](https://reference.wolfram.com/language/guide/WSTPCLanguageFunctions.html) (formerly known as MathLink
) and [LibraryLink](https://reference.wolfram.com/language/guide/LibraryLink.html). They provide interfaces for C programs to exchange data with the
Wolfram Language and to perform operations on either side. Both frameworks are documented with lots of examples.
Let us take a look at a sample WSTP function for sending list of integers from the Wolfram Language to a C program:
[![WSTPFunction][1]](https://reference.wolfram.com/language/ref/c/WSGetInteger32List.html)
If your code already uses error codes, manual memory management, out-parameters, etc. both WSTP and LibraryLink will
perfectly fit into your codebase.
However, if you prefer to use exceptions for error handling, RAII for resource management, namespaces, templates and other modern C++ techniques, you might
find the integration slightly more challenging.
The remaining part of this text assumes that the reader has basic knowledge of LibraryLink.
# LibraryLink Utilities
One thing that can help you achieve seamless integration of C++ and WL is called *LibraryLink Utilities* (abbr. **LLU**) which is a set of modern C++ wrappers
over
LibraryLink and WSTP. LLU is developed and maintained by Wolfram Research and it has been released under the MIT license on GitHub:
https://github.com/WolframResearch/LibraryLinkUtilities
With documentation and examples also available online:
https://wolframresearch.github.io/LibraryLinkUtilities/
We do not provide prebuilt binaries of LLU, so the only way to use it is by building locally from sources. The most common approach is to clone the repository
from one of the following URLs:
- **[ssh]:** `git@github.com:WolframResearch/LibraryLinkUtilities.git`
- **[https]:** `https://github.com/WolframResearch/LibraryLinkUtilities.git`
Alternatively, a zip package can be downloaded from GitHub containing a snapshot from any branch.
# Demo
Let's see how to write a complete C++ package for Wolfram Language using LLU. We will analyze a toy paclet named "Demo" which is shipped with LLU (under
`/tests/Demo`). This paclet exposes two functions to the Wolfram Language:
- `CaesarCipherEncode[message_String, shift_Integer]` - encodes given message by shifting every character by shift positions in the English alphabet
- `CaesarCipherDecode[cipherText_String, shift_Integer]` - restores the original message encoded with Caesar's cipher given the encoded text and the shift
Such simple functionality in practice would easily be implemented directly in WL without invoking library functions implemented in C++, but when you learn
the general structure of paclets written with LLU, you can have arbitrarily complex C++ code with all the rest being as simple as in this demo.
## Paclet structure
The demo paclet resides at the `/tests/Demo` directory inside the LLU repo root directory and has the following structure.
```
Demo
├── Demo
│ ├── Kernel
│ │ └── Demo.wl
│ └── PacletInfo.wl
├── Sources
│ └── demo.cpp
├── Tests
│ └── test.wl
└── CMakeLists.txt
```
The first thing you may notice is that Demo paclet uses CMake as a build system (via the presence of CMakeLists.txt). This is the most natural choice because
LLU
itself uses CMake and it provides a number of useful CMake utilities. It is possible to build and install LLU with CMake but then use different build system
in paclets, or even just ``CCompilerDriver`CreateLibrary``.
As you can see, our Demo follows the general paclet structure described in the
[Paclet Development Guide](https://www.wolframcloud.com/obj/tgayley/Published/PacletDevelopment.nb) - we have a Kernel directory with WL sources and the
required `PacletInfo.wl` with paclet's metadata. Additionally, we store C++ sources in `Sources` directory and a minimal set of tests in `Tests`. Having
unit tests in a paclet is not required but often a good practice.
## Installation
To be able to use the Wolfram Language functions that the Demo provides, it is enough to follow these steps:
1. Configure, build and install LLU as described
[in the documentation](https://wolframresearch.github.io/LibraryLinkUtilities/basic/how_to_use.html#configure).
Let’s say you chose `/my/workspace/LLU` as the install directory.
2. Navigate to `tests/Demo` in the LLU source directory - this is where the sources of the Demo project reside.
3. Run the following commands (or equivalent for your system):
```commandline
cmake -DLLU_ROOT=/my/workspace/LLU -DWolframLanguage_ROOT=/path/to/WolframDesktop/ -B build
cd build/
cmake --build . --target install
```
(where `/path/to/WolframDesktop` is a path to the Wolfram product (i.e. Wolfram Desktop/Mathematica/Wolfram Engine) you have installed)
This will put a complete paclet directory structure under `tests/Demo/build/Demo`. You can copy this directory into
[\$UserBaseDirectory/Applications](https://reference.wolfram.com/language/ref/$UserBaseDirectory.html) and then load the paclet by calling
Needs["Demo`"]
in a notebook.
However, a preferred way to make the paclet discoverable by the Wolfram Language is to build another target called `paclet`
```commandline
cmake --build . --target paclet
```
When built, the `paclet` target will take the directory structure created by the `install` target and turn it into a proper `.paclet` file. It can optionally
validate paclet contents, run a test file or install the paclet to a directory where the Wolfram Language can automatically find it. Investigate the
`tests/Demo/CMakeLists.txt` file for details on how to create and use this target.
Finally, after building the `paclet` target or manually copying the Demo paclet into the user paclet directory, you should be able to run the following code
in a notebook:
```mathematica
In[1]:= Needs["Demo`"]
In[2]:= Demo`CaesarCipherEncode["HelloWorld", 5]
Out[2]= "Mjqqtbtwqi"
In[3]:= Demo`CaesarCipherDecode[%, 5]
Out[3]= "HelloWorld"
```
## Using Demo as paclet template
Demo paclet is intended to serve as a short showcase of LLU capabilities but also as a convenient base for developers to create custom paclets. Let's see
what changes are necessary to turn Demo into a project of your own.
1. Copy all the sources to a new directory named the same as your new paclet, e.g.
```commandline
cd /path/to/LLU
cp -r tests/Demo ~/projects/MyPaclet
```
2. Rename all occurrences of "Demo" in directory and file names to "MyPaclet" (or whatever name you chose in step 1). On a Linux system, this step may be
accomplished as follows:
```commandline
cd ~/projects/MyPaclet
mv Demo MyPaclet
mv MyPaclet/Kernel/Demo.wl MyPaclet/Kernel/MyPaclet.wl
mv Sources/demo.cpp Sources/mypaclet.cpp
```
3. Modify source files (you can use `sed` on Linux or `Find & Replace` function in your favorite IDE):
1. Change the ``Demo` `` context in *MyPaclet.wl* and *Tests/test.wl* to ``MyPaclet` ``
2. Rename the main target and variable names in *CMakeLists.txt*
3. In *MyPaclet.wl* change the library name in line `` `LLU`InitializePacletLibrary["Demo"];`` to `"MyPaclet"`.
4. Unleash your creativity! Remove the two library functions defined by the Demo project and write your own code instead.
# Examples
Now that you know how to easily create WolframLanguage paclets that use LLU, let's see a number of common use cases and applications where LLU is especially
handy compared to plain LibraryLink or WSTP. All the C++ code snippets shown below require `LLU/LLU.h` and possibly other header files to be included but
this is omitted for brevity. For the WL snippets, whenever a library function is loaded and used we assume that this was preceded with a proper paclet
loading routine which includes an evaluation of ``LLU`InitializePacletLibrary["LibraryName"]``. This part should be clear after following the instructions in
the previous section.
## Working with numeric data
Exchanging numerical data between C++ and the Wolfram Language is one of the most popular tasks for LibraryLink. Initially, one could transfer scalars or
tensors of type: integer, real or complex. Although this seems to cover all the common cases, it is often desired to have a finer control over the data type.
For instance, if we know that all our data is in range 0 - 100 we can store each number in 1 byte, whereas tensors (a.k.a packed arrays) use 4 or 8 bytes
per element. For this reason
LibraryLink now supports [NumericArrays](https://reference.wolfram.com/language/ref/NumericArray.html).
Consider a simple function that takes a packed array of real numbers and returns a copy of the input array but with all the numbers clipped to
[0, 1] interval:
EXTERN_C DLLEXPORT int ClipToUnitInterval(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
// Create MArgumentManager to manage all the input and output arguments for the library function
LLU::MArgumentManager mngr(libData, Argc, Args, Res);
// The first argument should be our tensor of real numbers, otherwise the following line will throw an exception
const auto data = mngr.getTensor<double>(0);
// We will return a copy of the input array with some values modified
auto clipped = data.clone();
// We iterate over all elements of the container in a single loop even if the Tensor is multidimensional
for (auto& d : clipped) {
d = std::clamp(d, 0., 1.);
}
// Set function result
mngr.set(clipped);
return LLU::ErrorCode::NoError;
}
or, if one is not afraid to use a macro, we can hide all the boilerplate code:
```cpp
LLU_LIBRARY_FUNCTION(ClipToUnitInterval) {
const auto data = mngr.getTensor<double>(0);
auto clipped = data.clone();
for (auto& d : clipped) {
d = std::clamp(d, 0., 1.);
}
mngr.set(clipped);
}
```
Such library function can be loaded and used in the Wolfram Language as follows:
```mathematica
In[1]:= LLU`PacletFunctionSet[ClipToUnitInterval, "ClipToUnitInterval", {{Real, 1}}, {Real, _}];
In[2]:= ClipToUnitInterval[{{0.2, 0.3}, {-0.1, 1.4}}]
Out[2]= {{0.2, 0.3}, {0., 1.}}
```
Our `ClipToUnitInterval` works great but is limited to the input arrays of type Real. Oftentimes we want to have a function that works for any type of
tensor. The simplest solution is to provide 3 almost identical functions: for integers, reals and complex numbers. This gets less feasible for NumericArray,
which supports 12 different element types. LLU comes to the rescue here with "generic" containers. For every container `X` that is templated with element type
(such as `LLU::NumericArray` or `LLU::Image`) an `LLU::GenericX` is provided, which is a type-unaware counterpart of `X`.
For instance, we can write
```cpp
const auto numericArr = mngr.getNumericArray<double>(0);
```
if the only thing that we want to accept as the first argument passed to the function is a NumericArray of type "Real64". If we want to accept any NumericArray,
we can instead write:
```cpp
const auto numericArr = mngr.getGenericNumericArray(0);
```
our code is thus more flexible but we cannot access the underlying data of `numericArr` directly (because we don't know its type). We could write a `switch`
statement that would detect the data type at run-time and act accordingly; we could also use a helper function from LLU called `LLU::asTypedNumericArray` which
takes
a generic NumericArray, and an action to perform on this array, which behaves as if the data type of the array was known. For instance
LLU_LIBRARY_FUNCTION(Reverse) {
// Receive a NumericArray of any type as the first argument to the library function
auto numericArr = mngr.getGenericNumericArray(0);
// Perform an action (second argument) on the numericArr. The typedNA passed to the lambda is a strongly typed LLU::NumericArray.
LLU::asTypedNumericArray(numericArr, [&mngr](auto&& typedNA) {
// Extract data type from the argument, this can be simplified in C++20 with a template parameter list for the lambda
using T = typename std::remove_reference_t<decltype(typedNA)>::value_type;
// Create a "reversed" copy of the array preserving the type and dimensions.
auto reversedArr = LLU::NumericArray<T>(std::crbegin(typedNA), std::crend(typedNA), LLU::MArrayDimensions {typedNA.getDimensions(), typedNA.getRank()});
// Set the reversed array as the result
mngr.set(std::move(reversedArr));
});
}
## Error handling
If every function that we write always succeeded with no errors, programming would be much easier. Alas, errors do happen for many various reasons;
proper handling and reporting to the caller is one of the most challenging tasks, especially for API functions. LibraryLink, like many C libraries,
uses integer error codes with [7 predefined values](https://reference.wolfram.com/language/LibraryLink/tutorial/LibraryStructure.html#394079419).
In C++, however, an often preferred method of error handling is via exceptions. LLU conforms with this practice and uses exceptions both to report problems that
occurred in its own code and to allow programmers to throw exceptions in their library functions. Such exceptions may even be propagated up to
the Wolfram Language and transformed into `Failure` objects. This is described in more details in the
[LLU documentation](https://wolframresearch.github.io/LibraryLinkUtilities/modules/error_handling.html).
Imagine we expect two different exceptional situations that may occur in our code. For each of them, we register a separate kind of exception, with different
name and short textual description which may contain template slots populated at run time. The registration should be done inside `WolframLibrary_initialize`
and may look like this:
```cpp
EXTERN_C DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData) {
LLU::LibraryData::setLibraryData(libData);
LLU::ErrorManager::registerPacletErrors({
{"NoSourceError", "Requested data source does not exist."},
{"EmptySourceError", "Requested data source has `elemCount` elements, but required at least `elemReq`."}
});
return LLU::ErrorCode::NoError;
}
```
With this simple setup we can now throw exceptions from the function that reads data:
```cpp
void readData(DataSource* source) {
if (!source) {
LLU::ErrorManager::throwException("NoSourceError");
}
if (source->elemCount() < 3) {
LLU::ErrorManager::throwException("EmptySourceError", source->elemCount(), 3);
}
//...
}
```
Each call to `LLU::ErrorManager::throwException` causes an exception of class `LLU::LibraryLinkError` with predefined name and error code to be thrown. All
parameters of `throwException` after the first one are used to populate consecutive template slots in the error message. The only thing left to do now
is to catch the exception. Usually, you catch only in the interface functions (the ones with `EXTERN_C DLLEXPORT`), extract the error code from exception
and return it:
```cpp
EXTERN_C DLLEXPORT int ReadData(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
auto err = LLU::ErrorCode::NoError; // no error initially
try {
//...
} catch (const LLU::LibraryLinkError& e) {
err = e.which(); // extract error code from LibraryLinkError
} catch (...) {
err = LLU::ErrorCode::FunctionError; // to be safe, one may want to handle non-LLU exceptions as well and return generic error codes
}
return err;
}
```
LLU can later recognize this error code on the WL side and create a proper `Failure` object corresponding to the exception that was thrown on the C++ side.
#### Custom exception class
It is possible to make `ErrorManager` throw exceptions of different class than `LibraryLinkError` as long as this class defines a publicly accessible
constructor which takes `const LibraryLinkError&` as the first argument. This feature might be useful when we want a custom action to happen right before
throwing, for instance logging some information to a text file. Consider a simple example:
// Define custom error class. LoggingError inherits from LibraryLinkError and additionally logs file name, function name and line number to a file.
struct LoggingError : LLU::LibraryLinkError {
// Where to log exception details
static constexpr const char* logFileName = "LLUErrorLog.txt";
template<typename... T>
LoggingError(const LLU::LibraryLinkError& e, WolframLibraryData ld, int line, const std::string& file, const std::string& func, T&&... params)
: LLU::LibraryLinkError(e) {
// Pass LibraryLinkError's parameters to top-level
setMessageParameters(ld, std::forward<T>(params)...);
sendParameters(ld);
// Extend debug info to contain file and function names and append to the log file
setDebugInfo("Exception " + name() + " in " + file + ":" + std::to_string(line) + " in " + func + ": " + debug());
std::ofstream log {logFileName, std::ios_base::app};
log << debug() << "\n";
}
};
// Helper macro to conveniently throw LoggingErrors with current line, filename and function name
#define THROW_LOGGING_ERROR(name, ...) LLU::ErrorManager::throwCustomException<LoggingError>(name, libData, __LINE__, __FILE__, __func__, __VA_ARGS__)
This class can be used as follows:
```cpp
EXTERN_C DLLEXPORT int ReadDataWithLoggingError(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
auto err = LLU::ErrorCode::NoError;
try {
LLU::MArgumentManager mngr(Argc, Args, Res);
auto fileName = mngr.getString(0);
if (fileName.find(':') != std::string::npos) {
THROW_LOGGING_ERROR("DataFileError", fileName, 0, R"(file name contains a possibly problematic character ":")");
}
auto fileNameLen = static_cast<wsint64>(fileName.length());
if (fileNameLen > 16) {
THROW_LOGGING_ERROR("DataFileError", fileName, fileNameLen, "file name is too long");
}
THROW_LOGGING_ERROR("DataFileError", fileName, fileNameLen, "data type is not supported");
}
// The error handling code can remain as before, because our custom exception class inherits from LLU::LibraryLinkError
catch (const LLU::LibraryLinkError& e) {
err = e.which();
} catch (...) {
err = LLU::ErrorCode::FunctionError;
}
return err;
}
```
#### Top-level layer
Apart from the C++ code, paclets often have nontrivial amount of Wolfram Language code where errors might also occur. In order to achieve uniform
error reporting across C++ and WL, one needs to register errors specific to the WL layer of the paclet:
```mathematica
`LLU`RegisterPacletErrors[<|
"InvalidInput" -> "Data provided to the function was invalid.",
"UnexpectedError" -> "Unexpected error occurred with error code: `errCode`."
|>];
```
`RegisterPacletErrors` takes an `Association` of user-defined errors of the form
error_name -> error_message
Such registered errors can later be issued from the Wolfram Language part of the project like this:
```mathematica
status = DoSomething[input];
If[Not @ StatusOK[status],
`LLU`ThrowPacletFailure["UnexpectedError", "MessageParameters" -> <|"errCode" -> status|>]
]
```
Alternatively, one can use ``LLU`CreatePacletFailure`` which returns the `Failure` object as the result instead of throwing it.
## Monitoring progress
If functions in your library are meant to work on potentially large amounts of data, and are likely to take considerable amount of time to complete, it is a
good idea in terms of user experience to provide some kind of indication to the user of how far the function is from completing and to allow users to abort
the functions execution at any point. LibraryLink offers the latter in the form of
[AbortQ](https://reference.wolfram.com/language/LibraryLink/ref/callback/AbortQ.html)
function which lets you check whether an abort was requested and then act accordingly (e.g. stop the computation, clean up, return).
LLU goes one step further and provides a mechanism for functions to report progress to the caller on the Wolfram Language side. The basic form of reporting
progress is simply a real number that is shared between the WL and C++ code. During a library function execution, the number is updated from the C++ code and
those
changes can be reflected on the WL side in many different ways to give user a visual indication of current progress, e.g. using
[ProgressIndicator](https://reference.wolfram.com/language/ref/ProgressIndicator.html).
Imagine we want to write a simple function `SleepTight` that simply sleeps in a loop moving the progress bar in a steady pace. From the caller's perspective
, this
function takes a single argument - total time (in seconds) for the function to complete. We could load this function in the WL like this:
```mathematica
LLU`PacletFunctionSet[SleepTight, "SleepTight", {Real}, "Void", "ProgressMonitor" -> MyPaclet`PM`SleepTight];
```
Notice that for progress reporting to work on the Wolfram Language side as expected, the library function must be loaded with extra option
`“ProgressMonitor” -> x`, where `x` is a Symbol. Every time your library function reports progress, the new progress value will be assigned to `x`.
By default, `“ProgressMonitor” -> None` is used. It is a good idea to make sure the name for the monitoring symbol will be unique. One suggestion is to use
``PacletName`PM`` as the context, and the name of the symbol to be the same as the function name.
Now, one can run the library function with simple progress bar:
```mathematica
Monitor[
(* the function will run for 5 seconds *)
SleepTight[5],
(* check the value of progress every 0.2 second and display as progress bar *)
ProgressIndicator[Dynamic @ First @ Refresh[MyPaclet`PM`SleepTight, UpdateInterval -> 0.2]]
]
```
The remaining task of implementing the function on the C++ side is fairly straightforward:
```cpp
EXTERN_C DLLEXPORT int SleepTight(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
// Create MArgumentManager to manage all the input and output arguments for the library function
LLU::MArgumentManager mngr(libData, Argc, Args, Res);
// Get the first argument which determines how many seconds should this function take to evaluate
auto totalTime = mngr.getReal(0);
// Calculate number of steps for the progress bar, we want 10 steps per second
auto numOfSteps = static_cast<int>(std::ceil(totalTime * 10));
// Get ProgressMonitor instance, initialize with the number of seconds per step
auto pm = mngr.getProgressMonitor(1.0 / numOfSteps);
// Sleep in a loop, increase progress in each iteration. Increasing progress also automatically checks for Abort.
for (int i = 0; i < numOfSteps; ++i) {
std::this_thread::sleep_for(100ms);
++pm;
}
return LLU::ErrorCode::NoError;
}
```
## Using WSTP link as a stream
WSTP links can be used to sequentially transfer data between C++ code and the Wolfram Language. It makes them perfect candidates to be represented as streams
in the C++ sense (with "stream operators" `>>` and `<<`). LLU provides such a stream class for WSTP, called `WSStream`. The class takes two non-type template
parameters which represent default encodings to be assumed for C-string and `std::string`s that sent or received via the stream.
A sample function that receives a number of UTF8 strings from the link and puts back a total byte-length of all the strings, could look like this:
```cpp
LIBRARY_WSTP_FUNCTION(StringsByteLength) {
WSStream<WS::Encoding::UTF8> stream {wsl};
std::vector<std::string> strs;
stream >> strs;
auto total = std::accumulate(std::cbegin(strs), std::cend(strs), 0UL, [](auto sum, const std::string& s) { return sum + s.length(); });
stream << total << WS::EndPacket;
}
```
Which could then be loaded and used in the Wolfram Language as follows:
```mathematica
In[1]:= LLU`WSTPFunctionSet[StringsByteLength, "StringsByteLength"];
In[2]:= StringsByteLength["a", "bb", "ccc"]
Out[2]= 6
```
A slightly more complicated example could involve a function that accepts a list of integers and returns an Association with a list of divisors for each of the
input integers, or `$Failed` if the number has more than 15 divisors. This example demonstrates how to send expressions whose length is not known a-priori:
```cpp
LLU_WSTP_FUNCTION(FactorsOrFailed) {
WSStream<WS::Encoding::Byte> stream {wsl, 1}; // we expect 1 input argument - a list of integers
std::vector<int> numbers;
stream >> numbers;
stream << WS::Association(numbers.size()); // the output Association will have the same length as the input list even if the list contains duplicates
for (auto n : numbers) { // duplicates will be automatically taken care of on the WL side
stream << WS::Rule << n;
stream << WS::BeginExpr("List"); // we do not know the number of factors so we have to use WS::BeginExpr
int divisors = 0;
for (int j = 1; j <= n; ++j) {
if (n % j == 0) {
if (divisors < 15) {
stream << j;
divisors++;
} else {
stream << WS::DropExpr(); // Drop the List that and send $Failed instead
stream << WS::Symbol("$Failed");
break;
}
}
}
stream << WS::EndExpr();
}
stream << WS::EndPacket;
}
```
We can now load our function and use it like this:
```mathematica
In[1]:= LLU`WSTPFunctionSet[Factors, "FactorsOrFailed"];
In[2]:= Factors[{1, 3, 15, 10000000, 15}]
Out[2]= <|1 -> {1}, 3 -> {1, 3}, 15 -> {1, 3, 5, 15}, 10000000 -> $Failed|>
```
WSTP is very flexible and lets you exchange arbitrary expressions between the Wolfram Language and C++ code but it comes with a price - bigger overhead compared
to data transfer via LibraryLink arguments.
## Passing and returning heterogeneous lists
LibraryLink has a fixed list of types that can be used as library function arguments and return values. For instance, see the "Details and Options" section in
the documentation of [LibraryFunctionLoad](https://reference.wolfram.com/language/ref/LibraryFunctionLoad.html). As you can see, it is straightforward to
pass an array of numeric data, a string or an image but already sending a list of images is hard. One can conform all the images and transfer them via
LibraryLink as a single `Image3D` but this puts a toll on efficiency. Sending a list of strings is simply impossible without tricks or workarounds.
This topic has been brought up by the community quite a few times:
- [Returning multiple results from a LibraryLink function](https://mathematica.stackexchange.com/questions/31545/returning-multiple-results-from-a-librarylink-function)
- [Is it possible to let LibraryLink return multiple results?](https://mathematica.stackexchange.com/questions/121825/is-it-possible-to-let-librarylink-return-multiple-results)
- [LibraryFunctionLoad for multivariate C++ function containing several arguments](https://mathematica.stackexchange.com/questions/128723/libraryfunctionload-for-multivariate-c-function-containing-several-arguments)
- [LibraryFunctionLoad with variable number of arguments](https://community.wolfram.com/groups/-/m/t/366849)
- [Working with a variable number of arguments in LibraryLink](https://community.wolfram.com/groups/-/m/t/190560)
It turns out that the solution to this problem already exists in LibraryLink and is called `DataStore`. The only problem is that for now it remains
undocumented. Below is a quick glimpse of what `DataStore` is, which should also help understand the C++ wrappers of `DataStore` that LLU has to offer.
![DataStore structure][2]
As can be seen in the picture, `DataStore` is a simple unidirectional linked list but with limited functionality exposed, compared for instance to
`std::forward_list`. The API provided in the Wolfram Library allows for
- creating an empty `DataStore`
- copying or deleting existing `DataStore`
- appending new nodes at the end
- iterating over nodes
- obtaining the length of the store (in constant time)
Each node of the `DataStore` carries a value of type `MArgument`, which is a union type of all types that LibraryLink can handle as function arguments or
return values. The `MArgument` union include `DataStore` which means that the store can be nested. Additionally, every node contains an optional "name" which
can be any string. Names do not have to be unique.
A simple library function that takes a list of strings (in the form of a `DataStore`) and returns a new list with each of the input strings reversed,
implemented in pure LibraryLink may look like this:
EXTERN_C DLLEXPORT int StringsReversedLibraryLink(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
// get DataStore which is the first input argument to the library function
DataStore ds_in = MArgument_getDataStore(Args[0]);
mint length = libData->ioLibraryFunctions->DataStore_getLength(ds_in);
// create new DataStore to hold the result
DataStore ds_out = libData->ioLibraryFunctions->createDataStore();
if (ds_out == nullptr) {
// error handling
}
// start traversing the DataStore from the first node
DataStoreNode dsn = libData->ioLibraryFunctions->DataStore_getFirstNode(ds_in);
while (dsn != nullptr) {
MArgument node_data;
if (libData->ioLibraryFunctions->DataStoreNode_getData(dsn, &node_data) != 0) {
// error handling and cleanup
}
if (libData->ioLibraryFunctions->DataStoreNode_getDataType(dsn) != MType_UTF8String) {
// error handling and cleanup
}
// reverse the order of characters in the string and push to the output DataStore
std::string_view s {MArgument_getUTF8String(node_data)};
std::string outStr(s.rbegin(), s.rend()); // create reversed copy
libData->ioLibraryFunctions->DataStore_addString(ds_out, outStr.data());
// move to the next node
dsn = libData->ioLibraryFunctions->DataStoreNode_getNextNode(dsn);
}
// set the newly created DataStore as the result of this library function call
MArgument_setDataStore(Res, ds_out);
return LIBRARY_NO_ERROR;
}
It is clear how to operate on `DataStore`s in C code now, but the Wolfram Language side of LibraryLink also uses a representation of this structure and it is
defined as follows:
```mathematica
Developer`DataStore[node_expr$1, node_expr$2, ..., node_expr$n]
```
where each `node_expr` is of the form `string -> expr` or just `expr` with an extra requirement that `expr` must be an expression supported in LibraryLink.
For example, ``Developer`DataStore["abc", "de", "f"]`` passed to the function implemented above would result in ``Developer`DataStore["cba", "ed", "f"]`` being
returned from the library.
LLU provides two direct wrappers over `DataStore`: `LLU::GenericDataList` and `LLU::DataList<T>`. The former equips `DataStore` with a proper container
interface including methods like `push_back()`, `front()`, `back()` or `length()`. It also offers easier iteration over the list with `begin()` and `end()`.
The second wrapper is templated with a node type and should be used whenever we expect a homogeneous `DataStore` (with all nodes of the same type). The types
that can be passed as template parameter are not the raw LibraryLink types included in the MArgument union but rather their LLU counterparts. For convenience,
they are enclosed in `LLU::NodeType` namespace with following members:
```cpp
/// Boolean type, corresponds to True or False in the Wolfram Language
using Boolean = bool;
/// Machine integer type
using Integer = mint;
/// Double precision floating point type
using Real = double;
/// Complex number type, bitwise-compatible with mcomplex defined in WolframLibrary.h
using Complex = std::complex<double>;
/// Tensor stands for a GenericTensor - type agnostic wrapper over MTensor
using Tensor = MContainer<MArgumentType::Tensor>;
/// SparseArray type corresponds to the "raw" MSparseArray as LLU does not have its own wrapper for this structure yet
using SparseArray = MSparseArray;
/// NumericArray stands for a GenericNumericArray - type agnostic wrapper over MNumericArray
using NumericArray = MContainer<MArgumentType::NumericArray>;
/// Image stands for a GenericImage - type agnostic wrapper over MImage
using Image = MContainer<MArgumentType::Image>;
/// String values from LibraryLink (char*) are wrapped in std::string_view
using UTF8String = std::string_view;
/// DataStore stands for a GenericDataList - type agnostic wrapper over DataStore
using DataStore = MContainer<MArgumentType::DataStore>;
```
Additionally, there is also `LLU::NodeType::Any` which can be used to make `LLU::DataList` work with a heterogeneous `DataStore`. `LLU::DataList` compared to
`LLU::GenericDataList` provides more iteration options (iteration over node values or node names only) and a function to immediately create a `std::vector` out
of the stored values.
Let us implement the same function as above using `LLU::GenericDataList`:
EXTERN_C DLLEXPORT int StringsReversedGeneric(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
auto err = LLU::ErrorCode::NoError;
try {
using NodeT = LLU::NodeType::UTF8String;
LLU::MArgumentManager mngr(libData, Argc, Args, Res);
auto dsIn = mngr.getGenericDataList(0);
GenericDataList dsOut;
// iterate over the GeneridDataList with a range-based for loop
for (auto node : dsIn) {
// we are dealing with generic DataList, so we need to be explicit about the actual node type
std::string_view s = node.as<NodeT>();
std::string reversed {s.rbegin(), s.rend()}; // create reversed copy
dsOut.push_back(std::string_view(reversed)); // passing a view is fine because DataStore will copy the string immediately
}
mngr.set(dsOut);
} catch (const LLU::LibraryLinkError& e) {
err = e.which();
} catch (...) {
err = LLU::ErrorCode::FunctionError;
}
return err;
In the function above we know that all nodes should contain strings, so we could use `LLU::DataList<LLU::NodeType::UTF8String>` instead of the generic data
list. The code would stay almost the same, except we could simply call `node.value()` to get the corresponding `std::string_view`.
For a different example, consider a function `SeparateKeysAndValue` that takes a data store of named complex numbers and separates it into two data stores:
one holding names of the original data store and another one with values. For instance,
```mma
Developer`DataStore["a" -> 1 + 2.5 * I, "b" -> -3. - 6.I, 2I, "d" -> -4]
```
would be transformed into
```
Developer`DataStore[
"Keys" -> Developer`DataStore["a", "b", "", "d"],
"Values" -> Developer`DataStore[1. + 2.5 * I, -3. - 6.I, 2.I, -4.]
]
```
The implementation of such library function may look as follows:
EXTERN_C DLLEXPORT int SeparateKeysAndValues(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
LLU::MArgumentManager mngr(libData, Argc, Args, Res);
auto dsIn = mngr.getDataList<LLU::NodeType::Complex>(0);
DataList<LLU::NodeType::UTF8String> keys;
DataList<LLU::NodeType::Complex> values;
// we use structured bindings to immediately split each node into name and value
for (auto [name, value] : dsIn) {
keys.push_back(name);
values.push_back(value);
}
DataList<GenericDataList> dsOut; // the output type is a DataList of DataLists
dsOut.push_back("Keys", std::move(keys));
dsOut.push_back("Values", std::move(values));
mngr.set(dsOut);
}
Instead of a single loop with structured bindings, we could also make two passes over the input data list:
DataList<LLU::NodeType::UTF8String> keys;
for (auto name : LLU::NameAdaptor {dsIn}) {
keys.push_back(name);
}
DataList<LLU::NodeType::Complex> values;
for (auto value : LLU::ValueAdaptor {dsIn}) {
values.push_back(value);
}
Notice that we use iterator adaptors `LLU::NameAdaptor` and `LLU::ValueAdaptor` to iterate over a specific property of the nodes (either name or value,
respectively).
# Summary
LibraryLink Utilities is a new addition to the Wolfram ecosystem providing a convenient way for developers to integrate the Wolfram Language with external
libraries written in modern C++.
It builds upon existing LibraryLink framework making it more fitting into C++ codebases and providing a number of utilities for easier paclet development
including strongly typed container classes, enhanced error reporting and progress monitoring capabilities, a way to bring object-oriented paradigm into the
Wolfram Language with Managed Library Expressions, lazy loading of library functions and much more.
All of this comes with a good integration with CMake and a rich documentation in 2 flavors: doxygen comments for all public entities in the library
([see here](https://wolframresearch.github.io/LibraryLinkUtilities/doxygen/)) and tutorial-like Sphinx-based documentation
([here](https://wolframresearch.github.io/LibraryLinkUtilities/)) that covers
the most important aspects of the projects with examples and detailed instructions.
Last but no least, LLU is free and open-source, so anyone is welcome to contribute to this project on GitHub.
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=WSTPFunction.png&userId=824316
[2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=DataStore.png&userId=824316Rafal Chojna2020-12-09T10:47:37ZHow do I extract the condition from a conditional expression?
https://community.wolfram.com/groups/-/m/t/3070301
I have a calculation which at a certain point produces a conditional expression like
CE= (expression involving x) if 1<= x <= 3.
I want to extract the fact that the variable which is conditional is x, and that the bounds are 1 and 3.
Then I can continue the calculation by getting the values of the expression when x is 1 and when x is 3.
I guess I could do this by turning CE into a character string, searching for "if" and so on. But is there an easier (non-acrobatic) way?
ThanksDavid Golber2023-11-23T15:32:51ZComputing Eigenvalue Density of Wishart matrices?
https://community.wolfram.com/groups/-/m/t/3070419
There's a [paper][1] which claims to implement Eigenvalue density computation for finite size Wishart matrices in Mathematica, but the authors don't include link/contact information in the paper, has anyone see this implementation by chance?
For large matrices it's Marchenko Pastur, but for finite size matrices there are spikes --
&[Wolfram Notebook][2]
[1]: https://arxiv.org/pdf/1006.0812.pdf
[2]: https://www.wolframcloud.com/obj/bfcf3252-2f51-4eed-876e-ec235d8a36efYaroslav Bulatov2023-11-23T17:17:27ZHow do I make a tensor product operator for like the stabilizers?
https://community.wolfram.com/groups/-/m/t/2993896
I want to make the projection operator of 5-qubit code with the stabilizers. But QuantumTensorProduct[X,Z,Z,X,I] shows error(I have defined the X,Z,I operators beforehand). How can I make an operator that will act on 5-qubits?
Or do I need to use QuantumCircuitOperator and how?Fahim Bin Selim2023-08-20T01:43:46ZEnthalpy of the modified white hole
https://community.wolfram.com/groups/-/m/t/3070538
We analyze the following equation, concerning the enthalpy of the modified white hole. All calculations were carried out using the WolframAlpha.
![formulas][1]
![formulas][2]
![formulas][3]
![formulas][4]
![formulas][5]
![formulas][6]
![formulas][7]
![formulas][8]
![formulas][9]
![formulas][10]
![formulas][11]
![formulas][12]
![formulas][13]
![formulas][14]
![formulas][15]
![formulas][16]
![formulas][17]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=.jpg&userId=2483773
[2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_2.jpg&userId=2483773
[3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_3.jpg&userId=2483773
[4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_4.jpg&userId=2483773
[5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_5.jpg&userId=2483773
[6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_6.jpg&userId=2483773
[7]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_7.jpg&userId=2483773
[8]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_8.jpg&userId=2483773
[9]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_9.jpg&userId=2483773
[10]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_10.jpg&userId=2483773
[11]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_11.jpg&userId=2483773
[12]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_12.jpg&userId=2483773
[13]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_13.jpg&userId=2483773
[14]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_14.jpg&userId=2483773
[15]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_15.jpg&userId=2483773
[16]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_16.jpg&userId=2483773
[17]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700779780978-a2568bef-9e72-4f89-bfb7-ae5cc3ceb5b0_17.jpg&userId=2483773Michele Nardelli2023-11-23T23:11:49ZIs it possible to solve the control theory problem in wolfram alpha
https://community.wolfram.com/groups/-/m/t/3070613
Is it possible to solve the control theory problem in wolfram alpha, namely, to find a control function for a system that will provide it with asymtotic stability?Petro Bilyk2023-11-23T21:07:40ZElastic bouncing in circular bowl
https://community.wolfram.com/groups/-/m/t/3070150
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/2e8ef962-e944-4ab2-a759-bbae56a38670Sander Huisman2023-11-23T18:24:14ZGuide 1: The Wolfram Plugin for ChatGPT
https://community.wolfram.com/groups/-/m/t/3070428
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/ce4a1961-7813-4d3f-a60e-9ec2b6ae3595Michael Trott2023-11-23T17:51:06ZOn Graph Products and Matrix Products
https://community.wolfram.com/groups/-/m/t/3069212
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/44d6ce0c-5cac-473e-80a1-edc40c56592bJ. M.2023-11-22T08:14:10ZHow get GPT-4 to show a homotopy of a loop on a 2-sphere?
https://community.wolfram.com/groups/-/m/t/3069576
I first asked ChatGPT (GPT-4), through MacGPT, to write Mathematica code that shows a 2-sphere with a simple closed curve on it. That gave a totally reasonable response the first time:
(*Define the 2-sphere*)
sphere =
ParametricPlot3D[{Sin[theta]*Cos[phi], Sin[theta]*Sin[phi],
Cos[theta]}, {theta, 0, Pi}, {phi, 0, 2*Pi},
PlotStyle -> Opacity[0.5], Mesh -> None];
(*Define a simple closed curve on the sphere's surface*)
curve = ParametricPlot3D[{Sin[theta]*Cos[theta],
Sin[theta]*Sin[theta], Cos[theta]}, {theta, 0, 2*Pi},
PlotStyle -> {Thick, Red}];
(*Combine the sphere and the curve*)
Show[sphere, curve]
![simple closed curve on 2-sphere][1]
But what I really want, and next asked GPT-4 to do, is write dynamic Mathematica code that shows such a simple closed curve being shrunk continuously to a point on the sphere, with a particular "base point" remaining fixed.
At this, GPT-4 failed over around 10 attempts at producing a `Manipulate` to do this. Some of the attempts just shrunk the given 2-sphere to a point; others had the curve wandering far off thd surface of the sphere; and still others produced images whose meaning I don't comprehend.
While I would be interested in seeing Mathematica code to shows this, I am more interested in what GPT-4 prompt might actually create such code.
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=curveOnSphere.png&userId=50264Murray Eisenberg2023-11-22T20:12:55ZParabola on the complex plane
https://community.wolfram.com/groups/-/m/t/3069438
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/3e2e8157-fd16-48db-b260-fb5bc682a51fRobert Rimmer2023-11-22T18:30:02ZHow store ResourceObject on local machine?
https://community.wolfram.com/groups/-/m/t/3062279
I evaluate `ResourcObject["`*someName*`"]`, where the object i question is somewhere in the Cloud, I have access to it in the current Mathematica session. But how can I get it to be stored on my local machine, so that I subsequently have access to it there, without resorting to the Cloud again?
And where exactly does that object get stored on the local machine? (Presumably somewhere in `$UserBaseDirectory`?
Consider the preceding questions in the situation, for example, with:
ResourceObject["Sample Data for Query Book"]Murray Eisenberg2023-11-08T23:17:26ZHow to generate a list that represents a multinormal distribution?
https://community.wolfram.com/groups/-/m/t/3069070
I want to generate a list that represents a multinormal population, with 20 samples of size 10. With the means:{7,4,5,6} and covariances {{1,0.5,0.5,0.5}, {0.5,1,0.5,0.5}, {0.5,0.5,1,0.5}, {0.5,0.5,0.5,1}}. I expect to get a 10 x 20 array. The current status of my none-working code:
Needs["MultivariateStatistics`"]
dist=RandomReal[MultinormalDistribution[{7,4,5,6},{{1,0.5,0.5,0.5},{0.5,1,0.5,0.5},{0.5,0.5,1,0.5},{0.5,0.5,0.5,1}}]
exampleList = RandomChoice[dist,{20,10}]Jürgen Kanz2023-11-22T10:20:37ZHow to replace subset of Dataset with another Dataset?
https://community.wolfram.com/groups/-/m/t/3069028
Hi there,
I am hoping to assign a Dataset to a subset of another Dataset, as described below. This is pretty straightforward in Python, so sorry if this is somewhat basic.
Any help is greatly appreciated. Thanks in advance!
Toby
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/7bda80cc-ad18-43f3-93f8-fe42ec581bf6Toby Im2023-11-22T02:41:54ZNone of the countries bordering Poland before 1990 exist today: the fall of the Berlin Wall and USSR
https://community.wolfram.com/groups/-/m/t/3067969
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=_2_ezgif.com-optimizecopy.gif&userId=11733
[2]: https://www.wolframcloud.com/obj/34c7205a-40cd-4587-b6a1-f1fadc1ed8ceVitaliy Kaurov2023-11-21T01:14:10ZArtificial intelligence and the skill premium
https://community.wolfram.com/groups/-/m/t/3068735
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/b347ea32-be91-4666-8721-584a93daa737Jamel Saadaoui2023-11-21T19:37:19Z3D Plotting of Time Dilation near Black holes
https://community.wolfram.com/groups/-/m/t/3067368
Hi there,
I have some basic code for plotting the gravitional time dilation as one moves closer to the event horizon of a black hole of a given mass. See code:
ClearAll["Global`*"]'; tp = 1; G =
6.67408*10^(-11); M = (6.5*10^9)*(1.989*10^30); c =
2.99*10^8; rs = (2*G*
M)/(c^2); Print[]; Print[" Black hole mass:", M, "kg", " \
Schhwarzchild radius:", rs, "m"]; Print[]; Plot[
tp*Sqrt[1 - (rs/r)], {r, 0, 2*10^14}, AxesLabel -> {" ", "Time (s)"},
AxesLabel -> {Style[" (m)", Bold, 26],
Style["Time (s)", Bold, 16]}, LabelStyle -> Directive[Black, 16],
AxesOrigin -> {0, 0}, GridLines -> {{rs}, {}},
GridLinesStyle -> {Directive[{Dashed, Thick}, Red],
Directive[Thick, Red]}, ColorFunction -> "NeonColors"]
I want to create a 3D plot for different masses of black holes ranging from the one given in the code to one a thousand times more massive, with a 3D sheet. I am not great at this, and while my basic code works fine, when I go to do a 3D plot, nothing seems to work.Estelle Asmodelle2023-11-20T03:35:09ZUnsure how to format data for NN training
https://community.wolfram.com/groups/-/m/t/3067848
Hi There,
I would like to take MobileNet and train it to classify different features (deleting any previous training).
I have input the data and formated it to what I think is required. I have also modified the NN to manage the parameters.
I have attached the code rather than pasting it. To me the training data format looks ok but it tells me it's not. Any help or advice would be really appreciated. This is my first NN so I'm not sure what I'm doing wrong.
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/9628e506-128e-427a-b436-00d43f97c088S G2023-11-20T17:22:45ZPhase Transitions in the Quantum Transverse Field Ising Chain
https://community.wolfram.com/groups/-/m/t/2554882
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/f3893352-8dea-43e3-a022-3dc23c33fce7Aditya Dhumuntarao2022-06-21T20:40:21ZFinding all structural breaks in time series
https://community.wolfram.com/groups/-/m/t/1749226
## Introduction
In this document we show how to find the so called "structural breaks",
[[Wk1](https://en.wikipedia.org/wiki/Structural_break)],
in a given time series.
The algorithm is based in on a systematic application of Chow Test,
[[Wk2](https://en.wikipedia.org/wiki/Chow_test)],
combined with an algorithm for local extrema finding in noisy time series,
[[AA1](https://mathematicaforprediction.wordpress.com/2015/09/27/finding-local-extrema-in-noisy-data-using-quantile-regression/)].
The algorithm implementation is based on the packages
["MonadicQuantileRegression.m"](https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicQuantileRegression.m),
[[AAp1](https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicQuantileRegression.m)],
and ["MonadicStructuralBreaksFinder.m"](https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicStructuralBreaksFinder.m),
[[AAp2](https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicStructuralBreaksFinder.m)].
The package [[AAp1](https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicQuantileRegression.m)]
provides the software monad QRMon that allows rapid and concise specification of
[Quantile Regression](https://en.wikipedia.org/wiki/Quantile_regression) workflows.
The package
[[AAp2](https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicStructuralBreaksFinder.m)]
extends QRMon with functionalities related to structural breaks finding.
### What is a structural break?
It looks like at least one type of "structural breaks" are defined through regression models, [[Wk1](https://en.wikipedia.org/wiki/Structural_break)]. Roughly speaking a structural break point of time series is a regressor point that splits the time series in such way that the obtained two parts have very different regression parameters.
One way to test such a point is to use Chow test, [[Wk2](https://en.wikipedia.org/wiki/Chow_test)]. From [[Wk2](https://en.wikipedia.org/wiki/Chow_test)] we have the definition:
The Chow test, proposed by econometrician Gregory Chow in 1960, is a test of whether the true coefficients in two linear regressions on different data sets are equal. In econometrics, it is most commonly used in time series analysis to test for the presence of a structural break at a period which can be assumed to be known a priori (for instance, a major historical event such as a war).
### Example
Here is an example of the described algorithm application to the data from [[Wk2](https://en.wikipedia.org/wiki/Chow_test#/media/File:Chowtest4.svg)].
QRMonUnit[data]?QRMonPlotStructuralBreakSplits[ImageSize -> Small];
![IntroductionsExample](https://github.com/antononcube/MathematicaForPrediction/raw/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Introductions-example.png)
## Load packages
Here we load the packages [AAp1] and [AAp2].
Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/MonadicProgramming/MonadicQuantileRegression.m"]
Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/MonadicProgramming/MonadicStructuralBreaksFinder.m"]
## Data used
In this section we assign the data used in this document.
### Illustration data from Wikipedia
Here is the data used in the Wikipedia article "Chow test", [[Wk2](https://en.wikipedia.org/wiki/Chow_test#/media/File:Chowtest4.svg)].
data = {{0.08, 0.34}, {0.16, 0.55}, {0.24, 0.54}, {0.32, 0.77}, {0.4,
0.77}, {0.48, 1.2}, {0.56, 0.57}, {0.64, 1.3}, {0.72, 1.}, {0.8,
1.3}, {0.88, 1.2}, {0.96, 0.88}, {1., 1.2}, {1.1, 1.3}, {1.2,
1.3}, {1.3, 1.4}, {1.4, 1.5}, {1.4, 1.5}, {1.5, 1.5}, {1.6,
1.6}, {1.7, 1.1}, {1.8, 0.98}, {1.8, 1.1}, {1.9, 1.4}, {2.,
1.3}, {2.1, 1.5}, {2.2, 1.3}, {2.2, 1.3}, {2.3, 1.2}, {2.4,
1.1}, {2.5, 1.1}, {2.6, 1.2}, {2.6, 1.4}, {2.7, 1.3}, {2.8,
1.6}, {2.9, 1.5}, {3., 1.4}, {3., 1.8}, {3.1, 1.4}, {3.2,
1.4}, {3.3, 1.4}, {3.4, 2.}, {3.4, 2.}, {3.5, 1.5}, {3.6,
1.8}, {3.7, 2.1}, {3.8, 1.6}, {3.8, 1.8}, {3.9, 1.9}, {4., 2.1}};
ListPlot[data]
![DataUsedWk2](https://github.com/antononcube/MathematicaForPrediction/raw/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Data-used-Wk2.png)
### S&P 500 Index
Here we get the time series corresponding to [S&P 500 Index](https://en.wikipedia.org/wiki/S%26P_500_Index).
tsSP500 = FinancialData[Entity["Financial", "^SPX"], {{2015, 1, 1}, Date[]}]
DateListPlot[tsSP500, ImageSize -> Medium]
![DataUsedSP500](https://github.com/antononcube/MathematicaForPrediction/raw/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Data-used-SP500.png)
## Application of Chow Test
The Chow Test statistic is implemented in [AAp1].
In this document we rely on the relative comparison of the Chow Test statistic values: the larger the value of the Chow test statistic,
the more likely we have a structural break.
Here is how we can apply the Chow Test with a QRMon pipeline to the [Wk2] data given above.
chowStats =
QRMonUnit[data]?
QRMonChowTestStatistic[Range[1, 3, 0.05], {1, x}]?
QRMonTakeValue;
We see that the regressor points $\text{$\$$Failed}$ and $1.7$ have the largest Chow Test statistic values.
Block[{chPoint = TakeLargestBy[chowStats, Part[#, 2]& , 1]},
ListPlot[{chowStats, chPoint}, Filling -> Axis, PlotLabel -> Row[{"Point with largest Chow Test statistic:",
Spacer[8], chPoint}]]]
![ApplicationOfChowTestchowStats](https://github.com/antononcube/MathematicaForPrediction/raw/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Application-of-Chow-Test-chowStats.png)
The first argument of QRMonChowTestStatistic is a list of regressor points or Automatic.
The second argument is a list of functions to be used for the regressions.
Here is an example of an automatic values call.
chowStats2 = QRMonUnit[data]?QRMonChowTestStatistic?QRMonTakeValue;
ListPlot[chowStats2, GridLines -> {
Part[
Part[chowStats2, All, 1],
OutlierIdentifiers`OutlierPosition[
Part[chowStats2, All, 2], OutlierIdentifiers`SPLUSQuartileIdentifierParameters]], None}, GridLinesStyle -> Directive[{Orange, Dashed}], Filling -> Axis]
![ApplicationOfChowTestchowStats2](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Application-of-Chow-Test-chowStats2.png)
For the set of values displayed above we can apply simple 1D outlier identification methods,
[[AAp3](https://github.com/antononcube/MathematicaForPrediction/blob/master/OutlierIdentifiers.m)],
to automatically find the structural break point.
chowStats2[[All, 1]][[OutlierPosition[chowStats2[[All, 2]], SPLUSQuartileIdentifierParameters]]]
(* {1.7} *)
OutlierPosition[chowStats2[[All, 2]], SPLUSQuartileIdentifierParameters]
(* {20} *)
We cannot use that approach for finding all structural breaks in the general time series cases though as exemplified with the following code using the time series S&P 500 Index.
chowStats3 = QRMonUnit[tsSP500]?QRMonChowTestStatistic?QRMonTakeValue;
DateListPlot[chowStats3, Joined -> False, Filling -> Axis]
![ApplicationOfChowTestSP500](https://github.com/antononcube/MathematicaForPrediction/raw/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Application-of-Chow-Test-SP500.png)
OutlierPosition[chowStats3[[All, 2]], SPLUSQuartileIdentifierParameters]
(* {} *)
OutlierPosition[chowStats3[[All, 2]], HampelIdentifierParameters]
(* {} *)
In the rest of the document we provide an algorithm that works for general time series.
## Finding all structural break points
Consider the problem of finding of **all** structural breaks in a given time series.
That can be done (reasonably well) with the following procedure.
1. Chose functions for testing for structural breaks (usually linear.)
2. Apply Chow Test over dense enough set of regressor points.
3. Make a time series of the obtained Chow Test statistics.
4. Find the local maxima of the Chow Test statistics time series.
5. Determine the most significant break point.
6. Plot the splits corresponding to the found structural breaks.
QRMon has a function, QRMonFindLocalExtrema, for finding local extrema; see [AAp1, AA1].
For the goal of finding all structural breaks, that semi-symbolic algorithm is the crucial part in the steps above.
## Computation
### Chose fitting functions
fitFuncs = {1, x};
### Find Chow test statistics local maxima
The computation below combines steps 2,3, and 4.
qrObj =
QRMonUnit[tsSP500]?
QRMonFindChowTestLocalMaxima["Knots" -> 20,
"NearestWithOutliers" -> True,
"NumberOfProximityPoints" -> 5, "EchoPlots" -> True,
"DateListPlot" -> True,
ImageSize -> Medium]?
QRMonEchoValue;
![ComputationLocalMaxima](https://github.com/antononcube/MathematicaForPrediction/raw/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Computation-local-maxima.png)
### Find most significant structural break point
splitPoint = TakeLargestBy[qrObj?QRMonTakeValue, #[[2]] &, 1][[1, 1]]
### Plot structural breaks splits and corresponding fittings
Here we just make the plots without showing them.
sbPlots =
QRMonUnit[tsSP500]?
QRMonPlotStructuralBreakSplits[(qrObj? QRMonTakeValue)[[All, 1]],
"LeftPartColor" -> Gray, "DateListPlot" -> True,
"Echo" -> False,
ImageSize -> Medium]?
QRMonTakeValue;
The function QRMonPlotStructuralBreakSplits returns an association that has as keys paired split points and Chow Test statistics; the plots are association's values.
Here we tabulate the plots with plots with most significant breaks shown first.
Multicolumn[
KeyValueMap[
Show[#2, PlotLabel ->
Grid[{{"Point:", #1[[1]]}, {"Chow Test statistic:", #1[[2]]}}, Alignment -> Left]] &, KeySortBy[sbPlots, -#[[2]] &]], 2]
![ComputationStructuralBreaksPlots](https://github.com/antononcube/MathematicaForPrediction/raw/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Computation-structural-breaks-plots.png)
## Future plans
We can further apply the algorithm explained above to identifying time series states or components.
The structural break points are used as knots in appropriate Quantile Regression fitting. Here is an example.
The plan is to develop such an identifier of time series states in the near future.
(And present it at [WTC-2019](https://www.wolfram.com/events/technology-conference/2019/).)
![FuturePlansTimeSeriesStates](https://github.com/antononcube/MathematicaForPrediction/raw/master/MarkdownDocuments/Diagrams/Finding-all-structural-breaks-in-time-series/Future-plans-time-series-states.png)
## References
### Articles
\[Wk1\] Wikipedia entry, [Structural breaks](https://en.wikipedia.org/wiki/Structural_break).
\[Wk2\] Wikipedia entry, [Chow test](https://en.wikipedia.org/wiki/Chow_test).
\[AA1\] Anton Antonov, ["Finding local extrema in noisy data using Quantile Regression"](https://mathematicaforprediction.wordpress.com/2015/09/27/finding-local-extrema-in-noisy-data-using-quantile-regression/), (2019), [MathematicaForPrediction at WordPress](https://mathematicaforprediction.wordpress.com/2015/09/27/finding-local-extrema-in-noisy-data-using-quantile-regression/).
\[AA2\] Anton Antonov, ["A monad for Quantile Regression workflows"](https://github.com/antononcube/MathematicaForPrediction/blob/master/MarkdownDocuments/A-monad-for-Quantile-Regression-workflows.md), (2018), [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction/).
### Packages
\[AAp1\] Anton Antonov, [Monadic Quantile Regression Mathematica package](https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicQuantileRegression.m), (2018), [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction/).
\[AAp2\] Anton Antonov, [Monadic Structural Breaks Finder Mathematica package](https://github.com/antononcube/MathematicaForPrediction/blob/master/MonadicProgramming/MonadicStructuralBreaksFinder.m), (2019), [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction/).
\[AAp3\] Anton Antonov, [Implementation of one dimensional outlier identifying algorithms in Mathematica](https://github.com/antononcube/MathematicaForPrediction/blob/master/OutlierIdentifiers.m), (2013), [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction/).
### Videos
\[AAv1\] Anton Antonov, Structural Breaks with QRMon, (2019), YouTube.Anton Antonov2019-07-31T19:26:28ZFindMinimum problem with Interpolated curve
https://community.wolfram.com/groups/-/m/t/3067258
Here's is a simple example. &[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/14a83161-1a0a-4cf0-9082-0451ae15dc80
If the curve is defined as c[t], interpolating 2D points, FindMinimum doesn't work. If it is defined as {c1[t],c2[t]}, where c1 interpolates the x-values, and c2 interpolates the value, it works. Is there some reason I should expect this, from some (deeper than I understand) nature of Mathematica, or should we call this (gasp!) a bug?David Golber2023-11-19T01:30:22ZRequest to add operations on money accounts in Wolfram Alpha
https://community.wolfram.com/groups/-/m/t/3063787
One thing that could be added is the combined operations that are needed for money accountsjordi p2023-11-12T11:37:42ZSystemModelLinearize cannot linearize normally
https://community.wolfram.com/groups/-/m/t/3067562
The **SystemModelLinearize** instruction cannot be linearized properly, what is the reason for the following error after execution?
![enter image description here][1]
model = ImportString[
"model Circuit\n parameter Modelica.SIunits.Resistance r = 22 \
\"Resistance\";\n parameter Modelica.SIunits.Inductance l = 1 / 5 \
\"Inductance\";\n parameter Modelica.SIunits.Capacitance c = 1e-5 \
\"Capacitance\";\n parameter Boolean k1 = true \"If true, switch \
open\";\n parameter Boolean k2 = false \"If true, switch1 open\";\n \
Modelica.Electrical.Analog.Basic.Inductor inductor1(L = l, i.start = \
6 / 5) annotation(Placement(visible = true, transformation(origin = \
{-20, 15}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));\n \
Modelica.Electrical.Analog.Basic.Capacitor capacitor1(C = c, v.start \
= 24) annotation(Placement(visible = true, transformation(origin = \
{-0.043, -0.088}, extent = {{-10, -10}, {10, 10}}, rotation = -90)));\
\n Modelica.Electrical.Analog.Basic.Resistor resistor1(R = r) \
annotation(Placement(visible = true, transformation(origin = {40, \
-0}, extent = {{-10, -10}, {10, 10}}, rotation = -90)));\n \
Modelica.Electrical.Analog.Basic.Ground ground1 \
annotation(Placement(visible = true, transformation(origin = {80, \
-40}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));\n \
Modelica.Electrical.Analog.Ideal.IdealOpeningSwitch switch(Ron = 0, \
Goff = 0) annotation(Placement(visible = true, transformation(origin \
= {-60, 15}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));\n \
Modelica.Electrical.Analog.Ideal.IdealOpeningSwitch switch1(Ron = 0, \
Goff = 0) annotation(Placement(visible = true, transformation(origin \
= {-40, 0}, extent = {{-10, -10}, {10, 10}}, rotation = -270)));\n \
Modelica.Electrical.Analog.Sensors.VoltageSensor voltageSensor1 \
annotation(Placement(visible = true, transformation(origin = {80, 0}, \
extent = {{-10, 10}, {10, -10}}, rotation = 270)));\n \
Modelica.Electrical.Analog.Sources.SignalVoltage signalVoltage1 \
annotation(Placement(visible = true, transformation(origin = {-80, \
0}, extent = {{10, -10}, {-10, 10}}, rotation = -270)));\n \
Modelica.Blocks.Sources.BooleanConstant booleanConstant1(k = k1) \
annotation(Placement(visible = true, transformation(origin = {-80, \
40}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));\n \
Modelica.Blocks.Sources.BooleanConstant booleanConstant2(k = k2) \
annotation(Placement(visible = true, transformation(origin = {-80, \
-40}, extent = {{-10, -10}, {10, 10}}, rotation = 0)));\n \
Modelica.Blocks.Interfaces.RealOutput Vc(final quantity = \
\"ElectricPotential\", final unit = \"V\") \"Measured voltage\" \
annotation(Placement(visible = true, transformation(origin = {120, \
0}, extent = {{-10, -10}, {10, 10}}, rotation = -360), \
iconTransformation(origin = {110, 0}, extent = {{-10, -10}, {10, \
10}}, rotation = 0)));\n Modelica.Blocks.Interfaces.RealInput \
Vi(final quantity = \"ElectricPotential\", final unit = \"V\") \
\"Input voltage\" annotation(Placement(visible = true, \
transformation(origin = {-120, 0}, extent = {{-20, -20}, {20, 20}}, \
rotation = 0), iconTransformation(origin = {-120, 0}, extent = {{-20, \
-20}, {20, 20}}, rotation = 0)));\nequation\n connect(ground1.p, \
voltageSensor1.n) annotation(Line(visible = true, origin = {80, -20}, \
points = {{0, -10}, {0, 10}}, color = {10, 90, 224}));\n \
connect(signalVoltage1.p, switch.p) annotation(Line(visible = true, \
origin = {-76.667, 13.333}, points = {{-3.333, -3.333}, {-3.333, \
1.667}, {6.667, 1.667}}, color = {10, 90, 224}));\n \
connect(switch.n, switch1.n) annotation(Line(visible = true, origin = \
{-43.333, 13.333}, points = {{-6.667, 1.667}, {3.333, 1.667}, {3.333, \
-3.333}}, color = {10, 90, 224}));\n connect(switch1.n, inductor1.p) \
annotation(Line(visible = true, origin = {-36.667, 13.333}, points = \
{{-3.333, -3.333}, {-3.333, 1.667}, {6.667, 1.667}}, color = {10, 90, \
224}));\n connect(inductor1.n, capacitor1.p) annotation(Line(visible \
= true, origin = {-3.362, 13.304}, points = {{-6.638, 1.696}, {3.319, \
1.696}, {3.319, -3.392}}, color = {10, 90, 224}));\n \
connect(capacitor1.p, resistor1.p) annotation(Line(visible = true, \
origin = {19.979, 12.478}, points = {{-20.021, -2.566}, {-20.021, \
2.522}, {20.021, 2.522}, {20.021, -2.478}}, color = {10, 90, 224}));\n\
connect(resistor1.p, voltageSensor1.p) annotation(Line(visible = \
true, origin = {60, 12.5}, points = {{-20, -2.5}, {-20, 2.5}, {20, \
2.5}, {20, -2.5}}, color = {10, 90, 224}));\n \
connect(voltageSensor1.n, resistor1.n) annotation(Line(visible = \
true, origin = {60, -12.5}, points = {{20, 2.5}, {20, -2.5}, {-20, \
-2.5}, {-20, 2.5}}, color = {10, 90, 224}));\n connect(resistor1.n, \
capacitor1.n) annotation(Line(visible = true, origin = {19.979, \
-12.566}, points = {{20.021, 2.566}, {20.021, -2.522}, {-20.021, \
-2.522}, {-20.021, 2.478}}, color = {10, 90, 224}));\n \
connect(capacitor1.n, switch1.p) annotation(Line(visible = true, \
origin = {-20.021, -12.566}, points = {{19.979, 2.478}, {19.979, \
-2.522}, {-19.979, -2.522}, {-19.979, 2.566}}, color = {10, 90, \
224}));\n connect(switch1.p, signalVoltage1.n) \
annotation(Line(visible = true, origin = {-60, -12.5}, points = {{20, \
2.5}, {20, -2.5}, {-20, -2.5}, {-20, 2.5}}, color = {10, 90, 224}));\n\
connect(booleanConstant1.y, switch.control) annotation(Line(visible \
= true, origin = {-63, 34}, points = {{-6, 6}, {3, 6}, {3, -12}}, \
color = {255, 0, 255}));\n connect(booleanConstant2.y, \
switch1.control) annotation(Line(visible = true, origin = {-61, -20}, \
points = {{-8, -20}, {-3, -20}, {-3, 20}, {14, 20}}, color = {255, 0, \
255}));\n connect(Vi, signalVoltage1.v) annotation(Line(visible = \
true, origin = {-103.5, 0}, points = {{-16.5, 0}, {16.5, 0}}, color = \
{1, 37, 163}));\n connect(Vc, voltageSensor1.v) \
annotation(Line(visible = true, origin = {105, 0}, points = {{15, 0}, \
{-15, 0}}, color = {1, 37, 163}));\n annotation(experiment(StopTime \
= 10.0), Diagram(coordinateSystem(extent = {{-130, -60}, {130, 60}}, \
preserveAspectRatio = true, initialScale = 0.1, grid = {5, 5})), \
Icon(coordinateSystem(extent = {{-100.0, -100.0}, {100.0, 100.0}}, \
preserveAspectRatio = true, initialScale = 0.1, grid = {10, 10}), \
graphics = {Text(visible = true, fillPattern = FillPattern.Solid, \
extent = {{-100.0, -150.0}, {100.0, -110.0}}, textString = \"%name\", \
fontName = \"Arial\"), Rectangle(visible = true, fillColor = {119, \
163, 37}, pattern = LinePattern.None, fillPattern = \
FillPattern.Solid, lineThickness = 0, extent = {{-100.0, -100.0}, \
{100.0, 100.0}}), Rectangle(visible = true, origin = {0.1741, \
4.1776}, fillColor = {170, 85, 0}, fillPattern = FillPattern.Solid, \
lineThickness = 1, extent = {{2.5, 16.49}, {17.5, 50.0}}), \
Rectangle(visible = true, origin = {0.1741, 4.1776}, fillColor = \
{170, 85, 0}, fillPattern = FillPattern.Solid, lineThickness = 1, \
extent = {{2.5, -33.51}, {17.5, 0.0}}), Rectangle(visible = true, \
origin = {0.1741, 4.1776}, fillColor = {170, 85, 0}, fillPattern = \
FillPattern.Solid, lineThickness = 1, extent = {{42.5, 16.49}, {57.5, \
50.0}}), Rectangle(visible = true, origin = {0.1741, 4.1776}, \
fillColor = {170, 85, 0}, fillPattern = FillPattern.Solid, \
lineThickness = 1, extent = {{42.5, -33.51}, {57.5, 0.0}}), \
Ellipse(visible = true, fillColor = {170, 0, 0}, fillPattern = \
FillPattern.Solid, lineThickness = 1, extent = {{-70.0, -6.454}, \
{-40.0, 23.55}}), Line(visible = true, points = {{-70.0, -60.0}, \
{-40.0, -60.0}}, thickness = 2), Line(visible = true, origin = \
{-0.5222, -1.2185}, points = {{-62.51, -65.0}, {-47.1, -65.0}}, \
thickness = 2), Line(visible = true, origin = {-0.5222, -2.437}, \
points = {{-58.15, -70.0}, {-50.0, -70.0}}, thickness = 2), \
Line(visible = true, points = {{-54.657, 23.847}, {-55.0, 72.69}, \
{50.0, 72.69}, {50.132, 54.309}}, thickness = 0.5), Line(visible = \
true, points = {{9.922000000000001, 54.135}, {9.922000000000001, \
72.76000000000001}}, thickness = 0.5), Line(visible = true, origin = \
{-0.1741, 4.1776}, points = {{10.0, 0.0}, {9.922000000000001, \
16.536}}, thickness = 0.5), Line(visible = true, origin = {-0.1741, \
4.1776}, points = {{50.0, 0.0}, {50.0, 16.57}}, thickness = 0.5), \
Line(visible = true, points = {{49.783, -29.243}, {50.0, -45.0}, \
{-55.0, -45.0}, {-55.0055, -6.4405}}, thickness = 0.5), Line(visible \
= true, points = {{10.096, -29.243}, {10.0, -45.0}}, thickness = \
0.5), Line(visible = true, points = {{-55.0, -45.0}, {-55.0, -60.0}}, \
thickness = 0.5)}));\nend Circuit;", "MO"];
SystemModelLinearize[model,
Method -> {"SymbolicDerivative",
"SymbolicParameters" -> {"r", "l", "c", "k1", "k2"}}]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1700463109150.jpg&userId=2839169James James2023-11-20T06:55:32ZThe Enigma Machine
https://community.wolfram.com/groups/-/m/t/1066381
Below is an implementation of the German Enigma Machine which the German forces used to communicate encrypted messages during WWII. The machine was an ingenious design. It used a series of rotors and an elaborate electromechanical coupling to encrypt messages in German.
About the mechanism, the first rotor moved with each click of the keyboard; the second rotor moved once the first rotor completed 26 moves or one complete turn; and the third rotor once the first moved 26*26 steps (one can easily understand where this is going).
Since the rotors could move during the encryption process the key to deciphering the text was the "key" or the initial state of the rotors. The code was finally broken by a team of cryptographers at Bletchley Park led by Alan Turing. Some believe this caused the war to shorten by a few years. A movie titled "The Imitation Game" was released in 2014 highlighting this code breaking feat.
ClearAll@rotateWheel;
SetAttributes[rotateWheel, HoldFirst];
rotateWheel[wheel_] := Block[{},
wheel = RotateLeft[wheel]];
The immediate block of code above enables me to make in-place modification i.e. to rotate and preserve the state of the rotors.
EnigmaEncryption[string_, staterot1_, staterot2_, staterot3_] :=
Module[{count = 0, RotorIn, leftRotor, middleRotor, rightRotor, reflector, reflectorOutput,
rotateMiddleCheck, rotateRightCheck, inputToNext, reverseOutput},
RotorIn = ToLowerCase@CharacterRange["A", "Z"];
{leftRotor, middleRotor, rightRotor} = MapThread[Function[{x, y}, (z \[Function]
RotateLeft[z, First@Position[z, ToLowerCase@y] - 1])@
Characters@ToLowerCase[x]], {{"BDFHJLCPRTXVZNYEIWGAKMUSQO",
"AJDKSIRUXBLHWTMCQGZNPYFVOE", "EKMFLGDQVZNTOWYHXUSPAIBRCJ"},
{staterot1, staterot2, staterot3}}];
reflector = Characters@ToLowerCase@"YRUHQSLDPXNGOKMIEBFZCWVJAT";
inputToNext[rotor_, input_] := First@Cases[Thread[{RotorIn, rotor}], {input, map_} :> map ];
reverseOutput[rotor_, input_] := First@Cases[Thread[{RotorIn, rotor}], {map_, input} :> map ];
rotateMiddleCheck := If[count~Mod~26 == 0, rotateWheel@middleRotor, middleRotor];
rotateRightCheck := If[count~Mod~676 == 0, rotateWheel@rightRotor, rightRotor];
StringJoin@Table[
If[FreeQ[input, Alternatives[" ", ",", "'", "?" ]],
count += 1;
reflectorOutput =
Fold[inputToNext[#2, #1] &, input, {rotateWheel@leftRotor, rotateMiddleCheck, rotateRightCheck, reflector}];
Fold[reverseOutput[#2, #1] &, reflectorOutput, {rightRotor, middleRotor, leftRotor}], input]
, {input, Characters@ToLowerCase@string}]
]
Now lets assume that the Germans encrypt a message with state "A", "A","A" for the three moving rotors:
Style[text = EnigmaEncryption["this is the SS, Identify yourself, are you a German or are you Alan Turing?", "A", "A", "A"], {Bold, FontSize -> 24}]
**uubf jw dif oo, jctjgmbn nbtqrang, pvs vsh o orgiya lq lyw svn ssui zcxuxs?**
If the cryptographers at Bletchley have the incorrect key "B","A","E" they will not be able to decipher the text (it will be gibberish).
Style[EnigmaEncryption[text, "B", "A", "E"], {Bold, FontSize -> 24}]
**pgyy yd gnu nw, etlisxnw fnkniizh, tgy wde u gqkabx ma foe alc aifb cmavmt?**
However, with the right key:
Style[EnigmaEncryption[text, "A", "A", "A"], {Bold, FontSize -> 24}]
**this is the ss, identify yourself, are you a german or are you alan turing?**
We can make a small animation of the rotor states. For visual purposes, blue represents the forward states of the system and red the backward state.
![enter image description here][1]
the code below can be used to generate the animation sequence:
list = (Rasterize@*Grid /@
Module[{out, states, mergedstates, rotorstates, riffle, first, last, text = text,
textout = StringReplace[text[[1]], Alternatives[" ", ",", "'", "?"] :> ""]},
out = Characters@textout;
states = Partition[text[[2, 1]], 7];
mergedstates = Table[Join[states[[i]], {out[[i]]}], {i, Length@states}];
rotorstates = text[[2, 2]];
riffle = MapAt[Reverse, (Partition[#, 4] & /@ mergedstates), {All, 2}];
riffle = Apply[Composition[Partition[#, 2] &, Riffle], riffle, {1}];
Do[{first, last} = Flatten@Position[rotorstates[[j, i]], #] & /@ riffle[[j, i]];
rotorstates[[j, i, first]] = Style[First@rotorstates[[j, i, first]], {Blue, Bold, Background -> LightBlue}];
rotorstates[[j, i, last]] = Style[First@rotorstates[[j, i, last]], {Red, Bold, Background -> LightRed}];
, {j, Length@riffle}, {i, 4}];
rotorstates
]);
[1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=1479animate.gif&userId=942204Ali Hashmi2017-04-20T02:10:34Z