Community RSS Feed
https://community.wolfram.com
RSS Feed for Wolfram Community showing any discussions in tag Wolfram Language sorted by activeGuide 5: The Wolfram Plugin for ChatGPT
https://community.wolfram.com/groups/-/m/t/3078038
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/30387b08-4ff1-4022-ace3-8d742cf12177Michael Trott2023-12-05T15:41:14ZUnexpected output from Integrate
https://community.wolfram.com/groups/-/m/t/3077558
Hello, I am desperately trying to evaluate the following integral for several days now:
Integral[(1/(1+Exp[-x]))*Log[1/(1+Exp[-(x*w+b)])],{x,-Infinity,Infinity}]
I can't get Mathematica to do it; so I tried - at some point - this:
Integrate[(1/(1 + Exp[-x]))*Log[1/(1 + Exp[-x])], x]
That results in a formula, but it doesn't seem to make any sense?!
It returns:
https://gyazo.com/8ebda5752271450bbcf73067974f84a8
(I can't get Mathematica to copy anything to clipboard)
This contains `Log[-Exp[x]]` which can't exist since Exp[] is always positive?!
Can someone please help,
Carlo Wood
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/66667d6c-e09d-4a57-b907-9f5c2c5b4171Carlo Wood2023-12-04T18:08:04ZHow to remove noise in the plot generated from ListContourPlot?
https://community.wolfram.com/groups/-/m/t/3073060
I am trying to do ListContourPlot a list of three-dimensional data. The notebook file and the dependent file are attached here.
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/886cabfd-bc5d-4b21-bedf-b882bb08dc1c
I think the portion marked in the red-colored box is noise generated from the Mathematica module. Is there any way to remove these gittery plots from the original plot?Bibekananda Patra2023-11-28T07:24:30ZBuilding multi-qubit quantum channels
https://community.wolfram.com/groups/-/m/t/3064068
Does anyone have experience building multiqubit quantum channels? I can make a simple one with a single Krauss operator, but am having trouble defining a multiqubit channel with multiple Krauss operators. In this example, I'm applying a stochastic XX operator on qubits 1&2 and the identity on qubits 3&4.
This works:
channel =
QuantumChannel[
Sqrt[0.5] QuantumTensorProduct[QuantumOperator["X", {1, 2}],
QuantumOperator["I", {3, 4}]]]
Now if I want to apply XX or YY with equal probability on qubits 1&2, I fail.
This doesn't work:
channel =
QuantumChannel[{Sqrt[0.5]
QuantumTensorProduct[QuantumOperator["X", {1, 2}],
QuantumOperator["I", {3, 4}]],
Sqrt[0.5]
QuantumTensorProduct[QuantumOperator["Y", {1, 2}],
QuantumOperator["I", {3, 4}]]}]
I get this error message:
Thread::tdlen: Objects of unequal length in {4}->{1,2,3,4} cannot be combined.
Thanks in advance for any help.David Hayes2023-11-12T23:30:14Z[WSS18] Generating Music with Expressive Timing and Dynamics
https://community.wolfram.com/groups/-/m/t/1380021
![Cover][1]
## Goal ##
There are many ways to generate music and one of them is algorithmic, where music is generated with the help of a list of handcrafted rules.
The approach in this project is different - I build a neural network that knows nothing about music but learns it from thousands of songs given in MIDI format.
_Apart from just generating a meaningful sequence of notes I also wanted to add **dynamics** in loudness and humanlike mistakes in timing with **no restrictions for note durations**._
- **Why dynamics and timing?**
There is no human who is able to play on a musical instrument with precisely the same loudness and strictly in time with a metronome(at least I can't). People do mistakes, but in the case of music, they are helping in creating what we call more alive music. It is a fact that dynamic music with slight time shifts sounds more interesting, so even when you write music in a program you supposed to add these "mistakes" by yourself.
- **Why performances?**
The dataset that I use for the project contains performances of [Yamaha e-piano competition][2] participants. This gives us a possibility to learn the dynamics and mistakes in timings.
__Here's an [example][3] generated by the model.__
All the code, data and trained models can be found on [GitHub][32].
The examples will be attached to this post as files just in case.
----------
## Inspiration ##
This is not an original work and mostly it's an attempt to recreate the work of [Magenta][4] team from their blog [post][5].
Nevertheless, in this post, I will try to add more details to the **preprocessing** steps and how you can build a similar neural network model in Wolfram Language.
## Data ##
I've used a [site][6] that has the Yamaha e-piano performances but also contains a set of classic and jazz compositions.
In the original [work][7] Magenta team has used only the Yamaha dataset but with a heavy augmentation on top of that: Time-stretching (making each performance up to 5% faster or slower), Transposition (raising or lowering the pitch of each performance by up to a major third).
Also, you can create your own list of MIDI files and build a dataset with the help of the code provided below in the post.
Here are links to find a lot of free MIDI songs: [The Lakh MIDI Dataset][8](very well prepared a dataset for ML projects), [MidiWorld][9] and [FreeMidi][10]
## MIDI ##
MIDI is short for Musical Instrument Digital Interface. Its a language that allows computers, musical instruments, and other hardware to communicate.
MIDI carries event messages that specify musical notation, pitch, velocity, vibrato, panning, and clock signals (which set tempo).
_For the project, we need only events that denote where is every note starts/ends and with what are velocity and pitch._
## Preprocessing The Data ##
Even though MIDI is already a digital representation of music, we can't just take raw bytes of a file and feed it to an ML model as in the case of the models working with images. First of all, images and music are conceptually different tasks: the first is a single event(data) per item(an image), the second is a sequence of events per item(a song). Another reason is that raw MIDI representation and a single MIDI event itself contain a lot of irrelevant information to our task.
Thus we need a special data representation, a MIDI-like stream of musical events. Specifically, I use the following set of events:
- 88 **note-on** events, one for each of the 88 MIDI pitches of piano range. These events start a new note.
- 88 **note-off** events, one for each of the 88 MIDI pitches of piano range. These events release a note.
- 100 **time-shift** events in increments of 10 ms up to 1 second. These events move forward in time to the next note event.
- 34 **velocity** events, corresponding to MIDI velocities quantized into 32 bins. These events change the velocity applied to subsequent notes.
The neural network operates on a one-hot encoding over these 310 different events. This is the very same representation as in the original work but the number of note-on/note-off is fewer, I encode 88 notes in piano range instead of 127 notes in MIDI pitch range to reduce one-hot encoding vector size and make the process of learning easier.
**For example**, if you want to encode 4 notes from C major with durations of a half second and with different velocities your sequence of events would be somewhat like this(for clarity I use only indices instead of the whole one-hot encoding):
`{288, 60, 226, 148, 277, 62, 226, 150, 300, 64, 226, 152, 310, 67, 226, 155}`
![Preprocessing encoding C major example][11]
In this particular example:
- _60, 62, 64, 67_ are **note on** events(C5, D5, E5, G5). Values in a range from 1 to 88.
- _148, 150, 152, 155_ are **note off** events. Values in a range from 89 to 176.
- _226_ is a half second **time shift** event. Values in a range from 177 = 10 ms to 276 = 1 sec.
- _288, 277, 300, 310_ are **velocity** events. Values in a range from 277 to 310.
In this way, you can encode music that is expressive in dynamics and timing.
Now, let's take a look on another example with a chord from the same notes but with different durations:
`{300, 60, 62, 64, 67, 226, 152, 155, 226, 150, 226, 148}`
![C major chord][12]
As you can see, if you want to play more than one note at once you just need to put them in a single bunch of note-on events(60, 62, 64, 67).
Then you add time shift and note-off events as you needed. If you need a duration longer than 1 sec you can stack together more than one time-shift events({310, 310} = 2 sec time-shift).
**WL and MIDI**
Wolfram Language has a **built-in** support of MIDI files what is really simplifying initial work.
To get data from MIDI file you need to import it with specific elements:
![WL MIDI Import Elements][13]
In the code below I also extract and calculate needed information related to a tempo of a song.
{raw, header} = Import[path, #]& /@ {"RawData", "Header"};
tempos = Cases[Flatten[raw], HoldPattern["SetTempo" -> tempo_] :> tempo];
microsecondsPerBeat = If[Length@tempos > 0, First[tempos], 500000]; (* If there is no explicit tempo we use default 120 bpm *)
timeDivision = First@Cases[header, HoldPattern["TimeDivision" -> division_] :> division];
(* Convert timeDivision value to base of 2 *)
timeDivisionBits = IntegerDigits[timeDivision, 2];
(* Pad zeros at the beginning if the value takes less then 16 bits *)
timeDivisionBits = If[Length@timeDivisionBits < 16, PadLeft[timeDivisionBits, 16], timeDivisionBits];
(* The top bit responsible for the type of TimeDivision *)
timeDivisionType = timeDivisionBits[[1]];
framesPerSecond = timeDivisionBits[[2 ;; 8]];
ticksPerFrame = timeDivisionBits[[9 ;; 16]];
ticksPerBeat = If[timeDivisionType == 0, timeDivision, 10^6 /(framesPerSecond * ticksPerFrame)];
secondsPerTick = (microsecondsPerBeat / ticksPerBeat) * 10^-6.;
An example of raw data and header info from MIDI file in Wolfram Language:
![Raw MIDI output][14]
**SetTempo** is a number of microseconds per beat(microseconds per quarter note).
**Time Division** has two type of interpreting. If the top bit is 0 then the type is "ticks per beat" (or pulses per quarter note) otherwise, the type is "frames per second". We need those two values to calculate time per one **MIDI tick** that used in MIDI events as a time measurement.
One MIDI event in WL representation looks like this
`{56, {9, 0}, {46, 83}}`
- 56 is a number of **MIDI ticks** that means the total amount of time that must pass from the previous MIDI event.
It represents our **time-shift** event by simple multiplication of this number with **secondsPerTick**.
- 9 is a status byte of MIDI events(9,8 are **note-on**, **note-off** respectively).
- 0 is MIDI channel(irrelevant for us).
- 46 indicates what is a pitch of this note(related to **note-on**/**note-off** events).
- 83 is a number we encode in a **velocity** event.
If you want to understand how a real raw MIDI data structured, this [blog][15] is specifically useful.
Now, what we need is to parse a sequence of MIDI events and filter them only for events that are **note-on**, **note-off** and all the events that have the number of **MIDI ticks** greater than 0. Some of the meta-messages have irrelevant MIDI ticks thus we need to exclude them from final sequence - we just skip the events with value **F**(Meta message) in the MIDI status byte.
After filtering MIDI data you get a sequence that is ready to be encoded to the final representation and will be fed to the model.
![Filtered MIDI events][16]
To encode the sequence of MIDI events to the final representation I use the code below:
EncodeMidi[track_, secondsPerTick_] := Block[{lastVelocity = 0},
ClearAll[list];
Flatten[
Map[
Block[{list = {}},
(* Add time shifts when needed *)
If[TimeShiftByte[#, secondsPerTick] > 0, list = Join[list, EncodeTimeShift[TimeShiftByte[#, secondsPerTick]]]];
(* Proceed with logic only if it's a note event *)
If[StatusByte[#] == NoteOnByte || StatusByte[#] == NoteOffByte,
(* Add velocity if it's different from the last seen *)
If[lastVelocity != QuantizedVelocity[VelocityByte[#]] && StatusByte[#] == NoteOnByte,
lastVelocity = QuantizedVelocity[VelocityByte[#]];
list = Join[list, List[EncodeVelocity[VelocityByte[#]]]];
];
(* Add note event *)
list = Join[list, List[EncodeNote[NoteByte[#], StatusByte[#] == NoteOnByte]]];
];
(* Return encoded list*)
list
]&,
track]
, 1]];
This code has a lot of functions that I've written during the summer school but they are mostly utility short functions.
You can check them and complete implementation on [GitHub][17].
When the code for the preprocessing is ready it's time to build a dataset.
**Building Dataset**
I've made a [notebook][18] that takes care of preprocessing of MIDI files and encode them into the final representation.
(* Take all files names in Midi folder *)
files = FileNames["*", NotebookDirectory[] <> "Midi"];
dataset = Flatten[EncodeTrack /@ files, 1];
During the encoding, each track is partitioning into smaller segments:
encodings = Partition[EncodeMidi[GetMidiEvents[raw, secondsPerTick], secondsPerTick], 500];
In the original work, Magenta team split each song into 30-second segments to keep each example of manageable size. The problem is that partition by equal time doesn't give you the equal size of examples. Even though you can use varying input size in sequence models I wanted to use a static size of examples to speed up the training process. I was told that internally in WL(or maybe everywhere) it's more efficient to have the same size of every example for a model.
However, I believe this kind of partition has a drawback, in a way that an equal number of encoded events could have a different duration in time thus adding inconsistency in the dataset.
In my case, I've divided each song into segments of 500 encoded events.
![One Song Final Encoding][19]
To reduce the size of the final dataset I use only indices for one-hot encodings.
As the result, the final dimension of my dataset was **{99285, 500}**
If you want to try partition by the time you need to edit `EncodeTrack` function in [`Midi.m`][20].
With this code, you will find positions of where to split a sequence on equal time segments:
GetTimePositions[track_, seconds_, secondsPerTick_] :=
Block[{positions = {}, time = 0},
Do[
time = time + track[[i]][[1]] * secondsPerTick;
If[time > seconds, positions = Append[positions, i]; time = 0;],
{i, Length@track}];
positions
]
Where parameter `track` is a sequence of MIDI events. Then you split the same `track` with the positions you've got from the function.
segments = FoldPairList[TakeDrop, track, positions];
After that, you need to encode `segments` with the help of `EncodeMidi` function. If you do that there is one thing left - rework the model to accept varying input size but the next part will cover how to build a model with a static size of example.
----------
## Building a Model ##
Because music data is a sequence of events we need an architecture that knows how to remember, and predicts what is the next event based on all previous. This is exactly what Recurrent Neural Networks try to do - RNNs can use their internal state (memory) to process sequences of inputs. If you want to check more details I would recommend to watch this [introduction][21] video.
On the abstract level, RNN learns the probabilities of events that follow after each other. Take for example this language model from Wolfram Neural Repository, it predicts the next character of a given sequence.
NetModel["Wolfram English Character-Level Language Model V1"]["hello worl"]
The output is **d**.
You can get top 5 probabilities if you want.
NetModel["Wolfram English Character-Level Language Model V1"]["hello worl", {"TopProbabilities", 5}]
You will get:
{"d" -> 0.980898, "e" -> 0.00808785, "h" -> 0.0045687, " " -> 0.00143807, "l" -> 0.000681855}
In my work, I needed similar behavior but instead of characters, I wanted to predict encoded MIDI events. That is why the basis of the model I build is [Wolfram English Character-Level Language Model V1][22]. Also, after reading a [guide][23] about sequence learning with neural networks in WL I've decided to improve the training process with "teacher forcing" technique.
**Teacher Forcing**
In a simple language model, a model takes the last prediction from an input sequence and compute the class of it. But for "teacher forcing" we need to get classes of all predictions.
![Model comparison][24]
Comparatively to the language model I've removed one `GatedReccurentLayer` and `Dropoutlayer` due to the not so big dataset(as precautions to avoid overfitting). Another benefit of using "teacher forcing" is that you don't need to separately create labels for every example. To compute the loss we make out of an input example two sequences:
1. Everything but the **last** element(Sequence**Most**Layer)
2. Everything but the **first** element(Sequence**Rest**Layer)
![Teacher Forcing Net][25]
As you can notice the input is only one vector of indices with size 500 and labels for computing the loss are generating inside of a `NetGraph`.
Here is a visualized example of the flow with simple input:
![Input flow explanation][26]
You can find the code for creating the model in this [PerfrormanceRnnModel][27] notebook.
After all the data is ready and the model is finalized we can start training.
NetTrain[teacherForcingNet,
<|"Input" -> dataTrain|>,
All,
TrainingProgressCheckpointing -> {"File", checkPointDir, "Interval" -> Quantity[5, "Minutes"]},
BatchSize -> 64,
MaxTrainingRounds -> 10,
TargetDevice -> "GPU", (* Use CPU if you don't have Nvidia GPU *)
ValidationSet -> <|"Input" -> dataValidate|>
]
A friendly advice - it's better to use **"Checkpoining"** during the training. This will keep your mental health safe and will work as assurance that all training progress is saved.
I was training the model 30 rounds and it took around 4-5 hours on AWS' GPUs.
First 10-15 rounds weren't showing any sight of problems but later training clearly started to overfit.
![Training loss][28]
Unfortunately, I haven't had time to fix this problem because of the limited time but to overcome this problem I might reduce the size of GRUs from 512 to 256 and return Dropout layer.
## Generate Music ##
To generate music we need a model that predicts the next event in a sequence as it was in the language model. To do that I take the trained model and extract out of it "PerformanceRNN Predict Model" part.
predictNet = NetExtract[trainedNet, "predict"];
Next step is to convert this `predictNet` to a model that takes varying input size and return the class of the next event.
generateModel = NetJoin[NetTake[predictNet, 3], {
SequenceLastLayer[],
NetExtract[predictNet, {4, "Net"}],
SoftmaxLayer[]},
"Input" -> Automatic,
"Output" -> NetDecoder[{"Class", Range[310]}]
]
The resulting architecture is pretty the same as the language model from which I've started - it takes a sequence with varying size of encoded MIDI events `{177, 60, 90}` and predicts what could be next event `{177, 60, 90, ?}`.
![Model Comparison(Generation)][29]
**Now, let's the fun begin!**
generateDemo[net_, start_, len_] := Block[{obj = NetStateObject[net]},
Join@NestList[{obj[#, "RandomSample"]} &, start, len]
]
This small function is all we need to generate a sequence of the desired length.
`NetStateObject` helps to keep track of all sequences that were applied to the network, meaning every next prediction is the result of all previous events not only the recent one.
`start` should be a sequence of encoded MIDI events. It also can be a single item sequence, say you want to start from a pause or a particular note. This is a possibility to some extent put the generation process in a particular direction.
Okay, two lines of code left and you can hear play with generating of music:
generatedSequence = Flatten[generateDemo[generateModel, {60, 216, 148, 62, 200, 150, 64, 236, 152, 67, 198, 155}, 500]];
ToSound[generatedSequence]
These are other examples: [2][30], [3][31].
You can generate your own demos if download [repository][32] and open [PerformanceRNN][33] notebook.
## Further Work ##
That was a very fun and challenging task for me. I can't say that I'm satisfied with the results but this a good start and I have a direction now.
What I want to explore is Variational Autoencoder, especially [MusicVAE][34] that is made by the same Magenta team.
However, I'll start with improving the existing model by changing the architecture and cleaning the dataset to have only performances from the Yamaha dataset.
Thank you for reading the post, and feel free to ask any questions.
![Peace!][35]
[1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=2433article_cover.png&userId=1352227
[2]: http://www.piano-e-competition.com
[3]: https://drive.google.com/open?id=1I7l6hrecWsuMxqvEdUiRWtg6N6NCW34R
[4]: https://magenta.tensorflow.org/
[5]: https://magenta.tensorflow.org/performance-rnn
[6]: http://www.kuhmann.com/Yamaha.htm
[7]: https://github.com/tensorflow/magenta/tree/master/magenta/models/performance_rnn
[8]: http://colinraffel.com/projects/lmd/
[9]: http://www.midiworld.com/
[10]: https://freemidi.org/
[11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=5707Preprocessing_explanation.png&userId=1352227
[12]: http://community.wolfram.com//c/portal/getImageAttachment?filename=6109Prep_ex_2.png&userId=1352227
[13]: http://community.wolfram.com//c/portal/getImageAttachment?filename=MIDIimportelements.png&userId=1352227
[14]: http://community.wolfram.com//c/portal/getImageAttachment?filename=7717raw_midi_output.png&userId=1352227
[15]: http://www.recordingblogs.com/wiki/musical-instrument-digital-interface-midi
[16]: http://community.wolfram.com//c/portal/getImageAttachment?filename=filtered_midi_events.png&userId=1352227
[17]: https://github.com/Apisov/Performance-RNN-WL/blob/master/Project/Midi.m
[18]: https://github.com/Apisov/Performance-RNN-WL/blob/master/BuildData.nb
[19]: http://community.wolfram.com//c/portal/getImageAttachment?filename=One_track_final_encoding.png&userId=1352227
[20]: https://github.com/Apisov/Performance-RNN-WL/blob/master/Project/Midi.m
[21]: http://www.wolfram.com/wolfram-u/catalog/wl036/
[22]: https://resources.wolframcloud.com/NeuralNetRepository/resources/Wolfram-English-Character-Level-Language-Model-V1
[23]: http://reference.wolfram.com/language/tutorial/NeuralNetworksSequenceLearning.html#1013067167
[24]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Modelcomparison.png&userId=1352227
[25]: http://community.wolfram.com//c/portal/getImageAttachment?filename=3750Teacher_forcing.png&userId=1352227
[26]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Teacher_forcing_explanation.png&userId=1352227
[27]: https://github.com/Apisov/Performance-RNN-WL/blob/master/PerformanceRNNModel.nb
[28]: http://community.wolfram.com//c/portal/getImageAttachment?filename=raw_midi.png&userId=1352227
[29]: http://community.wolfram.com//c/portal/getImageAttachment?filename=Modelcomparison%28Generation%29.png&userId=1352227
[30]: https://drive.google.com/open?id=1GtlaOtTF_9rHiDVsrqmLnUaKCSAva1dP
[31]: https://drive.google.com/open?id=1sEihbFJw4XbVZveYl8efoM8Ivq8781ar
[32]: https://github.com/Apisov/Performance-RNN-WL
[33]: https://github.com/Apisov/Performance-RNN-WL/blob/master/PerformanceRNN.nb
[34]: https://magenta.tensorflow.org/music-vae
[35]: http://community.wolfram.com//c/portal/getImageAttachment?filename=d2eedc8a1ea8fc6a62e23b151c7fb3675c8153cc.png&userId=1352227Pavlo Apisov2018-07-11T21:20:56ZParametricNDSolve error: Dependent variables cannot depend on parameters
https://community.wolfram.com/groups/-/m/t/3077127
I'm trying to solve numerically this system of non linear DEs and there's some trouble with the numerical solution of the last differential equation, because while running the last line of the code that uses
`ParametricNDSolveValue` appears the followin message ParametricNDSolveValue::pdvar: Dependent variables {G,F[t,a,b]} cannot depend on parameters {a,b}.
I'm not quite sure about the evaluation in the F function but I tried. Here's the code:
t0 = 7;
u0 = 1.0;
v0 = -0.5;
z0 = Sin[t0];
sol = ParametricNDSolveValue[{
u'[x] == -Log[10] (-u[x] + a v[x]),
v'[x] == -Log[10] (u[x] - (a w[x])^2),
w'[x] == -Log[10] (w[x] (1 + a) - Cos[x]),
z'[x] == -Log[10] (4 a Sin[x]),
u[7] == u0, v[7] == v0, w[7] == w0, z[7] == z0}, {u, v, w, z}, {x,
7, -2}, {a}, MaxSteps -> Infinity];
dynsys[x_?NumericQ, a_?NumericQ] := sol[a][x];
x[t_] := Log[1 + t]/Log[10];
F[t_?NumericQ, a_, ?NumericQ, b_?NumericQ] :=
Sqrt[(b (1 + t)^3)/(
1 - u[x[t]]^2 - v[x[t]]^2 - w[x[t]]^2 - z[x[t]]^2)] /.
dynsys[x[t], a][[1]];
Gsol = ParametricNDSolveValue[{G'[t]/(1 + t) - G[t]/(1 + t)^2 - 1/
F[t, a, b] == 0, G[0] == 0}, {G}, {t, 0, 3}, {a, b},
MaxSteps -> Infinity];Carles Lopez2023-12-04T01:12:43ZPINN: Physics Informed Neural Networks for Laplace PDE on L-shaped domain
https://community.wolfram.com/groups/-/m/t/2557040
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=sdaf4q3egdfa.jpeg&userId=11733
[2]: https://www.wolframcloud.com/obj/41cb7cc4-a2ed-41ca-8fb1-16127a035be6Gianluca Argentini2022-06-24T09:51:13ZGuide 4: The Wolfram Plugin for ChatGPT
https://community.wolfram.com/groups/-/m/t/3077667
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/197b107a-cd1d-4a2e-b2a4-3b77ffa82d23Michael Trott2023-12-05T09:52:49ZUnable to Simplify non-commutative Expression Further
https://community.wolfram.com/groups/-/m/t/3076430
Dear Wolfram Community,
I hope this message finds you well. I require some assistance with a problem I encountered while using Wolfram Mathematica.
I have been trying to resolve problem 1.1 from the second edition of Sakurai's Modern Quantum Mechanics, using Mathematica. The problem's solution is shown below:
![enter image description here][1]
Here is the code I have been using:
In[3]:= commutator[x_, y_] := x ** y - y ** x
In[9]:= A ** commutator[C, B] ** D - A ** C ** commutator[D, B] +
commutator[C, A] ** D ** B -
C ** commutator[D, A] ** B // FullSimplify
Out[9]= -A ** C ** (-B ** D + D ** B) + A ** (-B ** C + C ** B) ** D -
C ** (-A ** D + D ** A) ** B + (-A ** C + C ** A) ** D ** B
However, I'm unsure as to why the result isn't further expandable and simplifiable. Am I missing something here? Any insights into this issue would greatly be appreciated.
Thank you for taking the time to read my query. I look forward to hearing from you soon.
Best regards,
Zhao
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Selection_031.png&userId=2470387Hongyi Zhao2023-12-02T11:51:05ZWolfram Neural Networks Boot Camp 2023
https://community.wolfram.com/groups/-/m/t/3077604
![enter image description here][1]
Hi everybody! Just about one month from now, our [online Neural Networks Boot Camp][2] will begin. This two-week boot camp will feature lectures from Wolfram instructors and developers from the machine learning team. These lectures will cover topics ranging from what a neural network even is all the way to describing generative pre-trained transformers (GPTs) and neural network applications in image, audio and video analysis.
Earn certification for:
- **Program completion** (by attending and actively participating in the sessions)
- **Level 1 proficiency in neural networks** (by completing graded exercises)
- **Level 2 applied expertise in neural networks** (by building and training your own neural networks)
> [**REGISTER HERE**][3]
![enter image description here][4]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot_2023-12-0415.12.16.png&userId=1711324
[2]: https://www.wolfram.com/wolfram-u/boot-camp-neural-networks/
[3]: https://www.wolfram.com/wolfram-u/boot-camp-neural-networks/
[4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=WolframUBanner.jpeg&userId=1711324Arben Kalziqi2023-12-04T21:34:41ZCorrect understanding of Wolfram Alpha continued fraction notation
https://community.wolfram.com/groups/-/m/t/3077320
Hello, I am new to Wolfram Alpha and this forum.
I am now trying to synthesize Low pass Chebyshev filter from a transfer function which is verified to be correctly calculated.
Based on this transfer function I am trying to count input impedance of the filter, which should be
Divide[4.4392Power[s,3]+1.58559Square[s]+3.20288s+0.49693,12.4Power[s,4]+4.4392Power[s,3]+13.9856Square[s]+3.20288s+2.04693]
I am trying to calculate the continuous fraction from the above formula:
https://www.wolframalpha.com/input?i2d=true&i=continued+fraction+Divide%5B4.4392Power%5Bs%2C3%5D%2B1.58559Square%5Bs%5D%2B3.20288s%2B0.49693%2C12.4Power%5Bs%2C4%5D%2B4.4392Power%5Bs%2C3%5D%2B13.9856Square%5Bs%5D%2B3.20288s%2B2.04693%5D
I transcribed the above form of output to this fraction:
0,242768+Divide[1\(44)18486s,-2\(44)81876*Power[10,16]s-Divide[7\(44)94538*Power[10,32]Square[s],1-Divide[2\(44)81876*Power[10,16]s,1+Divide[2.22045*Power[10,-16]s,0\(44)986281-Divide[0\(44)735921s,1+Divide[1,0\(44)51732s]]]]]]
Did I do it correctly? I am not sure if I understand the notation of the calculation of continued fraction correctly.
Thank you very much in advance for reply!
VaclavVaclav Zajic2023-12-04T12:31:30ZFinding the relationship between variables in PDEs?
https://community.wolfram.com/groups/-/m/t/3077137
Hello,
There are several equations need to be solve together.
1. `v'[t] == k*s*Sqrt[2*9.8*h]`
2. `r == Sqrt[1 - (1 - h)^2]`
3. `v'[h] == -Pi*r^2`
4. `k == 0.62`
5. `s == 10^-4`
6. When t equals 0, `h(t)==1`
So, I make out code as
DSolve[{D[v[t, h], t] == k*s*Sqrt[2*9.8*h], r == Sqrt[1 - (1 - h)^2],
D[v[t, h], h] == -Pi*r^2, k == 0.62, s == 10^-4, h[0] == 1}, v, {t,
h}]
I know this is not correct, at least `h[0]==1` is not correct. How to find the relationship between t and h?
This one can work but not give correct answer
DSolve[{D[v[t, h], t] == k*s*Sqrt[2*9.8*h],
D[v[t, h], h] == -Pi*r^2}, v, {t, h},
Assumptions -> {r == Sqrt[1 - (1 - h)^2], k == 0.62, s == 10^-4,
h == 1 /; t == 0}]
The answer should be `t == 1.068*10^4*(1 - 10/7*h^(3/2) + 3/7*h (5/2))`
Thanks.Zhenyu Zeng2023-12-04T02:11:16ZGuide 3: The Wolfram Plugin for ChatGPT
https://community.wolfram.com/groups/-/m/t/3077476
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/0dea27ec-856f-4032-afaa-1ef89c90cb47Michael Trott2023-12-04T20:02:42ZHow does Wolfram Language compile patterns for pattern matching?
https://community.wolfram.com/groups/-/m/t/3077464
I have been trying to understand how Mathematica is able to compute the values it does. The core evaluation process is very well documented and can be found online, but they completely lack the internal structure that deals with pattern matching.
For what I've found, some internal functions are used to (1) compare different patterns and arrange the argument matching order in the same function pattern (e.g. F[x_ + 1, x_] would first match the second argument (x_) and only then try match the first (x_ + 1)), and (2) which rule should be matched first, when multiple rules can be used. These functions are always changing with each version, but it does not matter to the question.
My computer science intuition would tell me to create some sort of prefix tree for the patterns and always try to match from there, though this would instantly go against the (2). Snooping around the documentation, I have found over the "Some Notes on Internal Implementation" that:
> Any transformation rule—whether given as x->y or in a definition—is automatically compiled into a form that allows for rapid pattern matching. Many different types of patterns are distinguished and are handled by special code.
>
> A form of hashing that takes account of blanks and other features of patterns is used in pattern matching.
>
> ...
>
> When a large number of definitions is given for a particular symbol, a hash table is automatically built using a version of Dispatch so that appropriate rules can quickly be found.
My guess is that they find common patterns in the rules and it does contribute for a special kind of hash they might be using, but this is just speculation. Does anyone knows to what form the patterns are compiled? What the Dispatch is actually doing, or how a hash table is used for pattern matching?Rui Gonçalves2023-12-04T19:58:48ZResidue Curve Map and Bubble Temperature Distribution for MTBE Chemistry at 8 atm
https://community.wolfram.com/groups/-/m/t/3075169
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/e95e30bf-7baf-4a3d-80d3-a5027b46d913Housam Binous2023-12-01T11:54:49ZGet the segment number for a coordinate on measured rod
https://community.wolfram.com/groups/-/m/t/3077061
Hi,
let me try to explain what I want to do, with Wolfram Mathematica, with an example.
I have subdivided a 1.7 meter long rod in several segments with offsets
{0., 0.533, 0.617, 0.819, 1.131, 1.455, 1.511, 1.7}
Let me give a position, e.g. x=0.56, I want to know in which segment that x is located. In this case x belong to segment=2.
If I give x=1.15 I should get segment=5.
If I give x=1.61 I should get segment=7.
And so on.
Obviously avoiding loops.
Many thanks for the helpTeodoro Marinucci2023-12-03T19:28:35ZComputational implementation of the German Enigma Machine
https://community.wolfram.com/groups/-/m/t/1066381
![enter image description here][1]
Below is an implementation of the German Enigma Machine which the German forces used to communicate encrypted messages during WWII. The machine was an ingenious design. It used a series of rotors and an elaborate electromechanical coupling to encrypt messages in German.
About the mechanism, the first rotor moved with each click of the keyboard; the second rotor moved once the first rotor completed 26 moves or one complete turn; and the third rotor once the first moved 26*26 steps (one can easily understand where this is going).
Since the rotors could move during the encryption process the key to deciphering the text was the "key" or the initial state of the rotors. The code was finally broken by a team of cryptographers at Bletchley Park led by Alan Turing. Some believe this caused the war to shorten by a few years. A movie titled "The Imitation Game" was released in 2014 highlighting this code breaking feat.
ClearAll@rotateWheel;
SetAttributes[rotateWheel, HoldFirst];
rotateWheel[wheel_] := Block[{},
wheel = RotateLeft[wheel]];
The immediate block of code above enables me to make in-place modification i.e. to rotate and preserve the state of the rotors.
EnigmaEncryption[string_, staterot1_, staterot2_, staterot3_] :=
Module[{count = 0, RotorIn, leftRotor, middleRotor, rightRotor, reflector, reflectorOutput,
rotateMiddleCheck, rotateRightCheck, inputToNext, reverseOutput},
RotorIn = ToLowerCase@CharacterRange["A", "Z"];
{leftRotor, middleRotor, rightRotor} = MapThread[Function[{x, y}, (z \[Function]
RotateLeft[z, First@Position[z, ToLowerCase@y] - 1])@
Characters@ToLowerCase[x]], {{"BDFHJLCPRTXVZNYEIWGAKMUSQO",
"AJDKSIRUXBLHWTMCQGZNPYFVOE", "EKMFLGDQVZNTOWYHXUSPAIBRCJ"},
{staterot1, staterot2, staterot3}}];
reflector = Characters@ToLowerCase@"YRUHQSLDPXNGOKMIEBFZCWVJAT";
inputToNext[rotor_, input_] := First@Cases[Thread[{RotorIn, rotor}], {input, map_} :> map ];
reverseOutput[rotor_, input_] := First@Cases[Thread[{RotorIn, rotor}], {map_, input} :> map ];
rotateMiddleCheck := If[count~Mod~26 == 0, rotateWheel@middleRotor, middleRotor];
rotateRightCheck := If[count~Mod~676 == 0, rotateWheel@rightRotor, rightRotor];
StringJoin@Table[
If[FreeQ[input, Alternatives[" ", ",", "'", "?" ]],
count += 1;
reflectorOutput =
Fold[inputToNext[#2, #1] &, input, {rotateWheel@leftRotor, rotateMiddleCheck, rotateRightCheck, reflector}];
Fold[reverseOutput[#2, #1] &, reflectorOutput, {rightRotor, middleRotor, leftRotor}], input]
, {input, Characters@ToLowerCase@string}]
]
Now lets assume that the Germans encrypt a message with state "A", "A","A" for the three moving rotors:
Style[text = EnigmaEncryption["this is the SS, Identify yourself, are you a German or are you Alan Turing?", "A", "A", "A"], {Bold, FontSize -> 24}]
**uubf jw dif oo, jctjgmbn nbtqrang, pvs vsh o orgiya lq lyw svn ssui zcxuxs?**
If the cryptographers at Bletchley have the incorrect key "B","A","E" they will not be able to decipher the text (it will be gibberish).
Style[EnigmaEncryption[text, "B", "A", "E"], {Bold, FontSize -> 24}]
**pgyy yd gnu nw, etlisxnw fnkniizh, tgy wde u gqkabx ma foe alc aifb cmavmt?**
However, with the right key:
Style[EnigmaEncryption[text, "A", "A", "A"], {Bold, FontSize -> 24}]
**this is the ss, identify yourself, are you a german or are you alan turing?**
We can make a small animation of the rotor states. For visual purposes, blue represents the forward states of the system and red the backward state.
![enter image description here][1]
the code below can be used to generate the animation sequence:
list = (Rasterize@*Grid /@
Module[{out, states, mergedstates, rotorstates, riffle, first, last, text = text,
textout = StringReplace[text[[1]], Alternatives[" ", ",", "'", "?"] :> ""]},
out = Characters@textout;
states = Partition[text[[2, 1]], 7];
mergedstates = Table[Join[states[[i]], {out[[i]]}], {i, Length@states}];
rotorstates = text[[2, 2]];
riffle = MapAt[Reverse, (Partition[#, 4] & /@ mergedstates), {All, 2}];
riffle = Apply[Composition[Partition[#, 2] &, Riffle], riffle, {1}];
Do[{first, last} = Flatten@Position[rotorstates[[j, i]], #] & /@ riffle[[j, i]];
rotorstates[[j, i, first]] = Style[First@rotorstates[[j, i, first]], {Blue, Bold, Background -> LightBlue}];
rotorstates[[j, i, last]] = Style[First@rotorstates[[j, i, last]], {Red, Bold, Background -> LightRed}];
, {j, Length@riffle}, {i, 4}];
rotorstates
]);
[1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=1479animate.gif&userId=942204Ali Hashmi2017-04-20T02:10:34ZNEW book by Seth J. Chandler—Query: Getting Information from Data with the Wolfram Language
https://community.wolfram.com/groups/-/m/t/3025306
![Query: Getting Information from Data with the Wolfram Language by Seth J. Chandler][1]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=community-chandler-2.png&userId=2691292
[Seth Chandler][at0] has a new book coming out tomorrow from Wolfram Media!
Mathematica and the Wolfram Language provide an integrated environment for sophisticated programming, making them the ideal tools for calculus, data visualization, machine learning, and more. But how about using Wolfram as the linchpin for organizing data? In this book, Professor Seth J. Chandler, an award-winning, 30-year user of the Wolfram Language, guides intermediate users from simple cases to data in the wild and shows how each can be fed into the full range of Wolfram Language functionality. It's both a conceptual and practical approach filled with annotated examples.
> ORDER NOW: https://www.amazon.com/Query-Getting-Information-Wolfram-Language/dp/1579550851
Want to know more about associations or the organization and presentation of datasets? This book shows how it's done. Want to organize or reduce data in the Wolfram Language similarly to the way one might use SQL, pandas in Python, or dplyr in R's tidyverse? This book shows how to simplify data processing and go beyond what is possible with other tools using the **Query** function. It also explores how to import information from the places real-world data is most likely to be found: CSV, Excel, or JSON files.
[at0]: https://community.wolfram.com/web/schandlerPaige Bremner2023-10-02T14:09:46ZAI vision via Wolfram Language
https://community.wolfram.com/groups/-/m/t/3072318
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/c298ab8f-16de-4178-bb27-268c24bd8485Anton Antonov2023-11-27T03:45:43ZAre there other solutions for DSolve[{y'[x] == E^(2*x - y[x]), y[0] == 0}, y[x], x]?
https://community.wolfram.com/groups/-/m/t/3077094
Hello,
DSolve[{y'[x] == E^(2*x - y[x]), y[0] == 0}, y[x], x]
produces
{{y[x] -> Log[1/2 + E^(2 x)/2]}
which I think is the all solutions. But Mathematica shows
![enter image description here][1]
It seems there are other solution for this equation. What will be them?
Thanks.
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshotfrom2023-12-0414-05-58.png&userId=2471588Zhenyu Zeng2023-12-04T06:07:06ZI do not understand well this output from FiniteGroupData
https://community.wolfram.com/groups/-/m/t/3077188
Because (Z/55Z)* has no subgroups of order 8. Am I wrong ?
In[89]:= FiniteGroupData[{"CyclicGroupUnits", 55}, "Subgroups"]
Out[89]= {"Trivial", {"CyclicGroup",
2}, {"AbelianGroup", {2, 2}}, {"CyclicGroup",
4}, **{"AbelianGroup", {4, 2}}**, {"CyclicGroup",
5}, {"AbelianGroup", {5, 2}}, {"AbelianGroup", {5, 2,
2}}, {"AbelianGroup", {5, 4}}, {"AbelianGroup", {5, 4, 2}}}Antonio Horcas Serrano2023-12-04T08:22:40ZAlgorithm for finding the minimum spanning tree of a graph output the value Null?
https://community.wolfram.com/groups/-/m/t/3077111
Hello! I'm trying to write an implementation of Prim's algorithm for finding the minimum spanning tree of a graph, using the found implementation of this algorithm in C++ ( https://evileg.com/ru/post/524/ ). When I run my code, I get Null as a result. I tried to display not the spanning tree itself, but, for example, the noedge variable. The answer is displayed as correct, but Null is still assigned next to it. Could you please tell me where the error is in my code and why does Null appear in output regardless of what my function returns?Varvara Markova2023-12-03T23:12:11ZHow do you create a box plot without a complete dataset?
https://community.wolfram.com/groups/-/m/t/3076882
Hi all!
I'm a beginner at Mathematica and I've been trying to create some data visualizations for a school assignment. In particular, I want to make a box-and-whisker plot, however, the tutorial I found (here --> https://reference.wolfram.com/language/StatisticalPlots/ref/BoxWhiskerPlot.html) only uses an example with a dataset of randomly generated numbers. With the data I am given, I only have the minimum, first quartile, median, third quartile, and maximum, and not the entire dataset, so how can I use only those to create my graphic?
Thank you!Margaret Chen2023-12-02T19:44:14ZFindMinimum error related to NDSolve
https://community.wolfram.com/groups/-/m/t/3077028
I'm trying to fit some data into a model using the standard procedure of minimizing the chi-square function using this code:
ClearAll["Global`*"]
Needs["ErrorBarPlots`"];
a0 = 1;
b0 = 2;
sol[a_, b_] :=
NDSolve[{x'[t] == -a*x[t] + Sin[t], y'[t] == a*x[t] - b*y[t]^2,
z'[t] == b*y[t] - Cos[t], x[0] == 1, y[0] == 0, z[0] == 0}, {x, y,
z}, {t, 0, 10}, MaxSteps -> Infinity];
F[t_] := x[t] + y[t]^2 + z[t]^3;
wsol[a_,
b_] := (wsol[a, b] =
NDSolve[{w'[t] - w[t] - ((F[t] /. sol[a, b])[[1]]) == 0,
w[0] == 0}, w, {t, 0, 10}]);
model[t_, a_, b_] := (w[t] /. wsol[a, b])[[1]] // Chop;
SeedRandom[1264645];
rangoT = Range[0, 9, 0.5];
ndat = Length[rangoT];
DATA = Table[{t,
model[t, a0, b0] + RandomVariate[NormalDistribution[0, .1]],
RandomReal[{.05, .1}]}, {t, rangoT}];
chi2[a_, b_] :=
Sum[((DATA[[i, 2]] - model[DATA[[i, 1]], a, b])/DATA[[i, 3]])^2, {i,
1, ndat}]
FindMinimum[chi2[a, b], {w, 0.9}, {b, 1.1}]
But when I run it I get error messages from the FindMinimum command related to NDSolve and ReplaceAll. Everything runs just fine until using that command. I was wondering why's not working and what should be changed.Carles Lopez2023-12-03T03:58:01ZExpanding structures based on hinged cubes
https://community.wolfram.com/groups/-/m/t/3076939
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/0f4b5e1a-0f75-4587-9fb0-8d3fa57be8e2Sandor Kabai2023-12-03T09:05:16ZSarrus linkage
https://community.wolfram.com/groups/-/m/t/2714334
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=sarrus.gif&userId=20103
[2]: https://www.wolframcloud.com/obj/79c77edf-b71e-4a2f-93d0-3ff85e258388Sandor Kabai2022-12-02T08:21:36ZCurrent Font Size display in Notebook
https://community.wolfram.com/groups/-/m/t/3076909
I use font size change with 'Alt + -' and 'Alt + =' to increase and decrease. I miss a permanent display on notebook window, which size is current. As a workaround I type some characters, to see the size.Heinz Wanger2023-12-02T21:41:39ZHow can I add background colour programmatically to certain cells of a grid
https://community.wolfram.com/groups/-/m/t/3076113
Hi everyone,
I have a matrix of numbers that I want to display as a grid and have the cells containing numbers > 135 highlighted. My approach was to pre-compute the position of the cells to be highlighted, and I'm wondering if there is a slicker way to do it. This is what I did.
(*the data*)
d = {
{129, 62, 60, 123, 62, 58, 144, 75, 56, 145, 79, 57},
{124, 67, 57, 126, 72, 59, 138, 71, 59, 137, 74, 56},
{132, 69, 62, 119, 67, 61, 125, 66, 62, 122, 67, 65},
{133, 75, 60, 130, 71, 62, 136, 66, 67, 136, 67, 67},
{128, 66, 53, 116, 69, 53, 139, 66, 52, 135, 67, 53},
{121, 64, 57, 118, 67, 58, 134, 71, 54, 136, 72, 56},
{128, 70, 54, 123, 61, 51, 139, 72, 58, 134, 69, 54},
{114, 62, 56, 116, 59, 54, 126, 65, 52, 119, 64, 53},
{125, 65, 57, 122, 65, 54, 130, 63, 63, 131, 62, 61},
{124, 67, 59, 125, 67, 59, 122, 62, 54, 128, 64, 55}
};
(*position of cells with values > 135*)
highlightPos = Position[d, _?(# > 135 &)];
(*the grid*)
Grid[
d,
Background -> {None, None, Map[# -> LightGray &, highlightPos]},
Alignment -> Automatic
]Gregory Lypny2023-12-01T22:31:13Z⭐ [R&DL] Wolfram R&D Developers on LIVE Stream
https://community.wolfram.com/groups/-/m/t/2593151
**Introducing our brand new YouTube channel, [Wolfram R&D][1]! Our channel features livestreams, behind-the-scenes creator presentations, insider videos, and more.**
----------
Join us for the unique Wolfram R&D livestreams on [Twitch][2] and [YouTube][3] led by our developers!
You will see **LIVE** stream indicators on these channels on the dates listed below. The live streams provide tutorials and behind the scenes look at Mathematica and the Wolfram Language directly from developers.
Join our livestreams every Wednesday at 11 AM CST and interact with developers who work on data science, machine learning, image processing, visualization, geometry, and other areas.
----------
⭕ **UPCOMING** EVENTS
- Dec 13: Understanding Graphics by [Brett Champion][55]
- Jan 17: Nutrients by the Numbers by [Gay Wilson][56]
----------
✅ **PAST** EVENTS
- Oct 18 -- [Overview of Number Theory][54]
- Sep 27 -- [QMRITools: Processing Quantitative MRI Data][52]
- Sep 13 -- [Make High Quality Graph Visualization][51]
- Sep 6 -- [Insider's View of Graphs & Networks][53]
- Aug 30 -- [Labeling Everywhere][49]
- Aug 22 -- [Equation Generator for Equation-of-Motion Coupled Cluster Assisted by CAS][48]
- Aug 16 -- [Foreign Function Interface][4]
- July 26 -- [Modeling Fluid Circuits][6]
- July 19 -- [Geocomputation][5]
- July 5 -- [Protein Visualization][7]
- Jun 14 -- [Chat Notebooks bring the power of Notebooks to LLMs][8]
- May 31-- [Probability and Statistics: Random Sampling][9]
- May 24 -- [Problem Solving][10]
- May 17 -- [The state of Optimization][11]
- May 10 -- [Building a video game with Wolfram notebooks][12]
- April 26 -- [Control Systems: An Overview][13]
- April 19 -- [MaXrd: A crystallography package developed for research support][14]
- April 5th -- [Relational database in the Wolfram Language][15]
- Mar 29th -- [Build your first game in the Wolfram Language with Unity game engine] [16]
- Mar 22nd -- [Everything to know about Mellin-Barnes Integrals - Part II][17]
- Mar 15th -- [Building your own Shakespearean GPT - a ChatGPT like GPT model][18]
- Mar 8th -- [Understand Time, Date and Calendars][19]
- Mar 1st -- [Introducing Astro Computation][20]
- Feb 22nd -- [Latest features in System Modeler][21]
- Feb 15th -- [Everything to know about Mellin-Barnes Integrals][22]
- Feb 8th -- [Dive into Video Processing][23]
- Feb 1st -- [PDE Modeling][24]
- Jan. 25th -- [Ask Integration Questions to Oleg Marichev][25]
- Jan. 18th -- [My Developer Tools][26]
- Jan. 11th -- [Principles of Dynamic Interfaces][27]
- Dec. 14th -- [Wolfram Resource System: Repositories & Archives][28]
- Dec. 7th -- [Inner Workings of ImageStitch: Image Registration, Projection and Blending][29]
- Nov. 30th -- [Q&A for Calculus and Algebra][30]
- Nov. 23rd -- [xAct: Efficient Tensor Computer Algebra][31]
- Nov. 16th -- [Latest in Machine Learning][32]
- Nov. 9th -- [Computational Geology][33]
- Nov. 2nd -- [Behind the Scenes at the Wolfram Technology Conference 2022][34]
- Oct 26th -- [Group Theory Package (GTPack) and Symmetry Principles in Condensed Matter][35]
- Oct 12th -- [Tree Representation for XML, JSON and Symbolic Expressions][36]
- Oct. 5th -- [A Computational Exploration of Alcoholic Beverages][37]
- Sept. 28th -- [Q&A with Visualization & Graphics Developers][38]
- Sept. 14th -- [Paclet Development][39]
- Sept. 7th -- [Overview of Chemistry][40]
- Aug. 24th -- [Dive into Visualization][41]
- Aug. 17th -- [Latest in Graphics & Shaders][42]
- Aug. 10th -- [What's new in Calculus & Algebra][43]
> **What are your interests? Leave a comment here on this post to share your favorite topic suggestions for our livestreams.**
**Follow us on our live broadcasting channels [Twitch][44] and [YouTube][45] and for the up-to-date announcements on our social media: [Facebook][46] and [Twitter][47].**
[1]: https://wolfr.am/1eatWLcDA
[2]: https://www.twitch.tv/wolfram
[3]: https://wolfr.am/1eatWLcDA
[4]: https://www.youtube.com/watch?v=C82NHpy7D6k
[5]: https://community.wolfram.com/groups/-/m/t/2985580
[6]: https://community.wolfram.com/groups/-/m/t/2982197
[7]: https://community.wolfram.com/groups/-/m/t/2982114
[8]: https://youtu.be/ZqawtrWwE0c
[9]: https://community.wolfram.com/groups/-/m/t/2946101
[10]: https://community.wolfram.com/groups/-/m/t/2925156
[11]: https://community.wolfram.com/groups/-/m/t/2921756
[12]: https://community.wolfram.com/groups/-/m/t/2918746
[13]: https://community.wolfram.com/groups/-/m/t/2917597
[14]: https://community.wolfram.com/groups/-/m/t/2911327
[15]: https://community.wolfram.com/groups/-/m/t/2907390
[16]: https://community.wolfram.com/groups/-/m/t/2921593
[17]: https://community.wolfram.com/groups/-/m/t/2861119
[18]: https://community.wolfram.com/groups/-/m/t/2847286
[19]: https://community.wolfram.com/groups/-/m/t/2851575
[20]: https://community.wolfram.com/groups/-/m/t/2852934
[21]: https://community.wolfram.com/groups/-/m/t/2842136
[22]: https://community.wolfram.com/groups/-/m/t/2838335
[23]: https://community.wolfram.com/groups/-/m/t/2827166
[24]: https://community.wolfram.com/groups/-/m/t/2823264
[25]: https://community.wolfram.com/groups/-/m/t/2821053
[26]: https://youtu.be/istKGqpDUsw
[27]: https://community.wolfram.com/groups/-/m/t/2777853
[28]: https://youtu.be/roCkXVkDuLA
[29]: https://youtu.be/pYHAz-NatXI
[30]: https://youtu.be/r7Hjdr_D7c4
[31]: https://community.wolfram.com/groups/-/m/t/2713818
[32]: https://community.wolfram.com/groups/-/m/t/2705779
[33]: https://community.wolfram.com/groups/-/m/t/2701172
[34]: https://youtu.be/UrM-OBu3H9o
[35]: https://community.wolfram.com/groups/-/m/t/2678940
[36]: https://community.wolfram.com/groups/-/m/t/2649407
[37]: https://community.wolfram.com/groups/-/m/t/2635049
[38]: https://community.wolfram.com/groups/-/m/t/2618033
[39]: https://community.wolfram.com/groups/-/m/t/2616863
[40]: https://community.wolfram.com/groups/-/m/t/2613617
[41]: https://community.wolfram.com/groups/-/m/t/2605432
[42]: https://community.wolfram.com/groups/-/m/t/2600997
[43]: https://community.wolfram.com/groups/-/m/t/2596451
[44]: https://www.twitch.tv/wolfram
[45]: https://wolfr.am/1eatWLcDA
[46]: https://www.facebook.com/wolframresearch
[47]: https://twitter.com/WolframResearch
[48]: https://www.youtube.com/live/ElP55ZILxPw?si=nsAPOQ3u-RbvuGKX
[49]: https://community.wolfram.com/groups/-/m/t/3007543
[50]: https://community.wolfram.com/web/charlesp
[51]: https://community.wolfram.com/groups/-/m/t/3019288
[52]: https://www.youtube.com/live/KM1yWHRrF2k?si=g2R7rHB2IinVRpo6
[53]: https://community.wolfram.com/groups/-/m/t/3009184
[54]: https://community.wolfram.com/groups/-/m/t/3064700
[55]: https://community.wolfram.com/web/brettc
[56]: https://community.wolfram.com/web/gwilsonCharles Pooh2022-08-05T21:37:19ZThe restored bells of Notre Dame: masses and sizes
https://community.wolfram.com/groups/-/m/t/3075381
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/67ec3e0c-7cfc-44ab-9672-130a090249b4Ed Pegg2023-12-01T16:22:21ZElectrical signal propagation along Actin filaments in physiological and pathological conditions
https://community.wolfram.com/groups/-/m/t/3075476
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/a14c3a9a-054a-4ed8-a33b-64436435257aMd Mohsin2023-12-01T18:38:03ZGuide 2: The Wolfram Plugin for ChatGPT
https://community.wolfram.com/groups/-/m/t/3075538
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/102d9203-91b3-4719-aeb6-7f17a42f420dMichael Trott2023-12-01T17:19:37ZAdvent of Code 2023
https://community.wolfram.com/groups/-/m/t/3075430
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/129086ac-c247-4909-a13e-646d2c0651a2Samuel Martineau2023-12-01T17:10:59ZChange OpenAI API key for Chatbook
https://community.wolfram.com/groups/-/m/t/3075200
Hello
I'm trying to use a Chat-Enabled Notebook with the Chatbook Paclet. I have to change my OpenAI API key to a new one, but I can't get it to work.
I can delete `SystemCredential["OPEN_API_KEY"]` after which I should be asked about my API key I thought at least and I can even set it to the right value, but it uses the wrong key "sk-i...unO1" whatever I set in `SystemCredential["OPEN_API_KEY"]`. How can I change the API key Chatbook is using?
![enter image description here][1]
The OpenAI API has some credit balance, so I don't think it has something to do with OpenAI. I use the Wolfram Cloud if this is relevant.
Anyone some ideas?
Many Thanks!
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshot2023-12-01154705.png&userId=3075182Malte Weigelt2023-12-01T15:01:34ZMore efficient way to extract info about Wolfram symbols and their docs?
https://community.wolfram.com/groups/-/m/t/3071114
Hello! I am working on the Clojure -> Wolfram bridge lib, https://github.com/scicloj/wolframite . One of the things we want to do is to "import" information about available Wolfram symbols (functions, entities, etc.) and their docs. What we do now is asking Wolfram to evaluate
Map[Function[{e},{e["Name"],e["PlaintextUsage"]}],WolframLanguageData[]]
but it is really slow, taking something between 30 and 60s on my Mac. I wonder if there is some faster way to get this information?
Thank you! Cheers, JakubJakub Holý2023-11-24T12:33:24ZNDSolve error:Equation or list of equations expected
https://community.wolfram.com/groups/-/m/t/3074833
These are my equations that have to been solved:
![enter image description here][1]
and this is my notebook that i'm working in. But i don't know what is wrong with that
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=imagem_2023-11-30_162428422.png&userId=3021630
[2]: https://www.wolframcloud.com/obj/17e52c18-f8ef-489b-bc3e-90205b26a3e2Anderson Coelho2023-11-30T19:25:47ZFolds over Lists, Streams, and Asynchronous Observables
https://community.wolfram.com/groups/-/m/t/3074231
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/d5d3b4d5-7e5a-4612-ba41-7ee0c85722adBrian Beckman2023-11-29T19:51:24ZSquare Grid Universe: programming one as a hypergraph rewriting ruleset
https://community.wolfram.com/groups/-/m/t/3065902
About
-----
![Unfolding space teaser][1]
The following is an algorithm and a hypergraph rewriting ruleset that produces a single-connected square grid.
**How is this different from existing grid-making rulesets?** This one is designed, not “discovered”. The space unfolds in clear stages, and produces a regular grid; the rules are human-readable.
**What value does this bring?** While the algorithm itself is straightforward, programming it as a hypergraph rewriting ruleset is challenging. Rule misapplication is ubiquitous. Enforcement of an execution order is non-trivial. The only debugging tool is “trial and error”.
With this post, I want to spell out my solutions and thus provide guidance to those who might want to construct their own universes.
**Why was this made?** For fun. I’d like to make a bare-bones universe, consisting only of space and basic particle interactions – and this is the first step to it.
----------
Ruleset
-------
We have a rectangular space with four corners. Let’s differentiate it into two zones: “outer layer” (active) and “inside” (done). First, to each corner of the outer layer, add a new cell. Second, fill the gaps between the expanded corners. Third, reset the cycle resets.
The algorithm executes these three steps in order – that’s an unnecessary restriction, enforced for aesthetic reasons: without it, the growth of the corners will outpace the growth of the inner area, which leads to unappealing “tendrils”.
**Key ideas:**
1. Algorithm-specific information is stored with self-loops and edge patterns. For example, `{x,x,x}` marks an outer layer node – so if the rule needs to be applied only to the outer layer, it can reference this marker and avoid misapplication.
2. Aliasing: `OUTER[x,y] = {x,x,x}, {y,y,y}` – this drastically improves readability and eases the editing. I personally had a lot of trouble with edge alignment in patterns – so being able to encapsulate that in `CELL[...]` and then use both in input and output of the rules removed a lot of typo-based bugs for me.
3. Rule execution order is enforced by adding extra markers. For example, Rule A is dependent on having both OUTER and CORNER markers, and produces the RIMs – which are needed by Rules B and C. Rule A also moves apart the OUTERs and CORNERs – which blocks it from execution until other rules bring them back.
4. Accounting for symmetry. Our square space is symmetrical at four axes, which breaks it into eight equivalent sectors. Creating rules that are contained within a single sector is crucial for preventing
rule misapplication.
![Ruleset][2]
----------
Code
----
&[Wolfram Notebook: the ruleset][3]
----------
Other notes
-----------
**Extension vs subdivision**. Expansion of this universe is achieved by extension – which means that any two nodes in the “stable” inside area remain forever near each other. Contrast that with an idea of subdivision, when the space expands by adding a new node between two existing ones: node-sized particles would be pulled apart and the bigger-sized matter would be shredded – unless some pulling force would be constantly updating it and pulling together. This makes me think about our universe expanding and the existence of gravitation and other fundamental interactions.
**Algorithm side effects as proto-particles**. Below I link a previous version of the algorithm, that leaves “junk” during its execution. I like to think of them as the matter created during the space unfolding (although, I was unable to create a rule that can operate over them without breaking the expansion).
**"Aberrant Fringes"**. It is interesting, that the outer layer is almost separate reality from the other space: with its own unique structures and rules that apply nowhere else.
**Proper 2D**. In my code, the space is single-connected (two nodes share one directional edge). For a double connection, one would need extra markers to differentiate corners and edge-midpoints (now, they are marked with all in-pointing and all out-pointing vertices respectively). For that, one might use an extra edge on the outer layer, that would be removed during the expansion.
**Negation is not available**. Markers like `{x,x,x}` cannot be used to indicate the absence of something. If you do not include it in a rule, that rule will apply both to the patterns with it and without it. Absence needs to be either indicated with its own markers – or derived from the pattern outside.
----------
Older ruleset, with side effects
--------------------------------
&[Wolfram Notebook: older ruleset, with side effects][4]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=2Dgridteaser.png&userId=3061034
[2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=4189Ruleset.png&userId=3061034
[3]: https://www.wolframcloud.com/obj/9ec9fd3d-4e41-413c-88bd-46ddb3f74b4c
[4]: https://www.wolframcloud.com/obj/26c891a9-0aa9-43c9-a2bc-be5e6be9a461Myk Konovalow2023-11-15T18:26:09ZStrangeworks: submit quantum circuits to quantum hardware through the Strangeworks platform
https://community.wolfram.com/groups/-/m/t/3074999
![enter image description here][1]
&[Wolfram Notebook][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Hero_image3.png&userId=20103
[2]: https://www.wolframcloud.com/obj/06b256c6-e383-4b83-b241-737da255d41bStuart Flannigan2023-11-30T20:25:56ZSeeking techniques for detailed step-by-step solution of integral equation transformation
https://community.wolfram.com/groups/-/m/t/3071960
Hello Mathematica Community,
I am working on a mathematical problem where I need to transform an integral equation into a simpler form using Euler's Formula and the Error Function. My current Mathematica code successfully simplifies the equation, but I'm looking for ways to extract more detailed intermediate steps within Mathematica.
The original equation is:
![enter image description here][1]
And it is transformed into:
![enter image description here][2]
I have used Euler's formula for the sine function and Mathematica provides the simplified result directly as:
![enter image description here][3]
However, I'm interested in understanding the intermediate steps that Mathematica goes through to arrive at this result. Specifically, I am looking for:
1. Mathematica techniques or functions that can help me break down the transformation process into more granular steps.
2. Insights into how Mathematica handles integral transformations, especially involving complex functions and the error function.
3. Any special Mathematica tricks or commands that could make these intermediate steps more explicit.
I would greatly appreciate any advice or guidance on how to use Mathematica more effectively for this purpose. Understanding the intermediate steps is crucial for my learning process.
Thank you for your help!
&[Wolfram Notebook][4]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshotfrom2023-11-2616-36-29.png&userId=3071927
[2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshotfrom2023-11-2616-39-32.png&userId=3071927
[3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Screenshotfrom2023-11-2616-39-45.png&userId=3071927
[4]: https://www.wolframcloud.com/obj/dda2fad8-6402-412f-bf94-a2eabbc753f8Rob Leny2023-11-26T14:48:20ZOn Jain's Digital Piracy Model: Horizontal vs Vertical Product Differentiation
https://community.wolfram.com/groups/-/m/t/3074816
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/53a19ad7-5777-4453-999e-88da6df308e3Kresimir Zigic2023-11-30T18:52:40ZHTML documentation doesn't show images (Workbench Eclipse plugin + M13)
https://community.wolfram.com/groups/-/m/t/2613086
We are developing GTPack, a Mathematica group theory package with an extensive documentation. The workbench eclipse plugin has been powerful in writing this documentation and compiling it to the standard Mathematica form. However, I am recently facing trouble with the generation of HTML documentation pages. The "documentation build" generates all necessary images and code, but when I open the page they are not displayed.
What do I need to do to fix this issue? I am using Eclipse IDE for Java Developers (Version: 2019-12 (4.14.0)), Wolfram Workbench (10.2.43), Mathematica 13, and Ubuntu 22.04.
I provide an example below.
For example, the regular documentation page looks like this:
![regular Mathematica documentation][1]
In contrast, the html version is lacking the images and looks as follows:
![html version][2]
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Bildschirmfotovom2022-09-0721-45-49.png&userId=2613052
[2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Bildschirmfotovom2022-09-0721-46-38.png&userId=2613052Matthias Geilhufe2022-09-07T19:49:47ZCopy an interactivity from a Cloud notebook?
https://community.wolfram.com/groups/-/m/t/3074421
Question:
can you copy the Logic Gates interactive here https://www.wolframcloud.com/obj/online-courses/introduction-to-discrete-mathematics/proposition-logic.html
without having to download the whole notebook?
ThanksRoberto Catanuto2023-11-30T10:39:34ZExtracting Russian casualties in Ukraine data from Mediazona publications
https://community.wolfram.com/groups/-/m/t/3073859
&[Wolfram Notebook][1]
[1]: https://www.wolframcloud.com/obj/3b0593a6-6542-4224-ad77-d9cd053bd390Anton Antonov2023-11-29T15:58:10ZHow to pick/select the coefficients of the same constant from the output of 3 equations
https://community.wolfram.com/groups/-/m/t/3074010
I have 3 equations Xi[1][x,t], Xi[2][t] , Phi[x, t], whose outputs are expressed in terms of variables {t, x, u}.
I want to generate X1 (an equation expressed in terms of variables {t,x,u} and constants C1, C2, ... , C6), similarly X2, X3, ... , X6. Please see an attachment on the attempts.&[Wolfram Notebook][1]
My goal is to write a syntax to select / pick the coefficient of the same constant from the 3 equations as shown below:
X1 = {{1/2 E^(-t \[Beta]) (1 + E^(2 t \[Beta])), 0, u (-((E^(-t \[Beta]) (-1 + E^(
t \[Beta])) (4 E^(t \[Beta]) \[Alpha] \[Beta] - \[Sigma]^2 - E^(t \[Beta]) \[Sigma]^2))/(2 \[Beta] \[Sigma]^2)) + 1/\[Sigma]^2 (2 x \[Beta] (1/ 2 E^(-t \[Beta]) (1 + E^(2 t \[Beta]))) +
2 x (E^(t \[Beta]) \[Beta] - 1/2 E^(-t \[Beta]) (1 + E^(2 t \[Beta])) \[Beta] )))},
X2 = {(E^(-t \[Beta]) (-1 + E^(2 t \[Beta])))/(2 \[Beta]), 0,
u (-((E^(-t \[Beta]) (-1 + E^(
t \[Beta])) (4 E^(
t \[Beta]) \[Alpha] \[Beta] + \[Sigma]^2 -
E^(t \[Beta]) \[Sigma]^2))/(2 \[Beta]^2 \[Sigma]^2)) +
1/\[Sigma]^2 (2 x \[Beta] ((
E^(-t \[Beta]) (-1 + E^(2 t \[Beta])))/(2 \[Beta])) +
2 x (E^(t \[Beta]) -
1/2 E^(-t \[Beta]) (-1 + E^(2 t \[Beta])) )))},
X3 = {0, 0, u},
X4 = {0, 1, 0}
[1]: https://www.wolframcloud.com/obj/87bc885b-4e98-4aa0-b0c2-1e18f76e04e3Nomsa Ledwaba2023-11-29T17:47:18ZA primer on Association and Dataset
https://community.wolfram.com/groups/-/m/t/1167544
*NOTE: all Wolfram Language code and data are available in the attached notebook at the end of the post.*
----------
For my class this fall, I developed a little primer on Association and Dataset that I think might be useful for many people. So, I'm sharing the attached notebook. It's somewhat about the concepts embedded inside these features. It's intended for people at a beginner-intermediate level of Mathematica/Wolfram Language programming, but might be of value even to some more advanced users who have not poked about the Dataset functionality.
The sections of the notebook are:
1. The world before Associations and Datasets
2. Datasets without Associations
3. Enter the Association
4. Creating a Dataset from a List of Associations
5. Nice queries with Dataset
6. Query
7. Some Recipes
#The world before Associations and Datasets#
Here' s an array of data. The data happens to represent the cabin class, age, gender, and survival of some of the passengers on the Titanic.
t = {{"1st", 29, "female", True}, {"1st", 30, "male", False}, {"1st",
58, "female", True}, {"1st", 52, "female", True}, {"1st", 21,
"female", True}, {"2nd", 54, "male", False}, {"2nd", 29, "female",
False}, {"3rd", 42, "male", False}};
As it stands, our data is a List of Lists.
Head[t]
> List
Head /@ t
> {List, List, List, List, List, List, List, List}
Suppose I wanted to get the second and fifth rows of the data. This is how I could do it.
t[[{2, 5}]]
> {{"1st", 30, "male", False}, {"1st", 21, "female", True}}
Suppose we want to group the passengers by gender and then compute the mean age. We could do this with the following pretty confusing code.
Use and enjoy. Constructive feedback appreciated.
grouped = GatherBy[t, #[[3]] &];
justTheAges = grouped[[All, All, 2]];
Mean /@ justTheAges
> {189/5, 42}
Or I could write it as a one liner this way.
Map[Mean, GatherBy[t, #[[3]] &][[All, All, 2]]]
> {189/5, 42}
But either way, realize that I have to remember that gender is the third column and that age is the second column. When there is a lot of data, this can get hard to remember.
#Datasets without Associations#
I could, if I wanted, convert this data into a Dataset. I do this below simply by wrapping Dataset about t. You see there is now some formatting about the data. But there are no column headers (because no one has told Dataset what to use). And there are no row headers, again because no one has told Dataset what to use.
t2 = Dataset[t]
![enter image description here][1]
The head of the expression has changed.
Head[t2]
> Dataset
Now, I can now access the data in a different way.
Query[{2, 5}][t2]
![enter image description here][2]
Or, I can do this. Mathematica basically converts this expression into Query[{2,5}][t2]. The expression t2[{2,5}] is basically syntactic sugar.
t2[{2, 5}]
![enter image description here][3]
##Digression : Using Query explicitly or using syntactic sugar##
Why, by the way would anyone use the longer form if Mathematica does the work for you? Suppose you want to store a Dataset operation -- perhaps a complex series of Dataset operations -- but you want it to work not just on a particular Dataset but on any Dataset (that is compatible). Here's how you could do it.
q = Query[{2, 5}]
> Query[{2, 5}]
q[t2]
![enter image description here][4]
Now, let' s create a permutation of the t2 Dataset so that the rows are scrambled up.
t2Scrambled = t2[{1, 4, 8, 3, 2, 7, 5}]
![enter image description here][5]
We can now run the q operation on t2Scrambled. Notice that the output has changed even though the query has stayed the same.
q[t2Scrambled]
![enter image description here][6]
We can also generate Query objects with functions. Here's a trivial example. There are very few languages of which I am aware that have the ability to generate queries by using a function. The one other example is Julia.
makeASimpleQuery[n_] := Query[n]
makeASimpleQuery[{3, 4, 7}][t2]
![enter image description here][7]
##MapReduce operations on Dataset objects##
Now, if I want to know the mean ages of the genders I can use this code. This kind of grouping of data and then performing some sort of aggregation operation on the groups is sometimes known as a MapReduce. (I'm not a fan of the name, but it is widely used). It's also sometimes known as a rollup or an aggregation.
Query[GroupBy[#[[3]] &], Mean, #[[2]] &][t2]
![enter image description here][8]
Or this shorthand form in which the Query is constructed.
t2g = t2[GroupBy[#[[3]] &], Mean, #[[2]] &]
![enter image description here][9]
I think this is a little cleaner. But we still have to remember the numbers of the columns, which can be challenging.
By the way, just to emphasize how we can make this all functional, here's a function that creates a query that can run any operation (not just computing the mean) on the Dataset grouped by gender and then working on age.
genderOp[f_] := Query[GroupBy[#[[3]] &], f, #[[2]] &]
genderOp[Max][t2]
![enter image description here][10]
To test your understanding, see if you can find the minimum age for each class of passenger on the Titanic in our Dataset **t2**.
Query[GroupBy[#[[1]] &], Min, #[[2]] &][t2]
![enter image description here][11]
#Enter the Association#
##Review of Association##
If you feel comfortable with Associations, you can skip this section; otherwise read it carefully. Basically the key to understanding most Dataset operations is understanding Associations.
###Construction of Associations###
Now let' s alter the data so that we don't have to remember those facts. To do this we will create an **Association**. Here's an example called **assoc1**. Notice that we do so by creating a sequence of rules and then wrapping it in an Association head. Notice that the standard output does not preserve the word "Association" as the head but, just as List is outputted as stuff inside curly braces, Association is outputted as stuff inside these funky "<|" and "|>" glyphs.
assoc1 = Association["class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True|>
I could equivalently have created a list of rules rather than a sequence. Mathematica would basically unwrap the **List** and create a sequence.
assoc1L = Association[{"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True}]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True|>
We can use **AssociationThread** to create Associations in a different way. The first argument is the list of things that go on the left hand side of the Rules -- the "keys" -- and the second argument is the list of things that go on the right hand side of the Rules -- the "values".
assoc1T = AssociationThread[{"class", "age", "gender", "survived"}, {"1st", 29, "female", True}]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True|>
Now let's use **AssociationThread** function to create a list of Associations similar to our original data.
convertListToAssociation =
list \[Function]
AssociationThread[{"class", "age", "gender", "survived"}, list]
> Function[list, AssociationThread[{"class", "age", "gender", "survived"}, list]]
I start with t and Map the **convertListToAssociation** function over the rows of the data. I end up with a list of Associations.
t3 = Map[convertListToAssociation, t]
![enter image description here][12]
###Keys and Values###
Associations have keys and values. These data structures are used in other computer languages but known by different names: *Python* and *Julia* call them dictionaries. *Go* and *Scala* call them maps. *Perl* and *Ruby* call them hashes. *Java* calls it a *HashMap*. And *Javascript* calls it an object. But they all work pretty similarly. Anyway, the keys of an **Association** are the things on the left hand side of the Rules.
Keys[assoc1]
> {"class", "age", "gender", "survived"}
And the values of an Association are the things on the right hand side of the Rules.
Values[assoc1]
> {"1st", 29, "female", True}
That' s about all there is too it. Except for one thing. Take a look at the input and output that follows.
assoc2 = Association["a" -> 3, "b" -> 4, "a" -> 5]
> <|"a" -> 5, "b" -> 4|>
You can' t have duplicate keys in an Association. So, when Mathematica confronts duplicate keys, it uses the last key it saw. You might think this is a minor point, but it is actually very important in coding. We will see why soon.
###Nested Associations###
A funny thing happens if you nest an **Association** inside another **Association**.
Association[assoc1, assoc2]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True, "a" -> 5, "b" -> 4|>
You end up with a single un - nested (flat) association. That's a little unusual for Mathematica, but we can exploit this flattening as a way of adding elements to an Association.
Association[Association["dances" -> False], assoc1]
> <|"dances" -> False, "class" -> "1st", "age" -> 29, "gender" ->
> "female", "survived" -> True|>
Or, here' s a function that exploits the flattening to add elements to an **Association**.
addstuff = Association[#, "dances" -> False, "sings" -> True] &
> Association[#1, "dances" -> False, "sings" -> True] &
addstuff[assoc1]
> <|"class" -> "1st", "age" -> 29, "gender" -> "female", "survived" -> True, "dances" -> False, "sings" -> True|>
###Extracting Values from Associations###
Just as the values contained in a **List** can be accessed by using the **Part** function, the values contained in an **Association** can likewise be accessed. Suppose, for example that I wanted to compute double the age of the person in **assoc1**.
It turns out there are a lot of ways of doing this. The first is to treat the Association as a list except that the indices, instead of being integers, are the "keys" that are on the left hand side of the rules.
2*Part[assoc1, "age"]
> 58
2*assoc1[["age"]]
> 58
A second way is to use Query. We can wrap the "key" in the head **Key** just to make sure Mathematica understands that the thing is a Key.
2*Query[Key["age"]][assoc1]
> 58
Usually we can omit the Key and everything works fine.
2*Query["age"][assoc1]
> 58
A third way is to write a function that has an association as its argument.
af = Function[Slot["age"]]
> "#age &"
Now look what we can do.
2*Query[af][assoc1]
> 58
We can shorten this approach by using a simpler syntax for a function.
2*Query[#age &][assoc1]
> 58
Note, though that this still will not work. Basically, Mathematica is confused. It thinks the function itself is the key.
2*assoc1[af]
> 2 Missing["KeyAbsent", #age &]
But here' s a simple workaround. For very simple functions, I can just use the name of the key.
2*assoc1["age"]
> 58
##A Note on Slot Arguments##
And please pay attention to this : sometimes the Mathematica parser gets confused when it confronts a "slot argument" written as #something. If you see this happening, write it as Slot["something"].
Slot["iamaslot"] === #iamaslot
> True
Here' s another problem. What if the key in the association has spaces or non-standard characters in it. Any of these, for example, are perfectly fine keys: the string "I have a lot of spaces in me", the string "I_have_underscores", the symbol True, the integer 43. But if we try to denote those keys by putting a hash in front of them, it will lead to confusion and problems.
problemAssociation = Association["I have a lot of spaces in me" -> 1, "I_have_underscores" -> 2, True -> 3, 43 -> 4]
> <|"I have a lot of spaces in me" -> 1, "I_have_underscores" -> 2,
True -> 3, 43 -> 4|>
{Query[#I have a lot of spaces in me &][problemAssociation],
Query[#I _have _underescores &][problemAssociation]}
![enter image description here][13]
Here' s a solution.
{Query[Slot["I have a lot of spaces in me"] &][problemAssociation],
Query[Slot["I_have_underscores"] &][problemAssociation]}
> {1, 2}
Here' s how we solve the use of True and an integer as keys. We preface them with **Key**.
{Query[#True &][problemAssociation], Query[#43 &][problemAssociation]}
![enter image description here][14]
{Query[Key[True]][problemAssociation],
Query[Key[43]][problemAssociation]}
> {3, 4}
##Working with Associations and Lists of Associations##
Here' s something we can do with the data in the form of an Association. I could ask for the gender of the person in the third row as follows. Notice I did not have to remember that "gender" was generally in the third position.
t3[[3]][["gender"]]
> "female"
So, even if I scramble the rows, I can still use the same code.
t3Scrambled = Map[convertListToAssociation, t[[All, {4, 1, 3, 2}]]]
![enter image description here][15]
t3Scrambled[[3]][["gender"]]
> female
I could also group the people according to their cabin class. Here I use Query on a list of Associations.
Query[GroupBy[#class &]][t3]
![enter image description here][16]
Again, the following code, which does not explicitly use **Query**, won' t work. Basically, nothing has told Mathematica to translate t3[stuff___] \[RightArrow]Query[stuff][t3]. If t3 had a head of Dataset, Mathematica would know to make the translation.
t3[GroupBy[#class &]]
![enter image description here][17]
I can also get certain values for all the Associations in a list of Associations.
Query[All, #age &][t3]
> {29, 30, 58, 52, 21, 54, 29, 42}
I can also map a function onto the result. I don't have to go outside the Query form to do so.
Query[f, #age &][t3]
> f[{29, 30, 58, 52, 21, 54, 29, 42}]
Or, without exiting the Query form, I can map a function onto each element of the result.
Query[Map[f], #age &][t3]
> {f[29], f[30], f[58], f[52], f[21], f[54], f[29], f[42]}
I could also do the same thing as follows.
Query[All, #age &, f][t3]
> {f[29], f[30], f[58], f[52], f[21], f[54], f[29], f[42]}
#Creating a Dataset from a List of Associations#
To get full use out of Query and to permit syntactic shorthands, we need for Mathematica to understand that the list of Associations is in fact a Dataset. Here' s all it takes.
d3 = Dataset[t3]
![enter image description here][18]
We can recover our original list of associations by use of the **Normal** command.
t3 === Normal[d3]
> True
With the data inside a Dataset object we now have pretty formatting. But we have more.
We can still do this. We get the same result but in a more attractive form.
d3g = Query[GroupBy[#class &]][d3]
![enter image description here][19]
But now this shorthand works too.
d3g = d3[GroupBy[#class &]]
![enter image description here][20]
And compare these two elements of code. When the data is in the form of a dataset, Mathematica understands that the stuff in the brackets is not intended as a key but rather is intended to be transformed into a Query.
{Query[#age &][t3[[1]]], d3[[1]][#age &]}
> {29, 29}
##A Dataset that is an Association of Associations##
Let' s look under the hood of **d3g**.
d3gn = Normal[d3g]
![enter image description here][21]
Note : if you *really* want to look under the hood of a **Dataset** ask to see the **Dataset** in **FullForm**. You can also get more information by running the undocumented package Dataset`, but this is definitely NOT recommended for the non-advanced user.
What we see is an Association in which each of the values is itself a list of Associations.
We can map a function over d3gn.
Map[f, d3gn]
![enter image description here][22]
I can of course do the mapping within the **Query** construct.
Query[All, f][d3gn]
![enter image description here][23]
If I try synactic sugar, it doesn' t work because d3gn is not a Dataset.
d3gn[All, f]
> Missing["KeyAbsent", All]
But, if I use the Dataset version, it does work. (The first line may be an ellipsis depending on your operating system and display, but if you look under the hood it looks just like the values for 2nd and 3rd. I have no idea why an ellipsis is being inserted.
d3g[All, f]
![enter image description here][24]
##A Dataset that just has a single Association inside.##
We can also have a Dataset that just has a single Association inside. Mathematica presents the information with the keys and values displayed vertically.
Dataset[d3[[1]]]
![enter image description here][25]
In theory, we could have a Dataset that just had a single number inside it.
Dataset[6]
![enter image description here][26]
#Nice queries with Dataset#
Now I can construct a query that takes a dataset and groups it by the gender column. It then takes each grouping and applies the Mean function to at least part of it. What part? The "age" column part. Notice that I no longer have to remember that gender is the third column and age is the second column.
qd = Query[GroupBy[#gender &], Mean, #age &]
> Query[GroupBy[#gender &], Mean, #age &]
Now I can run this query on t3.
qd[d3]
![enter image description here][27]
We can now learn a lot about Query. So long as our data is in the form of a Dataset we can write the query as either a formal Query or use syntactic sugar.
#Query#
A major part of working with data is to understand **Query**. Let's start with a completely abstract **Query**, that we will call **q1**.
q1 = Query[f];
Now let' s run q1 on t3.
q1[t3]
![enter image description here][28]
We end up with a list of Associations that has f wrapped around it at the highest level. It's the same as if I wrote the following code.
f[t3] === q1[t3]
> True
Now, let' s write a **Query** that applies the function g at the top level of the list of associations and the function **f** at the second level, i.e. to each of the rows. Why does it work at the second level? Because it's the second argument to **Query**.
q2 = Query[g, f];
q2[t3]
![enter image description here][29]
The result is the same as if I mapped **f** onto t3 at its first level and then wrapped **g** around it.
g[Map[f, t3, {1}]] === q2[d3]
Query[All, MapAt[StringTake[#, 1] &, #, {{"class"}, {"gender"}}] &][d3]
Here' s a function **firstchar** that takes the first character in a string.
firstchar = StringTake[#, 1] &
> StringTake[#1, 1] &
Now, let' s construct a query **cg1** that applies **firstchar** to the class and gender keys in each row.
cg1 = Query[All,
a \[Function] MapAt[firstchar, a, {{"class"}, {"gender"}}]]
> Query[All, Function[a, MapAt[firstchar, a, {{"class"}, {"gender"}}]]]
We apply **cg1** to our little dataset **d3**.
cg1[d3]
![enter image description here][30]
What if we want to apply the same function to every element of the Dataset. We just apply it at the lowest level. Here's one way.
Query[Map[f, #, {-1}] &][d3]
![enter image description here][31]
We can also combine it with column wise and entirety wise operations. For reasons that are not clear, Mathematica can't understand this as a Dataset and returns the Normal form.
Query[(Map[f, #, {-1}] &) /* entiretywise, columnwise][d3]
![enter image description here][32]
Here' s how we could actually a multilevel **Query**.
Suppose we want to write a function that computes the fraction of the people in this little dataset that survived. The first step is simply going to be to extract the survival value and convert it to 1 if True and 0 otherwise. There's a built in function Boole that does this.
{Boole[True], Boole[False]}
> {1, 0}
q3 = Query[something,
assoc \[Function] assoc["survived"] /. {True -> 1, _ -> 0}]
> Query[something, Function[assoc, assoc["survived"] /. {True -> 1, _
> -> 0}]]
q3[t3]
> something[{1, 0, 1, 1, 1, 0, 0, 0}]
So, now we have something wrapping a list of 1 s and 0 s. By making **something** the **Mean** function, we can achieve our result.
q4 = Query[Mean, Boole[#survived] &]
> Query[Mean, Boole[#survived] &]
q4[d3]
> 1/2
We can also examine survival by gender. Notice that **Query** is a little like **Association**: it gets automatically flattened.
Query[GroupBy[#gender &], q4][t3]
> <|"female" -> 4/5, "male" -> 0|>
If the data is held in a **Dataset**, we can also write the final step as follows.
d3[GroupBy[#gender &], q4]
![enter image description here][33]
Notice that even if we omit the "Query", this code works. Mathematica just figures out that you meant Query.
The code immediately above is in the form we typically see and often use.
#Some Recipes#
titanic = ExampleData[{"Dataset", "Titanic"}]
![enter image description here][34]
How to add a value to the Dataset based on values external to the existing columns.
Here' s some additional data. Notice that the data is the same length as the titanic dataset.
stuffToBeAdded =
Table[Association["id" -> i,
"weight" -> RandomInteger[{80, 200}]], {i, Length[titanic]}]
![enter image description here][35]
We use **Join** at level 2.
augmentedTitanic = Join[titanic, stuffToBeAdded, 2]
![enter image description here][36]
##How to add a column to a Dataset based on values in the existing columns and to do so row-wise##
Notice that the query below does NOT change the value of the titanic dataset. To change the value of the titanic dataset, one would need to set titanic to the result of the computation. Remember, Mathematica generally does not have side effects or do modifications in place.
Query[All, Association[#, "classsex" -> {#class, #sex}] &][titanic]
![enter image description here][37]
We can add multiple columns this way.
Query[All,
Association[#, "classsex" -> {#class, #sex},
"agesqrt" -> Sqrt[#age]] &][titanic]
![enter image description here][38]
##How to change the value of an existing column : row - wise##
Age everyone one year.
Query[All, Association[#, "age" -> #age + 1] &][titanic]
![enter image description here][39]
How to change the value of columns selectively.
Query[All,
Association[#,
"age" -> If[#sex === "male", #age + 1, #age]] &][titanic]
![enter image description here][40]
How to create a new column based on some aggregate operator applied to another column.
With[{meanAge = Query[Mean, #age &][titanic]},
Query[All,
Association[#, "ageDeviation" -> #age - meanAge] &]][titanic]
![enter image description here][41]
Can you develop your own recipes?
[1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=17751.png&userId=20103
[2]: http://community.wolfram.com//c/portal/getImageAttachment?filename=47222.png&userId=20103
[3]: http://community.wolfram.com//c/portal/getImageAttachment?filename=60813.png&userId=20103
[4]: http://community.wolfram.com//c/portal/getImageAttachment?filename=83664.png&userId=20103
[5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=44435.png&userId=20103
[6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=105416.png&userId=20103
[7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=49777.png&userId=20103
[8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=16898.png&userId=20103
[9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=28239.png&userId=20103
[10]: http://community.wolfram.com//c/portal/getImageAttachment?filename=262710.png&userId=20103
[11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=749611.png&userId=20103
[12]: http://community.wolfram.com//c/portal/getImageAttachment?filename=209912.png&userId=20103
[13]: http://community.wolfram.com//c/portal/getImageAttachment?filename=925313.png&userId=20103
[14]: http://community.wolfram.com//c/portal/getImageAttachment?filename=851614.png&userId=20103
[15]: http://community.wolfram.com//c/portal/getImageAttachment?filename=627315.png&userId=20103
[16]: http://community.wolfram.com//c/portal/getImageAttachment?filename=679516.png&userId=20103
[17]: http://community.wolfram.com//c/portal/getImageAttachment?filename=674717.png&userId=20103
[18]: http://community.wolfram.com//c/portal/getImageAttachment?filename=707518.png&userId=20103
[19]: http://community.wolfram.com//c/portal/getImageAttachment?filename=1053319.png&userId=20103
[20]: http://community.wolfram.com//c/portal/getImageAttachment?filename=579820.png&userId=20103
[21]: http://community.wolfram.com//c/portal/getImageAttachment?filename=980221.png&userId=20103
[22]: http://community.wolfram.com//c/portal/getImageAttachment?filename=450322.png&userId=20103
[23]: http://community.wolfram.com//c/portal/getImageAttachment?filename=113723.png&userId=20103
[24]: http://community.wolfram.com//c/portal/getImageAttachment?filename=932624.png&userId=20103
[25]: http://community.wolfram.com//c/portal/getImageAttachment?filename=754825.png&userId=20103
[26]: http://community.wolfram.com//c/portal/getImageAttachment?filename=836826.png&userId=20103
[27]: http://community.wolfram.com//c/portal/getImageAttachment?filename=393227.png&userId=20103
[28]: http://community.wolfram.com//c/portal/getImageAttachment?filename=577928.png&userId=20103
[29]: http://community.wolfram.com//c/portal/getImageAttachment?filename=158329.png&userId=20103
[30]: http://community.wolfram.com//c/portal/getImageAttachment?filename=984930.png&userId=20103
[31]: http://community.wolfram.com//c/portal/getImageAttachment?filename=664831.png&userId=20103
[32]: http://community.wolfram.com//c/portal/getImageAttachment?filename=1090332.png&userId=20103
[33]: http://community.wolfram.com//c/portal/getImageAttachment?filename=450733.png&userId=20103
[34]: http://community.wolfram.com//c/portal/getImageAttachment?filename=740234.png&userId=20103
[35]: http://community.wolfram.com//c/portal/getImageAttachment?filename=143835.png&userId=20103
[36]: http://community.wolfram.com//c/portal/getImageAttachment?filename=36.png&userId=20103
[37]: http://community.wolfram.com//c/portal/getImageAttachment?filename=37.png&userId=20103
[38]: http://community.wolfram.com//c/portal/getImageAttachment?filename=38.png&userId=20103
[39]: http://community.wolfram.com//c/portal/getImageAttachment?filename=39.png&userId=20103
[40]: http://community.wolfram.com//c/portal/getImageAttachment?filename=40.png&userId=20103
[41]: http://community.wolfram.com//c/portal/getImageAttachment?filename=41.png&userId=20103Seth Chandler2017-08-20T19:26:30Z