https://community.wolfram.com RSS Feed for Wolfram Community showing any discussions tagged with Operations Research sorted by most replies. https://community.wolfram.com/groups/-/m/t/478669 Here's today's New York Times 4x4 KenKen solved in Mathematica. The approach is to first construct all possible arrays with different numbers in each row, from that select all arrays that have different numbers in each column, then apply the constraints on various cells. For this particular example, it was not necessary to apply all the constraints to get the unique answer. It is necessary to take the absolute value of some difference and ratio constraints, since it the order of the difference or ratio is not known. Dimensions[p4 = Permutations[Range[4]]] Out[1]= {24, 4} Dimensions[a1 = Flatten[Outer[List, p4, p4, p4, p4, 1], 3]] Out[2]= {331776, 4, 4} Dimensions[ a2 = Select[ a1, (And @@ (And @@ Unequal[Sequence @@ #] &) /@ Transpose[#]) &]] Out[3]= {576, 4, 4} Dimensions[a3 = Select[a2, #[[1, 1]]*#[[1, 2]]*#[[2, 2]] == 12 &]] Out[4]= {112, 4, 4} Dimensions[a4 = Select[a3, Abs[Log[#[[1, 3]]/#[[1, 4]]]] == Log[2] &]] Out[5]= {64, 4, 4} In[6]:= Dimensions[ a5 = Select[ a4, #[[2, 1]] + #[[3, 1]] + #[[3, 2]] + #[[4, 1]] == 11 &]] Out[6]= {8, 4, 4} In[7]:= Dimensions[a6 = Select[a5, Abs[#[[2, 3]] - #[[3, 3]]] == 2 &]] Out[7]= {6, 4, 4} In[8]:= Dimensions[a7 = Select[a6, #[[2, 4]] == 2 &]] Out[8]= {1, 4, 4} In[10]:= a7 Out[10]= {{{1, 3, 2, 4}, {3, 4, 1, 2}, {4, 2, 3, 1}, {2, 1, 4, 3}}} Frank Kampas 2015-04-13T20:56:51Z https://community.wolfram.com/groups/-/m/t/178439 Hi, everybody, I would like to ask you guys a little question. I have three maximization problems, they are Q1, Q2 and Q3, respectively, as you can see below:[mcode]Clear["`*"]; d2 = a - b*p2;Q1 = Maximize[{(p2*(a - b*p2) - (w2*(a - b*p2) - (v2*((a - b*p2))^2)/ 2)), p2 >= a/b - (w2 - c)/(b*v2) && c < w2 < (a - b*c)*v2 + c && a/b > p2 > w2 > 0 && 2/b > v2 > 0}, {p2}]Q2 = Maximize[{(p2*d2 - (w2*d2 - (v2*(d2)^2)/2)), p2 >= a/b - (w2 - c)/(b*v2) && (a - b*c)*v2 + c <= w2 < a/b && 2/b > v2 > 0 && a/b > p2 > w2 > 0}, {p2}]Q3 = Maximize[{(p2* d2 - (((w2)^2 - c^2)/(2*v2) + c*(d2 - (w2 - c)/v2))), p2 <= a/b - (w2 - c)/(b*v2) && 0 < c < w2 < (a - b*c)*v2 + c && 2/b > v2 > 0 && a/b > p2 > w2 > 0}, {p2}][/mcode]For Q1, I want to find the optimal p2 to maximize the first expression, i.e., p2*(a - b*p2) - (w2*(a - b*p2) - (v2*((a - b*p2))^2)/ 2)) , the same for Q2 and Q3.  In fact, it is easy to get the results for them respectively. However, my final purpose is going to find the global optimality, which means I have to compare the first expressions of the three problems. Therefore, I tried to use command Max[Q1, Q2, Q3], the result is odd. So there are two important questions: Firstly, how to pick up the optimal value of the first expressions from Q1, Q2 and Q3; secondly, which comand can be used to solve this problem of comparison since "Max" doesn't work at all. Thank you very much in advance.  Shaoyan Robert 2014-01-04T02:50:48Z https://community.wolfram.com/groups/-/m/t/1703087 Hello community. I have created a function and would like to know if it is worth submitting in Function Repository or if there is already something simpler that does this same job? If anyone can give any opinion on this I will be very grateful. I modestly have created a function that can interleave two lists by alternating their terms (unlike Riffle, which only fits the terms into gaps, this function does this keeping the same number of terms as it replaces them by both functions simultaneously). - It works like this: If the third term inside the function ("c_") is {} the function does this automatically in a 1 to 1 pattern of each group: Alternate[a_, b_] := Alternate[a, b, {}] Alternate[a_, b_, c_] := PadRight[a* PadRight[ Take[Flatten@Table[If[c == {}, {1, 0}, c], Count[a, _]], Min[Count[a, _], Count[b, _]]], Count[a, _], 1], Max[Count[a, _], Count[b, _]]] + PadRight[b* PadRight[ Take[Flatten@Table[Abs[If[c == {}, {1, 0}, c] - 1], Count[b, _]], Min[Count[a, _], Count[b, _]]], Count[b, _], 1], Max[Count[a, _], Count[b, _]]] r = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20}; s = {3, 5, 7, 9, 11, 13, 15, 17, 19, 21}; Alternate[r, s] Alternate[s, r] ![ie1][1] The function works even with lists of different sizes, keeping the terms in excess unchanged: p = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20}; q = {3, 5, 7, 9, 11, 13, 15, 17}; Alternate[p, q] Alternate[q, p] ![ie2][2] Or you can change the third term ("c") in the function to any pattern (eg: {0,1,1,1}). Where "1" refers to the first term ("a") from within the function while "0" refers to the term ("b") from within the function: t = {2, 4, 6, 8, 10, 12, 14, 16, 18, 20}; u = {3, 5, 7, 9, 11, 13, 15, 17}; Alternate[t, u, {0, 1, 1, 1}] Alternate[u, t, {0, 1, 1, 1}] ![ie3][3] I would like to know if is this a good idea or there is a simpler way to do this? Is it worth sending a repository request? Thank you. [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=ie1.png&userId=1316061 [2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=ie2.png&userId=1316061 [3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=ie3.png&userId=1316061 Claudio Chaib 2019-06-12T19:48:51Z https://community.wolfram.com/groups/-/m/t/488913 Hi, I'm solving a non-linear system of equations with FindRoot. The system is very sparse, if I can say that about a non-linear system. After a few seconds of computation, Mathematica outputs the answer and gives me the error FindRoot::lstol: The line search decreased the step size to within tolerance specified by AccuracyGoal and PrecisionGoal but was unable to find a sufficient decrease in the merit function. You may need more than MachinePrecision digits of working precision to meet these tolerances. >> I have played around with DampingFactor, WorkingPrecision and AccuracyGoal to no avail: When I verify the solutions that Mathematica gives me, at best about 1/3 of them are not satisfied. Can you help me figure out what should I be looking at? Miguel Olivo-V 2015-04-30T05:55:25Z https://community.wolfram.com/groups/-/m/t/1895675 *MODERATOR NOTE: coronavirus resources & updates:* https://wolfr.am/coronavirus &[Wolfram Notebook][1] [1]: https://www.wolframcloud.com/obj/75ec2a4b-b7bd-4c7f-9a14-6e543085fc0e Anton Antonov 2020-03-10T20:51:05Z https://community.wolfram.com/groups/-/m/t/1065416 Is anyone interested in and actively using Wolfram to do Deep Learning research? I am doing research in all areas of Deep Learning with my focus currently on VGG style of convolution neural networks (CNN). I will be branching out very soon to RLN, RNN, and GAN research very soon. I would like to be able to compare experiences and lessons learned using Wolfram in these areas. Thanks for any interest in discussions on Deep Learning. Bryan Minor 2017-04-18T17:29:38Z https://community.wolfram.com/groups/-/m/t/251988 I'm trying to use mathematica to solve this linear integer program but I keep getting the input as the output. I deleted like 9 lines of the constraints to make it fit to a page. kiet ngo 2014-05-14T21:44:52Z https://community.wolfram.com/groups/-/m/t/169161 I wondered if FindMinimum always converges to the nearest minimum, so I minimized sin(x) cos(y)  with random starting points with x and y between -6 and 6 and colored the points by the location of the minimum. [img=width: 360px; height: 359px;]/c/portal/getImageAttachment?filename=chaoticmin.gif&userId=29126[/img] Frank Kampas 2013-12-12T13:36:07Z https://community.wolfram.com/groups/-/m/t/394537 Hello all, I am writing to tell you my idea. The process of splitting up a triangle either in four smaller triangles, do so by the larger angle as shown below: Graphics[Line[{{0, 0}, {7, -5}, {8, 10}, {0, 0}}]] After analyze the angles of the triangle we see that the larger angle is what is at the point {0,0}, subsequently we construct a straight line to the midpoint of the opposite side to the point {0,0 } Finally we estimate the midpoints of the sides adjacent to the point {0,0 } and the join by a straight line to the midpoint of the opposite side that we calculated above ,for finally getting the following figure: ![first step][1] If we do the same steps with each of the four triangles, we will be able to obtain more triangles 16 in total ![second step][2] Can anyone help me to do what I say, I have made some progress but I am having some problems in order to apply it several times on a initial triangle [1]: /c/portal/getImageAttachment?filename=montanafr.bmp&userId=133435 [2]: /c/portal/getImageAttachment?filename=montanagh.bmp&userId=133435 Luis Ledesma 2014-11-22T19:26:37Z https://community.wolfram.com/groups/-/m/t/598249 Packing ellipses into a circumscribing circle is more complex than packing circles, since there is no formula for the minimum distance between two ellipses or the maximum distance from the center of the circumscribing circle to an ellipse. This is overcome by embedding constraints using Lagrange multipliers into the problem to locate the most distant point from the origin of an ellipse and the points on pairs of ellipses closest to each other. A result of a packing, showing the ellipses and the determined points is shown here: ![enter image description here][1] [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=EllipsePackingUsingEmbeddedLagrangeMultipliers.jpg&userId=29126 The complete calculation is in the attached notebook. The optimization was carried out as a local search using MathOptimizer Professional http://www.wolfram.com/products/applications/mathoptpro as FindMinimum does not converge for the calculation. Frank Kampas 2015-10-26T13:32:54Z https://community.wolfram.com/groups/-/m/t/1907256 # Introduction Last weekend -- 2020-03-20 ÷ 2020-03-22 -- I participated in [the (Germany-centric) hackathon WirVsVirus](https://wirvsvirushackathon.org). (I friend of mine who lives in Germany asked me to team up and sign up. Participated together with Balint Badonfai and Diego Zviovich.) [Our idea proposal](https://github.com/antononcube/SystemModeling/blob/master/Projects/Coronavirus-propagation-dynamics/org/WirVsVirus-Hackathon-work-plan.org) was accepted, [listed in the dedicated overview table (see item 806)](https://airtable.com/shrs71ccUVKyvLlUA/tbl6Br4W3IyPGk1jt/viw7AlEju6qFtXJqL?blocks=hide). The title of our hackathon project is: **“Geo-spatial-temporal Economic Model for COVID-19 Propagation and Management in Germany”** [Nearly a dozen of people enlisted to help](https://devpost.com/software/geo-raumlich-zeitliches-wirtschaftsmodell-fur-covid-19). (We communicated through Slack.) ```mathematica WebImage["https://devpost.com/software/geo-raumlich-zeitliches-\ wirtschaftsmodell-fur-covid-19"] ```  Multiple people helped with the discussion of ideas, directions where to find data, with actual data gathering, and related documented analysis. Of course, just discussing the proposed solutions was already a great help! # What was accomplished ## Work plans The following mind-map reflects pretty well what was planned and done:  There is also a related [org-mode file with the work plan](https://github.com/antononcube/SystemModeling/blob/master/Projects/Coronavirus-propagation-dynamics/org/WirVsVirus-Hackathon-work-plan.org). ## Data I obtained Germany city data with Mathematica’s build-in functions and used it to heuristically derive a traveling patterns graph, [AA1]. Here is the data: ```mathematica dsCityRecords = ResourceFunction["ImportCSVToDataset"][ "https://raw.githubusercontent.com/antononcube/SystemModeling/master/Data/dfGermanyCityRecords.csv"]; Dimensions[dsCityRecords] (*{12538, 6}*) ``` Here is Geo-histogram of that data: ```mathematica cellRadius = Quantity[70, "Kilometers"]; aCoordsToPopulations = AssociationThread[Values /@ Normal[dsCityRecords[All, {"Lat", "Lon"}]], Normal[dsCityRecords[All, "Population"]]]; GeoHistogram[aCoordsToPopulations, cellRadius, ColorFunction -> (Opacity[#, Blue] &), PlotLegends -> Automatic] ```  We considered a fair amount of other data. But because of the time limitations of the hackathon we had to use only the one above. ## Single-site models During the development phase I used the model SEI2R, but since we wanted to have a “geo-spatial-temporal epidemiological economics model” I productized the implementation of SEI2HR-Econ, [AAp1]. Here are the stocks, rates, and equations of SEI2HR-Econ: ```mathematica Magnify[ModelGridTableForm[SEI2HREconModel[t]], 0.85] ```  ## Multi-site SEI2R (SEI2HR-Econ) over a hexagonal grid graph I managed to follow through with a large part of [the work plan for the hackathon](https://github.com/antononcube/SystemModeling/blob/master/Projects/Coronavirus-propagation-dynamics/org/WirVsVirus-hackathon-Geo-spatial-temporal-model-mind-map.pdf) and make multi-site scaled model that “follows the money”, [[AA1](https://github.com/antononcube/SystemModeling/blob/master/Projects/Coronavirus-propagation-dynamics/Documents/WirVsVirus-hackathon-Multi-site-SEI2R-over-a-hexagonal-grid-graph.md)]. Here is a diagram that shows the travelling patterns graph and solutions at one of the nodes:  Here is an interactive interface for studying and investigating the solution results:  For more details see the notebook [[AA1](https://github.com/antononcube/SystemModeling/blob/master/Projects/Coronavirus-propagation-dynamics/Documents/WirVsVirus-hackathon-Multi-site-SEI2R-over-a-hexagonal-grid-graph.md)]. Different parameters can be set in the “Parameters” section. Especially of interest are the quarantine related parameters: start, duration, effect on contact rates and traffic patterns. I also put simulations results code and programmed visualization routines in R, [AA2]. (In order other members of team to be able to explore the results.) # References \[DP1\] [47_wirtschaftliche Auswirkung_Geo-spatial-temp-econ-modell](https://devpost.com/software/geo-raumlich-zeitliches-wirtschaftsmodell-fur-covid-19), DevPost. \[WRI1\] Wolfram Research, Inc., [Germany city data records](https://github.com/antononcube/SystemModeling/blob/master/Data/dfGermanyCityRecords.csv), (2020), [SystemModeling at GitHub](https://github.com/antononcube/SystemModeling). \[AA1\] Anton Antonov, ["WirVsVirus hackathon multi-site SEI2R over a hexagonal grid graph"](https://github.com/antononcube/SystemModeling/blob/master/Projects/Coronavirus-propagation-dynamics/Documents/WirVsVirus-hackathon-Multi-site-SEI2R-over-a-hexagonal-grid-graph.md), (2020), [SystemModeling at GitHub](https://github.com/antononcube/SystemModeling). \[AA2\] Anton Antonov, ["WirVsVirus-Hackathon in R"](https://github.com/antononcube/SystemModeling/tree/master/Projects/Coronavirus-propagation-dynamics/R/WirVsVirus-Hackathon), (2020), [SystemModeling at GitHub](https://github.com/antononcube/SystemModeling). \[AAp1\] Anton Antonov, ["Epidemiology models Mathematica package"](https://github.com/antononcube/SystemModeling/blob/master/Projects/Coronavirus-propagation-dynamics/WL/EpidemiologyModels.m), (2020), [SystemsModeling at GitHub](https://github.com/antononcube/SystemModeling). Anton Antonov 2020-03-24T21:59:30Z https://community.wolfram.com/groups/-/m/t/1762584 <h2>Introduction</h2> On Friday 9th August 2019, Britain suffered one of the worst power outages in recent years. Over 1 million people in England and Wales were cut off from power for nearly 9 hours, preventing numerous rail services from running during the rush hour. ![Photograph of the distruption in King's Cross station][1] Was this a statistical misfortune, or a systemic failure which caused the recent power outages? <h2>Getting the data</h2> To obtain the information required to do analysis of the events leading up to the catastrophic power outage, we will we will use a fantastic website, [GridWatch Templar][3], which has realtime data as to the different power supplies, resources and frequencies. From the download page, I downloaded all the data from 27th May 2011 to the current time. Be patient; it will take a considerable amount of time to download the entire dataset (and as it is too large, I have not attached this csv file to this page). Also, as we will be carrying out analysis on such a large dataset, some cells will take a considerable amount of time to run. <h2>Analysis of the frequencies</h2> First, we import the dataset: ``` In[1]: data = Import["D:\\Programming\\GridWatch\\gridwatch.csv"]; ``` The data which it contains: ``` In[2]: data[[1]] Out[2]: {"id", "timestamp", "demand", "frequency", "coal", "nuclear", "ccgt", \ "wind", "pumped", "hydro", "biomass", "oil", "solar", "ocgt", \ "french_ict", "dutch_ict", "irish_ict", "ew_ict", "nemo", "other", \ "north_south", "scotland_england"} ``` To look at the frequencies: ``` In[3]: frequencies = If[#[[4]] == 0, Nothing, {DateObject[#[[2]]], #[[4]]}] & /@ data[[2 ;;]]; ``` To show the minimum value: ``` In[4]: MinimalBy[frequencies, Last] Out[4]: {{DateObject[{2019, 8, 9, 15, 55, 37}, "Instant", "Gregorian", 1.], 48.889}} In[5]: DateListPlot[frequencies, PlotRange -> All, AxesLabel -> {"Time", "Hz"}] ``` ![The date plot][4] This is not particularly enlightening; although it does show how dramatic the recent drop of frequency is. In fact, it is known that if the frequency drops below 49.5 Hz, a blackout will occur - and this is the first time it has happened (at least since 2011). Let's dive deeper - we'll use a normal distribution in approximation to the distribution: ``` In[6]: distribution = NormalDistribution[Mean[Last /@ frequencies], StandardDeviation[Last /@ frequencies]] Out[6]: NormalDistribution[50.0012, 0.0584802] ``` Now plotting it gives: ``` In[7]: Show[Histogram[Last /@ newfreqs, Automatic, "ProbabilityDensity"], Plot[PDF[distribution, x], {x, 49.7, 50.3}, PlotStyle -> Thick]] ``` This returns: ![histogram][5] Observe how quickly this tapers off. Assuming the frequencies follow that normal distribution, the probability of this occurring is: ``` In[8]: Probability[freq <= 49.5, freq \[Distributed] distribution] Out[8]: 5.1191*10^-18 ``` So we can be very confident that this was not just a statistical mishap - there was a genuine, systemic cause for this event to occur (see the comment below for explanation of this). Now let's take a closer look at what happened that day. <h2>Focusing in further</h2> A National Grid spokesperson said: > The root cause of yesterday’s issue was not with our system but was a rare and unusual event, the almost simultaneous loss of two large generators, one gas and one offshore wind, at 4.54pm. We are still working with the generators to understand what caused the generation to be lost. (courtesy of the [the Guardian][2] for this report). We will verify this report below. First, we will zoom in on the interval between one day before and one day after the event occurred. ``` In[9]: timeinterval = Select[data[[2;;]], DateObject[{2019, 8, 9, 15, 55, 37}, "Instant", "Gregorian", 1.`] - Quantity[1, "Days"] <= DateObject[#[[2]]] <= DateObject[{2019, 8, 9, 15, 55, 37}, "Instant", "Gregorian", 1.`] + Quantity[1, "Days"] &] ``` (This specific time interval data is attached to this post). To visualise it the frequencies: ``` In[11]: timeintervalfreqs = {DateObject[#[[2]]], #[[4]]} & /@ timeinterval; In[12]: DateListPlot[Out[51], PlotRange -> All, Epilog -> {Red, Text[Style["\[SixPointedStar]", 20], #] & /@ {{DateObject[{2019, 8, 9, 15, 55, 37}, "Instant", "Gregorian", 1.`], 48.889`}}}] ``` which shows: ![2 day interval][6] Now let's look at the specific types of energy generation: ``` In[13]: DateListPlot[ Table[{DateObject[#[[2]]], #[[k]]} & /@ timeinterval, {k, 5, 21}], PlotLegends -> data[[1]][[5 ;;]], PlotRange -> All] ``` which returns ![nuclear, coal etc.][7] Now let's reduce down to the two energy sources which were mentioned in the report: namely, gas (or CCGT, OCGT), and offshore wind. ``` In[14]: indices = {3, 7, 8, 14}; DateListPlot[ Table[{DateObject[#[[2]]], #[[k]]} & /@ timeinterval, {k, indices}], PlotLegends -> data[[1]][[indices]], PlotRange -> All, Epilog -> {Directive[{Thick, Red, Dashed}], Line[{{DateObject[{2019, 8, 9, 15, 55, 37}, "Instant", "Gregorian", 1.`], 0}, {DateObject[{2019, 8, 9, 15, 55, 37}, "Instant", "Gregorian", 1.`], 40000}}]}] ``` ![graph_6][8] This clearly shows a drop in CCGT and wind power (and also the demand) subsequent to the incident. Finding peaks for the CCGT data to find when the drop occurs: ``` In[15]: ccgtdata = {AbsoluteTime[#[[2]]], -#[[8]]} & /@ timeinterval; In[16]: inter = Interpolation[ccgtdata[[All, 1]]]; In[17]: peaks = FindPeaks[ccgtdata[[All, 2]]]; In[18]: peak = Select[{inter[#1], #2} & @@@ peaks, #[[1]] >= AbsoluteTime[ DateObject[{2019, 8, 9, 15, 55, 37}, "Instant", "Gregorian", 1.`]] &][[1, 1]] (*This calculates the first trough since the event took place*) Out[18]: 3774356135 ``` This means that, 20 minutes after the frequency collapses, the CCGT (and the Wind power) supply decrease dramatically. <h2>Conclusion</h2> At 3:55 PM GMT +1 (so really 4:55 PM, confirming the report - this is on account of BST) on Friday the 9th of August, for the first time since 2011 (and probably many years before), frequency reached levels below 49.5 Hz, at which point blackouts would take place, affecting nearly 1 million people and disrupting multiple rail services across England and Wales. According to the National Grid, this had almost happened 3 times before this one took place; our data can confirm this (see the graph below input 5). We can be certain that this was not a statistical oddity; in fact, around 20 minutes later, we begin to see that the CCGT and offshore wind supplies decrease, which confirms the report stating that both had cut out simultaneously. Hopefully this will never happen again. With this dataset at hand, it seems likely that these outages could be more foreseen in the future. [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=image1.jpg&userId=1719931 [2]: https://www.theguardian.com/business/2019/aug/10/uk-energy-watchdog-demands-answers-after-major-power-cut-england-wales [3]: https://gridwatch.templar.co.uk/ [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=graph_1.jpg&userId=1719931 [5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=graph_2.jpg&userId=1719931 [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=graph_4.jpg&userId=1719931 [7]: https://community.wolfram.com//c/portal/getImageAttachment?filename=graph_3.PNG&userId=1719931 [8]: https://community.wolfram.com//c/portal/getImageAttachment?filename=graph_6.PNG&userId=1719931 Henry Jaspars 2019-08-13T18:25:04Z https://community.wolfram.com/groups/-/m/t/1736350 I would be grateful if someone could help me with this code. I want to check whether all the entries in any column of the generated table is equal to one If the condition is satisfied in any column, the result shall be the column's heading. If the condition is not satisfied in all columns, the result shall be none. For example, I am attaching a file that outputs a table. This table contains no columns with all values are equal to one, so I am expecting the result to be none. Nehal Elshaboury 2019-07-13T17:13:12Z https://community.wolfram.com/groups/-/m/t/941017 ## Introduction This post is for the package \[[2](https://github.com/antononcube/MathematicaForPrediction/blob/master/ROCFunctions.m)\] that provides *Mathematica* implementations of [Receiver Operating Characteristic](https://en.wikipedia.org/wiki/Receiver_operating_characteristic) (ROC) functions calculation and plotting. The ROC framework is used for analysis and tuning of binary classifiers, \[[3](https://en.wikipedia.org/wiki/Receiver_operating_characteristic)\]. (The classifiers are assumed to classify into a positive/true label or a negative/false label. ) The function `ROCFuntions` gives access to the individual ROC functions through string arguments. Those ROC functions are applied to special objects, called ROC Association objects. Each ROC Association object is an `Association` that has the following four keys: "TruePositive", "FalsePositive", "TrueNegative", and "FalseNegative" . Given two lists of actual and predicted labels a ROC Association object can be made with the function `ToROCAssociation` . For more definitions and example of ROC terminology and functions see \[[3](https://en.wikipedia.org/wiki/Receiver_operating_characteristic),[4](https://ccrma.stanford.edu/workshops/mir2009/references/ROCintro.pdf)\]. ## Minimal example Note that here although we use both of the provided Titanic training and test data, the code is doing only training. The test data is used to find the best tuning parameter (threshold) through ROC analysis. ### Get packages These commands load the packages \[[1](https://github.com/antononcube/MathematicaForPrediction/blob/master/MathematicaForPredictionUtilities.m),[2](https://github.com/antononcube/MathematicaForPrediction/blob/master/ROCFunctions.m)\]: Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/MathematicaForPredictionUtilities.m"] Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/ROCFunctions.m"] ### Using Titanic data Here is the summary of the Titanic data used below: titanicData = (Flatten@*List) @@@ExampleData[{"MachineLearning", "Titanic"}, "Data"]; columnNames = (Flatten@*List) @@ExampleData[{"MachineLearning", "Titanic"}, "VariableDescriptions"]; RecordsSummary[titanicData, columnNames] [![Titanic1][1]][1] This variable dependence grid shows the relationships between the variables. Magnify[#, 0.7] &@VariableDependenceGrid[titanicData, columnNames] [![VariableDependencies][7]][7] ### Get training and testing data data = ExampleData[{"MachineLearning", "Titanic"}, "TrainingData"]; data = ((Flatten@*List) @@@ data)[[All, {1, 2, 3, -1}]]; trainingData = DeleteCases[data, {___, _Missing, ___}]; Dimensions[trainingData] (* {732, 4} *) data = ExampleData[{"MachineLearning", "Titanic"}, "TestData"]; data = ((Flatten@*List) @@@ data)[[All, {1, 2, 3, -1}]]; testData = DeleteCases[data, {___, _Missing, ___}]; Dimensions[testData] (* {314, 4} *) ### Replace categorical with numerical values trainingData = trainingData /. {"survived" -> 1, "died" -> 0, "1st" -> 0, "2nd" -> 1, "3rd" -> 2, "male" -> 0, "female" -> 1}; testData = testData /. {"survived" -> 1, "died" -> 0, "1st" -> 1, "2nd" -> 2, "3rd" -> 3, "male" -> 0, "female" -> 1}; ### Do linear regression lfm = LinearModelFit[{trainingData[[All, 1 ;; -2]], trainingData[[All, -1]]}] [![Regression1][2]][2] ### Get the predicted values modelValues = lfm @@@ testData[[All, 1 ;; -2]]; Histogram[modelValues, 20] [![Prediction1][3]][3] RecordsSummary[modelValues] [![Prediction2][4]][4] ### Obtain ROC associations over a set of parameter values testLabels = testData[[All, -1]]; thRange = Range[0.1, 0.9, 0.025]; aROCs = Table[ToROCAssociation[{1, 0}, testLabels, Map[If[# > \[Theta], 1, 0] &, modelValues]], {\[Theta], thRange}]; ### Evaluate ROC functions for given ROC association N @ Through[ROCFunctions[{"PPV", "NPV", "TPR", "ACC", "SPC", "MCC"}][aROCs[[3]]]] (* {0.513514, 0.790698, 0.778689, 0.627389, 0.53125, 0.319886} *) ### Standard ROC plot ROCPlot[thRange, aROCs, "PlotJoined" -> Automatic, "ROCPointCallouts" -> True, "ROCPointTooltips" -> True, GridLines -> Automatic] [![ROCPlot1][5]][5] ### Plot ROC functions wrt to parameter values rocFuncs = {"PPV", "NPV", "TPR", "ACC", "SPC", "MCC"}; rocFuncTips = Map[# <> ", " <> (ROCFunctions["FunctionInterpretations"][#]) &, rocFuncs]; ListLinePlot[ MapThread[Tooltip[Transpose[{thRange, #1}], #2] &, {Transpose[Map[Through[ROCFunctions[rocFuncs][#]] &, aROCs]], rocFuncTips}], Frame -> True, FrameLabel -> Map[Style[#, Larger] &, {"threshold, \[Theta]", "rate"}], PlotLegends -> rocFuncTips, GridLines -> Automatic] [![ROCPlot2][6]][6] ### Finding the intersection point of PPV and TPR We want to find a point that provides balanced positive and negative labels success rates. One way to do this is to find the intersection point of the ROC functions PPV (positive predictive value) and TPR (true positive rate). Examining the plot above we can come up with the initial condition for $x$. ppvFunc = Interpolation[Transpose@{thRange, ROCFunctions["PPV"] /@ aROCs}]; tprFunc = Interpolation[Transpose@{thRange, ROCFunctions["TPR"] /@ aROCs}]; FindRoot[ppvFunc[x] - tprFunc[x] == 0, {x, 0.2}] (* {x -> 0.3} *) ### Area under the ROC curve The Area Under the ROC curve (AUROC) tells for a given range of the controlling parameter "what is the probability of the classifier to rank a randomly chosen positive instance higher than a randomly chosen negative instance, (assuming 'positive' ranks higher than 'negative')", [3,4] Calculating AUROC is easy using the Trapezoidal quadrature formula: N@Total[Partition[Sort@Transpose[{ROCFunctions["FPR"] /@ aROCs, ROCFunctions["TPR"] /@ aROCs}], 2, 1] /. {{x1_, y1_}, {x2_, y2_}} :> (x2 - x1) (y1 + (y2 - y1)/2)] (* 0.474513 *) It is also implemented in \[[2](https://github.com/antononcube/MathematicaForPrediction/blob/master/ROCFunctions.m)\]: N@ROCFunctions["AUROC"][aROCs] (* 0.474513 *) ## References \[1\] Anton Antonov, [MathematicaForPrediction utilities](https://github.com/antononcube/MathematicaForPrediction/blob/master/MathematicaForPredictionUtilities.m), (2014), source code [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), package [MathematicaForPredictionUtilities.m](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/MathematicaForPredictionUtilities.m). \[2\] Anton Antonov, [Receiver operating characteristic functions Mathematica package](https://github.com/antononcube/MathematicaForPrediction/blob/master/ROCFunctions.m), (2016), source code [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), package [ROCFunctions.m](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/ROCFunctions.m) . \[3\] Wikipedia entry, [Receiver operating characteristic](http://en.wikipedia.org/wiki/Receiver_operating_characteristic). \[4\] Tom Fawcett, [An introduction to ROC analysis](https://ccrma.stanford.edu/workshops/mir2009/references/ROCintro.pdf), (2006), Pattern Recognition Letters, 27, 861-874. [1]:http://i.imgur.com/VTVyV9P.png [2]:http://i.imgur.com/d663I98.png [3]:http://i.imgur.com/bBXsDp2.png [4]:http://i.imgur.com/mzWjhZc.png [5]:https://i.imgur.com/Sam8wX5.png [6]:https://i.imgur.com/lrGRgJp.png [7]:http://i.imgur.com/DSkPQOH.png Anton Antonov 2016-10-12T13:21:04Z https://community.wolfram.com/groups/-/m/t/441131 In some cases, evaluation of expressions containing matrices can be sped up by wrapping the expression in Expand. Here's an example of a 5 fold gain in speed for an expression containing 50x50 matrices: In[12]:= n = 50; In[13]:= SeedRandom[0]; Dimensions[a = RandomReal[{-1, 1}, {n, n}]] Out[13]= {50, 50} In[14]:= SeedRandom[0]; Dimensions[b = RandomReal[{-1, 1}, n]] Out[14]= {50} In[15]:= v = Array[x, n]; In[16]:= AbsoluteTiming @ NMaximize[-1/4 v.a.Transpose[a].v - b.v, v] Out[16]= {7.294668, {51.9929, {x[1] -> -8.79, x[2] -> -5.14004, x[3] -> 4.77431, x[4] -> -33.5074, x[5] -> -17.8675, x[6] -> -41.973, x[7] -> -14.8, x[8] -> 7.17246, x[9] -> 40.5797, x[10] -> -1.30352, x[11] -> 15.2247, x[12] -> 24.7157, x[13] -> 20.5358, x[14] -> -37.6259, x[15] -> -23.1615, x[16] -> 5.04749, x[17] -> 0.716219, x[18] -> -9.14941, x[19] -> 18.7447, x[20] -> 14.4649, x[21] -> 9.51407, x[22] -> -23.4491, x[23] -> 1.5991, x[24] -> 6.41455, x[25] -> 2.53314, x[26] -> 17.8861, x[27] -> -13.0104, x[28] -> 10.2575, x[29] -> -16.721, x[30] -> 15.549, x[31] -> -1.68674, x[32] -> 12.4654, x[33] -> 24.0295, x[34] -> -17.7568, x[35] -> 9.24454, x[36] -> -11.9835, x[37] -> -5.83835, x[38] -> 0.33071, x[39] -> 16.9103, x[40] -> -30.5781, x[41] -> -18.2771, x[42] -> -29.8421, x[43] -> -7.07211, x[44] -> -14.7736, x[45] -> -14.2617, x[46] -> 39.7569, x[47] -> 48.0993, x[48] -> -3.96191, x[49] -> 9.70208, x[50] -> -21.7266}}} In[17]:= AbsoluteTiming @ NMaximize[Expand[-1/4 v.a.Transpose[a].v - b.v], v] Out[17]= {1.386620, {51.9929, {x[1] -> -8.79, x[2] -> -5.14004, x[3] -> 4.77431, x[4] -> -33.5074, x[5] -> -17.8675, x[6] -> -41.973, x[7] -> -14.8, x[8] -> 7.17246, x[9] -> 40.5797, x[10] -> -1.30352, x[11] -> 15.2247, x[12] -> 24.7157, x[13] -> 20.5358, x[14] -> -37.6259, x[15] -> -23.1615, x[16] -> 5.04749, x[17] -> 0.716219, x[18] -> -9.14941, x[19] -> 18.7447, x[20] -> 14.4649, x[21] -> 9.51407, x[22] -> -23.4491, x[23] -> 1.5991, x[24] -> 6.41455, x[25] -> 2.53314, x[26] -> 17.8861, x[27] -> -13.0104, x[28] -> 10.2575, x[29] -> -16.721, x[30] -> 15.549, x[31] -> -1.68674, x[32] -> 12.4654, x[33] -> 24.0295, x[34] -> -17.7568, x[35] -> 9.24454, x[36] -> -11.9835, x[37] -> -5.83835, x[38] -> 0.33071, x[39] -> 16.9103, x[40] -> -30.5781, x[41] -> -18.2771, x[42] -> -29.8421, x[43] -> -7.07211, x[44] -> -14.7736, x[45] -> -14.2617, x[46] -> 39.7569, x[47] -> 48.0993, x[48] -> -3.96191, x[49] -> 9.70208, x[50] -> -21.7266}}} Frank Kampas 2015-02-12T16:23:11Z https://community.wolfram.com/groups/-/m/t/3245846 Looking for a Mathematica implementation of the M/M/c retrial queueing system Goran Djuknic 2024-08-11T05:02:40Z https://community.wolfram.com/groups/-/m/t/220551 I just published a blog post proclaiming the implementation of the function MosaicPlot that gives visual representation of the contingencies of categorical variables in a list of records. The blog post has examples and explanations: [url=http://mathematicaforprediction.wordpress.com/2014/03/17/mosaic-plots-for-data-visualization/]http://mathematicaforprediction.wordpress.com/2014/03/17/mosaic-plots-for-data-visualization/[/url] If we consider the census income data set known as the [url=http://archive.ics.uci.edu/ml/datasets/Census+Income]"adult data set"[/url] that is summarized in this table: [img=width: 800px; height: 375px;]http://mathematicaforprediction.files.wordpress.com/2014/03/adult-census-income-data-summary.png[/img] we visualize the co-occurence of (categorical variable) values with mosaic plots like this one: [img=width: 500px; height: 506px;]http://mathematicaforprediction.files.wordpress.com/2014/03/adult-census-income-data-sex-education-colored-mosaic-plot.png[/img] By comparing the sizes of the rectangles corresponding to values “Bachelors”, “Doctorate”, “Masters”, and “Some-college” on the “sex vs. education” mosaic plot we can see that the fraction of men that have finished college is larger than the fraction of women that have finished college. We can further subdivide the rectangles according the co-occurrence frequencies with a third categorical variable. We are going to choose that third variable to be “income”, the values of which can be seen as outcomes or consequents of the values of the first two variables of the mosaic plot. [img=width: 550px; height: 509px;]http://mathematicaforprediction.files.wordpress.com/2014/03/adult-census-income-data-sex-education-income-colored-mosaic-plot.png[/img] From the mosaic plot "sex vs. education vs. income" we can make the following observations.1. Approximately 75% of the males with doctorate degrees or with a professional school degree earn more than $50000 per year.2. Approximately 60% of the females with a doctorate degree earn more than $50000 per year.3. Approximately 45% of the females with a professional school degree earn more than $50000.4. Across all education type females are (much) less likely to earn more than $50000 per year. Anton Antonov 2014-03-17T21:29:53Z https://community.wolfram.com/groups/-/m/t/571131 Can PDF/Docs (with math equations) documents be imported into Mathematica? hady hakim 2015-09-26T05:49:19Z https://community.wolfram.com/groups/-/m/t/2223040 I am attempting to use Wolfram to reverse engineer a formula. Unfortunately, my background in advanced math is limited, so I've been struggling to identify which functions (or even area of mathematics) to try. Hopefully someone has a suggestion on how I could try to either recreate the formula or what sort of function would approximate a solution with the smallest degree of error. The problem is reverse engineering combat math of a video game, specifically the result of a battle involving two forces, where the formulas calculate the number of losses from the attacker and defender. The formula for the losses of the defender is linear and simple, the losses for the attacker however are nonlinear. Given inputs: x = # of Attackers y = # of Defenders d = Damage output constant/coefficient = 0.1 a = Damage absorption constant/coefficient = 0.01 The formula to determine dy (the losses of the defenders) is: f(x * d - y * a) = (# of attackers * damage coefficient - # of defenders * damage absorption) Given the simplicity of the function to determine the losses of the defenders, it makes sense that the function for the loss of attackers is also simple, but it has me baffled. I've been playing with Wolfram trying to identify functions to try but I'm new to the tool, and like I said, don't have a strong math background. Some examples of the outcomes for the loss of x (attackers) and y (defenders) are: x, y, = dx, dy f(1100, 1000) = 90, 100 f(1200, 1000) = 89, 110 f(1300, 1000) = 88, 120 f(1400, 1000) = 87, 130 f(1500, 1000) = 86, 140 f(1600, 1000) = 86, 150 f(1700, 1000) = 85, 160 f(1800, 1000) = 84, 170 f(1900, 1000) = 84, 180 f(2000, 1000) = 83, 190 f(2100, 1000) = 82, 200 f(2200, 1000) = 81, 210 f(2300, 1000) = 81, 220 f(2400, 1000) = 80, 230 f(2500, 1000) = 80, 240 f(2600, 1000) = 79, 250 f(2700, 1000) = 78, 260 f(2800, 1000) = 78, 270 Anyway, I'm stuck. Does anyone have an idea of what I could try? Paul Loht 2021-03-18T18:47:14Z https://community.wolfram.com/groups/-/m/t/2026904 ![frontimage][1] &[Wolfram Notebook][2] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=imagefront.jpg&userId=1316061 [2]: https://www.wolframcloud.com/obj/88bd85e6-8a15-4fe1-ab85-7da71b0ce10b Claudio Chaib 2020-07-14T02:04:22Z