https://community.wolfram.com RSS Feed for Wolfram Community showing any discussions tagged with Tuning and Debugging sorted by most likes. https://community.wolfram.com/groups/-/m/t/1931315 [![enter image description here][2]][1] &[Wolfram Notebook][3] [1]: https://youtu.be/rOa5IntICFA [2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2020-04-09at12.54.33PM.png&userId=11733 [3]: https://www.wolframcloud.com/obj/afe2a2fb-ee55-4df5-a6fb-9bc16dd08af7 Brenton Bostick 2020-04-09T15:04:38Z https://community.wolfram.com/groups/-/m/t/1763688 **Update: This event is scheduled for 9/6/2019 at 3:30pm US Central timezone.** We are planning to have our live-streamed "Bugs Review" event soon, where Stephen Wolfram will discuss issues (bugs: incorrect behaviors, crashes, hangs, performance issues, etc.) that you care the most about. If you plan to attend and to help prepare for this event, please reply to this post with issues in the Wolfram Language (including the Notebook Interface) you think are the most important to discuss. Thanks! Arnoud Buzing 2019-08-14T17:04:55Z https://community.wolfram.com/groups/-/m/t/2763509 *Crossposted: https://mathematica.stackexchange.com/q/278250/13 * ![enter image description here][1] &[Wolfram Notebook][2] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=RealRational_560.jpg&userId=11733 [2]: https://www.wolframcloud.com/obj/4dd2dfe1-b258-4a55-a8c4-39f6b4f0cfe9 Vitaliy Kaurov 2023-01-06T18:30:28Z https://community.wolfram.com/groups/-/m/t/2330900 Daily Study Groups are back, now providing expertise on practical programming! After a short break, we're starting up with a series that picks up where Wolfram Language Basics sessions ended. This 3-week series will take you from basic programming concepts to package development. Need tips on improving your code’s speed or functionality? Programming Tutorials offer a great opportunity to level-up your skills while interacting with software development professionals. See hands-on examples and get questions answered by Wolfram Language experts. Plus: participants who pass weekly quizzes are awarded a program completion certificate. Level 1 certification is available for those who pass manually graded exercises. Join us any time between August 2 and August 20! Check out our [registration page][1] for more details. [1]: https://wolfr.am/Study_Group15 Wolfram U 2021-07-30T15:24:02Z https://community.wolfram.com/groups/-/m/t/2437685 I'm very happy with the continued improvement of the Dicom import and export functionality and speed. However, there is one issue with the current implementation that makes it very unuseful if you do quantitative image analysis. It might be that I miss an option if not I think this should be fixed. In the Mathematica documentation and examples, Dicom data is typically shown and imported as images which I understand for display purposes. But I consider the information in Dicom files as data, very well curated and standardized. For many (MRI) applications, the actual quantitative values of voxels stored in Dicom actually have meaning, values, and even units. For example in the image below each voxel value is actually a quantitative measure of T2 relaxation time in the heart, where the values are stored in milliseconds as voxel values as is also mentioned in the metadata. ![enter image description here][1] ![enter image description here][2] To get from the stored values to quantitative values (WV, DV or FP) the fields from the header and the equations are well defined. Header values: - SV = stored value of DICOM PIXEL DATA without scaling - WS = RealWorldValue slope (0040,9225) "RWVSlope" - WI = RealWorldValue intercept (0040,9224) "RWVIntercept" - RS = rescale slope (0028,1053) "RescaleSlope" - RI = rescale intercept (0028,1052) "RescaleIntercept" - SS = scale slope (2005,100E) "ScaleSlope" Outputs: - WV = real world value - FP = precise value - DV = displayed value Formulas: - WV = SV * WS + WI - DV = SV * RS + RI - FP = DV / (RS * SS) So my first try was that I want to obtain the "RawData" to access the SV pixel data, which does not output anything. ![enter image description here][3] Eventually, if I import this Dicom file into Mathematica I have to use a lot of tricks to get to the correct stored values. The Dicom images I use are stored as 12-bit Integers but are converted by Mathematica to a Numerical array with type Int16. Also, I have to specifically specify that I don't want any "DataTransformation" which by default rescales the data and actually changes some voxel values!!!! In[1]:= << QMRITools` {meta, data, bd} = Import[file, {"dicom", {"MetaInformation", "Data", "BitDepth"}}, "DataTransformation" -> None]; dataT = Import[file, {"dicom", {"Data"}}]; {dd = ToExpression[ StringJoin @@ StringCases[NumericArrayType[data], DigitCharacter]], bd} {ss, rs, ri} = meta /@ {"2005_100e", "RescaleSlope", "RescaleIntercept"} {data, dataT} = Normal@{data, dataT}; svT = 2.^bd (dataT/(2.^dd)); pfT = (rs svT + ri)/(rs ss); sv = 2.^bd (data/(2.^dd)); pf = (rs sv + ri)/(rs ss); PlotData[pfT, pf] Out[3]= {16, 12} Out[4]= {1.99854, 0.500366, -2.} For my current research, I need the PF values and to get them correctly I have to: 1. Use "DataTransformation"->None, which is not really well documented what it actually does. But based on what I see actual values of the data are changed by clipping the histogram, which for default handling of medical data is never OK!! ![enter image description here][4] 2. As far as I am aware the only way to obtain the actual stored values of my Dicom data (the actual binary values stored in the file itself) I have to find the actual BiteDepth and the imported data type and rescale my data accordingly. Below are the obtained PF valued data I need with and without DataTransformation. Although the image on the right might look less appealing with default range and scaling it is actually correct when scaled apropriately. ![enter image description here][5] ![enter image description here][6] Am I missing a correct option for getting the actual data stored? If not this should definitely be changed. Dicom is the international standard to transmit, store, retrieve, print, process, and display medical imaging information. With the current implementation, it is impossible to Import and Export such files without actually changing the stored data. Thanks, Martijn [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=dcmtag.png&userId=1332602 [2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=T2map.png&userId=1332602 [3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=raw.png&userId=1332602 [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=hist.png&userId=1332602 [5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=dataTrans1.png&userId=1332602 [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=dataTrans2.png&userId=1332602 Martijn Froeling 2022-01-05T09:42:43Z https://community.wolfram.com/groups/-/m/t/293403 TL;DR --- I have three issues with getting sun positions in Mathematica V10: - It is extremely slow compared to V9 - It doesn't *seem* to yield correct results - a) It needs an Internet connection, which b) is not assured to always return results and c) if not used optimally can easily consume your monthly allowance of W|A calls Long story === With the advent of V10 `AstronomicalData` has been deprecated, as shown on its documentation page: ![enter image description here][1]. I'm not an astronomer, so my main use of this function has been restricted to its capability to get the sun position using functions calls like this: { AstronomicalData["Sun", {"Azimuth", {2013, 3, 1, #, 0, 0}, {52.37`, 4.89`}}, TimeZone -> 1], AstronomicalData["Sun", {"Altitude", {2013, 3, 1, #, 0, 0}, {52.37`, 4.89`}}, TimeZone -> 1] } & /@ Range[0, 23] > {{341.47732, -43.93930}, {2.33417, -45.21747}, {22.94232, -43.19133}, > {41.52167, -38.28400}, {57.55208, -31.30276}, {71.47253, -23.03159}, > {84.02194, -14.08294}, {95.91940, -4.92723}, {107.80166, 4.03418}, {120.23770, 12.40167}, {133.72303, 19.72563}, {148.59433, > 25.48377}, {164.84179, 29.12209}, {181.93103, 30.19428}, {198.91284, > 28.55117}, {214.89602, 24.41962}, {229.46400, > 18.28613}, {242.70064, 10.70703}, {254.98998, > 2.19073}, {266.84760, -6.82566}, {278.85594, -15.94505}, {291.66723, -24.75464}, {306.00911, -32.75641}, {322.57956, > -39.29877}} So, this gets me the sun positions in steps of an hour during a particular day in Amsterdam (TZ 1). The same call still works in V10, though it now returns numbers with units; degrees in this case. On its first call, it reads some paclet information from a Wolfram server, but on any successive call no Internet connection is needed. I will be going into detail about timing further on, but I'll say here that the V10 function takes about three times longer than its V9 namesake. I blame the addition of units for that. With `AstronomicalData` apparently deprecated we are supposed to use its successors. In this case I need `SunPosition`. A direct translation of the above would be: SunPosition[GeoPosition[{52.37`, 4.89`}], DateObject[{2013, 3, 1, #, 0, 0}, TimeZone -> 1]] & /@ Range[0, 23] > {{95.7, -5.1}, {107.6, 3.9}, {120.0, 12.3}, {133.5, 19.6}, {148.3, 25.4}, {164.5, 29.1}, {181.6, 30.2}, {198.6, 28.6}, {214.6, 24.5}, {229.2, 18.4}, {242.5, 10.9}, {254.8, 2.4}, {266.6, -6.7}, {278.6, -15.8}, {291.4, -24.6}, {305.7, -32.6}, {322.2, -39.2}, {341.3, -43.5}, {2.0, -44.8}, {22.5, -42.9}, {41.1, -38.0}, {57.1, -31.1}, {71.0, -22.9}, {83.6, -13.9}} As with the new `AstronomicalData` the output is actually in degrees which I have removed in the above output for the sake of clarity. There are a few things to note: - `SunPosition` uses position and date objects, the latter being new in V10 - `SunPosition` does not have a `TimeZone` option, but you can set it in `DateObject` - `SunPosition` can use the old lat/long list position indication. It also can use a date list to enter the date instead of a `DateObject`. In the latter case you are out of options with respect to time zones and you have to add the appropriate amount of time offset - It is extremely slow, and it may even time-out: ![enter image description here][2] - Last but not least: the results seem to be plain wrong. It suggests that sunrise is somewhat before 1 am, which is -of course- incorrect. I assume that this has something to do with a `$GeoLocation` setting for the observer of the sun positions, but I haven't managed to sort out what I am supposed to enter to get the correct sun positions for the location provided in the same call. As to timing: I noticed very inconsistent timings for `SunPosition` compared to `AstronomicalData`, so I used the following code to collect a somewhat more statistical sound sample: SetAttributes[timingTest, HoldFirst]; timingTest[code_, repeats_Integer] := Table[ ClearSystemCache[]; code // AbsoluteTiming // First, {repeats} ] Using this, I collected timing of 20 calls to the following code snippets: - `AstronomicalData` V9 and V10: As above - `SunPosition`: As above - `SunPosition` without `GeoPosition`, just the lat/long list. - `SunPosition` without `GeoPosition`, and also without the `DateObject` date, just a classical date list (with the hour set to +1 to accommodate TZ 1) - `SunPosition` V10 without `GeoPosition` and with the `Map` (`/@`) gone and replaced by a `DateRange` inside the call. In the last case, the returned value is a `TimeSeries` object from which I extract the positions using the `"Paths"` method: SunPosition[{52.37`, 4.89`}, DateRange[{2013, 3, 1, 1, 0, 0}, {2013, 3, 1, 24, 0, 0}, "Hour"]]["Paths"][[1, All, 2]] The results were as follows: ![enter image description here][3] Clearly, the `SunPosition` results are very disappointing. Getting the sun positions with `SunPosition` is almost 40 times slower than using the old V9 method (which, I should add, wasn't particularly quick either. I have an implementation in Mathematica code which is faster). The V10 implementation of `AstronomicalData` is also more than three times slower than the V9 version. The `DateRange` version of the call saves a lot of communication overhead. Still, it is almost *five times slower* than in V9. The cause of all this slowness is that `SunPosition` simply does a call to Wolfram|Alpha. Sniffing the communication one sees the following string passed to the server: "1:eJxTTMoPSuNgYGAoZgESPpnFJcHcQEZwaV5AfnFmSWZ+XhoTsmxR/6GvGjH9wg4Qhr6XQxobsnzmXXYGhkxmIC+TEUSIgwggZihigIJgoAIGj/yizKr8PJigA5yBZtubwB1yrdxeDkXVIuvcH1aJOBRzAqUcS0vycxNLMpMBSAArww==" which can be turned into readable form using `Uncompress`: {"SunPosition", {4.89, 52.37}, {2013, 3, 1, 23, 0, 0.}, "Horizon", 2., 2., {52.09, 5.12}, Automatic} Here, we can recognize the lat/long of the position I used (but with lat/long reversed - Is this somehow significant?). At the end is my own `$GeoLocation`, but I don't believe it is used at all (and it shouldn't: I'm asking for the sun position over Amsterdam, not where I live). Changing it with `Block` I get the same results: Block[{$GeoLocation = GeoPosition[{52.37`, 40.89`}]}, SunPosition[{52.37`, 4.89`}, {2013, 3, 1, 1, 0, 0}]] Apart from the slowness, there's the issue of the necessary Internet connectivity (Want to give a demonstration and you don't have Internet? Sorry, you're out of luck). And what of the use of W|A calls? Each of the `SunPosition` tests (except the last one) took me 20 * 24 = 480 calls. So this part of my testing only already took 1440 calls, and one should be reminded that a typical Home Use license allows for only 3,000 calls per month. Things like this can go pretty fast. In fact, I once wrote an application that calculates the impact of building changes on shadows around your house throughout the year. It does in the order of 17,000 `AstronomicalData` calls. I couldn't implement that naively using `SunPosition` and have it actually work. Clearly, one should now use the `DateRange` version of the call as much as possible. ---------- To wrap up: I have one real question, i.e., how to get `SunPosition` to return the same values as `AstronomicalData`, and a request to the WRI team: please put `SunPosition` in the kernel and don't use W|A calls, because the situation as it is now is rather annoying and IMHO a real step backwards. [1]: /c/portal/getImageAttachment?filename=AstronomicalData.png&userId=43903 [2]: /c/portal/getImageAttachment?filename=timeout.png&userId=43903 [3]: /c/portal/getImageAttachment?filename=results.png&userId=43903 Sjoerd de Vries 2014-07-13T16:39:48Z https://community.wolfram.com/groups/-/m/t/1347913 This is a follow-up on [the discussion about the future of the `Graph` functionality in Mathematica](http://community.wolfram.com/groups/-/m/t/1321057). [@Charles Pooh][at0] was kind enough to respond in that thread and indicate that new developments are in the pipeline. However, the elephant in the room was not addressed: namely, will practical user needs be finally taken into account? Charles showed statistics that most bugs are reported internally, and every version has multiple bugfixes. The problem is that very serious bugs that surfaced when users tried to get actual work done have not been addressed through several versions. Problems with usability and workflow have also not been addressed. Bugs that hinder everyday work don't seem to be prioritized. A fundamental necessity for doing any non-trivial network analysis is to be able to associate attributes with edges and vertices of graphs, and then modify the graph (e.g. take a subgraph) while preserving attributes. This is near-unusable with Mathematica, and I am not exaggerating at all. If anyone believes otherwise, I challenge you to prove me wrong. Let's create a random graph of moderate size and associate attributes with its vertices and edges. To make the task not too challenging, I will use the built-in `EdgeWeight` property, which is easier to use than custom properties. rg = RandomGraph[{10000, 50000}, EdgeWeight -> RandomReal[1, 50000]]; Let's add a custom vertex property: rg = SetProperty[rg, Properties -> Thread[VertexList[rg] -> List /@ Thread["foo" -> RandomInteger[100, VertexCount[rg]]]]]; This looks pretty complicated. Why does it have to be so? With igraph's R interface, one would simply do `V(rg)$foo <- runif(vcount(g))`. Forget that some functions names are shorter, as that's mostly irrelevant. Look at how much simpler and easier the syntax is! Now let's add some vertex coordinates. Since `VertexCoordinates` is a built-in property, this is a bit simpler, but still not as simple as it should be. In[80]:= rg = SetProperty[rg, VertexCoordinates -> Thread[VertexList[rg] -> RandomReal[1, {VertexCount[rg], 2}]]]; // AbsoluteTiming Out[80]= {2.49243, Null} Also, it takes 2.5 seconds, which is unacceptably long considering that we are only processing 10000 pairs of machine reals! Now try something trivial: take a subgraph. This is probably the single most common operation one does on such datasets. Well, `Subgraph` doesn't support properties *at all*, so what do we do? Luckily we have `VertexDelete`, which *should* support them, but has been slow and buggy for as long as it has existed. sg = VertexDelete[rg, RandomSample[VertexList[rg], 5000]]; // AbsoluteTiming This took 6 seconds (!!!) on my computer. In igraph/R it takes no perceptible time, as it should. Imagine e.g. a very typical task where on would try to do some statistics on a large set of subgraphs. This is basically impossible in Mathematica (at least if we do use `Graph`). Furthermore, it turns out that `sg` is now corrupted and can't be used anymore. To make things more confusing, it is not immediately apparent that this is so, and that is was `VertexDelete` that broke it. `GraphQ[sg]` is still true. But now try In[82]:= {Length@PropertyValue[sg, EdgeWeight], EdgeCount[sg]} Out[82]= {50000, 12429} The edge weight vector is longer than the number of edges, so functions like. `EdgeBetweennessCentrality` will simply refuse to operate on the graph (without reporting any meaningful error BTW). At least I was able to check the length of the edge weight vector. With custom properties this isn't even possible (but they're also mishandled by `VertexDelete`). `VertexDelete` also frequently messes up vertex properties (not just edge properties), and again in a way that the symptoms will appear only much later in the workflow. In other words, we can't perform even the most trivial operations on graph with attributes. Finally, multigraphs simply don't work with properties, but there is absolutely no mention in the documentation that this is not supported. Things simply don't work, or break, or the `Graph` objects get corrupted. So, to sum up: It's great that `Graph` will get more attention. But will actual use cases be considered? Will the most fundamental problems, the problems that currently make it unusable for such work, be fixed? Is Mathematica trying to be suitable for this type of work at all, or, to ask once again, should we just give up and go to igraph and networkx, to R and Python, like nearly everyone else does? (I certainly wouldn't like to do that as I already invested a lot in Mathematica.) Bug(-fix) counts alone are not a good measure of progress. Effort should be concentrated where it actually matters. [at0]: http://community.wolfram.com/web/charlesp Szabolcs Horvát 2018-05-29T10:38:30Z https://community.wolfram.com/groups/-/m/t/2726860 &[Wolfram Notebook][1] [1]: https://www.wolframcloud.com/obj/d0094aa1-c4bc-47eb-b693-6f30a4f26f3c Hanfeng Zhai 2022-12-12T15:08:24Z https://community.wolfram.com/groups/-/m/t/1037730 *NOTE: Please see the original version of this post [**HERE**][1]. Cross-posted here per suggestion of [@Vitaliy Kaurov][at0].* ---------- Since Mathematica is a symbolic system, with symbolic evaluator much more general than in numerically-based languages, it is not surprising that performance-tuning can be more tricky here. There are many techniques, but they can all be understood from a single main principle. It is: *Avoid full Mathematica symbolic evaluation process as much as possible.* All techniques seem to reflect some facet of it. The main idea here is that most of the time, a slow Mathematica program is such because many Mathematica functions are very general. This generality is a great strength, since it enables the language to support better and more powerful abstractions, but in many places in the program such generality, used without care, can be a (huge) overkill. I won't be able to give many illustrative examples in the limited space, but they can be found in several places, including some WRI technical reports (Daniel Lichtblau's one on efficient data structures in Mathematica comes to mind), a very good book of David Wagner on Mathematica programming, and most notably, many Mathgroup posts. I also discuss a limited subset of them in [my book][3]. I will supply more references soon. Here are a few most common ones (I only list those available within Mathematica language itself, not mentioning CUDA \ OpenCL, or links to other languages, which are of course also the possibilities): 1. *Push as much work into the kernel at once as possible, work with as large chunks of data at a time as possible, without breaking them into pieces* 1.1. Use built-in functions whenever possible. Since they are implemented in the kernel, in a lower-level language (C), they are typically (but not always!) much faster than user-defined ones solving the same problem. The more specialized version of a built-in function you are able to use, the more chances you have for a speed-up. 1.2. Use functional programming (`Map, Apply`, and friends). Also, use pure functions in `#-&` notation when you can, they tend to be faster than Function-s with named arguments or those based on patterns (especially for not computationally-intensive functions mapped on large lists). 1.3. Use structural and vectorized operations (`Transpose, Flatten, Partition, Part` and friends), they are even faster than functional. 1.4. Avoid using procedural programming (loops etc), because this programming style tends to break large structures into pieces (array indexing etc). This pushes larger part of the computation outside of the kernel and makes it slower. 2. *Use machine-precision whenever possible* 2.1. Be aware and use Listability of built-in numerical functions, applying them to large lists of data rather than using `Map` or loops. 2.2. Use `Compile`, when you can. Use the new capabilities of `Compile`, such as `CompilationTarget->"C"`, and making our compile functions parallel and Listable. 2.3. Whenever possible, use vectorized operations (`UnitStep, Clip, Sign, Abs`, etc) inside `Compile`, to realize "vectorized control flow" constructs such as `If`, so that you can avoid explicit loops (at least as innermost loops) also inside `Compile`. This can move you in speed from Mathematica byte-code to almost native C speed, in some cases. 2.4. When using `Compile`, make sure that the compiled function doesn't bail out to non-compiled evaluation. See examples [in this MathGroup thread][4]. 3. *Be aware that Lists are implemented as arrays in Mathematica* 3.1. Pre-allocate large lists 3.2. Avoid `Append, Prepend, AppendTo` and `PrependTo` in loops, for building lists etc (because they copy entire list to add a single element, which leads to quadratic rather than linear complexity for list-building) 3.3. Use linked lists (structures like `{1,{2,{3,{}}}}` ) instead of plain lists for list accumulation in a program. The typical idiom is `a = {new element, a}`. Because a is a reference, a single assignment is constant-time. 3.4. Be aware that pattern-matching for sequence patterns (BlankSequence, BlankNullSequence) is also based on Sequences being arrays. Therefore, a rule `{fst_,rest___}:>{f[fst],g[rest]}` will copy the entire list when applied. In particular, don't use recursion in a way which may look natural in other languages. If you want to use recursion on lists, first convert your lists to linked lists. 4. *Avoid inefficient patterns, construct efficient patterns* 4.1. Rule-based programming can be both very fast and very slow, depending on how you build your structures and rules, but in practice it is easier to inadvertently make it slow. It will be slow for rules which force the pattern-matcher to make many a priory doomed matching attempts, for example by under-utilizing each run of the pattern-matcher through a long list (expression). Sorting elements is a good example: `list//.{left___,x_,y_,right___}/;x>y:>{left,y,x,right}` - has a cubic complexity in the size of the list (explanation is e.g. [here][5]). 4.2. Build efficient patterns, and corresponding structures to store your data, making pattern-matcher to waste as little time on false matching attempts as possible. 4.3. Avoid using patterns with computationally intensive conditions or tests. The pattern-matcher will give you the most speed when patterns are mostly syntactic in nature (test structure, heads, etc). Every time when condition `(/;)` or pattern test `(?)` is used, for every potential match, the evaluator is invoked by the pattern-matcher, and this slows it down. 5. *Be aware of immutable nature of most Mathematica built-in functions* Most Mathematica built-in functions which process lists create a copy of an original list and operate on that copy. Therefore, they may have a linear time (and space) complexity in the size of the original list, even if they modify a list in only a few places. One universal built-in function that does not create a copy, modifies the original expression and does not have this issue, is `Part`. 5.1. Avoid using most list-modifying built-in functions for a large number of small independent list modifications, which can not be formulated as a single step (for example, `NestWhile[Drop[#,1]&,Range[1000],#<500&]` ) 5.2. Use extended functionality of `Part` to extract and modify a large number of list (or more general expression) elements at the same time. This is very fast, and not just for packed numerical arrays (`Part` modifies the original list). 5.3. Use `Extract` to extract many elements at different levels at once, passing to it a possibly large list of element positions. 6. *Use efficient built-in data structures* The following internal data structures are very efficient and can be used in many more situations than it may appear from their stated main purpose. Lots of such examples can be found by searching the Mathgroup archive, particularly contributions of Carl Woll. 6.1. Packed arrays 6.2. Sparse arrays 7. *Use hash - tables.* Starting with version 10, immutable associative arrays are available in Mathematica (Associations) 7.1 Associations the fact that they are immutable does not prevent them to have efficient insertion and deletion of key-value pairs (cheap copies different from the original association by the presence, or absence, of a given key-value pair). They represent the idiomatic associative arrays in Mathematica, and have very good performance characteristics. For earlier versions,the following alternatives work pretty well, being based on internal Mathematica's hash-tables: 7.2. Hash-tables based on `DownValues` or `SubValues` 7.3. `Dispatch` 8. *Use element - position duality* Often you can write faster functions to work with positions of elements rather than elements themselves, since positions are integers (for flat lists). This can give you up to an order of magnitude speed-up, even compared to generic built-in functions (`Position` comes to mind as an example). 9. *Use Reap - Sow* `Reap` and `Sow` provide an efficient way of collecting intermediate results, and generally "tagging" parts you want to collect, during the computation. These commands also go well with functional programming. 10. *Use caching, dynamic programming, lazy evaluation* 10.1. Memoization is very easily implemented in Mathematica, and can save a lot of execution time for certain problems. 10.2. In Mathematica, you can implement more complex versions of memoization, where you can define functions (closures) at run-time, which will use some pre-computed parts in their definitions and therefore will be faster. 10.3. Some problems can benefit from lazy evaluation. This seems more relevant to memory - efficiency, but can also affect run-time efficiency. Mathematica's symbolic constructs make it easy to implement. A successful performance - tuning process usually employs a combination of these techniques, and you will need some practice to identify cases where each of them will be beneficial. [1]: http://mathematica.stackexchange.com/a/29351 [2]: https://groups.google.com/d/topic/comp.soft-sys.math.mathematica/XOXapJm_Q1Q/discussion [3]: http://www.mathprogramming-intro.org/ [4]: https://groups.google.com/d/topic/comp.soft-sys.math.mathematica/XOXapJm_Q1Q/discussion [5]: http://www.mathprogramming-intro.org/book/node355.html [at0]: http://community.wolfram.com/web/vitaliyk Leonid Shifrin 2017-03-22T20:05:20Z https://community.wolfram.com/groups/-/m/t/2409025 I know that calculating the digits of Pi is something very specific and probably irrelevant to the real practical problems, but somehow it feels appropriate to quickly test the performance of Wolfram Mathematica, especially if the machine is actually called Raspberry **Pi** :) So, here is the log of running `AbsoluteTiming[N[Pi,10^6];]` on a Raspberry Pi with different versions of Wolfram Mathematica, from which it is clear that 11.3, 12.1.1 and 12.2.0 give approximately the same timings, but 12.3.1 is substantially worse. $ math11 Wolfram Language 11.3.0 Engine for Linux ARM (32-bit) Copyright 1988-2018 Wolfram Research, Inc. In[1]:= AbsoluteTiming[N[Pi,10^6];] Out[1]= {2.03026, Null} -------------------------------------------------------- $ math12.1 Mathematica 12.1.1 Kernel for Linux ARM (32-bit) Copyright 1988-2020 Wolfram Research, Inc. In[1]:= AbsoluteTiming[N[Pi,10^6];] Out[1]= {2.08772, Null} -------------------------------------------------------- $ math12.2 Mathematica 12.2.0 Kernel for Linux ARM (32-bit) Copyright 1988-2021 Wolfram Research, Inc. In[1]:= AbsoluteTiming[N[Pi,10^6];] Out[1]= {3.25156, Null} -------------------------------------------------------- $ math Mathematica 12.3.1 Kernel for Linux ARM (32-bit) Copyright 1988-2021 Wolfram Research, Inc. In[1]:= AbsoluteTiming[N[Pi,10^6];] Out[1]= {7.66692, Null} Tigran Aivazian 2021-11-18T07:43:31Z https://community.wolfram.com/groups/-/m/t/1855244 Here are two simple but extremely convenient keyboard shortcuts that I've used for years. - This post is a favor to all my friends who type slowly (like me) - These shortcuts simply create a pair of brackets containing a template placeholder as the current new selection - Essentially, it's just a stripped-down version of the bulky (Command + Shift + K + Up/Down Arrow + Enter) completion **Favorite Shortcut #1:** Autocomplete single brackets by pressing `CommandKey + [` ![enter image description here][1] **Favorite Shortcut #2:** Autocomplete part brackets by pressing `CommandKey + ]` ![enter image description here][2] ## How to set them up ## On MacOS, first make a backup of the following file, in case you break it: FileNameJoin[{$InstallationDirectory, "SystemFiles/FrontEnd/TextResources/Macintosh/KeyEventTranslations.tr"}]; On my machine this path resolves to: *"/Applications/Mathematica.app/Contents/SystemFiles/FrontEnd/TextResources/Macintosh/KeyEventTranslations.tr"* Then, preferably in a non-Mathematica editor (e.g. Sublime), open the above file and copy-and-paste these following code into the list inside `EventTranslations`. For the first shortcut: Item[KeyEvent["[",Modifiers->{Command}],FrontEndExecute[{FrontEnd`NotebookWrite[FrontEnd`InputNotebook[],"[\[SelectionPlaceholder]]"],FrontEndToken["MovePreviousPlaceHolder"]}]] And for the second (of course making sure they are separated by a comma): Item[KeyEvent["]",Modifiers->{Command}],FrontEndExecute[{FrontEnd`NotebookWrite[FrontEnd`InputNotebook[],"\[LeftDoubleBracket]\[SelectionPlaceholder]\[RightDoubleBracket]"],FrontEndToken["MovePreviousPlaceHolder"],FrontEndToken["MovePreviousPlaceHolder"]}]] Finally, restart Mathematica, and presto, they should both be working. And to remove the shortcuts, simply remove those items or replace the .tr file with the backup. [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=own.gif&userId=900170 [2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=part.gif&userId=900170 Michael Sollami 2020-01-08T20:42:08Z https://community.wolfram.com/groups/-/m/t/849935 I found some unusual error in VoronoiMesh. I have the points *pts*, and first I compute the VoronoiMesh for that, Note the red circle: these are two close datapoints, and it correctly draws a line between them. dom=1.25\[Pi] (* domain size *) domspec=Transpose[{{0,0},{dom,dom}}] pts={{1.880951464600067`,1.473752017061138`},{1.8913812543459405`,2.319493673670671`},{1.7674367495794008`,3.5325088938533917`},{1.9320143413972677`,3.611151717969884`},{1.8107126970778207`,3.6790935311559876`},{1.804725058474518`,3.8520729911980185`},{1.805368854133445`,3.852728507949241`},{1.563939776616803`,3.9192958794919646`}}; VoronoiMesh[pts,domspec,Epilog->{Point[pts],Red,Circle[pts[[6]],0.1]},PlotRange->domspec,ImageSize->500,PlotRangePadding->Scaled[.05]] ![enter image description here][1] Now, I add points around it (to make the Voronoi periodic), leaving the old ones, and only adding more points *around* it. Then the voronoi is not correctly computed: allpts=Join@@Join@@Table[p+dom {i,j},{i,-1,1},{j,-1,1},{p,pts}]; (* add groups of points around it, making it periodic *) VoronoiMesh[allpts,Epilog->{Point[pts],Red,Circle[pts[[6]],0.1]},PlotRange->domspec,ImageSize->500,PlotRangePadding->Scaled[.05]] ![enter image description here][2] I have millions and millions of points and it took me several hours to 'minimize' the error and go to the core of it. Any help is welcome! Is there some option that controls when two points are considered the same? It looks like some points are deleted because they are too close? The distance between point 6 and 7 is: Log10[Norm[pts[[6]] - pts[[7]]]] -3.03678 So it should not be some machine-precision issue I presume! If someone can confirm it on different versions that would be appreciated. This is done in 10.4.1 and 10.3.1 on Mac, both show the error. [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2016-05-03at16.23.33.png&userId=73716 [2]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2016-05-03at16.24.30.png&userId=73716 Sander Huisman 2016-05-03T14:33:03Z https://community.wolfram.com/groups/-/m/t/2981586 Quadro M1200 Intel i7 32 Gb RAM I'm currently rendering some large 3D scenes for some projects. I'm aware I'm pushing my hardware and Mathematica to its limits. I previously [asked some questions][1] about frontend crashes when using Graphics3D and export and I now better understand why and can work around it. I am managing to get the results I want by closely monitoring my system when pushing the settings just below failure. But I'm wondering if some of the issues I'm working around can be prevented. Therefore I'm posting my experience here and hoping to get some tips to improve. But first the nice results before the boring evaluation. ![enter image description here][2] ![enter image description here][3] My laptop is a Intel i7 with 32 Gb RAM and and Nvidia Quadro M1200. for the graphs below red is memory usage and green is CPU usage both in %. For my scene I have 60 transparent iso-volumes and 4 solid ones. The streamlines I render are subsampled from around 500K lines i generate. For lines I select 12.5k lines with around 100 vertices each, for tubes I select 7.5 k lines. My `$HistoryLength` setting is always 0 since I'm working with sizable data. When exporting my scene using lines this is my system memory. I don't display the scene in the frontend only export it to *.png files. The export is clearly handled by the frontend and CPU and at the moment of export the memory spikes and the GPU does some work. However after everything is done and i quit the kernel the Mathematica frontend is still using between 3-4 Gb of RAM. Exporting one PNG takes around 2-3 Gb. ![enter image description here][4] When exporting tubes things get a bit more heavy and needs between 6-8 Gb of memory per image. after everything is done i have lost around 5 Gb of RAM that I can only get back by closing the frontend. For a clean system and Mathematica start i use around 25% memory. After loading and processing i have 35% in use. After one round of exporting lines and tubes, closing Mathematica reopening loading and processing the data I'm at 50% memory usage so somewhere I lost 15%. ![enter image description here][5] If I push everything to far the export will overflow the memory resulting in a frontend crash and sometimes a GPU driver crash. I now somewhat know the limits of what I can do but after working for a while exporting and optimizing visualization at some point my memory keeps growing. I now monitor this and when i come to the point of getting close to a crash the only thing i can do is restart everything to reset my memory. [1]: https://community.wolfram.com/groups/-/m/t/2920496?p_p_auth=86KqZmOz [2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=line.gif&userId=1332602 [3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=tube.gif&userId=1332602 [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=lines.png&userId=1332602 [5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=6379tubes.png&userId=1332602 Martijn Froeling 2023-08-02T10:10:50Z https://community.wolfram.com/groups/-/m/t/181641 Wanted to share with my kids the classical experiment on a rolling object down an incline plane using legos, arduino and Mathematica for the Raspberry Pi. Using a window valance and some lego technic pieces we proceeded to build the inclined plane platform. [img=width: 300px; height: 400px;]/c/portal/getImageAttachment?filename=inclinedplane1.JPG&userId=78214[/img] Using legos was great to allow to attach both leds and photorresistors to the inclined plane. It also allowed us to set up build a platform from where to attach a servo motor and a protractor to measure the angle of inclination. [img=width: 400px; height: 533px;]/c/portal/getImageAttachment?filename=inclinedplane2.JPG&userId=78214[/img] We'll detect when the ball reaches a certain point of the platform by the the disminution of the LED light received at the photoresistors. [img=width: 300px; height: 400px;]/c/portal/getImageAttachment?filename=8270photo(1).JPG&userId=78214[/img] The adafruit web site is a very good site to explain the details on how to connect the photoresistors and leds to the Arduino. [url=http://learn.adafruit.com/adafruit-arduino-lesson-2-leds/leds]http://learn.adafruit.com/adafruit-arduino-lesson-2-leds/leds[/url] [url=http://learn.adafruit.com/photocells/using-a-photocell]http://learn.adafruit.com/photocells/using-a-photocell[/url] As the readings of the photocells require of analog inputs, we decided to use a spare arduino we had at our disposal, and to have more fun, use XBEE modules to connect the arduino to the raspberry pi. [img=width: 300px; height: 225px;]/c/portal/getImageAttachment?filename=6471photo(2).JPG&userId=78214[/img] The following is the code at the Arduino.[code] #include <Servo.h> int servoPin = 9; int LightSensor[4]={A1,A2,A3,A4}; int current[4]={0,0,0,0}; double timers[4]={0,0,0,0}; int previous[4]={0,0,0,0}; Servo servo; int i=0; double factor=0.7; double timer = 0; int inByte =0; void setup() { Serial.begin(19200); servo.attach(servoPin); servo.write(0); delay(2000); for (i=0;i<4;i++){ previous[i]=analogRead(LightSensor[i]); } } void loop() { if(Serial.available()>0){ inByte=Serial.read(); if (inByte==65) { servo.write(90); timer=millis(); trackBall(); for (i=0;i<4;i++){ Serial.println(timers[i]); } servo.write(0); } } } void trackBall(){ int m =0; while (m<4){ current[m]=analogRead(LightSensor[m]); if (current[m]<factor*previous[m]){ timers[m]=millis()-timer; m++; } } }[/code] Mathematica code and results to follow [mcode]serial = DeviceOpen["Serial", {"/dev/ttyUSB0", "BaudRate" -> 19200}] lengths = {0., 0.175, 0.43, 0.69} ;(*Distance between light sensors in meters*) data = {};(*List to capture the time between sensors*) ping := Module[{}, DeviceWriteBuffer[serial, {"A"}]; Pause[3]; ToExpression[StringSplit[FromCharacterCode[DeviceReadBuffer[serial]]]]/1000] (*Function pings sends an "A" to the arduino to release the ball and \ collect timing*) Button["Collect Data", data = Append[data, ping]] Dynamic[tdata = Transpose@data; dataPoints = Flatten[Table[{tdata[[i, j]], lengths[[i]]}, {i, Length[lengths]}, {j, Length[data]}], 1]; lm = Fit[dataPoints, {1, x, x^2}, x]; Plot[lm, {x, -0.3, 1.2}, Epilog -> {Red, PointSize[Large], Point[dataPoints]}, AspectRatio -> 1, PlotLabel -> lm]][/mcode]Using a hollow ball, we can work out the gravity value [img=width: 440px; height: 500px;]/c/portal/getImageAttachment?filename=Untitled.png&userId=78214[/img] Using a golf ball [img=width: 500px; height: 482px;]/c/portal/getImageAttachment?filename=InclinedPlane.png&userId=78214[/img] A couple of videos of the experiment in action [url=https://www.youtube.com/watch?v=4eRZ48N3vM8]https://www.youtube.com/watch?v=4eRZ48N3vM8[/url] [url=https://www.youtube.com/watch?v=eLVPpEcyw_0]https://www.youtube.com/watch?v=eLVPpEcyw_0[/url] [url=https://www.youtube.com/watch?v=53WaEcRWY_0]https://www.youtube.com/watch?v=53WaEcRWY_0[/url] Diego Zviovich 2014-01-08T01:09:54Z https://community.wolfram.com/groups/-/m/t/3531785 Wolfram Video Processing is too slow in real-life situations. I love using Wolfram Language for real-life applications, and I appreciate that Wolfram has recently focused on first-class video functionality. However, it’s a pity that while it works fine for demonstrations, it is far too slow for practical use. For example, when I use CapCut to edit and render a 5-minute video, it takes only a few seconds. But in Wolfram Mathematica, the same process is extremely slow. For a simple 7-minute video (with only text overlay), Wolfram takes 6 minutes just to render. That is unusable! Imagine processing a **30-minute video—it could take 45 minutes**, while CapCut would finish in under a minute. ![enter image description here][1] I have also tried adjusting encoders, frame rates, and other settings, but it didn’t help. Can someone suggest a solution? Thank you very much. [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1755454340--0Codec.nb_-Wolfram.jpg&userId=138946 Jaques Secretin 2025-08-17T18:15:27Z https://community.wolfram.com/groups/-/m/t/3252532 ![Wolfram Language Runtime (SDK) demo in Zig][1] &[Wolfram Notebook][2] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=1169Hero.png&userId=20103 [2]: https://www.wolframcloud.com/obj/5c80768d-a495-4c9a-baa0-655d8dc47903 Daniel Sanchez 2024-08-21T02:25:51Z https://community.wolfram.com/groups/-/m/t/1108896 One of the well-known algorithm of doing the inverse square root: $$\frac{1}{\sqrt{x}}$$ is the so-called "fast inverse square root" algorithm, see [wikipedia][1]. This code gives a very good approximation of this function, possibly good enough for lighting in video-games. Even now, with modern CPUs, with many new instructions, this is still quite a bit faster than the actual inverse square root. The most surprising thing about this code is the appearance of a magic constant (note that this is C code, including the original comments) float Q_rsqrt( float number ) { long i; float x2, y; const float threehalfs = 1.5F; x2 = number * 0.5F; y = number; i = * ( long * ) &y; // evil floating point bit level hacking i = 0x5f3759df - ( i >> 1 ); // what the f*ck? y = * ( float * ) &i; y = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration return y; } As you can see, the input is a float (a 32 bit floating point number), which is cast in to a long integer, the bits are shifted, and the result subtracted from a magic constant, this number is then re-interpreted again as a float. To understand why this works is actually quite a long story, but realise that floats are stored as 32 bits as follows: SEEEEEEEEMMMMMMMMMMMMMMMMMMMMMMM where S= sign, E = exponent, and M = Mantissa. So once you interpret this as a long, and start shifted bits, you can get very unpredictable behavior. The question that I'm after is: can this constant be improved? Let's create a library function in c and link it back in to Mathematica, such that we can try different magic constants and different values x: Needs["CCompilerDriver`"] src=" #include \"WolframLibrary.h\" DLLEXPORT mint WolframLibrary_getVersion(){ return WolframLibraryVersion; } DLLEXPORT int WolframLibrary_initialize( WolframLibraryData libData) {return 0;} DLLEXPORT void WolframLibrary_uninitialize( WolframLibraryData libData) {return;} DLLEXPORT int constantzero(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res){ MArgument_setInteger(Res, 0); return LIBRARY_NO_ERROR; } DLLEXPORT int fastinvsqrt(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) { double in = MArgument_getReal(Args[0]); int magic = MArgument_getInteger(Args[1]); float x = in; float halfx = 0.5f * x; int i = *(int*)&x; i = magic - (i >> 1); x = *(float*)&i; x = x*(1.5f-(halfx*x*x)); double I1; I1 = x; MArgument_setReal(Res, I1); return LIBRARY_NO_ERROR; }"; Quiet@LibraryFunctionUnload[fastinvsqrt] fastinvsqrtlib=CreateLibrary[src,"fastinvsqrt"] fastinvsqrt=LibraryFunctionLoad[fastinvsqrtlib, "fastinvsqrt",{Real,Integer},Real] magicconst=16^^5f3759df Plot[{1/Sqrt[x],fastinvsqrt[x,magicconst]},{x,0.1,5},PlotRange->{0,2}] LogLogPlot[{1/Sqrt[x]-fastinvsqrt[x,magicconst],1/Sqrt[x]},{x,0.01,100000},PlotRange->{All,{10^-6,10}},Frame->True,PlotRangePadding->None,PlotPoints->1200,MaxRecursion->3] ![enter image description here][2] The match is indeed great! and we can see that the error (second plot, in blue) is of the order 0.1% of the result for the original magic constant! Let's introduce the relative error: ClearAll[relativeerror] relativeerror[x_Real, magic_Integer] := (1/Sqrt[x] - fastinvsqrt[x, magic])/(1/Sqrt[x]) and plot it: LogLogPlot[relativeerror[x, magicconst], {x, 0.01, 100000}, Frame -> True, PlotRangePadding -> None, PlotRange -> {10^-5, 1}, PlotPoints -> 1200, MaxRecursion -> 3] ![enter image description here][3] This plot is periodic because of the way floating point numbers are defined/implemented, we can zoom in on one of the parts: LogLinearPlot[relativeerror[x, magicconst], {x, 1, 4}, Frame -> True, PlotRangePadding -> None, PlotRange -> All, PlotPoints -> 1200, MaxRecursion -> 3] ![enter image description here][4] We can plot slight deviations from the magic constant to see the new relative errors: LogLinearPlot[{relativeerror[x, magicconst], relativeerror[x, magicconst - 20000], relativeerror[x, magicconst + 20000]}, {x, 1, 4}, Frame -> True, PlotRangePadding -> None, PlotRange -> All, PlotPoints -> 1200, MaxRecursion -> 3, PlotLegends -> {"0", "-20000", "+20000"}] ![enter image description here][5] If the constant increases, the center peaks go up, but the far-right and far-left peak go down. Let's try a bunch of magic-constants, and check the maximum (relative) error we find. I do this by sampling many point and looking for the maximum error I find in those points: Dynamic[d] data = Table[{d, Max[Table[With[{x = 2^s}, ((1/Sqrt[x] - fastinvsqrt[x, d + magicconst])/(1/Sqrt[x]))], {s, 0, 2, 0.00001}]]}, {d, -100000, 100000, 1000}]; data // ListLogPlot TakeSmallestBy[data, Last, 1] ![enter image description here][6] {{0, 0.00175225}} If there is a better constant, it sure is close to the original constant. Let's zoom in a bit: Dynamic[d] data = Table[{d, Max[Table[With[{x = 2^s}, ((1/Sqrt[x] - fastinvsqrt[x, d + magicconst])/(1/Sqrt[x]))], {s, 0, 2, 0.00001}]]}, {d, -1000, 1000, 10}]; data // ListLogPlot TakeSmallestBy[data, Last, 1] ![enter image description here][7] {{160, 0.00175121}} We can zoom in even further: Dynamic[d] data=Table[{d,Max[Table[With[{x=2^s},((1/Sqrt[x]-fastinvsqrt[x,d+magicconst])/(1/Sqrt[x]))],{s,0,2,0.0000001}]]},{d,160,170,1}]; data//ListLogPlot TakeSmallestBy[data,Last,1] ![enter image description here][8] {{166, 0.00175129}} magicconstant + 166 seems to have a slightly better worst-case performance. This corresponds to the follow new magic constant: BaseForm[magicconst + 166, 16] or in C (in hex): 0x5f375a85 We could also use the built-in FindMaximum: ClearAll[FindMaxima] FindMaxima[d_Integer]:=Module[{x}, Max[Table[First@FindMaximum[((1/Sqrt[x]-fastinvsqrt[x,d+magicconst])/(1/Sqrt[x])),{x,x0,1,4},MaxIterations->5000,WorkingPrecision->50],{x0,1.0,4.0,0.03}]] ] Now using this function and look around our probable minimum: Dynamic[d] data=Quiet@Table[{d,FindMaxima[d]},{d,150,175,1}]; data//ListLogPlot TakeSmallestBy[data,Last,1] ![enter image description here][9] Which also find 166. So that seems to be in agreement. Another [paper][10] found 0x5f375a86 which 16^^5f375a86 - magicconst = 167. So perhaps he/me is wrong with rounding somehow… ## Average error ## So far, we've been looking at maximum error possible, but what about minimising the average error? Since we're using floating point numbers that can range over many many decades I would like to minimize the following integral of the error: Integrate[relativeerror[2^s]^2,{s,0,2}] Note that I do not sample the domain x 1 to 4 linearly. Again we can run for large deviations from the magic constant: Dynamic[d] data=Table[{d,Total[Table[With[{x=2^s},((1/Sqrt[x]-fastinvsqrt[x,d+magicconst])/(1/Sqrt[x]))^2],{s,0,2,0.00001}]]},{d,-250000,250000,10000}]; data//ListLogPlot TakeSmallestBy[data,Last,1] ![enter image description here][11] Let's zoom in a bit more: Dynamic[d] data=Table[{d,Total[Table[With[{x=2^s},((1/Sqrt[x]-fastinvsqrt[x,d+magicconst])/(1/Sqrt[x]))^2],{s,0,2,0.00001}]]},{d,-96000,-94000,20}]; data//ListLogPlot TakeSmallestBy[data,Last,1] ![enter image description here][12] And zooming a bit more and using a smaller 'integration' steps: Dynamic[d] data=Table[{d,Total[Table[With[{x=2^s},((1/Sqrt[x]-fastinvsqrt[x,d+magicconst])/(1/Sqrt[x]))^2],{s,0,2,0.00000005}]]},{d,-94900,-94800,1}]; data[[All,2]]=Rescale[data[[All,2]]]; ListPlot[data,PlotRange->All] TakeSmallestBy[data,Last,1] ![enter image description here][13] The lowest value happens at magicconstant - 94863: LogLogPlot[{relativeerror[x,magicconst],relativeerror[x,magicconst-94863]},{x,0.01,100000},Frame->True,PlotRangePadding->None,PlotRange->{10^-5,1},PlotPoints->1200,MaxRecursion->3] LogLinearPlot[{relativeerror[x,magicconst]^2,relativeerror[x,magicconst-94863]^2},{x,1,4},Frame->True,PlotRangePadding->None,PlotRange->All,PlotPoints->1200,MaxRecursion->3] ![enter image description here][14] Here (in orange) the average (square) relative error is better as compared to the original constant. Hope you enjoyed this little exploration of using C code inside Mathematica and to optimize low-level algorithms interactively using Mathematica's library functionality and plotting functionality. The original code includes a single Newton iteration to further hone in to the correct answer, the plots I've showed are for a single iteration. Other people have optimized the constant for not a single iteration or two iterations… So there are different constants out there. And, since the values are always under-estimated, the Newton iteration can also be modified to account for this. [1]: https://en.wikipedia.org/wiki/Fast_inverse_square_root [2]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at01.36.58.png&userId=73716 [3]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at01.40.10.png&userId=73716 [4]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at01.41.11.png&userId=73716 [5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at01.43.40.png&userId=73716 [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at01.49.01.png&userId=73716 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at01.47.42.png&userId=73716 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at01.50.33.png&userId=73716 [9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at01.54.12.png&userId=73716 [10]: https://cs.uwaterloo.ca/~m32rober/rsqrt.pdf [11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at02.04.47.png&userId=73716 [12]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at02.05.39.png&userId=73716 [13]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at02.07.26.png&userId=73716 [14]: http://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2017-05-27at02.13.55.png&userId=73716 Sander Huisman 2017-05-27T00:25:47Z https://community.wolfram.com/groups/-/m/t/1065699 It seems that `Map`ing a function over a `SparseArray` can produce a result where one of the explicit values in the array is the same as the background value. It seems to me that this is a bug because other functions appear to assume that an explicit value will never be the same as the background value. ## Example Let us create a sparse array, In[1]:= sa = SparseArray[{1, 0, 2, 0, 3}]; and check its explicit values. In[2]:= sa["NonzeroValues"] Out[2]= {1, 2, 3} Let us now change the first element (which is nonzero) to have the background value. In[3]:= sa[[1]] = 0; And check explicit values again: In[4]:= sa["NonzeroValues"] Out[4]= {2, 3} As you can see, the sparse array has been recomputed so that none of the explicit values is 0. It is in a consistent state. This is what I expected. Now let us modify elements in a different manner. Let us map a function over the sparse array in such a way that it will transform some of the explicit values to zero: In[5]:= sa = SparseArray[{1, 0, 2, 0, 3}]; In[6]:= sa2 = Map[If[# > 2, 0, #] &, sa]; Check explicit values in the result: In[7]:= sa2["NonzeroValues"] Out[7]= {1, 2, 0} Oops! What is that `0` doing in the list? Is should have been removed from the explicit values list. **Is this a bug?** Do decide, let us take a look at how other functions handle this sparse array. `ArrayRules` works on both sparse and dense arrays. `ArrayRules[arr, b]` gives the positions of elements that are different from `b`. It works fine even if `b` is different from the background element of a sparse `arr`. But here `ArrayRules` fails to give the result I expect, even though the second element was given explicitly: In[8]:= ArrayRules[sa2, 0] Out[8]= {{1} -> 1, {3} -> 2, {5} -> 0, {_} -> 0} That `{5} -> 0` should not be in the list. See how ArrayRules works if the default element is set to `1`. There is no problem this time: In[9]:= ArrayRules[sa2, 1] Out[9]= {{2} -> 0, {3} -> 2, {4} -> 0, {5} -> 0, {_} -> 1} This suggests that either `ArrayRules` or `SparseArray` is misbehaving here. Which one should be considered the buggy one? Does this `SparseArray` have a broken internal state? Or does `ArrayRules` fail to take a special case into account? ---- This array can be fixed using `SparseArray[sa2]`. Szabolcs Horvát 2017-04-19T08:36:10Z https://community.wolfram.com/groups/-/m/t/967397 Hello everyone, In this post I would like to share a programming technique I find interesting, as well as ask for feedback on it. ## The problem The [MaTeX package][1] typesets LaTeX directly within Mathematica. Unfortunately, every call to the `MaTeX` function involves one run of LaTeX, which is very slow: it takes about 0.5 seconds. Typesetting 20 formulae would take 10 seconds—much too slow.  To mitigrate this, MaTeX can typeset a list of expressions with a single LaTeX run. This is much faster: In[37]:= ClearMaTeXCache[] MaTeX@Range[20]; // AbsoluteTiming Out[38]= {0.480417, Null} But it is not always convenient to put everything we want to compile into a single list. MaTeX is meant for creating figure labels and each label would usually appear in a different position in the Mathematica code we write. **Could we still gather all these inputs, appearing in different places in the code, and compile them with a single run?** ## The solution Here's how we can do this: delay the evaluation. Let's have a function called `delayedMaTeX` that doesn't evaluate immediately, but simply acts as a placeholder object. It also collects the inputs into a queue for later evaluation. Then we can trigger batch-evaluation with one command, which assigns the results to the placeholders. We also want to prevent the unlimited accumulation of compiled results, which take up memory. Let's get rid of them as soon as they are no longer referenced anywhere. This can be achieved using the rarely used [`Temporary`](http://reference.wolfram.com/language/ref/Temporary.html) attribute. For the following proof-of-concept example, let's forget that `MaTeX` can take options—taking that into account would just make the code longer and harder to understand without being educational. Let's load MaTeX and set some options right away, just so the examples below will look a bit nicer. << MaTeX` SetOptions[MaTeX, "DisplayStyle" -> False, FontSize -> 16]; Turning off history prevents unwanted references to symbols. Then we can observe how unreferenced results get cleared. $HistoryLength = 0; The inputs will be queued up for evaluation in this variable: queue = <||>; Let the placeholder be called `delayed`. It holds a unique `Temporary` symbol that allows for reference tracking, as well as the TeX input itself. If the input is not yet in the queue, it adds it: expr : delayed[s_Symbol, texcode_] /; Not@KeyExistsQ[queue, ToString[s]] := (AppendTo[queue, ToString[s] -> texcode]; expr) When the input is processed, the result will be assigned to the temporary symbol. The result from MaTeX always has head `Graphics`. If `delayed` sees this, it simply evaluates directly to the result: delayed[g_Graphics, ___] := g Finally let's add a nice printed form for these `delayed` expressions. This is irrelevant for the functioning of the system; it's purely for beautification. delayed /: MakeBoxes[expr : delayed[_, texcode_], StandardForm | TraditionalForm] := With[{boxes = ToBoxes@Tooltip[ Panel[ Style["$\[Ellipsis]$", "Code", ShowStringCharacters -> False, Background -> None], FrameMargins -> 2 ], texcode]}, InterpretationBox[boxes, expr] ] The `delayedMaTeX` function simply creates the temporary symbol (named as `id$...`) and places it within a `delayed` expression along with the TeX input: delayedMaTeX[texcode_] := With[{id = Unique[id]}, SetAttributes[id, Temporary]; delayed[id, texcode] ] Finally we will have an `update[]` function that processes the queue: update[] := Module[{}, queue = KeySelect[queue, NameQ]; (* drop keys that don't have a corresponding temporary symbol *) With[{syms = Symbol /@ Keys[queue]}, syms = MaTeX[Values[queue]] ]; queue = <||>; ] ---- We are ready to try it out now. Let's put some MaTeX tick labels in a plot: plot = Plot[Sin[x], {x, 0, 2 Pi}, Ticks -> { Table[{x, delayedMaTeX[x]}, {x, 0, 2 Pi, Pi/3}], Table[{y, delayedMaTeX[y]}, {y, -1, 1, 1/2}] } ]  Only the placeholders show for now. Let's process the queue: In[25]:= update[] // AbsoluteTiming Out[25]= {0.455054, Null} It was fast. Let's look at the plot again: plot  Everything is in place now. At this moment we have lots of temporary symbols holding the results: In[17]:= Names["id*"] Out[17]= {"id", "id$", "id$4271", "id$4272", "id$4273", "id$4274", \ "id$4275", "id$4276", "id$4277", "id$4278", "id$4279", "id$4280", \ "id$4281", "id$4282"} They are gone as soon as `plot` is cleared: In[18]:= plot =. In[19]:= Names["id*"] Out[19]= {"id", "id$"} ## To wrap up I hope you found this technique interesting. Any comments, improvements, additions, criticisms are welcome. If you use MaTeX yourself and you would like to have this feature properly implemented and integrated into MaTeX, let me know. It would eliminate any remaining disadvantages MaTeX has relative to [MathPSFrag](http://wwwth.mpp.mpg.de/members/jgrosse/mathpsfrag/). But given that MaTeX already uses a caching system, and that I personally don't find performance to be a problem, I'm not sure it's worth the effort. [1]: https://github.com/szhorvat/MaTeX Szabolcs Horvát 2016-11-21T19:49:25Z https://community.wolfram.com/groups/-/m/t/2741009 &[Wolfram Notebook][1] [1]: https://www.wolframcloud.com/obj/bcaad1e3-6a91-4b39-8aec-c829b4e94d11 Roman Maeder 2022-12-21T16:17:47Z