Message Boards Message Boards

GROUPS:

Why is some Graph functionality underdocumented to the point of unusabiliy?

Posted 1 year ago
2693 Views
|
6 Replies
|
23 Total Likes
|

There are several pieces of functionality related to Graph which are unsatisfactorily documented, to the point that they are simply unusable for real work. Even after repeated requests to support, the responsible developers simply refused to clarify these points. "There is no information available." This is a common response I receive when asking about functionality in the domain of graphs and networks.

How can Mathematica be taken seriously as a tool for network analysis or graph theory, if we don't even know what some of the functions do?

The last response I received suggested that for support, I might consider Wolfram Community, so I am posting here.

DegreeGraphDistribution

What is DegreeGraphDistribution? There is basically no information about it.

DegreeGraphDistribution[dlist] represents a degree graph distribution with vertex degree dlist.

"Degree graph distribution" is not a commonly used term. In fact, a google search only returns hits to the Mathematica documentation (or forums). So it's no exaggeration to say that the documentation says nothing at all—the description is a tautology.

Of course, I can make guesses about what DegreeGraphDistribution is supposed to be, but who among the readers of this site would be willing to publish a paper based on results from a tool whose function they are just guessing about? And when I make those guesses and try to verify them, I find many inconsistencies.

I won't repeat the comments I made about this function over the years. Here's one example: https://mathematica.stackexchange.com/questions/167450/does-degreegraphdistribution-sample-uniformly Be aware that the sampling may not even be close to uniform, depending on the input.

RandomGraph[DegreeGraphDistribution[{5, 5, 2, 2, 1, 1}], 10000] // 
   CountsBy[AdjacencyMatrix] // KeySort // Values

FindGraphCommunities

There is basically zero documentation about the methods used by this function. The only problem is that with community detection, "what the function does" and "how it does it" are basically the same question. Thus the function is literally useless—we don't know what it does.

It's not like computing eigenvalues where I can define what an "eigenvalue" is without detailing a procedure to compute it, and even verify that an eigenvalue/eigenvector result is correct without knowing how it was obtained. With community detection, the precise definition of the problem is the method itself (the rest is just an intuitive but far from precise statement).

When I ask about the methods of FindGraphCommunities, there is usually no satisfactory answer. I did receive references to general papers such as https://arxiv.org/abs/0906.0612 but it was without specific statements on what is being used from this paper. I was no more knowledgeable after receiving this link than before I asked ...

Other users have the same experience (e.g. link1, link2).

Did you receive a response from support? – Szabolcs May 24 '18 at 7:42

@Szabolcs Nope :( – becko May 24 '18 at 8:31

 


These are the experiences that make me consider again and again if using Mathematica for working with graphs is a good idea at all, and a safe investment for the future. I wrote about this before and @Charles Pooh responded:

We are looking forward to dramatic expansions in Graphs & Networks functionality. We are even more excited about its many applications across different domains and the graph related functions in the pipeline.

Indeed, a class of important bugs were fixed in M12.0: now we can use EdgeDelete/EdgeAdd and VertexDelete/VertexAdd without breaking the graphs. But then consider how long that took (from version 8.0 to 11.3 this is broken), and how fundamental these operations are... These were overdue fixes, not dramatic improvements.

Additionally, several functions now (finally!) preserve graph properties, a critical functionality. Yet none of this was documented for Mathematica 12.0, and it's still unclear how properties get combined by functions such as VertexContract. I could only learn about this change from one of the live-streamed meetings on Twitch. (They were also mentioned in the Graphs Twitch talk aired a week or so ago, which makes it even stranger that there's no documentation.)

What I am looking for is not for claims that issues will be dealt with, but for actual actions that show that our concerns are not dismissed. I find the responses relayed by support to be very discouraging. They definitely do not inspire confidence in the product. I understand that developers are busy sometimes, but the responses have been like this throughout the years (see e.g. the links above), not just in a busy period.

6 Replies

I should mention that despite stepping up the Graph bugfixes a bit for M12.0, there are still multiple bugs of the sort that should never survive for longer than a single release if they are known ...

For those who are unaware, don't ever trust any result from FindMaximumFlow or FindVertexCut ...

Szabolcs, please answer me frankly: is there any serious reason why to use Mathematica 12 for any kind of networks or graphs research?

From my point of view the Mathematica is going to be a more and more the magic black box tool, which is able to solve nearly everything, but surprisingly often not enough effectively and/or precisely.

Underdocumented functions are only one tip of problems iceberg.

If I didn't want to use Mathematica to work with networks, I wouldn't be complaining ... I would just jump ship.

There is, of course, a lot of value in Mathematica. I think that Mathematica really is a much better integrated system than what you might find elsewhere. This is not just marketing. Most functions will easily work together, even if they are from very different application areas. Working with networks usually involves a lot more than just manipulating Graph expressions. I might also want to do statistics on their features, visualize results, do linear algebra on graph matrices, convert between images / meshes / graphs, encode graph problems into ILP or SAT and solve them, etc. Having all these functions work well and easily together has great value in it.

Also, the Graphs & Networks functionality is broad, with lots of graph algorithms implemented. There is huge potential here. This is precisely why the problems I described above are so frustrating. Unfortunately, I still have the impression that Graph doesn't get as much attention as other functionality areas in Mathematica, and that practical user needs do not always seem to be taken into account.

Regarding the black box functions, we all know that this is one of the major criticisms of Mathematica, but I think that the criticism doesn't apply in every case. There was a question on StackExchange recently, asking whether FindIndependentEdgeSet uses the blossom algorithm. Of course, it would be nice to know what it uses, but I don't really need to know that in order to be able to use the function. The mathematical problem that the function solves ([graph matching](https://en.wikipedia.org/wiki/Matching_(graph_theory))) is clearly defined without reference to any algorithm. The solutions it returns are easily verified without knowing how they were obtained.

FindGraphCommunities is different. It is not very useful without knowing how it works—because "how it does it" defined "what it does" (more or less). What the function does is only described in an intuitive way, not in a precise way.

Actually, I have alternatives both for DegreeGraphDistribution and FindGraphCommunities in IGraph/M (based on igraph). The other day I was working with a network where FindGraphCommunities returned results that intuitively seemed more reasonable than what I could get with igraph. I would like to use these results. But in this case I can't because I don't know what the function does to formalize the intuitive concept of "communities".

(For those who want to nitpick, I know that I can compute the modularity score of a result, and use it to evaluate how good it is, regardless of how it was obtained. But modularity is not the end of the story when it comes to community detection ... and the documentation doesn't even claim that all FindGraphCommunities tries to do is maximize modularity.)

Actually, looking at the names of the available methods lets me make guesses about what each of them does ... but when I try to verify those guesses, then small things don't match up. How can I even verify the guesses without re-implementing the whole method? If I do, then what would be the point of using Mathematica?

This is why the lack of responses from the developers is so frustrating. It would take so little to make this function useful, but we are denied this, for no apparent reason.

Not everything in Mathematica is like this. Look at the old graph drawing tutorial (written by Yifan Hu perhaps?) It has references to the papers describing the algorithms that it implements (at least the original versions of these algorithms). Why couldn't this be done for FindGraphCommunities?

Newly added graph drawing methods, like "GravityEmbedding", have no such information. I asked support how "GravityEmbedding" worked (and what it's "RootVertex" suboption does) and received no clear response. Of course, for graph drawing, knowing the method is not critical ... but it's still very useful. What I do not understand is why would the response be denied? It's a newly implemented method (new in M12.0), so the developer who created it must still be around and could answer ... all it would take is to send a link to the paper describing the method. I assume there is a paper because during the Twitch stream (live-streamed meeting) it was mentioned that this graph layout was added based on a user request. It couldn't have been requested unless it was a published method.

Denying the response just leaves a bad taste, and diminishes my confidence in both the system and WRI.

Another symptom of such problems is that "GravityEmbedding" is designed in such a way that it gets in my way: it doesn't only change the layout, but also changes the EdgeShapeFunction, as if it weren't just a vertex layout, but a PlotTheme or GraphStyle ... It was claimed that this is so that it would return a "nice" and immediately usable result, but in fact it does exactly the opposite: it forces me to spend extra time and effort to set the edge shape back to something that looks similar to the default. That takes a lot more effort than you'd expect because the possible settings for EdgeShapeFunction are not properly documented (same problem again) ... and I still don't know what the correct setting is to get the default.

I remember that during the Twitch stream even Stephen Wolfram asked why GravityEbedding changes the edge shapes (and no convincing response was given), so I'm not the only one who thinks that this is not reasonable.

The example of GravityEmbedding illustrates well a disconnect with actual user needs, and a lack of willingness to communicate with the users of the product.

Update

I admit that the above was an impulse post stemming from frustration due to responses (or lack of) relayed by support. Now that it's out it can't be removed. I hope it did not antagonize anyone, as that wouldn't improve anything.

In fact, I think what could improve the situation would be more public interaction between the Graph developers and users. I think many of us feel like our feedback about this functionality area is often falling on deaf ears. By interaction, I do not mean generic statements that work is being done, but concrete comments on the specific issues that we bring up.

To be fair to the developers of the Graph functionality, I should note that the bug fixes that went into Mathematica 12.0 were well chosen. I really do wish that more issues had been fixed, but if WRI's resources only allowed for fixing this much, the prioritization was correct in my opinion. (Data corruption by [Edge|Vertex][Delete|Add] was indeed the most pressing issue.)

I also think that a big opportunity to gain goodwill was missed by not documenting the biggest Graph changes (property inheritance) for M12.0.

If they are undocumented then they do nothing except what the examples show. (well programmed unix things, like Motif, have a variable exposed for anything that could change, and three ways to get at changing that exposed variable - almost to the point of monotony - yet ending up a compact program)

I liked earlier versions that had all .nb (or Put or Needs) for all kernel libraries.

However that makes it easy competition to steal and offer the same product at a lower $ (please internationalize that if need be).

Wolfram is outreaching to community and does answer such questions about some given function when the need arises though.

In my case I made a raytrace front end for mm and got really burned because many functions not only are undocumented but do not give Graphics output: they ONLY give opengl output internally so to replicate them it meant writing a new function that does the same thing for each one.

I probably should have asked wolfram a few questions to save time instead of doing certain things "the hard way". And that's my point. You have only to ask them.

((No matter if I went forward I would code it all in MM anyway, but because mm's 3D has gone new directions that are good but very different from rendering software abilities - i likely won't do it even as a hobby - because then i'd be also coding all of modern opengl use as well as all mm un-exposed things - a bit of a task for one unpaid person i'd say!))

My complaint would be Mathematica is great as a math swiss army pocket knife (the functions allow any specific goal to be achieve without providing a function for every conceivable goal). But Wolfram seems to have paid employees to add many small program functions that do very little (nothing that mathematica couldn't do without them), all in a name. On the other hand my PC is still running fast enough so I can't complain. The addition of Geo and Chem and other things are great.

However your answer may be this (an educated guess). DegreeGraphDistribution[] is used to provide a service to a higher Mathematica function that IS well documented, so you don't need to know how it works, because it is a service function. That's typically the kind of answer I've seen over the years which may apply here.

Isn't graph theory a mathematical theory about mathematical structures that can be ENTIRELY represented by symbolic structures WITHOUT any reference to graphical representations on a screen or piece of paper?

Then the mathematical representation might be an Association or a nested structure of Associations but have no information at all pertaining to a graphical representation such as vertex locations or their graphical style or the style of links - but only their mathematical graph properties.

Then there might be high level routines for the graphical representation of some common types of mathematical graphs. But this would not be expected to handle everything and so a user could use the mathematical representation to build up primitives for whatever graphical representation he wanted. The mathematical graph would be manipulated by Mathematica statements - but never through the graphical representation, which is derivative.

Wouldn't this make it much easier to work with mathematical graph theory, and make it much easier to tailor graphical representations from graphical primitives up rather than from a complicated structure down?

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract