Message Boards Message Boards

8
|
4383 Views
|
2 Replies
|
12 Total Likes
View groups...
Share
Share this post:

Idea: Well-specified consistent array/list interface

Posted 7 years ago

Mathematica has an increasing number of "things" that behave like lists. Some other languages would call these containers.

It would be useful to formalize (within the Wolfram Language) what it means to "be like an array/list", i.e. specify an interface that is guaranteed to work for all array-like objects. Then we could write functions which we are confident will work with any list type, even those that are not yet available in the current version.

A few versions ago this was not really necessary, as we only had List and sparse arrays (version 8.0). But now we have at least List, SparseArray, ByteArray, StructuredArray, QuantityArray, RawArray, Dataset, and maybe other things that I did not think of or have not yet encountered. Maybe we can even include associations.

These tend to support:

  • Indexing by Part
  • Extract
  • Length, Dimensions
  • Take, Drop
  • First, Last, Rest, Most
  • Append, Prepend
  • AppendTo, PrependTo
  • Join
  • Normal for converting to a plain List

But there is no full consistency between them.

Then there are many other functions which may work with some, but don't work with all of them, even though their name makes it reasonable that they could. Examples: VectorQ, LengthWhile, TakeWhile, and all the various things that operate on "lists", but aren't strictly list related: Tally, GroupBy, etc.

Most of these things are AtomQ, but some of them have their innards exposed to some functions (often pattern matching: see SparseArray), which makes it hard to predict their behaviour. Others look like they are one thing, but they are really something else. QuantityArray looks compound in its default display, but it is really atomic and its head is StructuredArray. In Mathematica "everything is an expression", which used to make things simple to understand. But things are no longer simple.

Some programming languages deal with this by having features to define and enforce interfaces. This is typical for object-oriented languages. Others, like C++, do not enforce the interfaces, but their specification still defines them precisely. See for example concepts in C++: the container concept describes what one can do with containers. Then the standard library provides a number of functions/algorithms which will work with any class that conforms to this concept. The fact that this concept is documented makes it easy to either develop new algorithms that will work with any container, or develop new containers that will work with the algorithms in the standard library. In Mathematica, one would typically want to do the former (develop new algorithms, not containers).

I do not think that enforced interfaces would fit well into Mathematica.

But something like C++'s concepts would be useful. Then we could write functions that will work with any "list", without having to test it individually with every single list type. There could also be an alternative to a ListQ function which will return True for any object that conforms to the "list concept". It could also serve as an interal guideline within WRI for the times when new "lists" are implemented (like ByteArray), so that the feature set they support wouldn't be arbitrarily picked. Consistency is always good.


What do people think about this idea? Would it be useful to document a "list concept" and provide a function to test for it? Or am I making a big deal out of nothing?

POSTED BY: Szabolcs Horvát
2 Replies

"Everything is an expression" is probably the general characteristic that I most love in Mathematica.

And then Images were born... The need for a different way to treat these "huge" objects was pretty obvious. The solution, in the case of the Images, in my opinion, was proven acceptable, but I wonder which other solutions were on the table.

And then Associations came, and a feeling that Mathematica had just developed a taste for Atoms... But I remember hearing someone say, when Association were live announced, that, "for now", Associations were Atom, but that it was still a possibility of making them non Atom on future releases. Not sure if that was meant as an implementation or as a global behaviour. But now that the new technology is being built on top of these structures, I guess that that option is probably out.

Besides the obscure nature of Atom structures, which I think goes against Mathematica "brand", they tend to become incomplete and messy on the edges. So, I'm completely with you. A solution is needed before it gets out of track.

Cheers,

POSTED BY: Pedro Fonseca

I would definitely include Associations. I was actually thinking the same thing some times ago, we get more and more 'types' of data. ByteArray being the newest addition. It reminded me a bit of labview wires types:

enter image description here

Over time they got more and more data types, that can be converted to each other and so on....

I'm not sure what the solution for Mathematica is though; all these new 'datatypes' are made for memory efficiency (most) or convenience/speed (Dataset/Associations).

EDIT: Yeah fully agree a function like ListQ that tells us if that object (data type) conforms to the 'list concept' (with a good definition of that somewhere in the documentation) would be useful! ListConceptQ?

POSTED BY: Sander Huisman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract