Message Boards Message Boards

Visual Interface to the Wolfram Language

GROUPS:

Hi everyone.

I'm working on a visual interface to the Wolfram Language called visX, and I'd like to ask what you all think of it.

Wolfram Language code can often be thought of as a set of blocks, each of which takes some inputs, does something, and produces an output. VisX lets you write WL code exactly this way - you draw a digram, connecting blocks with links. For example, say you want to count how many times each digit (0 to 9) occurs in the first 30 digits of Pi. With text-based WL code, you'd write

digits = RealDigits[N[Pi, 30]][[1]]
Count[digits, #] & /@ Range[0, 9]

In visX, you'd draw this:

digits of Pi

I guess it's pretty self-explanatory. In addition to using built-in WL blocks, you can write your own, like the CountInList block. Normally, blocks just transform inputs to output, but the CountInList block is mapped over its input which is indicated by the little brackets on the outside of its connection ports. (That's basically visual syntactic sugar for "/@" or Map.) The 4 in the upper-right corner indicates that results inside this block are showing results from the 4-th time through the map. The block with "digits" in it sets a variable, which is then referenced in the CountInList block.

You define blocks (which are basically functions) by just making an empty rectangle and dragging contents in then wiring them together, then you can use copies of the block wherever you want. A change in any copy of the block will be reflected in all other copies. There's no real difference between a defining a block and using it. Recursion can be specified by just including a copy of the block inside itself. Blocks can call other blocks in the same manner.

Just like regular WL code, visX blocks can be nested deeply, but with the visual interface, it's easy to zoom in and out. At any point, the UI will show you the right amount of detail for each block - sometimes no detail at all, sometimes its name and labels on its inputs, sometimes its actual contents (which can then be edited or further zoomed...).

visX is stand-alone software that runs locally on your machine, evaluates the diagram using your local Mathematica kernel, and receives the results and puts them back in the diagram. You can load data files using Import as usual.

One of the problems that I've seen with visual languages in the past is that while simple things are easy to do, the code quickly gets too complex to manage and the visual interface starts to get in the way. With the Wolfram Language in theory everything is an expression, and this can lead you to write functional-style programs which are easily thought of as a diagram, but that's not always the most natural way to express a computation. Sometimes you just need a little for loop. Consider calculating Fibonacci numbers. Start the sequence with 1, 1, ... then each element of the sequence is the sum of the previous two. Yes, you can write a recursive algorithm to do this, but most people just want to write a little for loop. In visX, you can do this (calculates the 6th Fibonacci number):

embedded code

I've tried to let you use blocks-and-links when that's the most natural thing (which is usually), and text-based code when that's better. Of course, you can mix them together however you want.

A second problem I've found with visual programming languages is that it can actually be much slower to use then writing out text, because you have to laboriously drag and drop every single block. Even simple algebraic expressions like

Sin[x]^2 + Cos[x]^2
2x^2 + 4x*y + 8y^2

would involve a lot of blocks because of all the Plus, Times, and Power blocks, as well as all the constants and symbols. With visX, you can enter Wolfram Language code snippets like those, and it will parse them and transform them into blocks which you can then insert into your diagram all at once and edit at will. This makes it much faster to get your idea onto the screen so that you can start evaluating it and developing it. I'm also working on the ability to take a visX block and give you back the Wolfram Language code that it represents.

The examples given here are simple, but of course you can use this interface for putting together a complex piece of code as well. I find it especially handy when building up a calculation with lots of intermediate results along the way, or to rapidly prototype an algorithm where I want to be able to easily switch the data flows around.

Does this project seem useful to anyone? I'd like to get some feedback - what do you think of it? Would you use it? For what?

If there's interest, I could do a small-scale alpha test in about a month from now.

More info at visx.io.

-Nicholas Hoff

edited to clarify block definitions and recursion

POSTED BY: Nicholas Hoff
Answer
3 months ago

Nicholas Hoff

I found this very interesting. I believe it would be an extraordinary way to teach programming or as you mentioned as a new way of looking at complex mathematical procedures. It may help to sell mathematical solutions to managers that do want to know the structure of such solutions but want to skip programming details. I want to teach programming to my 11 years old son, this may be great tool for that.
I wonder if visx could run inside a notebook??? I will visit the web page to look for more details

Answer
3 months ago

I hadn't thought about that. This might be a good way to let managers or other non-programmers in a company understand the interface for an algorithm (just look at the input ports) and even make minor changes at one level of zoom without having to look at a screen full of code. I guess the question is whether there are actually any managers or colleagues who want to do that.

I've also thought about using this to teach students to program, specifically with the Wolfram Language. It would probably be an effective tool, and might help them get into it without being scared away by syntax. I've never done any tests with students though.

As for running inside a notebook, I think some significant changes would be required. I don't know how the frontend is written, I assume it's C/C++, probably mostly platform-agnostic with some platform-specific tweaks. My code is mostly platform-agnostic JavaScript, and it wouldn't be too hard to rewrite or adapt. I'm sure it's possible, but I don't know how much work would be involved on either side. It sure would be neat though.

POSTED BY: Nicholas Hoff
Answer
3 months ago

This very much reminds me of labview (national instruments) and simulink (matlab). I've always found, with both of them, that it becomes quickly messy (i.e. spaghetti). But for some (simple) things it can be nice to visualize the data-flow.

How are function-definitions and function-calls implemented in your 'drawings'? This is very fundamental in Wolfram language, and needs to work very well to be useful. Not to mention recursion of a single functions or two functions that call each other…

Just out of curiosity, what language did you program that in? Seems already quite a bit of work!!

POSTED BY: Sander Huisman
Answer
3 months ago

Thanks for the feedback.

I've used Labview and Simulink in the past, and I never found them too useful beyond toy problems because of what you said - it just turns into spaghetti after a while. I had these problems in mind when designing visX, and here's how I avoid them:

  • The amount of detail shown is progressive - only as much as will fit nicely on the screen given the current zoom level. You can draw a block, test its inputs and outputs to make sure it works, then zoom out and not even see the contents.
  • You can use text-based code as input. Input blocks don't just have to contain single atom, they can contain a string of WL code like {a, a^2, a^3}. That already saves a bunch of blocks (List, Power, a, 1, 2, and 3).
  • You can use text-based code in the middle of an algorithm, as in the Fibonacci example. That saves lots of blocks when you need it.
  • The syntactic sugar for Map makes looping intuitive. Some Labview graphical for-loops I've seen were truly mind bending.
  • In the Wolfram Language, there's only one type, so you don't have to worry about conversions and different colored ports and wires. In general, the Wolfram Language lends itself nicely to having a visual interface.

So while some computations certainly involve a large number of blocks, I've found it to be fairly easy to deal with.

To define a function, you draw an empty rectangle and start dragging in blocks and wiring them together. You could then use your block in the same place by wiring some values to its inputs. If you want to use the same block again in a different place, create another instance of it (just like you can have multiple instances of Plus or any other function). Changes in one instance are reflected in all instances, and there is no one "special" instance which constitutes the definition. So, there's no distinction between where you define the block and where you use it.

Recursion can be handled by including a copy of the block inside itself like this:

infinite recursion

The infinite recursion block is only defined once, but the definition includes itself, just as in text-based code. Functions that call each other or mutual recursion work in the same way. Basically, whatever's inside the block will be compiled to Wolfram Language code. You could also imagine setting attributes like HoldFirst on a block, although I haven't implemented that yet. It wouldn't be hard.

The user interface and the diagram-to-WL compiler come to about 7k lines of JavaScript/Coffeescript combined. There are 100 or so lines of Wolfram Language code and little bits of other glue.

POSTED BY: Nicholas Hoff
Answer
3 months ago

I think this idea is more related to the "systems of systems" view rather than views that represent problems as "planar networks" that very quickly become messy . In my modest opinion, the ideas is closer to the systems of systems view behind Wolfram SystemModeler rather than to ideas used in the Simulink or LabView interfaces. As an electrical engineer I prefer messy "circuit" representation to long, also messy, conventional programming codes such as m functions in Matlab or other programming languages.

Great job!

Regards Jesus

Answer
3 months ago

enter image description here - you have earned "Featured Contributor" badge, congratulations !

This is a great post and it has been selected for the curated Staff Picks group. Your profile is now distinguished by a "Featured Contributor" badge.

POSTED BY: Moderation Team
Answer
3 months ago

Today, by chance I stumbled across a visual programming tool for interactive data analysis workflows called Orange. Originally developed for the biosciences I see it's now also being used in digital humanities workshops. Below is a screenshot of a simple, labeled workflow ('recipe') using the text pre-processing component.

Textable recipe ps. Another example of a visual programming environment is Apple's (no longer supported?) Quartz Composer.

POSTED BY: Arno Bosse
Answer
3 months ago

Thanks for the comment. Orange is a fairly new one and the graphics look nice.

There are a lot of blocks-and-links interfaces out there for everything from data science to GUI design to 3D rendering. I've used a bunch of them, and one problem I've found with these tools is that they try to shield the user from the underlying complexity of the task, but the easy-looking interface makes the user think that the tool has actually reduced the underlying complexity. This leads to a situation in which non-technical users think "oh it looks simple, I can handle this" but in reality, they can't get past the examples. They want to make a plot and they see a function called "Plot" so they connect their raw SQL database dump to Plot (following a previous example in which "file" went into "Plot") and are confused why it doesn't work. Or they didn't realize that there's a difference between a number and a list containing that number as a single element, which causes some function to operate incorrectly on it. The simple examples look great, but when you try to change them to do your own work you run into those problems. Basically, graphical programming is still programming. Is this your experience as well?

My goal with visX is not to shield users from any complexity, but rather to give them another option for how to get their ideas onto the screen. You can enter code and transform it to blocks, you can draw blocks and transform them to code, and you can inter-mingle code and blocks however you want. I think visX would be useful for people with moderate familiarity with the Wolfram Language, and for students who want to learn. (I could be wrong though - that's what I'm here to learn!) Hopefully, though, they could use it beyond the examples.

The other thing that some graphical programming tools do is to make lots of assumptions so that the diagram can be simple. People find these tools useful in niche applications. For example, you can make tools that do allow you to push complex data formats (as often found in biosciences, or a SQL dump) into a block called "Plot" and get something useful out of it, but then you have a very specialized tool. It's powerful but not flexible.

The great thing about the Wolfram Language is that it's both powerful and flexible. Diagrams like the one you showed can be done just as simply (see below), but of course you can also do complex computations without the system making assumptions that get in your way when you try to go beyond the examples. That's the idea at least, what do you think?

word count

POSTED BY: Nicholas Hoff
Answer
3 months ago

Hopefully there is still time to add my feedback: My introduction to programming was through Autodesk's Dynamo platform and its node-base interface was the perfect push I needed to learn to program. Before I was a coder through, I was a supporter of more computational math and programming in our classrooms. I see a great educational benefit for this type of MMA plugin because it lowers the barriers for learning to code.

POSTED BY: B Birdsell
Answer
2 months ago

Hi, thanks for the post (and sorry for the delay in responding). Yes, I think education is a great application for visX, and specifically learning to code. Some people are intimidated, even by the friendly syntax of the Wolfram Language. It could really help them but they never get past the initial hump. As I wrote elsewhere, visX doesn't actually reduce any of the complexity of the task a person is trying to solve, instead it makes it easier to deploy the power of the Wolfram Language to that problem and get your ideas on the screen. It may even be the case that once people start using visX and see what WL can do for them, they then transition to a notebook and become frequent Mathematica users. In any case, I'm eager to involve some students in my alpha testing.

POSTED BY: Nicholas Hoff
Answer
2 months ago

This is definitely a nice feature. It's also interesting to do the reverse. Create code by drag and drop.

POSTED BY: Rubens Zimbres
Answer
1 month ago

Nice work. The advantage of graphical and text based programming can be efficiently used through this method. How to test VisX in the work?

POSTED BY: Suman Banerjee
Answer
1 month ago

Group Abstract Group Abstract