Message Boards Message Boards

[LiVE] "Functional Dataflow"? Wolfram Live Coding Session // Mar 19

Posted 5 years ago

For those interested in data science, please join us for a 1 hour Wolfram Live Coding Session Tuesday Mar 19 2019 at 4pm ET on Zoom (https://wolfram.zoom.us/j/4659236576) . Please see the Linkedin post:

https://www.linkedin.com/pulse/functional-dataflow-wolfram-live-coding-session-mar-19-calvitti-phd

Abstract

This live coding session will focus on functional methods and patterns to write compact but flexible pipes for data science using real-world examples. The Wolfram Language is ideal to data science due to its functional orientation, large-scale symbolic processing and pattern matching abilities. Many of these features have no direct counterpart in object-oriented languages like Python. Consequently WL workflow is more programmer efficient: line-count ratios of 5:1 and often much more, are common relative to equivalent Python (inclusive of its extensive libraries). Functional programming lets you do more with less code and is more convenient and safer to refactor. After a brief tour of the forthcoming book Functional Dataflow, the focus will be on data transformation, aka "data wrangling" - recognized to be a highly time-consuming aspect of workflow - rather than on math or statistical methods. Exposition will be adapted from Excursion chapters in FD. Key skills described include composing operator pipes, combining tabular, hierarchical, temporal and graphical datasets, exploring queries on data slices then scaling up to the whole dataset, creating insightful visualizations, and writing flexible utility (eg one-liner operator) libraries for reuse in future projects.

Overview Mathematica notebook containing links to data and dependencies: enter link description here

POSTED BY: Alan Calvitti
17 Replies

Hi Alan, I liked the functional dataflow stream! Would be great if you could share this stuff on the "functions" page somewhere. The now famous [Bullet] solution baffles me a bit so might deserve a new thread. I tried to work with it (following the above definition) but it seems to work only on self defined functions and not on build in. Anyway again look forward to follow a new stream.

POSTED BY: l van Veen
Posted 5 years ago

l van Veen

but it seems to work only on self defined functions and not on build in

Make sure you are using the definition in the reply from Andrew Steinacher.

Here is a very simple example of using the "magic bullet". Pick a function that does not have an operator form (most likely because it is varargs). To use it in postfix form

Range[10] // Take[#, {3, 5}] &
(* {3, 4, 5} *)

Using the "magic bullet" is a lot cleaner

Range[10] // Take[\[Bullet], {3, 5}]
(* {3, 4, 5} *)

There are a lot of more complex examples in Alan's notebook.

POSTED BY: Rohit Namjoshi
Posted 5 years ago

Since I'm on Windows and don't have a shortcut for 'bullet', I find \[NumberSign] a bit faster: esc # esc

POSTED BY: Updating Name

Use whatever you like - ideally this would be standardized in the language.

But I would advise against [NumberSign] in particular: (1) it looks too similar to Slot notation - especially if you share code, you can't control other users' fonts. (2) how is 3 keystrokes in esc # esc faster than 2 keystrokes in option 8 (alt 8 on Windows)?

POSTED BY: Alan Calvitti

Yes, agreed that the notation is too close to Slot.

Alt-8 on Windows wasn't producing a bullet for me...hum.

POSTED BY: Vincent Virgilio

For the bullet • character in Windows, try with ALT-7 where the 7 is from your numeric key pad. That is easier than ALT-0149 which also works.

POSTED BY: Gustavo Delfino

Windows Alt-7 on numeric keypad indeed produces a bullet.

Thank you!

POSTED BY: Vincent Virgilio

That's right, so that the bullet can be placed anywhere for any function taking arbitrary number of parameters.

POSTED BY: Alan Calvitti

Slick.

Seems like pre and post are heads of BlankNullSequence's? (three underscores)

POSTED BY: Vincent Virgilio

I find some unfamiliar syntax in Example "In[215]Â…". What is the solid dot following "10^" and in the first argument of StringJoin? I thought I knew functional notation...

Thank you,

Vince

POSTED BY: Vincent Virgilio

Vince,

That's a position-coded curry syntax that I invented and use extensively. It's found in Appendix A of my book but here it is.

[Bullet] /: f[pre, [Bullet] , post_] := With[{n = Length[<a href="mailto:List@pre]">List@pre], m = Length[List@post]} , Curry[f, Join[Range[n], {n + m + 1}, Range[m] + n]][pre, post] ];

Put the solid dot, aka bullet (Option-8 on most keyboards), in the position of a function's argument list to curry that parameter. As per operator forms in general, It streamlines syntax in data pipes.

Thanks to Jose Martin-Garcia from WRI for help in refactoring from an earlier version.

POSTED BY: Alan Calvitti
Posted 5 years ago

Wow, that is very cool. Looks like several characters were interpreted as markup and messed up the definition. I think this is the correct definition.

\[Bullet] /: f_[pre___, \[Bullet], post___] := 
  With[{n = Length[List@pre], m = Length[List@post]}, 
   Curry[f, Join[Range[n], {n + m + 1}, Range[m] + n]][pre, post]];
POSTED BY: Rohit Namjoshi

Thanks for the edit. And yes beyond cool, this bit of syntactic sugar simplifies readability. There's a good thread on mathematica.stackexchange about operator forms versus nested function/slot lookup (see especially Mr. Wizard's comments)

POSTED BY: Alan Calvitti

Thanks! But I think you missed one _ character after the f, which makes a big difference:

ClearAll[\[Bullet]];
\[Bullet] /: f_[pre___, \[Bullet], post___] := With[
   {
    n = Length[List@pre],
    m = Length[List@post]
    },
   Curry[f, Join[Range[n], {n + m + 1}, Range[m] + n]][pre, post]
   ];

I have to say, this is perhaps the coolest thing I've stumbled across in a long time. As another Windows user, I couldn't find an easy key combo to enter the bullet point, so instead I found that hitting Escape bu Escape to print out \[Bullet] in a notebook works pretty well (and should be OS-independent). It's perhaps not as fast as Option + 8, but it should be good enough.

As somebody who really likes to use RightComposition, this would be a fantastic feature to build into the language, as it's quite a pain to have to use parentheses (and the &) when chaining many "pure functions" together. Until then (assuming it would be implemented) I think I'll be including this definition into more of my projects.

Posted 5 years ago

Thanks Andrew. I updated my reply.

POSTED BY: Rohit Namjoshi
Posted 5 years ago

Alan,

Looking forward to this. BTW the link above is broken, I think it should be this.

Rohit

POSTED BY: Rohit Namjoshi

Rohit,

Thanks. Was the original link malformed or misspelled?

Al

POSTED BY: Alan Calvitti
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract