Group Abstract Group Abstract

Message Boards Message Boards

Wolfram Language vs. TensorFlow vs. Numpy - Coding on Simple Neural Network

2 Replies
Posted 8 years ago

I just want to clarify a few things about this post for

  • The timing results are completely misleading: for a real-world dataset, TensorFlow and the WL neural net framework should be faster than your NumPy code, and TensorFlow and the WL neural net frameworks should have very similar speeds for both GPU and CPU. Only in the special case of a super small dataset should you see these timing differences. There are many reasons: NetTrain starts plotting loss curves, both NetTrain and TensorFlow run a compilation step that tries to find any runtime optimizations possible by rewriting the computation graph of the net. All of this is completely unnecessary for tiny examples, but can produce major speedups for large nets, which is the case that people care about optimizing.
  • Lines of code versus TensorFlow: TensorFlow is designed to be a super low-level framework that gives maximal flexibility (it can do things that the WL neural net framework can't, due to this flexibility), whilst the WL neural net framework is designed to be as high-level as possible to be as simple to use as possible (it itself uses a low-level framework as a backend, MXNet). A much more interesting comparison is against Keras (a high-level framework built on top of TensorFlow), which you mention.
  • The comparison between NumPy and WL is strange: the analogue of using NumPy is writing a neural net from scratch using WL PackedArrays. Why not compare against that instead?
POSTED BY: Updating Name

Thanks for your comment.

1 - Yes, you are right about the timing, that's why I've pointed out that "we should not forget that this toy example is simple" in my post. In fact, the main focus of my post is the prototyping abilities instead of timing, and when I wrote the post I was thinking whether I should exclude the runtime in my post, or should I include it but pointing out that it is not really a good indicator for this case. Since timing is always what people eager to see, and people may even test it themselves especially in this simple case with just a few lines of codes at the appendix. Finally, I decided to include the runtime and point out the above consideration so that all of us are aware.

2 and 3 - The purpose of this post is to see the prototyping abilities of different approaches, from high to low level. This is why I picked these three languages/libraries including Numpy. I hope the codes in the appendix would help people to have some quick insight on how the tools abstract different concepts and can make good reference when comparing to the summary table. Perhaps in the future, we can have a comparison with another set of approaches when we want to look at another angle.

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard