I was surprised how pleasant these music tracks actually are. Thank you for sharing! How much time did this project take? Also I am curious about data -- did you publish your dataset, maybe on that GitHub repo? Is this all modern-classic composers based or there is some old classical music too for training? Any famous composers?

I built my dataset from a collection of my favorite piano scores from musescore. You want a lot of different .mid files so the network you train cannot simply memorize all of your training data and instead has to perform some amount of information generalization. Our data augmentation strategy helps with this, but it will not completely solve the issue of having too little data. I have found that 150 songs is acceptable when using smaller RNNs.

BTW about repos. You might consider for your work Wolfram's Neural Net and Data Repositores:

• https://resources.wolframcloud.com/NeuralNetRepository
• https://datarepository.wolframcloud.com

-- you have earned Featured Contributor Badge Your exceptional post has been selected for our editorial column Staff Picks http://wolfr.am/StaffPicks and Your Profile is now distinguished by a Featured Contributor Badge and is displayed on the Featured Contributor Board. Thank you!

I am glad you enjoyed this project, Michael. It is encouraging that you were thinking of a similar project: great minds think alike, as they say!

Your suggesting of classifying patterns and transposing / inverting / etc. them brings to mind the analogy of using words-level encoding vs character-level encoding for tokenizers in language models. Word-level encodings typically result in higher performance in language processing tasks, so I would expect it to perform better on musical data as well.

I have thought some about creating word-level encodings for music (e.g. an extension of BPE/SentencePieces/WordPieces), but I was unable to come up with some robust method of encoding the metadata for musical words (note timings and volumes for notes in the word). A successful approach would likely involve timing quantization and categorizing notes into timing patterns (e.g. triplet, quarter-note, 16th-note) as is done in sheet music. Even combining chords into the same token is tricky because it would likely need to have the data of whether-or-not to play the chord as an arpeggio.

Your comments about inversion, transposition, etc. also bring to mind another idea I was thinking of: encoding pitch values as {note-name, octave} pairs. The current implementation encodes note pitches as one-hot vectors of 120 possible pitches, but this encoding likely makes learning simple octave-shifts or inversions quite tricky for the model. By instead encoding the pitches as a one-hot length 12 vector for note-name (A, A#, B, ...) and a one-hot length 10 vector for octave, octave shifts and chord inversions should become much simpler/faster for the net to learn, even if we are still doing our 'character-level' note-by-note encoding of music data.

Honestly, I have not done any research into what others have done in the music space. I would assume some clever methods were used to encode notes into higher-level patterns when earlier music generation work was done using Markov models.

Thanks for the thought-provoking suggestions, and sorry for the lengthy reply!

Also, many works on MuseScore.com have been published into the public domain, including many great classical compsers' works. These public domain works can be downloaded for free without a 'pro' license since there are no royalties to pay, and they could form the basis of a dataset that could be freely distributed.

Such a dataset could be uploaded to the Wolfram Data Repository without issue!