Message Boards Message Boards

[WSS18] Generating Music with Expressive Timing and Dynamics

Posted 6 years ago
Attachments:
POSTED BY: Pavlo Apisov
9 Replies

enter image description here - Congratulations! This post is now a Staff Pick as distinguished by a badge on your profile! Thank you, keep it coming!

POSTED BY: Moderation Team

Very nice. You generated jazzz :) The Only possible outcome.

POSTED BY: l van Veen
Posted 6 years ago

Thank you!

Yeah...jazz was a little part of the dataset but it gave a big influence :)

POSTED BY: Pavlo Apisov

Nice project. You must realize that MIDI is absolutely awful at describing 'real' music. For a keyboard player, it might be sufficient to specify pitch and duration, using equal temperament, but for a wind player or especially a string player or singer, most of the stylistic information is missing. This is why a MIDI rendering of a solo violin piece (Bach's Chaconne, for example) or a flute piece is flat and mechanical.

I looked into inventing a replacement for MIDI, where you could specify the attack -- which is highly variable for most instruments -- and the way the sound evolves over time. However, it became apparent that even for 'simple' music, there were simply too many variables. It was easier to just learn the instrument. ;-). I am just an amateur flute player, but I can vary the note attack in 10 or 15 different ways, and change the way the sound developed, including adjusting the timbre during the note's elaboration is several more. A professional can do a lot more.

Professional musicians can play in strict time, of course. However, for many types of music, such as Chopin, the controlled deviation from strict time is an important performance criterion.

Posted 6 years ago

MIDI representation loses an enormous part of musical information. This is the price you pay when you want to generalize a music notation. To be honest, I don't know much about a classic music notation or MIDI notation but as I understand MIDI tries to "digitize" a classic one. Correct me if I'm wrong but a classic music notation also lacks those things that you've mentioned.

If we want to learn specific techniques of playing an instrument we must learn it from raw music data(sound). Here is the latest research in this directions: https://twitter.com/deepmindai/status/1012290879120429056

POSTED BY: Pavlo Apisov

I have read about and heard examples where recordings of piano music -- specifically Glenn Gould's recording of Bach's Goldberg variations -- were analyzed and encoded so that one of those automatic pianos (replacing fingers by actuators with an regular grand piano). The results sounded pretty good, and there were concerts (live performances) of the result.

However, a piano is a very simple instrument to model, since the only parameters are the time and velocity of key strike (plus pedal info, of course.) It's pretty useless for modern extended techniques, such as Cage's pieces for prepared piano or some of George Crumb's stuff. (I studied composition with George Crumb a while ago.)

Doing a flute, oboe, or violin is much harder. Complicating matters from a technical viewpoint is that performance details change depending on the acoustics of the concert hall. This says nothing of the interpretive variations that you get when two or more people play together. When you added extended techniques, such as multi-phonics for the flute (more rarely the oboe or clarinet), The only real solution would be to make a model based on the differential equations of sound generation and use the raw data to discover the particular solutions.

I don't see much point to this, other than to put a lot of musicians out of work, since a convincing performance would require a lot of work and expense.

I have dabbled in electronic music, and I think that its real strength is in finding new ways to generate organized sound (as Edgard Varèse called music) and not to try to imitate human produced music. As much as I like some of the music (which I have been listening to since the 1950s), I still prefer live performance.

sorry to get off topic. My point is that using MIDI as a way to characterize music is a bad model, and is likely to lead to wrong conclusions.

Not to say that algorithmic analysis is hopeless. Someone wrote a program that generated chorales in the style of Bach, and they are pretty good. I have use read that someone made an algorithm for generating pop hits, and considering how generic a lot f pop sounds, it is probably being used.

MIDI is not even useful for cataloging music by 'incipits' (the first few notes of a theme), since it does not differentiate between sharp and flat. Whenever I import a MIDI file into Finale ( music notation program), I have a couple hours of clean-up just to make the music look presentable -- and that is when I have the score available.

There is an opportunity to replace MIDI with something a bit less primitive, since there are no longer the old constraints on memory and storage. Making it easy to change temperaments is a start, as is a way to save the notations for sharp/flat or double-sharp/double-flat. These distinctions are not relevant to the piano, but are important for other instruments. Having a notation for microtones would be useful for modern music. Specifying the attack profile and sound envelope would make the standard useful enough to roughly render most music performances other than voice.

It would be able to, for example, make use of Harry Partch's intonation. It would also be able to properly render non-western music, which does not use equal temperament at all.

I am afraid that any effort that relies on MIDI will not reflect any of the main features of music, so I offer this as a suggestion.

Using actual performance as a source of data is problematic since the size of the dataset is pretty limited. Just using Western 'art-music' as an example, there are probably 10 or 15 recordings of each of the Beethoven symphonies, and a lot fewer of most other classical compositions. If you wanted to use the equivalent of ImageIdentify[] to guess the composer from the music, you might be able to tell the difference between Beethoven and Bartok, but not between Bach and Telemann, or Mahler and Richard Strauss. (Depending on the length of the passage, I sometimes still confuse the latter two.)

I did some work in this area back before MIDI, when you could fit all the composers who also knew how to program into a small room. It is a non-trivial problem. I am convinced that the starting point to gaining any real insight is to replace MIDI. Wolfram Language is certainly up to the task.

Attachment

Posted 6 years ago

Audio representation is just another digital version of music but it's much more detailed and closer to reality than midi. With this kind of data, you can learn a timbre of an instrument and sound of it in special conditions (techniques of playing).

As could be listened from the tweet I've posted above - the piano generated their model sound pretty realistically. The quality of sound is poor for now though.

POSTED BY: Pavlo Apisov
Posted 5 months ago

Wow, Pavlo. I can't believe I didn't come across this post until now! This is great work, clearly presented. Despite its shortcomings, MIDI is a great tool for identifying useful insight into the structure and properties of music (certainly in the case of Western music).

POSTED BY: Updating Name
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract