AI-Driven Transformation of Score MIDI into
Expressive Piano Performance

encodec architecture

System Overview: A MIDI-to-MIDI (M2M) model with a MIDI-to-Audio (M2A) model


This demo page showcases samples from our system that transforms symbolic music scores into expressive piano performance audio. The system integrates a Transformer-based Expressive Performance Rendering (EPR) model with a finely tuned neural MIDI synthesizer. It offers an efficient approach for converting plain score MIDI files into rich, expressive piano performances. Due to the data limitation, the system currently only focuses on handling Beethoven's works, especially Sonatas.


encodec architecture

Piano Sonata No. 18 in E-Flat Major, Op. 31 No. 3 'The Hunt'


Source
Audios
Human
Score MIDI (Pianteq)
Generation (M2M + Pianoteq)
Generation (M2M + M2A)

Extended Outputs from Our System (M2M + M2A)

Our system can generate longer, more complex audio performances using only the score MIDI as input.