MelodicBrush : a cross-modal link between ancient and digital art forms

Abstract

MelodicBrush is a novel cross-modal musical system that connects two ancient art forms: Chinese ink-brush calligraphy and Chinese music. Our system endows the process of calligraphy writing with a novel auditory representation in a natural and intuitive manner to create a novel artistic experience. The writing effect is simulated as though the user were writing on an infinitely large piece of paper viewed through a viewport. The real-time musical generation effects are motivated by principles of metaphoric congruence and statistical music modeling.