Moving with the Digital

Improvisation of music is all about the emotion, says one single expert, but scientists have now identified a way to comprehend the intricate interactions that take place between instrumentalists and singers during jam with the goal of utilizing those insights to add greater emotional expression to a performance involving digital instruments.

To industrial designer Gustavo Ostos Rios, music improvisation is about the entire emotion but he and his two supervisors in the Department of Industrial Design at the Eindhoven University of Technology in the Netherlands have now identified a method to comprehend the intricate interactions that occur between instrumentalists and singers during a jam session with the goal of using those insights to add greater emotional expression to a performance involving digital instruments.

“In human – computer – interaction, we are more and more moving away from designing interactions for single users, towards designing interactions for networked groups of single users. We move from one user technology paradigm to multiple users multiple technologies paradigm,” says Mathias Funk, one of the co-authors in the International Journal Arts and Technology.

Evaluation of EMjam during group improvisation

Figure: Evaluation of EMjam during group improvisation

Examples of such shift comprise the familiar social media and social networking sites, such as Facebook and Twitter, much individual use on a daily basis as well as the like of Wikipedia and other associative ventures, like citizen science projects.

In the department of Industrial Design, Funk and his team Bart Hengeveld study new musical instruments that transform such idea towards musical performance, More focus is levied on the solo performance in few of the creative arts, like sculpture and painting and the audience is usually detached from art, interacting and viewing it with the ‘product’ sometime after completion of creative process.

Live music is distinct, there are usually more than one people involved in creating a performance and the audience is present the entire time. As such, there is a linked emotional response that can, in the case of improvisational performance, take the music in novel directions. More popularly, the alterations in direction are driven by the musicians and how they link with each other, but audience response can nudge them too.

The group has introduced a three-layer model that highlights the relationship between band members and the audience as a system, where emotions, expressivity and generation of sound give shape to improvisation. The group has utilized this model to emphasis specifically on how individual emotional arousal can be utilized as input to regulate as a group their digital musical instrument, EMjam.

The system, the group says, “builds on the structure that when paying attention at a concert it is feasible to see performer’s expressions, a drummer makes accents with the entire body, and moreover, the audience responding to the performance.

The instrumentalists obtain a wrist band with skin conductance sensors that can in a sense estimate the musician’s emotional status. The percussionist holds a wrist band regulating the rhythm created by the EMjam, the bass guitarist has harmony regulator and the guitarist of keyboard player a regulator for melody.