The Effect of Visualisation Level and Situational Visibility in Co-located Digital Musical Ensembles

Digital Musical Instruments (DMIs) offer new opportunities for collaboration, such as exchanging sounds or sharing controls between musicians. However, in the context of spontaneous and heterogeneous orchestras, such as jam sessions, collective music-making may become challenging due to the diversity and complexity of the DMIs and the musicians’ unfamiliarity with the others’ instruments. In particular, the potential lack of visibility into each musician’s respective contribution to the sound they hear, i.e. who is playing what, might impede their capacity to play together. In this paper, we propose to augment each instrument in a digital orchestra with visual feedback extracted in real-time from the instrument’s activity, in order to increase this awareness. We present the results of a user study in which we investigate the influence of visualisation level and situational visibility during short improvisations by groups of three musicians. Our results suggest that internal visualisations of all instruments displayed close to each musician’s instrument provide the best awareness.

We are excited to announce the 1.0 release of Bf-Pd, a library for spontaneous collaboration between digital instruments. Bf-Pd is a set of abstractions for PureData (Pd) which enables communication and cooperation between digital instruments. Bf-Pd allows musicians to share parameter control data between instruments on the same local area network. In other words, Bf-Pd makes it easier to have spontaneous jam sessions! Bf-Pd consists of a set of Pd objects that can be easily integrated into an existing or new instrument, and it provides a “collaboration window” from which musicians can easily view each others’ activities and share control of instrument parameters and other musical data.

Bf-Pd can be downloaded at https://gitlab.cristal.univ-lille.fr/boeuf/bf-pd or installed with Deken. Bf-Pd comes with example instruments and documentation accessible from both the downloaded repository (in the 'doc folder') or online at https://gitlab.cristal.univ-lille.fr/boeuf/bf-pd/-/blob/master/doc/README.md .

Bf-Pd has been developed as part of the BOEUF project led by Florent Berthaut at Université de Lille and Luke Dahl at University of Virginia. More information on the project, including publications and demo videos can be found at https://bf-collab.net

We hope you find Bf-Pd useful in your collaborative music making, and we look forward to hearing the music you make!

Flo & Luke

Adapting & Openness: Dynamics of Collaboration Interfaces for Heterogeneous Digital Orchestras

Florent Berthaut, Luke Dahl
New Interfaces for Musical Expression, Jul 2020, Birmingham, United Kingdom
https://hal.archives-ouvertes.fr/hal-02561064

Advanced musical cooperation, such as concurrent control of musical parameters or sharing data between instruments, has previously been investigated using multi-user instruments or orchestras of identical instruments. In the case of heterogeneous digital orchestras, where the instruments, interfaces, and control gestures can be very different, a number of issues may impede such collaboration opportunities. These include the lack of a standard method for sharing data or control, the incompatibility of parameter types, and limited awareness of other musicians’ activity and instrument structure. As a result, most collaborations remain limited to synchronising tempo or applying effects to audio outputs. In this paper we present two interfaces for real-time group collaboration amongst musicians with heterogeneous instruments. We conducted a qualitative study to investigate how these interfaces impact musicians’ experience and their musical output, we performed a thematic analysis of inter-views, and we analysed logs of interactions. From these results we derive principles and guidelines for the design of advanced collaboration systems for heterogeneous digital orchestras, namely Adapting (to) the System, Support Development, Default to Openness, and Minimise Friction to Support Expressivity.

Introduction

In this experiential workshop we present two software systems, bf-pd and bf-web, which, when used together, allow non-co-located musicians to make connections and share data between instruments in real-time. In other words, they make it easier to have real-time spontaneous jam sessions, while accessing advanced modes of collaboration, using your own custom software instruments.

Bf-pd is a library for Pure Data, and was originally designed for co-located networked jamming. Bf-pd has been under development since 2016, and after various focus groups and studies, we are ready to release the beta version and invite more people to use it.

Our original plan for this workshop was to focus on co-located jamming using bf-pd. However, due to the current need for physical distance, we are in the process of developing bf-web, a desktop app which coordinates networking between bf-pd instruments, thus enabling full telematic real-time collaborations. Bf-web will be available as an alpha version at the workshop.

Workshop Outline

1. We will begin with introductions and installing software.
2. We will briefly present the Boeuf Conceptual Framework for describing heterogeneous orchestras of digital instruments, the modes of collaboration they afford, and a set of components for enabling these modes.
3. We will demonstrate how to use the bf-pd library, including the components: bf-session, bf-instrument, bf-param, bf-output, and the Collaboration Window.
4. Participants will engage in a brief hands-on exercise to to experience how data can be shared between instruments, using pre-made bf-pd instruments on their own computer.
5. We will then demonstrate using bf-web to communicate between bf-pd instruments at distant locations.
6. Participants will split into small groups of 3 or 4 and explore making music together with the collaborative functionality that bf-pd and bf-web afford. I.e. each group will have a “jam session”.
7. We will then briefly discuss the experience of making music together in this way.
8. If time allows, we will help participants integrate the bf-pd components into their own software instruments, and/or…
9. Participants may form new groups and engage in more on-line music-making together.

What to Bring

1. Some experience of using Pd or Max is very helpful but not required.
2. An interest in creating music in real-time with other digital musicians.
3. A computer that…
   1. Is connected to the internet
   2. Has vanilla Pd installed (start here https://puredata.info/)
   3. Can play audio
4. You are encouraged to bring instruments that you have built in Pd. We will also provide examples for you to work with.
5. You are welcome to bring MIDI controllers or other ways of controlling software instruments. We will help you integrate the controller into your bf-pd instruments.

Workshop Organizers

Luke Dahl is an Assistant Professor of Composition and Computer Technologies in the Music Department at University of Virginia where he teaches classes on music technology, audio signal processing, and music interaction design. His research is primarily situated in the interdisciplinary field of New Interfaces for Musical Expression (NIME) which investigates and explores the intersection of technology and musical practice through the activities of design, empirical research, and musical performance. He is especially interested in systems for digitally-mediated real-time music collaboration, and in the role of human gesture and movement in music.

Dr. Dahl earned his PhD in Computer-Based Music Theory and Acoustics from the Center for Computer Research in Music and Acoustics (CCRMA) at Stanford University, and a bachelor’s in Electrical Engineering at the University of Michigan. At CCRMA he was a founding member of the Stanford Laptop Orchestra and the Stanford Mobile Phone Orchestra. Before returning to academia he worked at the Joint E-mu/Creative Advanced Technology Center where he developed reverb algorithms for the SoundBlasterLive sound card products and co-authored five patents on audio signal processing, and at Apple where he worked on audio for iPod and laptop products.

Florent  Berthaut is an Assistant Professor at the University of Lille, France and Researcher in the MINT team of the CRIStAL. He obtained his PhD in Computer Science from the Universityof Bordeaux, in the SCRIME/LaBRI and the INRIA Potioc team. He then obtained a Marie CurieFellowship to lead the EU FP7 research project "IXMI", on improving the spectator experience with Digital Musical Instruments using Mixed-reality displays, at the University of Bristol, UK. His research focuses on building connections between the fields of 3D User Interfaces and New Interfaces forMusical Expression. In particular, he has been exploring 3D interaction techniques adapted for musical interaction, mixed-reality displays for augmenting digital musical instruments on stage and spatial interfaces for musical collaboration.

References & Links

• Berthaut, F. and L. Dahl.  2015.  “BOEUF: a Unified Framework for Modeling and Design-ing Digital Orchestras.”  In Proceedings of the International Symposium on Computer MusicMultidisciplinary Research (CMMR 2015), Plymouth UK. https://hal.archives-ouvertes.fr/hal-01380373/document  This paper presents the conceptual framework we developed for describing heterogeneous orchestras of digital instruments, the modes of collaboration they afford, and a set of components for enabling these modes.
• Dahl, L., F. Berthaut, A. Nau, P. Plenacoste. 2017. “bf-pd: Enabling Mediated Communication and Cooperation in Improvised Digital Orchestras.” In International Symposium on ComputerMusic Multidisciplinary Research (CMMR 2017), Matosinhos, Porto Portugal. https://hal.archives-ouvertes.fr/hal-01577942/document This paper describes the bf-pd software library for enabling communication, cooperation, and awareness between digital instruments.
• Bf-Pd: The main documentation can be found at https://gitlab.cristal.univ-lille.fr/boeuf/bf-pd/wikis/home . bf-pd comes with documentation (help files) for each object and a set of example instruments.