Presentation Video NIME 2020

Adapting & Openness: Dynamics of Collaboration Interfaces for Heterogeneous Digital Orchestras

Florent Berthaut, Luke Dahl
New Interfaces for Musical Expression, Jul 2020, Birmingham, United Kingdom
https://hal.archives-ouvertes.fr/hal-02561064

Advanced musical cooperation, such as concurrent control of musical parameters or sharing data between instruments, has previously been investigated using multi-user instruments or orchestras of identical instruments. In the case of heterogeneous digital orchestras, where the instruments, interfaces, and control gestures can be very different, a number of issues may impede such collaboration opportunities. These include the lack of a standard method for sharing data or control, the incompatibility of parameter types, and limited awareness of other musicians’ activity and instrument structure. As a result, most collaborations remain limited to synchronising tempo or applying effects to audio outputs. In this paper we present two interfaces for real-time group collaboration amongst musicians with heterogeneous instruments. We conducted a qualitative study to investigate how these interfaces impact musicians’ experience and their musical output, we performed a thematic analysis of inter-views, and we analysed logs of interactions. From these results we derive principles and guidelines for the design of advanced collaboration systems for heterogeneous digital orchestras, namely Adapting (to) the System, Support Development, Default to Openness, and Minimise Friction to Support Expressivity.

Workshop on Spontaneous Networked Jamming with Bf-Pd and Bf-Web @ NIME 2020

Introduction

In this experiential workshop we present two software systems, bf-pd and bf-web, which, when used together, allow non-co-located musicians to make connections and share data between instruments in real-time. In other words, they make it easier to have real-time spontaneous jam sessions, while accessing advanced modes of collaboration, using your own custom software instruments.

Bf-pd is a library for Pure Data, and was originally designed for co-located networked jamming. Bf-pd has been under development since 2016, and after various focus groups and studies, we are ready to release the beta version and invite more people to use it.

Our original plan for this workshop was to focus on co-located jamming using bf-pd. However, due to the current need for physical distance, we are in the process of developing bf-web, a desktop app which coordinates networking between bf-pd instruments, thus enabling full telematic real-time collaborations. Bf-web will be available as an alpha version at the workshop.

Workshop Outline

  1. We will begin with introductions and installing software.
  2. We will briefly present the Boeuf Conceptual Framework for describing heterogeneous orchestras of digital instruments, the modes of collaboration they afford, and a set of components for enabling these modes.
  3. We will demonstrate how to use the bf-pd library, including the components: bf-session, bf-instrument, bf-param, bf-output, and the Collaboration Window.
  4. Participants will engage in a brief hands-on exercise to to experience how data can be shared between instruments, using pre-made bf-pd instruments on their own computer.
  5. We will then demonstrate using bf-web to communicate between bf-pd instruments at distant locations.
  6. Participants will split into small groups of 3 or 4 and explore making music together with the collaborative functionality that bf-pd and bf-web afford. I.e. each group will have a “jam session”.
  7. We will then briefly discuss the experience of making music together in this way.
  8. If time allows, we will help participants integrate the bf-pd components into their own software instruments, and/or…
  9. Participants may form new groups and engage in more on-line music-making together.

What to Bring

  1. Some experience of using Pd or Max is very helpful but not required.
  2. An interest in creating music in real-time with other digital musicians.
  3. A computer that…
    1. Is connected to the internet
    2. Has vanilla Pd installed (start here https://puredata.info/)
    3. Can play audio
  4. You are encouraged to bring instruments that you have built in Pd. We will also provide examples for you to work with.
  5. You are welcome to bring MIDI controllers or other ways of controlling software instruments. We will help you integrate the controller into your bf-pd instruments.

Workshop Organizers

Luke Dahl is an Assistant Professor of Composition and Computer Technologies in the Music Department at University of Virginia where he teaches classes on music technology, audio signal processing, and music interaction design. His research is primarily situated in the interdisciplinary field of New Interfaces for Musical Expression (NIME) which investigates and explores the intersection of technology and musical practice through the activities of design, empirical research, and musical performance. He is especially interested in systems for digitally-mediated real-time music collaboration, and in the role of human gesture and movement in music.

Dr. Dahl earned his PhD in Computer-Based Music Theory and Acoustics from the Center for Computer Research in Music and Acoustics (CCRMA) at Stanford University, and a bachelor’s in Electrical Engineering at the University of Michigan. At CCRMA he was a founding member of the Stanford Laptop Orchestra and the Stanford Mobile Phone Orchestra. Before returning to academia he worked at the Joint E-mu/Creative Advanced Technology Center where he developed reverb algorithms for the SoundBlasterLive sound card products and co-authored five patents on audio signal processing, and at Apple where he worked on audio for iPod and laptop products.

Florent  Berthaut is an Assistant Professor at the University of Lille, France and Researcher in the MINT team of the CRIStAL. He obtained his PhD in Computer Science from the Universityof Bordeaux, in the SCRIME/LaBRI and the INRIA Potioc team. He then obtained a Marie CurieFellowship to lead the EU FP7 research project "IXMI", on improving the spectator experience with Digital Musical Instruments using Mixed-reality displays, at the University of Bristol, UK. His research focuses on building connections between the fields of 3D User Interfaces and New Interfaces forMusical Expression. In particular, he has been exploring 3D interaction techniques adapted for musical interaction, mixed-reality displays for augmenting digital musical instruments on stage and spatial interfaces for musical collaboration.

References & Links

  • Berthaut, F. and L. Dahl.  2015.  “BOEUF: a Unified Framework for Modeling and Design-ing Digital Orchestras.”  In Proceedings of the International Symposium on Computer MusicMultidisciplinary Research (CMMR 2015), Plymouth UK. https://hal.archives-ouvertes.fr/hal-01380373/document  This paper presents the conceptual framework we developed for describing heterogeneous orchestras of digital instruments, the modes of collaboration they afford, and a set of components for enabling these modes.
  • Dahl, L., F. Berthaut, A. Nau, P. Plenacoste. 2017. “bf-pd: Enabling Mediated Communication and Cooperation in Improvised Digital Orchestras.” In International Symposium on ComputerMusic Multidisciplinary Research (CMMR 2017), Matosinhos, Porto Portugal. https://hal.archives-ouvertes.fr/hal-01577942/document This paper describes the bf-pd software library for enabling communication, cooperation, and awareness between digital instruments.
  • Bf-Pd: The main documentation can be found at https://gitlab.cristal.univ-lille.fr/boeuf/bf-pd/wikis/home . bf-pd comes with documentation (help files) for each object and a set of example instruments.