Music Technology Tools – A Therapist-in-a-box? Human–Computer Interaction and the Co-Creation of Mental Health

The Co-agent

The term co-agent is inspired by the recent philosophical perspective of an ethical and dialogical approach to mental health care (Bøe & Thomassen, 2017). In recent decades, mental health care has developed into a term that covers diverse practices, but with the same goal: improving people’s mental health. Within this context, not only health professionals, such as psychiatrists or psychotherapists, but also other professionals, with a range of expertise, are considered to be part of a person’s recovery. A multitude of humanistic agents can work together with health professionals to promote mental health.² A music producer, for instance, can use the expertise and knowledge of music production to work with music to promote people’s mental health and wellbeing, as one of many possible resources for recovery. My background as a musician, music producer and popular musicologist working on various technological platforms, makes me a humanistic agent in this context. Although I am not a trained therapist, I take part in a person’s recovery by offering my expertise together with other health professionals. The humanistic, resource- and empowerment-oriented health approach recognizes health not as the absence of illness (pathogenetic perspective), but as a holistic and salutogenic (Antonovsky, 1987, 1996) perspective focusing both on factors that promote health and on a state of health and wellbeing that balances the body, mind, and soul (Cappelen & Andersson, 2014). This means that health is a subjective, experienced condition, existing as a continuum that can be influenced by our actions, participation, and self-actualization (Bruscia, 1998; Stensæth, 2010). A professional music producer can subsequently influence this subjective, experienced condition by offering knowledge of practice and access to technological tools that enable users to take part in health-promoting activities. Consequently, the music producer becomes a co-agent in an interdisciplinary teamwork, promoting mental health and wellbeing.

The iPad as Co-agent

In addition to human co-agents, several nonhuman co-agents are also present as resources in mental health care. One notion is the nonhuman co-therapist—a concept used to explain the role of objects as cooperative elements in the treatment of mental health. This concept is known from eHealth (Federici & Scherer, 2018; Smaradottir et al., 2015; van Velsen et al., 2013), nature-based or outdoor therapy (Berger & McLeod, 2006), and music therapy (Aigen, 2005; Rolvsjord, 2013; Weissberger, 2014; Wärja & Bonde, 2014). These disciplines share a basic idea of the interactive, nonhuman co-agent as an active partner in the recovery process in mental health care.

Moreover, the development in computer technology towards artificial intelligence (AI) has enabled deep learning technology that interacts and co-creates on a whole new level. In music production, computer-generated music-making, randomizers, generative or adaptive music generators, and automated audio mastering services are examples of how digital technology is more than just a passive tool. Whether the aim is to make a musical artifact or to use music-making as a means to obtain other objectives, computers can be co-agents in these processes. The iPad, being a versatile computer with numerous possibilities for musical interaction within the various apps available, is consequently a potential co-agent, both in making music and in turning the creative music-making process into a resource for recovery and treatment in mental health care.

For the latter to happen, co-creation (Eide, 2014; Stensæth, 2018) is emphasized as a vital quality to promote positive musicking (Small, 1998) experiences. The iPad does not make music, nor does it promote mental health and wellbeing all by itself; it is a co-agent, meaning it needs to collaborate with other co-agents. However, the co-agent perspective on technology means that the focus shifts from controlling the interface to motivating social interaction, musicking, and co-creation (Cappelen & Andersson, 2014). In a therapeutic setting, many agents are involved in a patient’s recovery and wellbeing. These agents (both human and nonhuman) are engaged in a socially constructed agency network that relies on co-creational skills and mutual interaction (Ansdell, 2014). In my research, working with the iPad as a co-agent in mental health, this notion suggests that further examination of this human-technology interaction is required to better understand the role of the iPad within this interaction and its potential health benefits.

The Human–Computer Interaction

The attempt to “humanize” the iPad and expand its status from being merely a tool to becoming a co-agent within an agency network is founded on interdisciplinary discussions going back to the beginning of computers. According to Brey (2005), discussing human–computer interaction (HCI) from a computational science perspective, the primary epistemic relation between humans and computers, as information processing and problem-resolving tools, has in recent years been supplemented by an ontic relation. This ontic relation stimulates virtual and social environments that extend the interactive possibilities found in our physical environment. When the computer functions as an enhancement of human cognition, humans and computers become a hybrid cognitive system, unlike any other human-technology interaction:

This hybrid cognitive system is part human, part artificial, integrating the cognitive functions of the two parts to cooperate in performing cognitive tasks. From this perspective, the computer is considered both autonomous and part of an agency network, capable of engaging in creative, co-creational relationship with humans.

In music production, such co-creational relationships exist on several levels. Humans and computers cooperate to perform cognitive tasks, but the relationship also has practical consequences, affecting how musicians work, and aesthetic consequences, shaping the sound and structure of the music (Bennett et al., 2006; Frith & Zagorski-Thomas, 2016; Hepworth-Sawyer et al., 2019; Zagorski-Thomas et al., 2020). In a similar way that Brey (2005) argues for an evolution in HCI towards a boundless virtual reality, Moorefield (2005) describes an analogous evolution in music recording, also driven by the underlying mechanism of technological development. He claims that “recording’s metaphor has shifted from one of the ‘illusion of reality’ (mimetic space) to the ‘reality of illusion’ (a virtual world in which everything is possible)” (p. xiii). This “virtual musical reality” that Moorefield refers to is co-created by human-technology interaction, within a “symbiotic relationship” between human and machine, where technology operates as a co-agent. Without crossing the borders of technological determinism, one might say that these metaphors suggest a notion of “humanized” technology (or technology that possesses “human” skills and qualities) that actively participates in co-creating artwork and making “decisions,” thereby stretching the boundaries of reality. Brown (2016) puts it in the following way when explaining the computer as a musical partner: “Coding enables automated behaviours that take on a life of their own” (p. 179).

This co-agent perspective on technology challenges current thinking and historic misconceptions about incorporating technology into music therapy. In a reflection about interactive musical media, Stensæth (2018) leads us away from the notion of technology as an object or tool, and instead suggests “human–computer interaction” as a frame for examining the relationship between technology and the user. Drawing on experience from the RHYME project (Stensæth, 2014a), she argues that the media became active agents of co-creation when they were “transformed from intermediaries into technical and musical actors that were able to learn, memorize and respond to the inputs the user made” (Stensæth, 2018, p. 314). Stensæth looked outside the music therapy discourse to substantiate this point of view: both Cooren et al. (2006; organizational communication) and Brown (2016; music technology) suggest that objects can do things and that they afford or allow different types of behavior on our part. These nonhuman agencies can act and hence produce a change or transformation in the chain of actions, making them agents (or contributors) to the emergence of social processes. Agency is not a trait that is reserved for human beings only. Inspired by the anthropologist Alfred Gell, Brown suggests that

This relational agency within HCI, described by Brown, offers a perspective on technology that embraces the ability for machines to be co-creative. This perspective resembles the symbiotic relationship described by Moorefield (2005) and the hybrid cognitive system described by Brey (2005), in that they all acknowledge technology as an autonomous and integrated co-agent. In music production, this means that machines can act and participate in a creative relation with other agents to co-create music. The consequence is a virtual world where everything is possible, as Moorefield (2005, p. xiii) implies. If we adopt this idea in a therapeutic setting, as suggested by Stensæth (2018), then the concept of co-agency opens additional possibilities. The virtual world enables dialogue between users and technology that could lead to meaningful experiences detached from the basic human relationship (for instance between patient and therapist). By offering new ways in which to communicate and interact, the technological co-agent facilitates new perspectives on music-making and emotional expression that are potentially health-promoting. Used as a therapeutic tool, technology might become a co-agent for mental health and wellbeing: a therapist-in-a-box. Before I discuss this any further, I present two case vignettes that illustrate this co-agency.

Case Vignette 1: Karen

Karen was a shy, introverted, socially anxious, 17-year-old girl who struggled with identity issues due to a demanding family situation. According to her therapist, she expressed fear of losing track of who she was. This uncertainty and instability led to lack of confidence and fear of trying new things. However, to challenge herself, she agreed to participate in the project. When we first met, she was acting nervous and restless and avoided eye contact. Moreover, when I introduced her to some of the apps, she barely touched the iPad, afraid of doing something wrong. She had low self-esteem and said she was afraid not to perform well enough, having no prior experience of electronic music making. Based on these observations, the therapeutic goal for our sessions was to build confidence and strengthen her identity.

We began to explore the iPad’s possibilities by experimenting in a very openminded way with the apps. This unstructured and free introduction to the sonic world of the iPad was a way for her to find what she liked and identified with. It became clear that she favored the calm, atmospheric soundscapes of apps such as Bloom, Trope and TC-11. These apps provided a starting point for our musical piece. We recorded long, improvised sections which we later edited and used to structure the piece. The apps were operated by tapping or swiping the full-screen multitouch performance area, and the timing and movement of the fingers controlled the sound. By observing her playing these apps, I discovered similarities between how she operated the touch screen moving her fingers around it, and dance. I knew that Karen was a very dedicated dancer. Now she was “dancing” on the iPad. We used that metaphor to incorporate other dance-related sounds, and by doing that, also connected the music to her already existing identity as a dancer. The opening of the piece is one example of this approach. She recorded dance steps with the iPad using AudioShare (a sample recording and filing app), and we edited and processed the sounds using sample processing apps such as Samplr and Borderlands, before bringing them into the GarageBand project file. By bringing her dance world into the music, identification was strengthened, and the personal samples expressed through the music became a statement: “This is who I am.” Her ownership of the music, the mood, the personal samples and the overall sonic expression, and the way we built on her interest in dance to reinforce her identity, gave her a new tool. She said that music making on the iPad provided her with “a new color in life.”

Moreover, it gave her confidence. She experienced an enormous development in her engagement with the iPad and the mobile technology. In the beginning she barely looked at the iPad, almost afraid to touch it. After four months, she took full control in the final mixing session, doing all the editing, volume and panning, automation, effect processing and use of plugins, and the final arranging of the song. She owned it. The process empowered her and gave her courage to show it to friends and family. In the research interview following the project, she said:

Case Vignette 2: Daniel

Daniel had a long history of complicated mental health problems, including depression, anxiety, aggression, anger management, and suicidal thoughts. His short temper, mood swings, and destructive mind-rush resulted in social maladjustment and destructive behavior, such as self-cutting, drug abuse, and criminal behavior. In addition to regular sessions with his psychotherapist, he had a history of short-time hospitalizations due to his unstable condition.

When we started working on our music piece, we did not have a single, specific therapeutic goal. However, as Daniel learned to use the iPad, certain beneficial features became evident. In sessions, the iPad became focus for his attention, and he managed to concentrate for long periods of time. Moreover, he participated and was activated by the music making on the iPad, despite the impression that he was acting both indifferent and uninterested when we first met. This engagement made the iPad a valuable tool at home as well, either as a time-filler or to eliminate his destructive mind-rush. He said that he could sit for hours “…and just try out a bunch of stuff.” Coming home from a session, he typically went up to his room and continued making music until bedtime. He also reported incidents of insomnia due to mind-rush, where the iPad became a distraction that turned his attention towards music making and away from the destructive thoughts that kept him awake.

Daniel’s use of apps was very different from Karen’s. While Karen preferred the ambient and reverberant soundscapes, Daniel preferred more energetic musical expressions, with intense beats and distorted sounds. He discovered an app called BlocksWave that matched his musical preferences and that enabled him to create beats using samples. BlocksWave is a sample and loop sequencer using either samples from internal sample packs, or personal samples imported from other apps or recorded directly into BlocksWave. The samples can be used in live improvisation or they can be combined and organized in different sections that can be played back as a song. Daniel made several sections at home by combining samples. Moreover, we used the recording function in BlocksWave to make personal samples that we integrated with the beat. These sample-combinations were then used to structure the musical piece into sections that were recorded into GarageBand and further developed by adding tracks from other apps.

Daniel was able to make music because of the affordances of the apps. He did not know how to play a keyboard or make a drumbeat. However, he made a personal piece of music using a sequencer and organizing samples into a musical structure of his choice. Moreover, he had ideas and suggestions about the musical expression, mood, and feeling of the music that I could help realize by introducing him to relevant apps. One example was when he played me a rap song from one of his Spotify playlists, featuring a male artist screaming phrases in Russian. Daniel liked the raw energy and the distorted sound. We included this element in our music by using a vocal sound generator app called VoxSyn to add spectral changes and sound modifications to a recording of a male voice, screaming the song title. The recording was made directly into the VoxSyn app, and the sound manipulations were triggered by placing fingers on virtual pads, spread out on the iPad screen. The outcome was an intense and distorted sound, adding personality and identity to the music.

Actor-Network Theory

The cases illustrate that the iPad facilitated HCI that resulted in distinctive musical artifacts. The role of the iPad in this process could be analyzed simply as a tool controlled by the music maker, operating the interface in a causal series of events. However, this way of thinking about technology in music-making is not consistent with the technology’s autonomous and co-creational properties put forward by both modern HCI research and research on technology in music production and music therapy, outlined in the background section. Technology is not merely an “instrumental” tool or a medium for expressing musical ideas; rather, it should be considered as an “organic” musical partner that contributes to the creative process of making music. This view is consistent with theories on creativity in popular music, discussing the systems approach to creativity and creative practice (Csikszentmihalyi, 2014; McIntyre, 2012,, 2020), a genetic model of creativity, as Warner (2009) suggests, and perspectives on technology as a compositional tool (Eno, 2004; Théberge, 1997; Thompson, 2019).

My research supports this perception based on the way in which the participants used and talked about the iPad. Musical relationships, therapeutic goals, inspiration, and creativity were triggered and developed by the iPad and its position as the nexus of the project. Furthermore, apps such as Bloom, Trope, and TC-11 were engaged as improvisational instruments, and because of their generative interface attributes, they acted as co-creating musical partners, enabling participants to make music that would never have been possible without the creative contributions made by the technology itself. In the interviews, the participants highlighted this when they described their dynamic relationship with the iPad: as “musicking partners.” This demonstrates how the iPad facilitated HCI and became a co-agent in the process of creating music.

To further explain and understand the role of technology in such symbiotic relationships, it would be necessary to consider the social context and the sociology of technology. Zagorski-Thomas (2014) examined this human-technology relationship, arguing that if music is a process and not a thing, as Small (1998) suggests, then musical activity (i.e., musicking) should be analyzed as a social process involving people, technology, and the environment. According to Zagorski-Thomas, this ecological perspective informs a notion of technology as a co-agent, providing “affordances” (Gibson, 1979) that have a tremendous impact on how musicians work, how the music sounds, and how it is perceived by the environment.

As a social theory, the actor-network theory (ANT; Latour, 2005) explores these relational ties between different agents within a network with the purpose of explaining how they interact. Since ANT insists on the capacity of nonhumans to be actors or participants in networks and systems, and it emphasizes the equality between all relevant actors (“mediators”) in the network (whether human or nonhuman), ANT offers a framework to explain how human and nonhuman participants (“actants”) configure each other through their perception of and action upon affordances. This configuration, moving back and forth between the “actants,” occurs when they start acting as a whole. When my participants found apps that they liked, felt attached to, or believed were applicable to our musicking activity, one finger on the iPad’s surface would initiate a series of sonic events (or actions), often unpredictable and “uncontrollable,” that led to a reaction from the human actor. The “actants” consequently began to act as a whole, developing symbiotic relations that, among other things, led to experiences of flow. Moreover, this HCI (or “configuration”) led to musical dialogue, joint human–computer improvisations, and a co-creational creativity between humans, technology, and the environment—equivalent to an actor-network system. The role of the iPad, as an entity that explicitly made a difference in this co-creational network, accentuates my point that according to ANT, the iPad must be considered as a relevant actor and a “mediator” in this mode of computational music-making. The iPad makes a difference; the benefits are inspiring ways of musical interaction; co-creation; and new virtual possibilities that stretch the boundaries of reality, where technology is capable of taking on “a life of its own.”

The intention of this brief analysis of the iPad’s role and value in my project is to position the iPad as a co-agent within a socially constructed actor-network system. The cases illustrate that the human-technology interaction enhances the music-making process and that people, technology, and the environment mutually benefit from the process. In the light of this, the following question then arises regarding the value of the iPad in terms of mental health and wellbeing: Apart from being a co-agent of music-making, could the iPad also be a co-agent and co-creator of mental health?

Agency

The cases illustrate that the iPad facilitates agency in the participants. Agency can be simply defined as the ability to produce an (intended) effect; it is the capacity to influence one’s action and to exercise control over one’s thought processes, motivation, and affect. One of the most influential socio-cognitive psychologists, Albert Bandura (2001, 2002), highlights the important health-promoting causal effects of strong personal agency. Being in control and exercising that control by being (pro-) active is empowering. According to Bandura, agency is strengthened through “intentionality and forethought, self-regulation by self-reactive influence, and self-reflectiveness about one’s capabilities, quality of functioning, and the meaning and purpose of one’s life pursuits” (Bandura, 2001, p. 1).

In this respect, the iPad is a strong facilitator and co-agent for strengthening agency. The immediate sonic effect of an intended action exercised towards a music app provides immediate feedback about the choice of action and the consequences of that choice, because the sound and the musical context change. The human actor consequently applies a self-reflective value judgement to the consequences of a decision—asking questions such as, 1) Do my actions provide the desired effect? and 2) Do they improve the music or not? —and he or she is held responsible for that decision. This self-reflectiveness regarding the causal effects of an action is an experience that, according to socio-cognitive theory, enhances the consciousness about one´s capabilities and quality of functioning. Vulnerable adolescents who suffer from mental illness often experience a lack of agency in general, and music could be a resource to strengthen that agency. “Small steps in ‘musical agency’ (e.g., understanding how music impacts me, how I would want it to impact me, and how I can achieve that) may lead to further steps in agency in general” (Saarikallio, 2019, p. 95). The interactive communication between the human agent and the co-agent (in this case, the iPad) is essential, and exemplifies the role of the nonhuman agent within this HCI.

Several of the participants reported a strong feeling of agency and control, especially at the end of the project. This was confirmed by my observations, because all participants demonstrated an increased belief in their own ability to influence their actions and to exercise control over those actions when working with music on the iPad. A prime example would be the way Karen took control of the mixing process, making artistic choices, executing technical changes and sound editing, and reflecting on the aesthetic result. I suggest that the ease of use and the multitude of possibilities and choices offered by the apps are major advantages in turning the iPad into a powerful tool for strengthening agency and experiencing empowerment.

From a health-promoting perspective, empowerment is about recognizing people’s self-understanding and competence in their own life, and it facilitates patient participation as a valuable resource for recovering from mental illness. This process involves dialogue, participation, and the mobilization of resources, with the goal of increasing patients’ capacity to take control of the factors that affect wellbeing and enabling them to make beneficial changes (Tveiten, 2017, pp. 48–49). I suggest that the iPad strongly facilitates this participation. First, tablets and smartphones are familiar tools for adolescents, which means that these individuals already inherit a strong agency regarding the tools, and the technological threshold is consequently low. Although some of my participants reported fear towards the music apps in the beginning of the project, the iPad itself (and particularly GarageBand) was a familiar starting point for exploring new possibilities for musical expression. Furthermore, the iPad contributes to leveling the underlying power structure between patient and therapist, and it transfers the role of the expert towards the patient. This renegotiation of power, where the individual’s lived experience and skills becomes a source of shared expertise, is highlighted in recovery and resource-oriented music therapy literature as a basic value and premise that empowers the patient (Anthony, 1993; Deegan, 1997, 2001; McCaffrey et al., 2018; Rolvsjord, 2010; Solli, 2012, 2015; Stensæth, 2014a). This is also strengthened by the co-creation structure of the project, where my role as participant and co-creator was to share my expertise, empower the participants, and strengthen their agency by uncovering resources that they could use in their recovery process. My research suggests that the multifaceted, mostly intuitive apps invited the participants to engage in creative activities that, for all of them, revealed new resources they did not know they had, hence mobilizing resources that could also be part of their recovery.

The previously described core features of agency enable people to participate in their recovery. Bandura (2001) suggests that activities that promote these features (such as this iPad project) consequently enhance self-efficacy. From this perspective, active participation in one´s own recovery seems to be crucial in promoting wellbeing. However, participation is dependent on agency: people need both an opportunity to act and a willingness to do so, based on prospects of a beneficial outcome; they need to believe that their actions matter. Therefore, efficacy beliefs “are the foundation of human agency. Unless people believe they can produce desired results by their actions, they have little incentive to act” (Bandura, 2001, p. 10).

In my experience, the co-creation model used in this project facilitated motivation from the co-agents (human and nonhuman), which contributed to the participant´s motivation and beliefs. The relational and dialogical attitude, as well as the amount of time spent together, seemed equally important. This was also highlighted by the participants’ therapists as a decisive quality of the project: They suggested that the benefits of individuals having an expert all to themselves for several months provides those individuals with a stable and secure environment for experimentation and reflection. In addition, I suggest that establishing a clear goal in the beginning of the project, and then following it by making an artifact—a personal piece of music—was equally important for motivation. This is consistent with some of the literature on music technology and therapeutic songwriting that highlights the benefits of recording (Kirkland & Nesbitt, 2019; McFerran et al., 2019; Sadnovik, 2014; Viega, 2018, 2019; Weissberger, 2014) and discusses the role of the artifact (Baker, 2015, pp. 22–23; Rolvsjord, 2010, pp. 194–196):

Affordance and Appropriation

To strengthen personal agency, one must be exposed to opportunities for goal-directed action. Activities that promote creativity and interaction, such as music-making with tablets or smartphones, are carriers of musical affordance that provide such opportunities. One perspective that examines this relationship between music and humans is the theory of affordance.

The concept of affordance dates back to the ecological psychologist James J. Gibson (1977, 1979), and it was later adopted by many scholars in other research fields to understand why we act the way we do and how we interact with our environment.⁶ Most relevant for this paper is analyzing how technological affordance influences the way we engage with music and how this in turn can benefit mental health and wellbeing. My cases illustrate that the interface technology of the iPad affords specialized, diverse, and often straightforward means of musical interaction that meet an adolescent’s individual needs, independent of former knowledge or skills. This can, in turn, be used to promote health.

Mooney (2010) suggests a model in which tools of music-making (so-called “frameworks,” including both physical and conceptual tools) are viewed in terms of what they allow us to do (their “affordances”). These frameworks and the affordance model allow us to see the impact that tools have on the creative process and the resulting music, based on how we engage with the tools to achieve musical outcomes. Mooney suggests that because every framework requires both knowledge and skills to be appropriated by a human agent, there will be individual differences both in perception and in the ability to realize the affordances. Some of them are easier to achieve than others, and by ordering the affordances from “easiest” to “most difficult,” we obtain the “spectrum of affordance” for that framework: “Put simply, every tool has a range of things it allows us to do, and some of those things can be done more easily than others” (Mooney, 2010, p. 146).

Although Mooney applies the model to music-making and education, the model may also help us to understand the benefits of music technology in relation to health. First, Mooney highlights that the choice of frameworks has an obvious influence on the musical result, since the tools shape the product. The choice of instrument, technical equipment, or concepts (such as notation style, tuning, harmonic modes, or structural principles) will provide a set of affordances, as well as a set of constraints. The iPad’s affordances (or more simply, what it offers) are a diversity of individually designed apps that offer various ways in which to make, process, and organize sound. These apps naturally influence the music that is made, both formally and expressively; however, in contrast to a piano, which has a limited spectrum of affordances, the iPad’s range of affordances is almost limitless. The versatile qualities of the iPad to make sound, ranging from environmental samples and acoustic instrument emulations to electronic soundscapes and drumbeats, offer different affordances compared to other frameworks. Working with music-making and mental health, the iPad offers infinite options for expressing mood, emotions, and identity that can be individually adapted. My research suggests that the participants appropriated these affordances in a highly individual manner, resulting in a personal piece of music towards which they expressed a high degree of ownership, pride, and recognition. The music resembled their personality and was considered an artifact that substantialized the process that they underwent. The iPad facilitated this process in a versatile and dynamic way, strengthening agency, empowerment, and self-efficacy.

A second point discussed by Mooney is that the process of making music is closely related to the spectrum of affordances—the tools also shape the process. It seems important that the affordances are perceptible and that they are not experienced as too difficult to use. As Gaver (1991) points out, this is partly a matter of design:

This implies that there is a causal connection between design, perception, the spectrum of affordance, and the way we act. Furthermore, this affects the product as well, as discussed above.

On the iPad, each app can be considered a framework of its own, with a spectrum of affordances. Some apps are organized as sequencers, playing rhythmic patterns and musical loops organized around a metric beat and a pre-selected number of bars, while others offer more expressive and improvisational ways of making soundscapes free from the metric system. A third group of apps take the form of virtual music instruments or synthesizer emulations, with piano keys, strings, or drumheads, while others are virtual effect-units or sound-processing devices with knobs, buttons, and faders. The design of the apps, the sounds, and the possibilities for interaction provide the spectrum of affordance.

My research suggests that the interaction between the user and the apps is strongly dependent on the individual perception of the affordances of the apps. Since the iPad has physical and tactile limitations because of the touchscreen, app design becomes important in making affordance perceptible. The apps that were most instinctive and that responded immediately with auditory perceptible changes were most appealing to my participants. However, they did not choose the same apps; given their individual musical preferences, prior experience, and skills, they all chose different methods of engagement. They rapidly left the apps they did not understand in favor of the ones they mastered. Moreover, they expressed frustration in relation to affordances that were too difficult to appropriate, and they expressed surprise when I demonstrated new features of the apps that they had missed on their own. This exemplifies the spectrum of affordance and how it seems important that an app’s design and technical interface present a manageable affordance for the individual user in order to be considered useful.

My experience was that the iPad’s affordance for music-making, especially when working with patients experiencing apathy and depression, was dependent on the co-creation model. My expertise and experience with music technology and music-making was consequently important to enable appropriation for the participants. However, as soon as they were presented with some opportunities, the iPad facilitated experimentation, empowerment, and mastery, and it functioned as a versatile and flexible co-agent in the process of creating music.

Co-creation of Mental Health

Co-creation means creating something together. It thus involves a process of creative activity (creating), socially contextualized as a collaboration between several agents (together), that consequently moves towards a goal (something). In addition to spontaneous and random creative factors, such as playing, listening, exploring, and composing, Eide (2014) recognizes that co-creation also involves more deliberate collaboration, where people are socially motivated to act towards a common goal: the creation of “something third” (p. 125). It is natural to assume that concerning music-making, “something third” refers to the artifact (a piece of music). However, music therapists also describe “something third” as an intersubjective moment of meeting, something that is both invented and discovered and that exists on its own terms (Eide, 2014, p. 125). Co-creation can hence result in meaningful meetings between people, changes in relationships, shared experiences of discovering personal resources and potential, and changes concerning our experience of ourselves. If we, according to the humanistic health approach, consider health as an experience of wellbeing, then we can assume that this healthy experience can be facilitated by being co-creative. The co-created third’s duality as both invented and discovered enables experiences that are potentially health-promoting. In other words, we might say that creating something together, being an agent in a co-creative process, can potentially result in experiences of wellbeing, and we could consequently talk about the co-creation of mental health.

I have argued that this co-creational activity takes place within a network of human and nonhuman agents, and my research suggests that the iPad’s affordances play a crucial role in facilitating the creation of “something third.” Furthermore, I suggest that for the participants, this co-creative process has resulted in meaningful meetings between people, where new personal resources and potential have been discovered, perspectives have changed, and agency has been strengthened, leading to health-promoting experiences of wellbeing. Such experiences of wellbeing, facilitated by co-creation between humans and technology, demonstrate that the iPad’s affordances and role in this process are crucial not only to co-create music, but also to co-create mental health.

Similar findings are presented in the RHYME project (Stensæth, 2014a). The co-creative tangibles studied in this project are recognized as actors in the process of co-creation; they establish relations with the other actors and are considered equal partners and co-agents in the collaboration, which leads to health-musicking and health-promoting co-creation. In other words, co-creation implies health-musicking (the process of continuously promoting health), a strengthening of agency and mastery, and empowering interaction both with the tangible and with other people (Stensæth, 2014b, p. 74). However, in the editor’s foreword, to further promote healthy interaction between humans and technology, Stensæth calls for device flexibility, universal design, interdisciplinarity, and a common ground for understanding. Accordingly, she asks, “Who knows, perhaps our future home environments will have musical and interactive media that can operate as agents of health promoting co-creation?” (Stensæth, 2014a, p. xiii).

I argue that music-making with tablets and smartphones is one answer to that question. These computers—being in our pockets and following us wherever we go—are not only part of our homes, but also a significant part of our lives. Furthermore, the flexibility of the iPad, the apps, and the affordance that is brought forward by sonic and visual design, leading to musical artifacts of great personal and artistic integrity, demonstrate that the iPad vastly surpasses traditional musical instruments in its interactivity. The iPad therefore possesses great potential to enable concrete and tangible health-promoting co-creation.