My Gesture-controlled interactive instrument program
I am responsible for the sound design in this project.
I want to create an immersive sound design. I believe the role of sound design is to help the audience immerse themselves in the story and to help them understand the theme. The goal of our group’s project is to make the audience feel the connection between the individual and the environment — through the “river” beneath London, to break down how power defines space and shapes people’s behaviour, to understand that a river can carry community and the plural meanings of life, and to find healing in a temporary utopia.
⬇️
So I asked myself: how do we connect people with space?
1. Immersion is one way. In the first half, live performance (for example, creating wind, tracks, ticket barriers, footsteps, water sounds) guides the audience into immersion.
2. Beyond that, interaction is another way to involve the audience in the work. In the second half, a camera is set up for each audience member to capture their gestures, and each audience member is assigned a different interactive instrument to play, producing different sounds through left/right hand and heartbeat sensors.
My design principle is that everyone is part of the river, everyone stirs up a splash, and in the end these come together into one river, making the audience reflect on their relationship with the river. When the audience personally plays different “splashes,” the rules of the space are rewritten by everyone together, and the temporary utopia shifts from “being watched” to “being co-created.”
⬇️
Research & Development: What exactly is the sound of water?
Smetana’s Vltava (The Moldau) gives me inspiration — it is also a representative piece in classical music for depicting a river. It not only uses sound to depict the Czech mother river from its source, over the calm and then the tank-like mountains, flowing through the whole Czech land, but also nurtures Czech civilisation, the continuation of life, and carries forward the Czech people’s unyielding will to resist. I need a musical model that has both a river narrative and can be broken down into interactive sound material — Vltava happens to provide a complete sound imagery from source to waves.
Therefore, I mainly chose four instruments — the triangle, representing the river lapping the banks and stirring up splashes; the violin, representing the main river; the flute, representing the source of the river; and the harp, representing the waves — to create interaction for four instruments, letting the audience perform a healing river beneath London. The four instruments correspond to different forms of the river; when the audience plays together, individual splashes merge into a communal river — exactly the “from isolation to connection” that the project wants to express.
⬆️The demonstration videos for the different interactive design programs have been uploaded to YouTube. Feel free to watch them.
Water carries memory and connection; the human body and nature are inherently inseparable. Yet space and historical narratives are often shaped by power and deliberately erased. We want to connect these two threads. The London Underground is precisely a space shaped by efficiency and rules—it dictates how you walk, where you go, and how fast you move. We want to reimagine it as an underground river: a river does not dictate direction; it connects, carries, and heals. Therefore, we plan to transform an abandoned Underground station into an immersive art space, using water, lighting, sound, and live performance to turn this space designed to “transport people” into a space where people “simply stop and exist.”
Drawing on hydrofeminism, we hope to explore the deep connections between water, the body, the environment, and power. By bringing water and performance into this space normally associated with movement and control, we hope to challenge how urban environments shape human behaviour, and encourage people to reconnect with their bodies and their surroundings. It will become a temporary utopia—a space to slow down, find healing, and build dignified connections.
At the same time, we keep in mind what our tutor said: do not preach to the audience, but use sensory experiences and metaphors to let people form their own connections; and avoid performative shock, maintaining care for the audience’s trauma.
The thread running through these classes moves from colonial history step by step to today’s creative practice. I think the teacher was talking about the past and the present, and the river is a living artefact — it flows endlessly, running through different periods, a bridge carrying different eras, and what it flows toward is the future, that is, how we speak out for it and how we change the future. Water here is not just a metaphor: hydrofeminism holds that the human body is mostly made of water, and that the body, water and nature are inherently inseparable; when water is polluted, the connection between humans and nature is broken too. So the river is both a carrier of memory and a connecting point between body and environment, which is what made me understand why “speaking out” must start from the body and the senses, not from slogans.
Beyond that, from history to cultural performance as resistance — the origins of Notting Hill Carnival, Claudia Jones’s line “a people’s art is the genesis of their freedom,” and carnival as a public event that transforms violent trauma into beauty, love and free bodily expression. This knowledge made me reflect on my creative approach. In the past, I thought art succeeded if it made the audience feel the same force and emotion, so in week two my sound design leaned Hollywood-style — using intense sound effects and emotion to “manufacture” impact, pressing the audience down with sound. After reflecting, the carnival the teacher talked about had a big influence on me. I realised that for people who have lived through trauma, art should not be another round of shock and consumption, but something beautiful, dignified, participatory and healing. So my sound design shifted from “shocking the audience with sound” to “letting the audience stir up sound themselves” — the immersive part gently receives the audience, and the interactive part makes everyone part of the river, replacing preaching and oppression with beauty and participation.
This image shows our group using bodily sounds—breathing, waves, footsteps, and friction—to narrate the daily suffering of people inside the cabin.
Our group’s sound narrative depicts how people endured suffering day after day in the ship’s hold. We used the body itself as an instrument — breath, waves, footsteps, the friction between the body and its restraints — to let the audience feel the weight of that era through sound rather than through graphic description, so that the body becomes a carrier of memory. But I now think this creative approach is problematic.
We need to reflect: content creation should not pursue a “performative shock” that deliberately heightens impact, but should place consideration for the audience in an important position. Especially when a work involves real historical trauma, sound is not just a tool for building atmosphere — it may also trigger the audience’s own unprocessed traumatic experiences. Therefore, rather than using the body to reproduce pain and subject the audience to an oppressive auditory shock, it is better to treat the body as an entry point for connection and healing — letting breath, heartbeat and other most basic rhythms of life become sounds the audience can enter and empathise with, rather than a heaviness imposed on them. This also made me reconsider the ethics of sound design: success is not about making the audience “feel the same pain,” but about letting the audience, on the premise of being informed and respected, choose for themselves how to approach this history.
1· Course in-class notes, from confusion to recording to reflection
Unit 1: Bank Robbery – Reflection NoteUnit 1: Bank Robbery – Reflection Note
In the first week of the course, we rehearsed and performed 17 versions of “Bank Robbery”. In the first few days, I was very confused about what intention, meaning, and purpose I actually wanted to convey to the audience, so I recorded the course content and conducted an initial reflection.
The teacher first spent a day and a half leading us to rehearse the initial version of “Bank Robbery”. Like traditional theatre, it included dialogue and action. Afterwards, the teacher guided us to brainstorm and gave this fixed script various completely different styles, including musical, modern dance, animal world, western, advertisement, etc…
I realized that this class leaned more towards improvisational performance design: first determine the style direction, and then based on this think about how lighting, music, and props assist in designing movement. In this process, performance replaced dialogue in traditional theatre, and the body, movement, and props turned into a kind of language, used to convey the speech, emotions between characters, and events happening on stage.
2· Physical Theatre Study – Association Based on an Oil Painting
Adolf Hirémy-Hirschl, The Souls of Acheron, 1898. Oil on canvas. The extreme physical tension and intertwining bodies serve as a visual language, conveying despair and agony without a single word.
After consulting materials, I realized that the core of this lesson revolves around Physical Theatre. As defined by Murray and Keefe in their book, physical theatre is a highly tense and innovative form of expression in contemporary theatre. It subverts the traditional theatre model that relies on dialogue to drive narrative, takes body movement itself as the core narrative language, and constructs story and emotional structures through actors’ postures, movements, spatial relationships, and rhythms (Murray & Keefe, 2016). Here, the body is not only an emotional carrier, but also “dramatic grammar” itself, by no means an appendage to dialogue.
As summarized by Mark Evans, the core of physical theatre lies in “taking the body and movement as the starting point for creative practice”, and presents “the dominance of movement over text and speech” (Evans, n.d., p.4). This kind of physical theatre is extremely attractive to me. It is like an oil painting, which does not rely on words to tell the viewer what happened, but expresses through a stronger, more sensory new language—the body.
More interestingly, the audience no longer relies solely on traditional textual narrative to experience, but perceives through intuition. Because the body often experiences emotions before our consciousness does: when we are anxious, the body will tremble involuntarily; when we are tired, language becomes redundant, and our expressions and postures will precede logic, honestly mapping out a low spirit.
In this context, the body is more honest than language, carrying those indescribable complex experiences. The audience relies on intuition and senses, directly touching that untranslatable tremor through the actor’s body language. This is precisely the moving power of physical theatre that transcends language and directly strikes the soul.
3· Viewing and Study of Physical Theatre Works
Kamoï Rei, a physical theatre production by tarinainanika and Hyōgensha Kōbō. Without dialogue, the actors’ bodies and gestures become the narrative language, embodying the painter’s inner struggle and the characters from his work.
Kamoï Rei was a painter who continuously depicted the “darkness of the human world”, spent his life groping for his “true self” in the gap between ideals and reality, and ultimately ended his own life at the age of 57. Throughout his career, he painted countless portraits of drunkards, clowns, disabled soldiers, and old women—obsessively searching for himself among these departed souls.
The He Theatre Troupe uses the physical theatre piece Kamoï Rei as a vehicle to bring this painter, who groped in the gap between the “darkness of the human world” and ideal reality, to the stage together with the souls from his paintings. The stage abandons traditional textual narrative and instead practices the core concept of physical theatre: “taking the body as the starting point for creation”. The artist is heavily surrounded by the drunkards, clowns, and disabled soldiers he created, while the canvas in front of him, which should have carried everything, is in a completely blank and barren state.
The blank canvas, the surrounded painter, the tense and exaggerated physical staging… The actors use Corporeal Mime to externalize the painter’s psychological despair into a visual sense of oppression. This expression transcends the limitations of language, just as physical theatre theory emphasizes, its core lies in “taking the body and movement as the starting point for creative practice” and advocates “the dominance of movement over text and speech” (Evans, n.d., p.4). On the stage of Kamoï Rei, the body is more honest than language; it is no longer an auxiliary to the narrative, but carries the painter’s unspeakable despair. Every twisted tremor and every loss of balance and fall in the actors’ bodies is the concretization of the painter’s unspeakable despair in physical space. The audience is no longer a passive “spectator”, but through intuition and senses, directly touches the painful tremor left by that real life, which cannot be retold.
Reference
Murray, S. and Keefe, J. (2016) Physical Theatres: A Critical Introduction. 2nd ed. Abingdon and New York: Routledge
Evans, M. (n.d.) A Concise Introduction to Physical Theatre [PDF]. Digital Theatre+. Available at: https://edu-digitaltheatreplus-com.arts.idm.oclc.org/content/guides/a-concise-introduction-to-physical-theatre(Accessed: 7 October 2026).