Ripples of Sound

C ontemporary German sound artist Carsten Nicolai is internationally known for his experimental music performances and installation works such as wellenwanne (2001/2003/2008), in which water trays visualize musical sounds through changing wave and interference patterns. In fall 2015, Nicolai presented a new work of immense proportions, unitape, at the Kunstsammlungen Chemnitz. ^[1] Twenty-five meters in length and four meters in height, unitape puts to work different encodings of sound and music, its visual structure recalling the look of punch cards as they had first been used in the weaving industry around Chemnitz in the mid-1800s. Cascading from the gallery’s ceiling, ever-changing arrangements of quadrilateral marks trigger the production of acoustical signals, their vibrations so intense that we can literally sense them with our entire bodies. Mirrors on each side of the projection screen create wondrous impressions of endlessness: a sense of openness resulting from the sheer infinite combination of basic elements, the simplicity of binary code. Whether you witness the installation for only few minutes or for hours on end, it is difficult not to experience unsettling wanderings and cross-overs of perception. Nicolai’s falling patterns invite us to listen with our eyes as much as with our ears. Hidden underneath the bench, the speaker system causes visitors to see with their ears as much as to attend to sound through their guts. Though seemingly based on unambiguous algorithms and bonding sound and image into seamless synchronicity, then, unitape suspends the ordinary work—the reliability—of sensory perception. As we scan Nicolai’s monumental projection, the visible seems to slip away from the audible. And as we listen closely to the installation’s sounds, we come to realize that projected image and sound space can never fuse into absolute unity, not least of all because the varied operations of our sensorium are much more complex, layered, messy, and unpredictable than a purely technical synchronization of acoustical and visual codes might suggest. Unitape’s beats and rhythms hit the viewer with the unswerving precision of a techno performance. Yet in the end the possibilities of how we translate different codes into each other emerge as seemingly endless while the space in-between sound and image unlocks a vast zone of indeterminacy, a zone of unscripted pleasure, attention, and resonance.

What we over the course of the last two decades have come to call sound art—understood as a curious blend of performance and installation art, of experimental music, and of new media practice—certainly witnessed a first awakening among modernist and avant-garde practitioners in the early decades of the twentieth century. Think of Futurism’s praise of industrial din and its construction of elaborate noise machines. Think of Dada performances such as Kurt Schwitter’s Ur-Sonate (1922-1932).

Think of experimental moving image work such as Hans Richter’s Rhythm 21 (1921), Walter Ruttmann’s Opus 1 (1921) and Weekend (1929), and Viking Eggeling’s Symphonie Diagonale (1924), all designed to explore synesthetic modes of perception and to make viewers hear the beats, rhythms, and echoes of visual shapes. But much of this early work was undertaken with means starkly different from what computational devices can do for artists today. Richter’s or Ruttmann’s experiments were dedicated to investigate the artistic specificity of the filmic medium prior to the coming of sound, its difference from other modes of artistic production. Today’s work, by contrast, often rests on the assumption that digital machines integrate the operations of all previous mediums and therefore allow us to move beyond modernism’s obsession with issues of medium specificity once and for all.

And yet, Nicolai’s desire to move sound and image into a zone of indeterminacy was certainly on the mind of German filmmakers, theorists, sound technicians, voice coaches, and musical composers as well as they, in the late 1920s, faced the transformation of synchronized sound into a new industrial standard of cinema. Many Weimar film theorists continue to receive a lot of bad press for arguing against the coming of synchronized sound. Their fear that sound could debase the aesthetic properties of film strikes many today as a strangely technophobic gesture amid their ambition to celebrate cinematic technology as the bedrock of twentieth-century art. But what some critics in the 1920s dreaded most about sound in the theater was less the mere addition of new technological components than the specter of redundancy: that cinema could disseminate the same thing through multiple channels at once and thus thwart what good art was considered to be all about. True art embodied creative uniqueness and non-conceptual singularity in each and every of its objects and moves. Sound film, on the other hand, was seen as simply adding information unnecessary for those able to watch with attentive and discerning eyes. It was feared to double information about the image on screen, and to duplicate information meant to undo any rigorous aesthetic aspiration.

The historical response to these misgivings about possible redundancy was threefold, as we know. Filmmakers were eager either (1) to build a fortress around the moving image to silence sound altogether; (2) to pursue the possibilities of contrapuntal sound in order to run a film’s acoustical dimensions against its visual properties, and vice versa, and thus explode possible duplication dialectically from within; or (3) to allow sound to engage with the image across the boundaries of a given frame, expanding or placing pressure on the visible and thereby enriching the cinematic experience. Each of these responses was meant to vaccinate the art of art cinema against the disease synchronized sound was alleged to bring to the theater. All of them certainly inspired creative practices and artistic departures, even if they in the end rested—as I will suggest in a moment—on false assumptions about the nature of sound. Yet none of these efforts seemed to entirely placate some of the most critical viewers appalled about all-too frequent situations during the early days of synchronized sound in which sound and image appeared at odds with each other, not for deliberate artistic reasons, but simply due to technological failures and technical lapses of their makers.

In an essay written in April 1929, shortly after the first full-length films had premiered in German theaters, film critic Béla Balázs described the most virulent danger of sound cinema as a quasi-schizophrenic disjunction of visual and acoustical perspectives, a painful sliding off of the sound image from the visual image:

The sound that issues from the fixed speaker, and therefore always comes from the same place, simply cannot follow the picture that moves straight across the space of the screen. Imagine I have realized that there is sound coming from the moving lips of a singer standing in the left corner of the screen. If he then moves to the right side of the screen, regardless of how wide he opens his mouth or how forcefully he uses his vocal instrument, the sound will not move to the right side with him. The acoustic image stays where it is, sliding off from the visual image, and what we are left with in the end is a ventriloquial effect. You hear the sound coming from an entirely different place than where you see the source. Without really knowing why, the audience experiences a sense of unease resulting from this general confusion of impressions.^[2]

Though known as a champion of cinema’s visual qualities, Balász was certainly no outright enemy of synchronized sound. Like some others, he was eager to envision a future of sound film art able to retrain the viewer’s hearing and make moviegoers listen to the formerly unheard, the acoustical unconscious. But mainstream sound film, as it came to fill the screens and theaters circa 1929, in Bálasz’s perspective, largely failed to live up to this task because filmmakers, in their very efforts to wed visual and acoustical image into perfect matrimony, produced bastard offsprings: films whose sounds did not want to stick to the images; films in which images made sounds stutter; and worst of all, films in which actors, contrary to intention, appeared as ventriloquist dummies.

Balász’s notion of sound cinema as a cinema of unaesthetic ventriloquism is as intriguing and instructive as his metaphor of a film’s sound image sliding off the visual image. Both are helpful to untangle the very etiologies Weimar critics sought to identify when synchronized sound entered the stage; and both are valuable to understand why all three cures suggested around 1930—no matter how path-breaking they may have been—often rested on erroneous conceptions of the nature of sound and auditory perception. Whether film practitioners argued for silence, counterpoint, or frame expansion as a cure to the ills of un-artistic sound: in most cases they endorsed Balász’s implicit view of synchronized sound as an acoustical image, a track similar to a film’s image track, their mutual relationship to be defined by the filmmaker and welded into predictable and lasting existence on the celluloid strip itself. Whatever their factual results, all three cures were conceived as strategies to oppose artistically mindless couplings of image and sound. They were designed either to exclude the possibility of redundant parallelism or to prevent seemingly inevitable schizo-dramas of visual and acoustical perspectivalism, the anti-aesthetic of exposed ventriloquism. As Balász’s underlying notion of the sound image suggests, however, Weimar’s crusade against audiovisual redundancy and its fear of the ventriloquial modelled its understanding of sound along the lines of what was known and familiar about the image. It expected sound to follow the principles of visual perspective, presupposed that the audible should travel from speaker to listener similar to how the visible image would travel from screen to viewer, and then cried havoc when sound suddenly seemed to assume a life of its own.

{}

There were certainly good reasons in the late-1920s to do so. What made synchronized sound possible was the ability to record sound as a visible trace on the actual film strip, right next to the frames of the image track—as nicely retold and entertainingly illustrated for historical audiences in Max Fleischer’s 1929 cartoon, Finding His Voice. Moreover, static microphones, mono-channel recordings, and single speakers centrally hidden behind the screen, made sound initially appear as immobile indeed, void of direction, depth, and relative location. What fueled Weimar misgivings about redundancy and ventriloquism, then, was nothing less than the fact that historically specific states of sound technology caused critics to graft their ideas of the visual onto the auditory, not to hear that even single-channel sound behaved quite differently than projected images, and as a result to present their skewered experience of sound as an ahistorical law of the acoustical, an anthropological constant of what it might me mean to hear. To think of sound as image created the very problem Weimar film practitioners and critics so eagerly and polemically sought to solve. It resulted from a thinking assuming that sound should address the ear analogous to how moving images addressed the viewer’s eyes through the window of a quadrilateral screen in front of the cinematic auditorium. And it produced cures to illnesses that even mono-channel sound never had. It thought film from the materiality and parallel logics of inscriptions on the film strip, whereas the true challenge would have been to think of how the human sensorium relates quite differently to visual and acoustical input while never keeping one channel of information entirely separate from the other.

The coming of stereophony in the course of the 1930s exposed what Weimar critics, compelled by the loaded but misleading metaphor of sound as image, were often unwilling to hear: sound waves do not behave like the rays of light; we do not encounter sound like something following the tenets of Renaissance perspective. As conventionally projected onto flat surfaces, moving images create powerful illusions and impressions of spatial depth. They offer portals to what the perceiving eye and body take as three-dimensional even if it, in fact, may exist only in two-dimensions in the world. Even prior to stereo and surround technologies, sound on the other hand produces actual spaces; invisible, albeit physical, environments of immersion; something that envelopes, floods, activates, and physically touches upon the body. Sound cannot but exist in space: it models three-dimensional experience, articulates time in space. There may be good reasons, as Jonathan Sterne has rightly argued, not to ontologize and naturalize the difference between the auditory and the visual all too much and hence proclaim laws for each independent of historically available recording and reproduction technologies.^[3] But once you actually abandon the idea of sound as image and start to recognize the different sensory dimensions of the auditory and the visual, once you desert the assumption that the ear’s operations simply emulate the eye’s functions, then neither Weimar’s fears about redundancy and ventriloquism, nor historical visions of audio-visual counterpoint or other cures to the alleged pathologies of the coming of sound make sense at all anymore. Dialectical audio-vision is categorically as impossible as any perception of sound perversely slipping off the image. The first plays out material against virtual perceptions of space; it juxtapose elements that are different anyway or that meet in much more messy ways than the iron fist of dialectical thinking might be willing to grant. The second ignores the fact that sound can actually never slip off any image unless you think of sound as an image in the first place, and not as something whose space for the perceiving subject is always already different as the space of the image.

Even before stereophony stressed the spatiality and dynamic character of sound, however, imaginative filmmakers such as Fritz Lang were already eager to reveal the inadequacies of what most contemporary critics had to say about the coming of synchronized sound. To be sure, Lang’s own reflections on cinema often ripped off the writing of critics such as Bálasz. But in some of his statements about the coming of sound film, and even more so in his first sound films, Lang left little doubt that sound cannot and should not be treated like an image—and that, therefore, Weimar’s qualms about the pathologies of redundancy and ventroquialism relied on flawed diagnoses. In 1931, curious readers of Film-Kurier were presented with this stunningly perceptive summary: “Lang does not believe that a generalized sound symbolism is attainable. The spoken word will remain as an unavoidable factor of realism, and even of the theatrical stage. For this reason, it will frequently be difficult to translate image montages into sonic form, for sound spreads as a spherical wave identical on all sides; no sound ‘shot’ can change the imprint of a sonic expression—whereas the visual can be approached from many different angles, and creates a different impression each time.”^[4] Unlike those who envisioned sound from the vantage point of the image and were dismayed when it violated the corset of visual perspectivalism, Lang’s understanding of the acoustical as spherical underwrote uses of sound in film no longer simply meant to prevent possible ills and cure existing contagions. Sound, for Lang, was an animal altogether different from the visual image—and filmmaking at its best embraced this tension between the perspectival and the spherical as a source of infinite artistic opportunity, as something that freed sound film categorically from the specters of redundancy and ventriloquism.

The very first sequence of Lang’s very first sound film, M (1931), certainly knows of and details this. When Elsie’s mom leaves the stage-like setting of the balcony, assuming that she has succeeded in shutting down the children’s song, and when after a few moments of pregnant silence we hear the song again without witnessing the camera redirect its gaze: what we certainly do not experience here is a Balászian instance of sound sliding off the image, nor are we encouraged to understand Lang’s choreography of image and sound as intentionally contrapuntal or as an attempt of simply expanding the frame. Quite on the contrary. What we witness is that sound, due to its spherical nature, travels and behaves quite differently than the projected image; that it has little respect for the integrity of frames, stages, and screens to begin with; that it can have the power to overwrite the image; and that even moments of redundancy—the repeat singing of that “awful song”—can produce chilling aesthetic effects, reminding us of the fact that no one can ever fully control the spaces of urban life in each and every of their aspects. Sounds can clash with sounds and thus develop dialectical tensions; sounds and images can’t as they belong to partially different experiential orders. Sounds do not simply expand the frame of the image while leaving the principles of central perspective intact. They are hostile to the very idea of the frame and instead spread across and contaminate the visual, questioning the very reliability of monocular vision and the ideologies of sovereignty—of controlling space through imperial gazes, of being one’s master in the house, of speaking in one and only one firm voice— Western societies have come to attach to vision ever since the Renaissance.

{}

As any astute viewer of Lang or any listener of Nicolai’s unitape can discover, even the seemingly most redundant projection of visual and acoustical elements can produce bodily responses that are complex, differentiated, and unsettling. No matter what inscription you may see on a film strip or can visualize on today’s hard drives, the meeting of sound waves and light rays in the world cannot but produce relative zones of indeterminacy simply because our ears and eyes don’t operate like mechanistic channel readers, one solely focused on the first, the other solely on the second channel, both trying to weld everything that comes along their way into perfect harmony. Weimar viewers were probably much more playful, creative, and less prone to being duped than many of their critics feared. Like Lang himself and like Nicolai’s audience in Chemnitz, they may have understood that redundancy is never as redundant as wary critics and judicious watchdogs make it out to be, and that ventriloquism—far from being a negative, a haunting specter—is one of the most profound pleasures cinema has to offer. “Our senses have boundaries,” says Nicolai. “But one can enable translations and transfer sounds into a different medium one can see or sense.”^[5] The infinite, albeit never complete, translatability of images, sounds, movements, thoughts, and affects between and across different registers of the human sensorium remains perhaps cinemas greatest and most enduring promise.

Download this article as a PDF.

Lutz Koepnick is the Max Kade Foundation Chair of German Studies and Professor of Cinema and Media Arts at Vanderbilt University in Nashville. Koepnick has published widely on film, media theory, visual culture, new media aesthetic, and intellectual history from the nineteenth to the twenty-first century. He is the author of Resonant Matter: Sound, Art, and the Promise of Hospitality (2021); Fitzcarraldo (2019); Michael Bay: World Cinema in the Age of Populism (2018); The Long Take: Art Cinema and the Wondrous (2017); On Slowness: Toward an Aesthetic of the Contemporary (2014); Framing Attention: Windows on Modern German Culture (2007); The Dark Mirror: German Cinema between Hitler and Hollywood (2002); Walter Benjamin and the Aesthetics of Power (1999) and of Nothungs Modernität: Wagners Ring und die Poesie der Politik im neunzehnten Jahrhundert (1994). Koepnick is the co-author of Windows | Interface (2007), [Grid ‹ › Matrix] (2006), and the co-editor of various anthologies on ambiguity in contemporary art and theory, the culture of neoliberalism, German cinema, sound culture, new media aesthetics, aesthetic theory, and questions of exile. His current projects include a collaborative multi-season podcast on art and climate change, entitled The Art of Interference; and a book on expanded notions of listening.

Lutz Koepnick: http://lutzkoepnick.com

Please cite this article as:

Koepnick, Lutz. “Ripples of Sound.” The Promise of Cinema. 12-11-2015. ./index.php/ripples-of-sound/.

---

[1] Kulturstiftung des Bundes

[2] Béla Balász, “A Conviction: An Initial View of a New Art Form’s Perspectives and Limitations,” trans. Christopher M. Geissler, in: The Promise of Cinema: German Film Theory 1907-1933, eds. Anton Kaes, Nicholas Baer, Michael Cowan (Berkeley: University of California Press, 2017) 561.

[3] Jonathan Sterne, The Audible Past: Cultural Origins of Sound Reproduction (Durham: Duke University Press, 2003) 14-15.

[4] “Fritz Lang: Problems in Sound Film Design: Moving Away from Naturalism,” The Promise of Cinema 576.

[5] Carsten Nicolai, Parallel Lines Cross at Infinity (Chemnitz: Kunstsammlungen Chemnitz, 2015) 10.