As humans, we are driven to narrative. It is almost impossible for us to experience a sequence of events and not attempt to ascribe some kind of narrative arc to it. What just happened? What does it mean? How did it start? What will happen next?
There is a considerable literature on the theory and practice of creating narrative in different contexts: fiction or non-fiction; in print or on stage or screen; or via any other medium. Narrative can be language-based, as in a novel, or non-verbal, as in a choreographed dance, or a combination of both, as in a movie. But the basic structure of narrative mostly boils down to a few key points: there is a beginning, middle and an end; something changes along the way; it occurs in some kind of contextual framework. If any of these key elements is missing, we, the readers or viewers, inevitably will try to fill in the gaps.
Over the last 20 years, neuroscientists and cognitive scientists have made significant advances in understanding how – and to some degree, why – the brain creates narrative. Much of this new research complements well the ideas of theorists and practitioners concerning the role of narrative in literature, cinema, theatre and dance. Indeed, many authors have integrated data across the sciences and humanities to build new appreciations of how and why narrative works. Key foundation texts in this field include The Poetics of Mind: Figurative Thought, Language and Understanding by Raymond W Gibbs, Jr (1994); On the Origin of Stories: Evolution, Cognition and Fiction by Brian Boyd (2009); and How Literature Plays with the Brain: the Neuroscience of Reading and Art by Paul B Armstrong (2013).
From a cognitive point of view, we can consider the construction of a narrative as coming in different flavours: for example, we can describe a series of external events, that is, things that others can also describe from their own experiences of the same events; we can create a story from our own remembered experiences that is essentially private, since no one else can have those same memories; or we can create a fiction, a story that no one has ever actually experienced, even though such a narrative necessarily contains elements that are relatable to the external world.
In order to create a narrative, we must access several different elements of memory. There is no single phenomenon called “memory”. Rather, there are many forms of memory, some of which are so fundamental to neural processing that we are not consciously aware of them. The duration of different types of memory can range from a few tenths of a second (eg the early steps of processing visual or auditory stimuli) to a lifetime (eg learning to walk or knowing what a flower is). Furthermore, many forms of memory lie outside easy verbal description, eg how to button up a shirt; how you solved a crossword puzzle; how you felt at lunch-time yesterday.
The components of a traditional first-person narrative (“I did this, and then I did that…”) rely on what is generally known as “autobiographical memory”. This is memory of what you have done in the past. It is fully private, in that only you can access it. Unless you have a very specific type of brain injury, it is updated continuously as you consciously experience the world. There are several remarkable features of this memory system:
For the reader or viewer of a narrative, we automatically feed the story (such as it might be) through our own autobiographical memory processor. We identify the temporal sequence, the salience of each component, the relationships between the components, assign meaning to elements we recognise and make a guess at the meanings of elements new to us. Even the most abstract, abstruse, uneventful “narrative” will have a beginning, middle and an end, a context in which it is viewed, an emotional and cultural framework within which we will evaluate it. And so we create our own narrative version of “the story”, almost certainly incomplete when compared with the narrator’s intent, perhaps remembered only fleetingly, but more likely, generating a new entry of our own autobiographical memory (“Let me tell you about the movie I saw last night…”).
This ability for the viewer to make narrative out of minimal clues has been well recognised for many years. A famous experiment by Heider and Simmel (1944) showed how moving abstract shapes are commonly read as behaving like people within a strong narrative arc.
Similarly, short form texts such haiku, one-sentence poems, even Tweets (!!) can be commanding in their narrative intensity. Advertisers have known all this for a long time. Successful television advertisements must tell a story in as few as 15 seconds (“Look at this! See, it’s amazing! That’s why you need it… Buy one now! Get it here!!”). Indeed, some television advertisements have been recognised as stand-out examples of short-form video narration, for example, highly-regarded productions for Budweiser and Google.
https://www.youtube.com/watch?v=_ZhO57NBJm0
Most television advertising lacks such sophisticated production values, but even the most simple, hackneyed approaches can inspire a poetry video narrative. In my 42nds, the timing of the scenes and the amount of text per scene closely follows that of typical advertisements 15-45 seconds long. This video was commissioned for outdoor screening in a downtown shopping mall and has since been screened on other public locations mixed in with genuine advertisements.
I’ll finish with two examples of masterful, albeit unconventional, poetry videos, both of which have featured on Moving Poems previously, that incorporate many of the elements mentioned above. These episodic narratives subvert the tropes of commercial television whilst illustrating the highly mutable state of autobiographical memory: Profile by RW Perkins and Human Condition by Rich Ferguson and Mark Wilkinson.
The success of these videos relies totally on our ability to:
I’ve always loved language and languages. I did Latin, French and German at school and I could easily have ended up studying linguistics in a slightly different universe. For me, poetry and experimental writing are fundamental ways to explore the limits of language: to try to describe what cannot be put into words, to find out what happens when language is stripped down to its essentials (whatever they are…), to discover how the visual, oral and aural aspects of language interact.
Combining video with poetry and experimental writing has been a revelation in this context. In a video, text can be dynamic, as it changes and morphs in multiple dimensions. Voices can be added, distorted, re-timed, presented in counter-point to each other and to on-screen text; they can even be made to articulate the literally unspeakable via increasingly sophisticated text-to-speech algorithms. And then there are all the possible interactions between the text of the video and its audio-visual content.
Over the last few years, I’ve been increasingly interested in how we deal with translation in poetry videos. I have had many videos screened in non-English speaking festivals and installations where there is usually a requirement for subtitles in either English or the native language. But sometimes, there is an absolute requirement for subtitles in the native language. I also have collaborated with non-English speaking poets which has required me become familiar with at least some aspects their native tongue. And I have even created a genuine bi-lingual video poem.
So how do we deal with multi-lingualism? How should the translations work? There is a large literature on the nature of translation, as well as the underlying neuroscience of bi- and multi-lingualism. But I have been strongly influenced by a couple of books in particular, both of which have been written by authors who have translated some of the most influential experimental poetry and writing :
Poetry videos offer a unique slant on translation that is not available for written text: videos give us the option to hear the original language, and, indeed, read the original written text itself as well as a translation. And we still have access to all the same audio-visual material and its interactions with the text. One corollary of this is that the translated text does not necessarily have to be as “poetic” as the original. It may not even need to be a complete translation, if the sound of the original and its accompanying imagery allow.
It is very common for translated poetry videos to have subtitles added in a similar way to any other video or film. However, conventional subtitles can clash with the visual aesthetic of the video. Nearly all my poetry videos have at least some of the text embedded in them, as part of the overall visual design. When adding translated text, I try to use the same fonts, layouts and designs as the original, so that the look of the video is not changed and the translation is seen as a natural component of the work. Another option is to use closed captions for a translated version. It does not look very elegant, but it allows the user to turn the translation on or off. A further advantage is that the text exists as a separate underlying time-stamped file (eg .srt format) that can be translated by a third party as required.
So now to the big question: how do we actually get the translation? Ideally, you are sufficiently knowledgeable in the appropriate languages to do it yourself. I can do that well enough for German or French, and I have made a genuine bilingual English-French video based on a poem of mine, Signature, originally published in the French journal Recours Au Poème. But although I have become familiar with some aspects of other languages, most notably Spanish, Italian, Greek and Swedish, I cannot translate from English into them. Instead, I rely on machine translation, good dictionaries and, when necessary, advice from a native speaker.
My preferred machine translation system is DeepL which, when tested on languages I do know, performs better than Google Translate across the board, especially when the language becomes more idiomatic or figurative. The suggested translations can be checked and fine tuned in several ways. One is to simply back-translate the phrase to see if you get the same thing. Using a different system for the back-translation (such as Google Translate) can also be useful. Another good strategy is to replace some of the words in the original with synonyms or near-equivalents and compare how they get translated. If the translation offers options, I often look them up in the native language dictionary or thesaurus (eg Wiktionary) that shows how they understand the meaning of the word and how it should be used. And I always keep a grammar book at hand for the language in question to find out how it is structured and how things like tenses, cases, pronouns, adjectives, etc, actually function. These resources have been critical for most of my translations.
Sometimes, the translation simply will not work. An implied meaning in one language perhaps cannot be made in another, often because of the way the grammar rules operate. As is well known to translators, there are idioms and turns of phrase that do not cross languages or cultures. Experimental writing or visual poetry that relies on the intrinsic structure of words and their grammatical variations may be impossible to translate in any literal manner. What do we do then? We simply watch the video, listen to the ebbs and flows of words that even a native speaker may not understand, and revel in the uncertainty of it all.
Click here to see all Ian’s videos with translations.
This essay first appeared on Ian’s blog, which anyone with a serious interest in videopoetry ought to follow. —Dave B.