Archive for September, 2013

Translating the unseen into the ‘seen’ through sound.. visual experience mediated through sounds – using The vOICe

Tuesday, September 10th, 2013

Amplifying one sense over another through temporary sensory deprivation (e.g. self-imposed silence) or using exercises to focus attention on one particular sense (e.g. 360º listening) are tools I often use to develop my individual and participatory work. For this project we are using a digital tool (The vOICe) that has opened up a more complex set of questions for me, in terms of exploring the mechanisms of sensory perception than previous exercises – or at least how I perceive my immediate environment on both an everyday level & in relation to my creative methodology – as well as how others with or without sight might perceive theirs. I will attempt to share & discuss these over several blogs.

I’ve recently had individual conversations with each of the project team and am really looking forward to everyone meeting here online, broadening the blogging to include thoughts, ideas and documentation of experiments, as well as meeting in physical space in the future.

Michael (Proulx) and I recently had a long conversation about perception and using vOICe to explore the mechanisms of vision, amongst other things. Sian (Thomas) and I talked about how we discriminate aurally and what we tune in to or allow ourselves to hear in our soundscapes, dissonance and how we perceive the external influences of culture/language/landscape. We also touched on the shape of sounds in relation to dance. Sam (Randhawa) and I talked about how a dancer can move to describe space whereas as a tabla player with a specific percussive language (Bol) his movement is limited and focused on hand and drum resounding together, he has been thinking about using this language for ‘playing music effectively to represent a space’ and combining this analogue tradition with digital interfaces like a chaos pad to transform sound.

I chose to use The vOICe as a tool and stepping off point for the first part of our enquiry ‘How do we perceive our environment through sound’ because it uses a set unchanging ‘language’ or series of sounds with which to sonify the digital images it processes.¹ It’s tangible. Other members of the team might use other methods.

Putting on the blindfold, headphones and headcam all hooked up to the netbook running the software I felt very ‘other’ and physically disassociated from the material world, luckily Mike was present in the solid world and guiding me; we talked as we went about practicalities as well as the nature of learning new things and perceiving new things.

At first vOICe feels very hectic ² , just too much information, and as someone who is particularly aware of my immediate live soundscape I initially found it hard to adjust my focus to the auditory input from the headphones, my brain unsure which sonic information it should act on. Adjusting to this split aural-visual sensation by realising that using vOICe is a visual experience mediated through sounds not an auditory one.. more of this later I expect. But soon a set of sounds suggested a familiar shape and I settled in to the stereo sensation of scanning left to right within a mental frame. This frame represents the headcam field of vision, onto which my mind started to describe two-dimensional geometric shapes and began to categorise and store the sounds. Little by little constructing whole shapes in my mind’s eye that relate to real objects ³.

With frequent use I tune more deeply into the language, what seemed initially like a quite limited ‘frame of view’ seems to enlarge with ‘training’, it’s a satisfying brain work out – just like speaking another language is. The temptation to guess what I’m ‘seeing’ (or lift the blindfold) fades and I realise I am genuinely perceiving my environment in a sensorily different way that lies between seeing, hearing and mentally visualising. I’m aware of using my hands to draw shapes in the air as my mind perceives them rather than nervously waving them around to avoid bumping into obstacles.

Mike and I will be recording and documenting a vOICe walk to post here soon. I’m looking forward to discovering a new sonic shape to things.

¹These sounds are based on a lot of past experiments (1920s onwards)with sighted people to see what sort of correspondences they had between certain sounds and spatial locations.

²Michael Proulx cited the ‘blooming, buzzing, confusion’ William James described that a newborn might experience before it begins to categorise it’s experiences and therefore perceive – we all need to categorise a new experience at any age before we perceive it ‘the shock of the new’!

³Michael likes to use vOICe for his research particularly because it reveals the mechanisms of vision by causing the user to approach the raw input as new sensory information to be broken down, categorised and stored for future use.