The Contemporary Paradigm and Cognitive Neuroscience

Hi - sorry for the delay in writing. You know how it is: you have the best of intentions but things come up. So I gave a lecture in Oz  on the gnostic ideology underpinning Cognitive Neuroscience. There are papers available, there's also a journal call for this subject which will soon go large. Everyone loves CG and MFRI scanners because they seem to offer proof where before there was only conjecture.

Eye tracking only tells you where eyes are triggered to move to, which talks about design but not art, it talks about mechanics but not aesthetics. As it stands in the attached paper the researchers are careful to say that CG and MFRI scanners only imp lie certain things - and of course everyone's executed because the medieval latin scholar has access to 'proof' as much as the hard core scientist - but the truth is the proof is circumstantial.

The Arts and Humanities Research Council has called for the cultural values evaluation project to reveal ideas of the value of culture and art that are more empirical than flowery french prose and at back of that the work of the Frankfurt Group, already criticised as a misdirection by Thomas Crow in 'The Intelligence of Art'.

But - because society functions in faddist terms (as a friend of mine says: "academics are like cows in a field they hear a loud bang and all look in the same direction" - but you can substitute artists, cinematographers - whatever you like here) then we can be sure that rock and roll currently finds its resting place in Cognitive Neuroscience.

And then there's the problem of what art is in an age of integration (having superseded convergence). It's said by Emeritus Professors of Cognitive Neuroscience that the Artist is the cognitive formative node that sits within a cognitive distributive network - and that art is what it is to be human, not just a thing that humans do (note the Gnostic position) - so the question arises in an age when everyone has a platform for screeching their individuality (and here I always laugh at the Italian phrase: "Few are called; But many answer"): what is the function of art within a paradigm that few even recognise has superseded an older paradigm that no longer functions?

I make art work that is cognisant of this set of problems and have two new works that directly deal with the central problem of the role of the artist - one of which I have to premier which has taken a year and is a triptych which feature a reconstruction of Dali's Crucifixion (really). I'm also preparing to do a performance event where I engage with the celebrity Ted lecture stance which is called: 'An Anatomy of Light'. This is a lecture in-the-round which invites the audience to consider how the world is illuminated, how we use light to manifest 'moving images' and what happens when light ceases to operate in the universe (I say moving images but that's a complete fallacy - it is the mind that moves).

So there it is, that's what I've been thinking about lately - how to deal with a new era of human thought given that contemporarily we're still thinking in old pattens.

Understanding the Digital Realm - On coming to ISEA 2013

Duchamp must be laughing in his grave as we go forward towards the future, as we appear to be walking backwards staring into the past. Duchamp might laugh because he was always strong on pushing the concept rather than the material – or at least balancing the two. But now we are having real trouble disentangling the real from the unreal, the material from the immaterial. Trying to understand the things that flow past us as we walk backwards – effectively part of the past as soon as we notice them – we reach for descriptions of these things fashioned with the terms and ideas we understand so well, from the past. Ideas that once fitted like gloves. But these are of course inappropriate to grasp the naming of the needs of the new.

Were we facing the right way when walking forward we would see an unfamiliar landscape and we’d have to invent names for the things that we see coming towards us, some animate, some inanimate and sometimes we’d confuse the two. But we’d have a new language that would describe the form and behaviour of the new things that approached us – or as we approached them, because it is we that are moving.

We would have to remember Einstein’s dictum that everything is relative and what we think we see as we red-shift our way towards things that in the distance would appear different from the way we see them close-up, and then differently as they fall behind: we’d have to realise that they have a changing nature as we apprehend them and lose them from sight.

Right now in walking backwards as we go forwards we use the nomenclature of things we have become familiar with from past experience, because of their similarities with the new things we perceive – but of course there is a moment when our metaphors start to fail in their description of what we are seeing – and the language is no longer fit for purpose. But we cling to this language because it has served us until now. It will be our older selves or even our children who will laugh at the misconceptions we generate and wonder why we didn’t walk forwards, facing ahead, describing what we once saw, in new language rather than old.

The Ontology of Digital Cinematography

I’ve been reading reports in Senses of Cinema* on recent international film festivals and came upon this comment on the film Leviathan by Daniel Fairfax and Joshua Sperling during their review of the New York Film Festival:

By contrast, Leviathan is pure cinema. The fact that it, too, was shot on digital does not detract from such a status. Rather, it demonstrates that there are really two digital aesthetics: the fantasy digital practiced by the likes of Ang Lee, and the “ontological digital” at work in this film. Or rather, it demonstrates that the digital/analogue dichotomy is more a question of aesthetic principles, of philosophies towards filmmaking, than of technology – and in this case, Castaing-Taylor/Paravel’s work falls squarely on the analogue side of the divide.

‘The analogue side of the divide’ is the metaphor used to describe that which entrances by lack of guile – whereas the digital side of the divide is all guile. It used to be that the metaphor for the digital was ‘clinical ‘and in some senses the ‘fantasy digital’ has helped move the clinical into the mythic. But here, a documentary helps create another line of division, where the poetic and the clinical can mix and transmute the medium into what the authors describe as the ‘ontological digital’:

The waterproof prosumer cameras used to extract the extraordinary imagery of Leviathan are tasked simply with recording the real. They do so to such a visceral extent that at certain moments – when the masses of dead fish squirm about as they pile up before the camera – Leviathan can feel like a horror film, an effect which the Gothic writing on the film’s title card would suggest is intentional. In a way, it possesses a more truly three-dimensional quality than the tawdry gimmicks of Pi could ever hope to attain.

This associative thinking is reminiscent of much film theory of the French style where over elaborate and poetic descriptions are used to massage the reader into the belief that they are reading ‘truth’. But you can’t really blame the French for trying their best – after all they have a lifestyle to maintain and the rest of the world should celebrate their largesse, their joie de vivre. Any country that has such a surfeit of bakeries deserves respect even if it requires agricultural support way beyond others. The Anglo-Saxon critic owes a great debt to French theory with its twin polarities of articulation and obfuscation in pursuit of poetic truth.

This last comment really allows the digital through into cinema as it invokes one of the mythic gods of French theory to authenticate its position:

Our absorption in the film’s unrelenting diegesis is enhanced not only by the immersive camerawork, but also by the unsettling surround-sound audio, which I felt was reminiscent of Philippe Grandrieux’s Un Lac. And, lo! Grandrieux himself was in the audience for Leviathan’s press screening, having just embarked on a road-trip with the filmmaking duo, where, as Castaing-Taylor related, they whiled away the hours by discussing Deleuze.

The fantasy digital’ is actually a symbolic reference to what is in truth a combined commercial and technical description. What the digital has enabled in surpassing the capabilities of film, whilst at the same time finally imitating its ability to invoke ‘cinema’, is a set of economic benefits. If you wanted to shoot 35mm film at 48 frames per second – you would need a massive stock budget. With digital you simply dial up 48fps. Yes it has data ‘costs’ – that is management issues around the production of large amounts of data, which also has cost implications for storage – but due to the much-abused Moore’s Law, computational storage becomes cheaper over time. The issue is that one can dial in a specific enhancement, 3D, higher frame rate, uncompressed recording, higher dynamic range etc, without the pain of late Victorian industrial style costs.

Film grew out of sewing machines and fake teeth. Stop-start machinery could enable proper exposure and shifting film in the gate to move on without fogging. Material developed for Victorian dentists for replacement teeth could be stretched, flattened and made clear – a medium fit for holding virtual images (until they were developed). Even with Henry Ford’s intervention, it would still be expensive late into the twentieth century. The masses would not take up such a clunky modernist medium. With the advent of analogue, then its chimera digital video, the last gasp of the struggle for democratic production of images was to be heard, but the ‘industry’, the protective mechanism of ‘quality’ would see that Digital Cinematography, raw, progressive imaging, was sufficiently expensive to deter the great mass of untalented or rather, derivative creativity - as exhibited by user generated content. Until the great mass receives the education of the intelligentsia the end product would always be the accidents of a 1000 chimpanzees typing away, with the odd surprise on Utube.

Beneath this piece of blogging (unsubstantiated opinionating) is a conviction derived from many years of being at several ‘coal faces’. These are professional production in the UK TV and film industries, artistic practice, theorizing as an academic and lastly and tellingly,  as a socialist. This latter is about youthful conviction where I believed that the means of production should be placed in the hands of the populace so that its more varied stories and perspectives could be spoken aloud. I had that belief whilst being enabled as a speaker for that demographic – that is, I was a specialist who could ‘help’ the masses. As defined by Buckminster Fuller, E F Schumaker, Edmund Carpenter, Stafford Beer and their like I could be a ‘competent man’ (this term coined prior to the advent of feminism really means competent person). This competence began its like as a specialism, such as drawing, which when abstracted from the practice could be made universal in creative terms: if you were good at radio, you could be good at anything if you simply kept your wits about you and exported a set of taste functions (in most circumstances, ‘this’ is better than ‘that’, but in specific circumstances remember ‘this’ juxtaposition). But there was a high romanticism about this which included a belief in ‘great art’ -  that is some art was better than other art – and in that belief the socialist ideal fell. And it stands with me now that I do not believe everyone is as talented as everyone else – and the training of everyone to be excellent must also mean that everyone has to have a value system of excellence – which is both tautological and self-defeating.

Just look at Ang Lee for instance. He can certainly make movies, but frankly with his resources (and I include talent) so can anyone. The Life of Pi in movie form to me is trite and soporific. It has a confused message which advocates embracing symbolism over ‘reality’. I’m not sure about the book because I haven’t read it – but I suspect it as a piece of modern fiction as most modern fiction is victim of ‘cut and paste’. But that’s a different argument.

But back to ‘the fantasy digital’ and ‘the ontological digital’. These are consumerist descriptions as they can be applied to different cinematic experience on a consumer level. I like the idea of an ontological digital because I’ve been operating it since I first took up making experimental motion images – even in analogue form when it was equally a form of ontological video – and was appropriate in the naming because video is latin for ‘to see’. And ‘to see’ is important if understood from the vantage point of cognitive neuroscience where when one speaks of ‘seeing’ one is speaking of the combined eye/brain pathway. In these terms perceiving and understanding are a combined activity, the left eye being governed by right brain and vice versa…. And in this narrative, left-brain is the site of focused attention which is highly ratiocinatory in nature – or so the neuro-scientific community would currently have it.

Whilst looking through or within the electronic terms provided by video, I have seen a description of the world that has been reflective of my internal state. I have made works in this electronic as opposed to photo-chemical medium that have added meaning to the world I see in biological terms. Biology here means ontological in a certain sense – that sense if added to, is more full when combined with the extra viewpoint enabled by video.

Max Hastings once wrote ‘film is a long-distance telephone call whereas video is a call from the box round the corner’. This comment was made at a time when there was a qualitative difference in the two kinds of call. The distant one sounded so due to interference on the line, the inferior sound quality, the clicks and bumps and atmospherics. Hastings wasn’t saying that film was inferior; he was talking of the romanticism of distance – distant, unknowable lands where information had been brought back by a Marco Polo, as an ambassador that brought back tales of the unknown. Interestingly there are questions about the authenticity of Polo’s stories – but again, that’s another article.

It was in fact Hastings intent to describe something other than the lesser quality of video – that though the displayed image was lesser in quality it had a greater quality: video was live, it was here and it was now. Remember that video came along a long time after television, the parent medium with its ability to disseminate. So the currency of television was of presenting the world as it is NOW. Video inherited the connection of representing NOW. The feel of it was and is immediate and Digital Cinematography, when it became progressively based, shed the sense of the immediate and became THEN, elsewhen and  elsewhere. With Digital Cinematography came a greater possibility of the electronic capture of video as having a developed capacity for ontological use.

So our two authors bring up a definition of immediacy as if it were a Gothic artefact additional to the medium that has been introduced successfully as a medium of fantasy. I think there’s more to it than that description, that delineation of two levels. I think it has many more levels than two and this current description is unaware of the width of the medium. My evidence for this the developed plasticity of past media, which, when they go past the necessary period of remediation, always disclose their true nature outside of their ability to chameleon-like imitate the behaviour of other media. For a description of what that true nature is, watch this space.

Note to self, see 'Leviathan'.

*http://sensesofcinema.com/

The Expanding Horizon of Digital Cinematography

Recently I have been working with Faculty of Engineering at University of Bristol and BBC Research and Development to shoot some tests for the expanding capabilities of Digital Cinematography. My role on this project was to oversee the cinematography. In essence we have been trying to calibrate Higher Dynamic Range, Higher Frame Rate and Higher Resolutions to match the eye/brain pathway to create highest immersion for the viewing experience. The first tests were shot in November 2012 and since then have been ongoing.

In the human optical system we have a sensitivity of 14 orders of magnitude of which we can always access 5 orders of magnitude. These 5 orders slide upwards for bright desert sun and downwards to cope with low light levels of moon and starlight. The University of Bristol has a display that exhibits this dynamic range.

Digital Cinematography cameras capture around 12 – 14 stops and 35mm film captures around 18 stops. Standard displays, TV’s projectors & computer screens display between 10 and 16 Fstops – that’s between 2 to 3 orders of magnitude of the entire 14 available in the human system, so if you create a higher dynamic range image of 18 or above stops – it will display beneath its dynamic range on contemporary displays. The common response from people seeing this is: “the image looks ‘plastic’”.

The department of Experimental Psychology at Bristol has already undertaken 2D and 3D immersion tests – but these require of 30 minutes of footage. With a limited budget we decided to shoot the Somerset Carnival because of its high internal illumination and floats with internal movement. To shoot HDR an Epic would shoot 50 fps at 4k in HDR mode with a 4 to 6 stop difference giving 18 stops dynamic range.

We then considered mounting 2 Phantoms to shoot 200 fps at 2k in a 3D mirror rig, one camera exposing the high stop the other the low. Dr Marc Price & Alia Skeikh of BBC R&D placed two cameras on a rig at BBC London, but found that artifacts became evident as the tolerances necessary for HDR alignment are far higher than 3D because you need pixel accurate registration to eliminate these artifacts.

We calculated on Epic we could record 6 mins of 4k, 50 fps HDR (thats 100 fps) that would take 40 minutes to download. We decided that due to much higher levels of data output on the Phantom, we would shoot selected floats at the carnival exhibiting high levels of motion. Had we shot Phantom in a mirror rig at its highest speed you could easily generate 1 terabyte of data per minute and that would take 6 and half hours to download.

An HDR stop-motion test conducted by Aaron Fang of University of Bristol Engineering revealed that you need 7 exposures combined to display full higher dynamic range on the display. So using Red’s strategy of setting a correct Fstop to build upon for HDR did not exploit the full-potential available.

The hardest thing to expose in cinematography is a subject that emits light: At our first shoot in Burnham on Sea on the Epic I set the ASA at 320 at 50 fps with a shutter of 100^th, then used a spotmeter to calculate a stop. But following this through to display, we discovered that a full HDR image was not achieved using normal cinematographic judgment. I realised that the 100 year old maxim of exposing to protect highlights was no longer a correct rule for HDR. In fact you had to expose the ‘correct’ stop ‘virtually’ – What I mean by this is that if the Fstop should be F5.6 then we would have set the Iris to overexpose three stops over at F2 plus HDRx highlight protection of three stops under: making 18 stops in total. That might seem obvious now, but on the shoot, sphincters tightened, because the Epic images looked terribly overexposed.

We had planned a second shoot at Wells and collected material at 2 stops and 3 stops over and under e-exposed. In HDR terms that’s between 16 to 18 stops respectively. We are about to shoot more footage for immersion testing at the newly built lab at Bristol University. It should be remembered that the first tests worldwide with this expanded digital cinematographic form took place in the South West of the UK.

From the cinematographers viewpoint, it seems to me that counter-intuitively, lower lighting levels are where HDR will function with most impact on a photographic level. Obviously in the highlights colour formation will be held better than in standard systems, but this is a technical issue, rather than specifically an artistic one. But this is interesting because it may in fact require a renaissance of cinematic judgment – someone will need to know that the end result will be fine. With regard low light, the way we read an image may allow the cinematographer to offer clues when underscoring plot, story & emotional cues, in a far more subtle way than with standard dynamic range that the cinematographic arts have used for decades: that of trying to represent 5 orders of magnitude in a 2 – 3 orders of magnitude display.