ABSTRACT:
As with all form and content, there is an interrelationship between the technicalities of the form, in this case High Definition Video, and its output, be this art or entertainment. In this article I try to cover the technical, aesthetic and artistic issues that face those who would use High Definition as an educational, theoretical or production medium. A central premise that I shall propose is that for some years now we've been entering what could be termed 'the Digital Age' having travelled from the childhood of the Analogue Age, through the early Electrical age and onwards. Great things are now upon us as the Digital comes out of its first stage, into subsequent developments which will lead to changes in both the practices and aesthetics of the moving image form.
KEYWORDS:
High Definition, Analogue, Digital, Wavelet, Fourier, Discrete Cosine Transform.
HOW THE DIGITAL WORLD IS CHANGING DURING WEB 2.0
At this years National Association of Broadcasters convention in Las Vegas, HD changed forever - suddenly we leaped several generations and as from a few weeks ago (April 2007) we gained the capacity, on everybody's desktop, to manipulate the most high level of HD there currently is at the production level. Jim Jannard, a sunglasses manufacturer from Canada said to himself - 'I think I can change things for the better' - and he did. He managed to manufacture some equipment that one might previously pay $500,000 for, for $17,500 - and if that wasn't all, he got together with Steve Jobs at Apple and made Final Cut Pro Studio take this data and work with it... So, if NHK's Super Hi-Vision (as it is so quaintly named), a distant dream six months ago, with results that resembled the first Lumiers Brothers screenings - that of completely dumbfounding the audience who then ran from the cinema (the Japanese are reported to have produced copious amounts of Vomit from their test audience), then actually, this distant data future - is now absolutely in reach.
Any serious understanding of High Definition technologies require a basic understanding of digital technologies and the idea of 'compression'. Simply put, data is a representation of the artefact manufactured whether generated in or imported into the digital realm, and all representations have levels of veracity. Meaning, a full representation is all the data of the original, and anything else is something that resembles the original but has less data in that representation. Wavelets, a newish technology (the theories were in place in 1807) has helped prize open pandoras box:
"Wavelets are mathematical functions that cut up data into different frequency components, and then study each component with a resolution matched to its scale. They have advantages over traditional Fourier methods in analyzing physical situations where the signal contains discontinuities and sharp spikes. Wavelets were developed independently in the fields of mathematics, quantum physics, electrical engineering, and seismic geology. Interchanges between these fields during the last ten years have led to many new wavelet applications such as image compression, turbulence, human vision, radar, and earthquake prediction".
Amara Graps, http://www.amara.com/index.html
Here Walter Benjamin's 1936 essay comes to mind, on the question which concerns mechanical representations of the original - prints, photography etc - can the representation carry any of the authenticity of the original? Then in 1987 John Wyver carried the argument along with the help of Jean Baudrillard, Paul Virillio et al in his opening programme for the satellite station: La Sept - L'objet d'art a l'age electronique - at that time we were concerned with analogue representations which like most things actually decay in their passage from medium to medium, from copy to copy. We are now in the maelstrom of the following digital issue - where is meaning, significance and value in the digital domain, given that now, the media of copying and actually making things reside together in that realm ?
If one proceeded with previous forms of digital compression then Benjamin's insight would ring true, that to copy is to decrease (not only in terms of the Chinese whispers function - that things are changed in the act of copying, but also that the representation itself is simply a Borg - a copy without feeling. In other words the copy is without the 'true' sense of the original. It brings to mind the notion of the Golem:
"In Jewish folklore, a golem From the Hebrew word meaning material is an animated being created entirely from inanimate matter. In modern Hebrew the word golem literally means 'cocoon', but can also mean "fool", "silly", or even "stupid". The name appears to derive from the word gelem which means "raw material".
An abridged definition of the Golem from Wikipedia
These days, though the net, Hesse's Glass Bead game of meaning is available and so cross metaphors can be usefully applied - so I return again to Amara Graps description of the underlying principles of Wavelets:
"The wavelet analysis procedure is to adopt a wavelet prototype function, called an analyzing wavelet or mother wavelet. Temporal analysis is performed with a contracted, high-frequency version of the prototype wavelet, while frequency analysis is performed with a dilated, low-frequency version of the same wavelet. Because the original signal or function can be represented in terms of a wavelet expansion (using coefficients in a linear combination of the wavelet functions), data operations can be performed using just the corresponding wavelet coefficients. And if you further choose the best wavelets adapted to your data, or truncate the coefficients below a threshold, your data is sparsely represented. This sparse coding makes wavelets an excellent tool in the field of data compression".
Amara Graps, http://www.amara.com/index.html
It becomes clear that the Wavelet function is analogous to the use of a zoom lens as it picks which window to look on the world in either a large way or a small way, in that one zooms in on the data at a certain level and chooses at what level to represent what lies in front of the camera.
Do forgive a series of metaphors mixed, I am simply trying to get at some difficult technical concepts which no doubt the deep practitioners and theorist within these various forms would turn red with rage at my descriptions. And: A note on all the technical details in my postings - If I've got it wrong - write and tell me because everyone that is involved in this kind of information is liable to get things wrong.
--------------------
Lest numbers are not your comfortable place, I will cone to the aesthetics of the medium later on in the article.
The meta language of HD is replete with either numbers in some configuration 444, 311 for instance or made up words like Wavelet's, DPX, LUT's, Cineform, Codecs and so on.
This language requires a grasp as, if you hear someone saying they are shooting HD with an HDV camera, which has an original sensor resolution of 960 pixels as opposed to the full 1920 of 2k HD, then you need to know there's a misunderstanding going on. More importantly, you need to know what the issues are to make a proper judgment - like in any other are of concern.
Equally, you have to know that with a form like HD Cam (Sony's lower end HD system), which might have a sensor that produces 1920 x 1080 pixels, that very, very quickly in the processing chain in the camera, that 480 of those pixels are dispensed with so that the magic number of 1440 are recorded. (the HDV's 960 pixels up up-rezzed to 1440 to produce what might look like to some eyes, HD). Then - after so much is thrown already away in terms of data, does compression begin.
HDV for instance records an Mpeg signal which uses a long GOP of 15 frames which is a highly compressed codeces.... If I carry on like this I shall spend the rest of this article simple running numbers and trying to explain them. Suffice it to say that last year I gave a seminar on HD entitled: High Definition, No Mercy.
So that there's no sense of 'Datus Interruptus', I'll attempt to explain this issue then try to leave the technicalities alone so that we do not go off on too many back alley trips: Group of Pictures refers to basic compression notions derived from Fourier Transforms leading to Discrete Cosine Transforms or DCT's - In a long Gop, the group of pictures only changes the data where the picture itself changes, so for instance a person against a backdrop moves their eyes and mouth and sometimes their head, sometimes their hands, but the backdrop and the suit stay the same - so why renew these elements if they're the same all the way through the 15 frames. So they don't and the impact of this in recording images as can be seen with any HDV camera which records images using this system is that it handles motion badly. It is this kind of artefact that exercises the Director's of Photography, imaging technicians and post production technicians so much: Their eyes are attuned to pick up this stuff and yell like hell when they see them. If you allow these through then even the digital realm will disintegrate from copy to copy, or from tape to streaming or DVD or broadcast. This to my mind is one use of Discrete Cosine Transform which describes just how much damage can be done to the data/signal in its passage through the film making process.
WHERE REALITY ITSELF IS EXTREME RESOLUTION
Or where does compression begin and aesthetics end ?
It's been my practice as a DP on receiving a new script which it was then my responsibility to confer a look upon and also to quality control the production from inception through to release print, to make a plan.
Within the available contemporary aesthetics there are a series of tactics to 'say something' with light. These if listed become quotidian: A warm look for comfort, blue for coldness, alienation, and a variety of strategies which, if the DP were 'artistic' and masterly enough could be transcended and actually trip the production out of the commercial realm into the artistic.
So I would sit down with my script and make my notes to the best of my ability and my aesthetic whilst keeping a mind to innovation. Then the night before the shoot, in a growing state of anxiety I would throw away my plan as I suppose I believe, deep down, that if I approach the production as it happens on my wit and my unarticulated aesthetic then I shall reach deeper down into my creativity and find something that goes beyond technique and hopefully approaches art.
In the past as an artist, when I've made works of art - works which were intended only every to say something about the world outside of the act of theorising the world - then I would function solely intuitively responding to each moment whether in the shooting stage or in the editing stage to whatever the material and the software programme I was using 'said' to me to do. Therefore I was open to the practical acts of assembling the final work so that those final acts were also part of the creative process.
All of my previous low definition art worked in this way and I was often enraptured by the simple act of making the work with such wonderful technology. The technology like an internal combustion engine functioned faster than the eye or mind. If you think that a car uses a series of miniature controlled explosions many thousands of times a minute - you can't help but wonder. And video, even analogue video, I believe, took one 64 millionth of a second to 'write' a line, of which in PAL there were 625 of these in each frame and 25 of these every second. How amazing.
But now High Definition is faster and more fully resolved and I have some research goals to discover: I know there is such a thing as a photographic moment - this I have to find. I know that the higher resolutions available will enable me to disclose this photographic moment. Equally I know that the actual moment deserves dwelling upon and here I remember Bill Violas Zen-like observation:
Duration is to consciousness as light is to the eye.
I know what he means - the terms of the idea state deal with the rules of consciousness and duration is a key element within consciousness in that consciousness itself is only enacted not as one simple moment but in and through a series of moments where infinite difference registers. The issue is to unveil this thought and other developed thoughts about who, how and what we are to an audience. However, Viola's concerns are not my concerns. His roots seem to be in renaissance painting and though I've always aspired to renaissance ideals - to be knowledgeable of many different concerns and practices - my roots really grow out of the moment that someone noticed that all things record an image. From a lowly rock which if left in shadow long enough, to paper that has a leaf left on it j bright sunlight to metal with a chemical coating to celluloid that holds a coating, to tubes, chips and sensors that react to light.
I have also to remember Georgia O'Keeffe's stance on creative acts on becoming blind; she took up ceramics to continue being creative. (she died on 6th March 1986 at the age of 98!). This is a quote from my piece WINGS:
(http://video.google.co.uk/videoplay?docid=-5542788291542710223):
After Georgia O'Keeffe had lost her sight and could no longer paint, she took to pottery to continue creating... As she neared death she spoke of the creative act, of how the need to make something was like looking out over a precipice, of how one must leap into the abyss and of how fear itself would turn into wings...
Frida Khalo in the year of my birth had her feet amputated and said of this: Feet, what do I need feet for if I have wings to fly ?
--------------------
A SHORT EXCHANGE
KF: '... I think there's a place of surrender in which one has very little to do with either the journey of life or the exact outcome of art. Lots of the artists I know talk about a kind of alpha state in which they work. I know that place and both regard it highly and revere it deeply! I think, ideally, I would like to approach life in the same way; if the alpha state and the meditative state could only be amalgamated. Much of the pain of life seems to come from resistance and fear, and these are conditions which are projected, not of the moment....'
TF: 'I'd like to quote you on that - it's succinct and in terms of technology highlights an interesting problem: that often one has to work through others in a topography which is outside of the terrain of the initial idea. Or, said in another way, the place where the art is made is different and sometimes unsympathetic to the state that the idea was framed within'.
--------------------
A CREATIVE FELLOWSHIP IN HIGH DEFININTION IMAGING: THE ACTUAL, THE VIRTUAL AND THE HYPER REAL
And what went before
So - I am approaching my Creative Fellowship at Bristol University which begins in September (it is now May 2007) and in proposing to the British Arts and Humanities Research Council I had to create a three year work schedule to qualify what it was that I felt I had to say about the aesthetics of high definition Video and how this new and developing aesthetic might affect the work of the practitioners and the viewing experience of the audience.
Of course, I am now further on in my thinking than I was when I proposed my schedule and my artistic endeavours. So the work I am now having grow in my mind is beginning to differ from the original suggestions. That difference is about a realisation I had about what it is that needs exploring which centres on unveiling the deep and full resolution of the photographic moment.
I began my involvement with HD around 1992, albeit at arms length and previously there were other indicators in my practice that lead in this direction - my involvement with photography from as early as 1971 for instance with my concomitant exposure to sound from around 1966, but professionally from 1971). My current and growing understanding of the aesthetics that are potentially of the HD form have made me begin to think in a new way. Also I am beginning to realise that the term High Definition is not completely correct - maybe High Resolution Imaging is more appropriate - as High Definition is a relative term - relative to what has been.
I am coming to the end of the first stage of my work which began with my involvement in video in 1976. All of my standard definition work from this date to the present has been a preparation for the development of a concern which renders the previous work as sketches, investigations into image making. My yet to be premiered low definition long form piece 14, History Lessons, 18, Visions, 21, Beatification, represents the end of that stage. (2007, PAL, 77 minutes). In this work I seek to disclose the central and underlying experiences that have coloured the reasons that make me make art. Initially as a seventeen year old I explained my urge to make art to myself as just that - a creative urge and maybe that's exactly what it was. However, as I grew older I had some formative experiences that coloured my view of the world sufficiently to energise the kind of art that I made and currently make.
My most recent piece, The Unfurling represents a threshold I have to now go through. Suffice it to say that the Unfurling requires extreme slow motion High Definition images and though I have revelled in the artefacts that come about by extreme processing of a low definition image (meaning that it changes in a way that I couldn't predict) and thus enjoying the accidents that come about through stressing the parameters the equipment was manufactured to handle to the point where the image breaks down, I wish for more clarity as opposed to less.
Let me return to Viola's statement:
Duration is to consciousness as light is to the eye.
Through talking about the way we experience the world directly using the terms of the experience then we can realise that this is an element that the artist might grasp and manipulate, I believe his intuition was leading him to something that High Definition can deliver - not just duration, but articulation:
Definition is to consciousness as luminosity is to the eye.
And when I observe my own naming of pieces as an indicator to the underlying enquires that one as an artist makes - and given my chosen title for this most recent piece, this also represents an unfurling of a consciousness in my own mind about the way a series of images function when demonstrating movement. I was dimly aware in my previous use of photography through its higher resolutions than standard TV resolutions, that the lessening of picture quality was holding something back from my grasp.
I had explored most other elements within the moving image and have always thought narrative should not be excised from the artists palette, regardless of its heavy usage by the entertainments industry - artists of course need necessarily go in the opposite direction from commercial definitions and usages of elements with the available palette as a form of distancing from entertainment and therefore creating a chiracuso for their art.
I can articulate this 'something' by talking around it: though cinema requires the creation of atmosphere normally achieved through the distressing of the image, atmosphere was a form of impurity disavowed by artists who tend to seek the truth of their art through an ascetic attitude as a means of obtaining and delivering truth through a kind of purity.
Cinematographers on the other hand constantly distort the colour standards and definitions the film manufacturers seek to implement for their stock, to impose atmosphere. 'Atmosphere', like popcorn, shares a quality that allows the easier suspension of disbelief. If film manufactures say that development should occur at such and such a temperature, then heating up or cooling down the developer is a means by which the colour or grain or exposure may be changed in a pleasing way. All cinematographers seek to distinguish themselves from the others primarily to have a signature for themselves that delivers a unique selling point, therefore a higher income - however, as in all practices, there are some that completely transcend the monetary issues.
One issue arises here and that's an argument that has developed in contemporary high end HD photography which is beginning to be won by a faction that I suppose, because I'm a romantic, I disagree with.
High end HD work is far beyond the use of tape to capture the data necessary to render the image accurately. Sony became famous amongst those in the know in HD cinematography by winning the format war against Philips who'd developed a superior 1250 line system in the 1990's by developing a more manageable image of 1920 x 1080 pixels (1080 lines) and then making cameras which threw away about 500 pixels and only recorded 1440. They did this because it wasn't possible to record all 1920 pixels - the data pipelines were not wide enough. Even then, the 1440 pixels were only recorded in a way that lost much detail. A pixel is effectively a packet of data that is represented on screen by a changing luminosity and a changing colour identity - Sony's pixels in HD Cam are in 3.1.1 colour space - that's a description of what each pixel packet contains in terms of data that renders the luminosity and colour in tact - or not as the case may be. The ideal in terms of recording data (at least at the moment is 4.4.4.) though the previous Olympian peak of Standard Definition PAL was 4.2.2. What these numbers mean (and I will go into this at a later stage in a later paper) one can guess that the more samplings (copied bits) of the original there are - the more data can be displayed. As usual, bigger equals better.
At the high end of HD the goal is to record all of the pixels in 4.4.4 colour space. Compression or 'lossy' recording, soon became a solution to massive amounts of data (60 gigabytes per minute of decent HD) so that something of some value could be recorded on tape. In the highest form of HD image capture at the moment one has to record on hard disc - and not just any hard disc (though Sony's data deck, the SR, records a form of high end HD which is in general use). In short, contemporary drives run at 7200 revolutions per minute and read and write between 30 and 50 megabytes per second (MB/sec as opposed to Mb/sec). It doesn't really matter what that means in detail, unless like me you're a train spotter at heart, it simply matters that if you want to record Sony's system which loses 500 pixels at the outset, you need 220 MB/s of information captured and played back and for that you need a Redundant Array of Independent Discs.
If I throw you a ball you might be able to catch it. If I manage to throw you 20 balls at the same time you have no chance, If I throw 20 balls at you and another 19 friends - you have a chance of catching them. A RAID Array uses a group of discs to catch large amounts of data. This of course has a more complicated description and in some ways my metaphor is incorrect - but you get the idea.
If you want to record 1920 x 1080 pixels with their full complement of data then you need read and write speeds of 440 MBs and up. There are however cheap ways of doing this, but that's another issue. So, given all of that the HD world has spent its time thinking in terms of being true to the form and it really doesn't want to distress an image that is already distressed by being captured badly.
If you do work on the image in camera, in the colour matrices, then you limit how much data is recorded. If you crush the blacks to get a 'look' you automatically reduce the data that is output into the image display - therefore the cinematographer ever mindful of his or her employ wants every bit of data carried back into post production and when all is gathered in their swag bag and taken home - then the work on the image can begin. But of course if you've ever really watched an image that is produced like this you see that there is a thin patina over the image and the look itself is not inherent in the image you are looking at. As I said before, I'm a romantic so I want the look within the image. I spent 30 years shooting video as well as film and I know it's possible.
--------------------
People like Vitorio Storaro (Apocalypse Now, The Spiders Stratagem etc) whose famous colour theories render incomprehension and a throwing up of hands from the more prosaic and practical amongst the brotherhood of cinematographers - but everyone has to agree - whatever it is he is thinking is certainly 'bringing home the bacon' for him in terms of his invention and sheer artistry. (And by the way, I love the fact that my spellcheck renamed Storaro, Victoria Storeroom). Equally, if one is to talk about the cinematographers art, one has to mention Conrad Hall for his inventiveness and commitment to the photographic within the cinematic arts. Hall's use of an oppositional term is confusing - cinematography is the photography engendered by a constant flow of images that create movement - so why use the term photography when it refers to the still image ?
As his career progressed, and as he traversed the boundaries of contemporary wisdom about what constitutes good exposure, Hall came to understand that the act of generating the still image and something within the act of photography gets at something that cinematography rarely gets at and he was therefore concerned with finding the photographic moment amidst the haste and flow of images (Jean Baudrillard also noted something similar in his diatribes on the ubiquitous ness of the image). I'll leave definitions of what the photographic might be to others, as I don't want to be diverted from my present task, except to point up that the journalistic is a large element within finding the photographic within cinematic art. Here, I don't mean the prosaic happenings of the quotidian, I mean rather, their opposite, the finding of the extraordinary within the ordinary.
--------------------
One last thing on the cinematography that is taking place in Hollywood in High Definition shooting. Historically in the clash between film and video, the film users were seen as the craftsmen and the video users were seen as being artless. I remember being on a Ridley Scott set in 1983 as he shot the famous 1984 commercial. I was shooting for Apple and my footage was used by Apple to heighten awareness that the commercial that was to introduce the Apple Mac was only going to be shown once in the middle of that years Superbowl - this was a fantastic ploy that earned that strategy a massive audience that is still viewing this particular commercial. As we were viewing back our rushes checking focus and exposure (and the fact that we were shooting NTSC and sometimes the frequencies in the English electrical system created strobing patterns that need to be identified and therefore the footage might need re-shooting). At that moment of viewing the footage I became aware that about 20 people were standing behind us looking at our monitor.
This was a film crew that had never before seen what it had been shooting at the same time as shooting it. Usually the film would come back the next day so the past would always be viewed (by the more select in the hierarchy) not the 'present'. We all stared at each other - two alien tribes, previously at war with each other (the video people after all were going to destroy their medium in the long run and take their work). Then one of them grinned in pleasure at seeing this and suddenly, like the German and British troops in the first world war downing their rifles on Christmas day and playing football together, suddenly we were friends. From then on they stopped being horrible to us, even sometimes offering to move lights to let us have more illumination. In the end the artist in me took this footage and I made a piece called Prisoners (1984, Pal/NTSC lowband, 16 minutes), which Apple had some problem with because on the shoot there were 150 neo fascists from the bovver boots agency who were known to have rioted during the production of Pink Floyds 'The Wall'. Apple didn't want that particular association with their new product. I contacted Ridley Scott's office and at that time he backed my usage of the footage.
http://video.google.co.uk/videoplay?docid=4229365999366193762
But - historically film people are brought in to light video because they are seen as the artists. But they don't know the technology (film guys light, video guys to the technology for them, though of course HD knowledge is spreading like lightning) and that has meant that pure currently unaffected footage gets taken back into post to do the colour grading work.
So, though High Definition is a clear and clean medium (in the past film dirt went vertically and analogue video dirt went horizontally) and HD in its nature and form in essence is counter to the notions that lead to the distressing of the structure the image, for me now, HD is beginning to represent the finessing of the detail that is under investigation, and what that detail might mean on one level and do in terms of being an aesthetic, on another.
So, though I understand that creating the look as it's called in the business is the means by which the cinematographer creates atmosphere (as a kind of tour guide for the story) and as they lead their audience into the commercial entertainment that is the cinema and TV experience, in terms of producing art, the clarity and detail that is an essential part of high definition is now a direction that I have to pursue - especially as these two factors are the area of fastest and most sought after development in High Definition form.
I haven't come to this idea rapidly as I have long cherished the ability of video to create surprises for the maker, but with the development of the video form through its analogue past into its digital future I can now find surprises in the higher rendering of detail. This is one element of HD and my intuition and experience tell me that after a long exposure to HD, I have only just scratched the surface - and scratch it I must, with a pick-axe if necessary.
--------------------
One last description of Wavelet's as they have the power to metaphorise the Digital Moment - Amara Graps:
"The fundamental idea behind wavelets is to analyze according to scale. Indeed, some researchers in the wavelet field feel that, by using wavelets, one is adopting a whole new mindset or perspective in processing data.
Wavelets are functions that satisfy certain mathematical requirements and are used in representing data or other functions. This idea is not new. Approximation using superposition of functions has existed since the early 1800's, when Joseph Fourier discovered that he could superpose sines and cosines to represent other functions. However, in wavelet analysis, the scale that we use to look at data plays a special role. Wavelet algorithms process data at different scales or resolutions. If we look at a signal with a large "window," we would notice gross features. Similarly, if we look at a signal with a small "window," we would notice small features. The result in wavelet analysis is to see both the forest and the trees, so to speak.
This makes wavelets interesting and useful. For many decades, scientists have wanted more appropriate functions than the sines and cosines which comprise the bases of Fourier analysis, to approximate choppy signals (1). By their definition, these functions are non-local (and stretch out to infinity). They therefore do a very poor job in approximating sharp spikes. But with wavelet analysis, we can use approximating functions that are contained neatly in finite domains. Wavelets are well-suited for approximating data with sharp discontinuities.
The wavelet analysis procedure is to adopt a wavelet prototype function, called an analyzing wavelet or mother wavelet. Temporal analysis is performed with a contracted, high-frequency version of the prototype wavelet, while frequency analysis is performed with a dilated, low-frequency version of the same wavelet. Because the original signal or function can be represented in terms of a wavelet expansion (using coefficients in a linear combination of the wavelet functions), data operations can be performed using just the corresponding wavelet coefficients. And if you further choose the best wavelets adapted to your data, or truncate the coefficients below a threshold, your data is sparsely represented. This sparse coding makes wavelets an excellent tool in the field of data compression.
Other applied fields that are making use of wavelets include astronomy, acoustics, nuclear engineering, sub-band coding, signal and image processing, neurophysiology, music, magnetic resonance imaging, speech discrimination, optics, fractals, turbulence, earthquake-prediction, radar, human vision, and pure mathematics applications such as solving partial differential equations."
For much more information on wavelets go to: http://www.amara.com/current/wavelet.html
