Presentation to Jesse's class March 04, 2013.

1. encoding for the imagination
Douglas Goodwin
03/04/2013
Monday, March 4, 13
encoding for the imagination,
doug goodwin, march fourth, 2013

2. encoding strategy:
Parrhasius’ grapes
Zeuxis’ grapes fooled nature
Parrhasius’ grapes fooled Zeuxis!
Monday, March 4, 13
According to the Naturalis Historia of Pliny the Elder, Zeuxis and his contemporary Parrhasius (of Ephesus and later Athens) staged a contest to determine the greater artist. When Zeuxis unveiled his painting of grapes,
they appeared so real birds flew down to peck at them. But when Zeuxis asked Parrhasius to pull aside the curtain from his painting, the curtain itself turned out to be a painted illusion. Parrhasius won, and Zeuxis
admitted, 'I have deceived the birds, but Parrhasius has deceived Zeuxis.'

3. overview
1. how does digital video represent an image?
2. how can we tap into the system that lies between stimuli and
sensation?
3. how does video take advantage of imagination and memory?
4. the grapes will soon be forgotten. will we starve?
5. Lossless #3 & #4
6. q&a
Monday, March 4, 13
overview

4. how does digital video encode an image?
1. cull what cannot be seen (ex: everything below a certain
luminance value)
2. divide the x/y frame into a matrix of macroblocks
3. analyze temporal difference between current frame and the
next frame for each block (motion analysis)
4. if temporal difference < n then retain the current block
5. split the x/y frame luminance and chrominance
6. throw out a lot of chrominance (4:2:2)
7. reorder the frames for efficient storage
8. transform the frame into waveforms and then digitize them.
Monday, March 4, 13
MPEG Compression & how it works in IPTV and other digital media delivery systems.
http://youtu.be/P7abyWT4dss

5. we call this “lossy” compression because so
much of the image is discarded in the encoding
process.
Monday, March 4, 13
overview

6. are you missing that image data?
Monday, March 4, 13
overview

7. what can you offer to replace it?
Monday, March 4, 13
overview

8. the curtain.
Monday, March 4, 13
overview

9. what do I mean by that?
we may be enticed to forget about the high-resolution image.
how?
1. there may be images that suggest the image without
showing it,
2. images that please us more than the grapes,
3. and/or images that go around the problem altogether.
Monday, March 4, 13
overview

10. representing the invisible
Monday, March 4, 13

11. Monday, March 4, 13
here is an example. This image contains a symbol within it. once seen it is not easily forgotten. and more to our point: once seen it does not need to be shown
again.
ready?

12. Monday, March 4, 13
once seen, it can never be
unseen

13. Monday, March 4, 13
do I ever need to show you the arrow
again?

14. what is this?
how do you know?
Monday, March 4, 13
how do you interpret this image? After you find an anchor point your eyes track to one eye, then to the other, then to the mouth, back to the eye, then up to the
forehead, along the hairline to the side of the cheek, down to the chin, etc.

15. your eyes travel in a
predictable pattern
when you look at faces.
what could you do with that?
Monday, March 4, 13
you have done this before. Your saccades have travelled down this road. That’s how you know it’s a face.

16. discarding details
Monday, March 4, 13

17. Aoccdrnig to rscheearch at Cmabrigde
Uinervtisy, it deosn't mttaer in waht oredr the
ltteers in a wrod are, the olny iprmoatnt tihng is
taht the frist and lsat ltteer be at the rghit pclae.
The rset can be a toatl mses and you can sitll
raed it wouthit a porbelm. Tihs is bcuseae the
huamn mnid deos not raed ervey lteter by
istlef, but the wrod as a wlohe.
Monday, March 4, 13
can you read this? did you need to think about it?
could a machine?
where is the processing that makes this possible?
does it suggest ways you might hack the perceptual pipeline?

18. system hacks
Monday, March 4, 13

19. Monday, March 4, 13
say the color of each word as fast as possible.
would a computer have trouble with this?

20. Monday, March 4, 13
this image is -not- moving

21. placing the viewer with representational style
Monday, March 4, 13

22. Monday, March 4, 13
example from scott mccloud
also: mcluhan’s hot vs. cold media

23. The bedroom window—the one nearest the hallway— opens outward. The upper part of A .. ,'s
body is framed within it. She says "Hello" in the playful tone of someone who has slept well
and awakened in a good mood; or of someone who prefers not to show what she is thinking
about—if anything—and always flashes the same smile, on principle; the same smile, which
can be interpreted as derision just as well as affection, or the total absence of any feeling
whatever.
Besides, she has not awakened just now. It is obvious she has already taken her shower. She
is still wearing her dressing gown, but her lips are freshly made up—the lipstick color the same
as their natural color, a trifle deeper, and her carefully brushed hair gleams in the light from the
window when she turns her head, shifting the soft, heavy curls whose black mass falls over the
white silk of her shoulder.
She goes to the heavy chest against the rear partition. She opens the top drawer to take out a
small object and turns back toward the light. On the log bridge the crouching native has
disappeared. There is no one visible around the house. No cutting crew is working in this
sector, for the moment.
Monday, March 4, 13
robbe-grillet, jealousy
“objective” language

24. This room was too big, the ceiling was too high, the doors were too tall, and the white carpet
that went from wall to wall looked like a fresh fall of snow at Lake Arrowhead. There were full-
length mirrors and crystal doodads all over the place. The ivory furniture had chromium on it,
and the enormous ivory drapes lay tumbled on the white carpet a yard from the windows. The
white made the ivory look dirty and the ivory made the white look bled out. The windows
stared towards the darkening foothills. It was going to rain soon. There was pressure in the air
already.
I sat down on the edge of a deep soft chair and looked at Mrs. Regan. She was worth a stare.
She was trouble. She was stretched out on a modernistic chaise-longue with her slippers off,
so I stared at her legs in the sheerest silk stockings. They seemed to be arranged to stare at.
They were visible to the knee and one of them well beyond. The knees were dimpled, not bony
and sharp. The calves were beautiful, the ankles long and slim and with enough melodic line
for a tone poem. She was tall and rangy and strong-looking. Her head was against an ivory
satin cushion. Her hair was black and wiry and parted in the middle and she had the hot black
eyes of the portrait in the hall. She had a good mouth and a good chin. There was a sulky
droop to her lips and the lower lip was full.
She had a drink. She took a swallow from it and gave me a cool level stare over the rim of the
glass.
Monday, March 4, 13
raymond chandler, the big sleep

25. Monday, March 4, 13

26. Other systems: Tineye imagination
what does the web “see” in an image?
Monday, March 4, 13

27. Monday, March 4, 13

28. Monday, March 4, 13

29. lossless series
with rebecca baron
2008-09
Monday, March 4, 13
what happens when the keyframes are taken away in mpeg2?

30. artifact 1
2010
Monday, March 4, 13
what would it be like to have a very large temporal buffer?

31. thanks!
doug goodwin
dgoodwin@gmail.com
Monday, March 4, 13