1. Mult i-perspect ive Panoramas
Slides f r om a t alk by Lihi Zelnik-
Manor at I CCV’07 3DRR wor kshop

2. Pict ur es capt ure memories

3. Panoramas
Regist rat ion: Brown & Lowe, I CCV’05
Blending: Burt & Adelson, Trans. Gr aphics,1983
Visualizat ion: Kopf et al., SI GGRAPH, 2007

4. Bad panor ama?
Out put of Brown & Lowe sof t ware

5. No geomet rically consist ent solut ion

6. Scient ist s solut ion t o panoramas:
Single cent er of proj ect ion
Regist rat ion: Brown & Lowe, I CCV’05
Blending: Burt & Adelson, Trans. Graphics,1983
Visualizat ion: Kopf et al., SI GGRAPH, 2007
No 3D!!!

7. From sphere t o plane
Dist ort ions are unavoidable

8. Dist or t ed panoramas
Out put of Brown & Lowe sof t ware
Act ual appearance

9. Obj ect ives
1. Bet t er looking panoramas
2. Let t he camera move:
• Any view
• Nat ural phot ographing

10. St and on t he shoulder s of
giant s
Cartographers
Artists

11. Cart ographic proj ect ions

12. Common panorama pr oj ect ions
θ
φ
Cylindircal
Perspective Stereographic

13. Global Proj ect ions
Cylindircal
Perspective Stereographic

14. Learn f rom t he art ist s
Mult iple view point s
De Chirico “Myst ery and
Melancholy of a St reet ”,
1914
perspect iveperspect ive
Sharp
discont inuit
y

15. Renaissance paint ers solut ion
“School of Athens”, Raffaello Sanzio ~1510
Give a separat e t reat ment t o dif f erent part s of t he scene!!

16. Personalized pr oj ect ions
“School of Athens”, Raffaello Sanzio ~1510
Give a separat e t reat ment t o dif f erent part s of t he scene!!

17. Mult iple planes of proj ect ion
Sharp discont inuit ies can of t en be well hidden

18. Our multi-view result
Single view

19. Our multi-view result
Single view

20. Our multi-view result
Single view

21. Applying personalized proj ect ions
Foreground
Input images
Background
panorama

22. Single view
Our multi-view result

23. Single view
Our multi-view result

24. Obj ect ives - r evisit ed
1. Bet t er looking panoramas
2. Let t he camera move:
• Any view
• Nat ural phot ographing
Mult iple views can live t oget her

25. Mult i-view composit ions
David Hockney, Place Furst enberg, (1985)
3D!!

26. Melissa Slemin,
Place Furst enberg, 2003
Why mult i-view?
Mult iple viewpoint s Single viewpoint
David Hockney,
Place Furst enberg, 1985

27. Mult i-view panoramas
Single view Mult iview
Requires video input
Zomet et al. (PAMI ’03)

28. Long I maging
Agarwala et al. (SI GGRAPH 2006)

29. Smoot h Mult i-View
Google maps

30. What ’s wrong in t he pict ure?
Google maps

31. Non-smoot h
Google maps

32. The Chair
David Hockney (1985)

33. J oiners ar e popular
4,985 phot os mat ching j oiners.
4,007 phot os mat ching Hockney.
41 groups about Hockney
Thousands of members
Flickr st at ist ics (Aug’07):

34. Main goals:
Aut omat e j oiners
Generalize panoramas t o general
image collect ions

35. Obj ect ives
• For Ar t ist s:
Reduce manual labor
Manual: ~40min. Fully aut omat ic

36. Obj ect ives
• For Ar t ist s:
Reduce manual labor
• For non-ar t ist s:
Gener at e pleasing-t o-t he-eye j oiner s

37. Obj ect ives
• For Ar t ist s:
Reduce manual labor
• For non-ar t ist s:
Gener at e pleasing-t o-t he-eye j oiner s
• For dat a explorat ion:
Or ganize images spat ially

38. What ’s going on her e?

39. A cact i garden

40. Principles

41. Principles
• Convey t opology
Correct
I ncorrect

42. Principles
• Convey t opology
• A 2D layering of images
Blending:
blurry
Graph-cut :
cut s hood
Desired j oiner

43. Principles
• Convey t opology
• A 2D layering of images
• Don’t dist ort images
rot at e scalet ranslat e

44. Principles
• Convey t opology
• A 2D layering of images
• Don’t dist ort images
• Minimize inconsist encies
GoodBad

45. Algorit hm

46. St ep 1: Feat ure mat ching
Brown & Lowe, I CCV’03

47. St ep 2: Align
Large inconsist encies
Brown & Lowe, I CCV’03

48. St ep 3: Or der
Reduced inconsist encies

49. Or dering images
Try all orders: only f or small dat aset s

50. Or dering images
Try all orders: only f or small dat aset s
complexit y: (m+n)α
m = # images
n = # overlaps
α = # acyclic orders

51. Or dering images
Obser vat ions:
– Typically each image overlaps wit h only a
f ew ot her s
– Many decisions can be t aken locally

52. Or dering images
Appr oximat e solut ion:
– Solve f or each image independent ly
– I t er at e over all images

53. Can we do bet t er?

54. St ep 4: I mprove alignment

55. I t erat e Align-Order-
I mpor t ance

56. I t erat ive ref inement
I nit ial Final

57. I t erat ive ref inement
I nit ial Final

58. I t erat ive ref inement
I nit ial Final

59. What is t his?

60. That ’s me reading

61. Anza-Borrego

62. Tract or

63. Paolo Uccello, 1436
Art r eproduct ion

64. Paolo Uccello, 1436 Zelnik & Perona, 2006
Art r eproduct ion

65. Single view-point Zelnik & Perona, 2006
Art r eproduct ion

66. Manual by Phot ographer

67. Our aut omat ic result

68. Failur e?

69. GUI

70. The I mpossible Bridge

71. Homage t o David Hockney

72. • I ncorrect geomet ries are possible and f un!
• Geomet ry is not enough, we need scene analysis
• A highly r elat ed wor k:
"Scene Collages and Flexible Camera Arrays,”
Y. Nomura, L. Zhang and S.K. Nayar,
Eurographics Symposium on Rendering, J un, 2007.
Take home

73. Thank You

74. 15-463 Class Proj ect
f r om 2007
http://www.cs.cmu.edu/afs/andrew/scs/cs/1
5-463/f07/proj_final/www/echuangs/