Politecnico di Milano

1. Camera Fingerprint and its robustness
Polo Regionale di Como
Facoltà di Ingegneria dell’Informazione
Laurea Magistrale in Ing. Informatica
12 gennaio 2010
Francesco Forestieri
Giorgio Zinetti

2. 2
1. Forensics Overview
2. Application of Image Forensics
3. Image Acquisition
4. Type os Image Sensor
5. Camera Fingerprint
6. PRNU
7. PRNU Robustness
8. Conclusions
9. References

3. 3
Forensic science is the application of a broad
spectrum of sciences to answer questions of interest to a legal system.
This may be in relation to a crime or a civil action.
In modern use, the term "forensics" in the the place of "forensic
science" can be considered correct as term "forensic" is
effectively a synonym for "legal" or "related to courts".

4. 4
Digital forensics (sometimes Digital forensic science) is a branch Forensic science
encompassing the recovery and investigation of material found in digital devices,
Digital
Forensics
Computer
Forensics
Network
Forensics
Multimedia
Forensics
Image Video Audio

5. 5
Digital
Forensics
Computer
Forensics
Network
Forensics
Multimedia
Forensics
Image Video Audio
DIGITAL IMAGE FORENSICS
For a digital image:
• Wich camera brand took the picture?
• Is it forgered or manipulated?
• How it is capturede? Digital camera? Digital scanner o camcorder?
• Is a computer graphic rendering?
In a court a digital images and videos are not easily
acceptable because Is difficult to establish their integrety.
To avoid this problem we can use:

6. 6
Forgery Identification
Is it real ?

7. 7
Fake photo Original photo

8. 8

9. 9
Estimate geometrical processing
Visual representation of the detected cropping and scaling parameters .
The gray frame shows the original image size, while the blue frame shows
the image size after cropping and before resizing.

10. 10
Distinction between scan and digital image
Illegal copy of a digital content:
• Cinema recaptured video by a camcorder
• Scannered book

11. 11
Device
Identification
Device
Linking
Fingerprint
Matching
Device Identification
• Which camera brand took this photo?
• Is a natural image or a computer grapich rendering?

12. 12
Human Fingerprint Camera Fingerprint
Photo-Response NonUnifomity (PRNU) is an intrinsic property of all digital imaging
sensor due to slight variations among indivisual pixels in their ability to convert
photons to electorns. Consecuently every sensor cast a weak noise-like pattern onto
every image it takes and this pattern play the role of sensor fingerprint.

13. 13
Device Linking
To prove that two images were taken by the same device
or to know which camera brand took the photo.
Device
Identification
Device
Linking
Fingerprint
Matching

14. 14
Fingerprint matching
Corresponds to the situation in which we need to
decide whether or not two estimates of two
potentially different fingerprints are identical.
The presence of camera fingerprint in an image is
also indicative of the fact that the image under
investigation is natural and not a computer rendering
Device
Identification
Device
Linking
Fingerprint
Matching
Is it a natural photo?

15. 15
The process of acquiring a digital image is quite complex and varies greatly across
different camera models, some basic elements are common to most cameras.
…0101011010100101…
light

16. 16
Light
Projection
Amplification and
quantization of
energy
Sharpening
Interpolation
and
demosaiking
Color and gamma
correction
Colors
Storing
Signal
Adjusting
1. The light cast by the camera optics is
projected onto the pixel grid of the
imaging sensors.

17. 17
Light
Projection
Amplification and
quantization of
energy
Sharpening
Interpolation
and
demosaiking
Color and gamma
correction
Colors
Storing
Signal
Adjusting
2. The charge generated through interaction
of photons with silicon is amplified and
quantized

18. 18
Light
Projection
Amplification and
quantization of
energy
Sharpening
Interpolation
and
demosaiking
Color and gamma
correction
Colors
Storing
Signal
Adjusting
3. The signal from each color channel is
adjusted for gain (scaled) to achieve proper
white balance.

19. 19
Light
Projection
Amplification and
quantization of
energy
Sharpening
Interpolation
and
demosaiking
Color and gamma
correction
Colors
Storing
Signal
Adjusting
4. Because most sensors cannot register color,
the pixels are typically equipped with a
color filter that lets only light of one
specific color (red, green, or blue) enter
the pixel. The array of filters is called the
color filter array (CFA)

20. 20
Light
Projection
Amplification and
quantization of
energy
Sharpening
Interpolation
and
demosaiking
Color and gamma
correction
Colors
Storing
Signal
Adjusting
5. To obtain a color image, the signal is
interpolated or demosaicked.

21. 21
Light
Projection
Amplification and
quantization of
energy
Sharpening
Interpolation
and
demosaiking
Color and gamma
correction
Colors
Storing
Signal
Adjusting
6. Then the colors are further adjusted to
display correctly on a computer monitor
through color correction and gamma
correction

22. 22
Light
Projection
Amplification and
quantization of
energy
Sharpening
Interpolation
and
demosaiking
Color and gamma
correction
Colors
Storing
Signal
Adjusting
7. Cameras may also employ filtering
techniques, such as denoising or
sharpening.

23. 23
Light
Projection
Amplification and
quantization of
energy
Sharpening
Interpolation
and
demosaiking
Color and gamma
correction
Colors
Storing
Signal
Adjusting
8. Finally, the image is stored in the JPEG or
some other format, which may involve
quantization.

24. 24
Image sensors are devices that convert light in electical signals which will be trasformed
in bits that are the costituents of a digital image.
In the digital camera world there are basically two types of digital sensors:
(Charge-Coupled Devices)
Currently the most commonly used image sensor, CCDs capture light onto an array of
light-sensitive diodes, each diode representing one pixel.

25. 25
(Complementary Metal Oxide Semiconductor)
Like CCDs, these imagers are made from silicon, but as the name implies, the process
they are made in is called CMOS. This process is today the most common method of
making processors and memories, meaning CMOS Imagers take advantage of the
process and cost advancements created by these other high-volume devices.

26. CMOS CCD
Cost
inexpensive because CMOS
wafers are used for many
different types of
semiconductors
expensive to produce
because of special
manufacturing methods
employed
Power
low power consumption consumes upto 100x more
power than CMOS
Noise susceptible to noise high quality, low noise images
Extended Functionality
other circuitry easily
incorporated on same chip
technically feasible; other
chips are used
Fill Factor high low
26
Due to both design and manufacturing considerations, there are a number of
advantages that CMOS Imagers have over CCD:
• Integration
• Manufacturing Cost

27. 27
The noise in a taken picture in caused by shot noise and pattern noise.
This first one is unavoidable noise caused by fluctuating photons, the latter one
stays almost the same when pictures are taken in the same scene and over time.
This is why pattern noise can be used for identication.
Noises
Shot
Noise
Pattern
Noise
PRNU FPN

28. 28
I[i] the quantized signal registered at pixel i
Y[i] is the incident light intensity at pixel i
pixel indices have been dropped for better readability
g is the gain factor (different for each color channel) and is the gamma
correction factor (typically < 0.45)
The matrix K is a zero-mean noiselike signal responsible
for the PRNU K is the sensor fingerprint.
is a combination of the other noise sources
Q is the combined distortion due to quantization and/or JPEG compression

29. 29
In parts of the image that are not dark, the dominant term in the square bracket in
is the scene light intensity, Y.
By factoring it out and keeping the first two terms in the Taylor expansion of
we obtain
is the ideal sensor output in the absence
of any noise or imperfections
Is the PRNU term that absorbs also
is the modeling noise
where

30. 30
Device Linking
Device
Identification
Device
Linking
Fingerprint
Matching
Calculate the camera
reference pattern
Calculate the noise of an
image
Find out the correlation
between camera reference
pattern and image noise

31. 31
Fast Way:
• Average multiple images (approximation)
To speed up this process:
• Remove the scene content using a denoising filter
• Subtract the denoised images with the original one
• Average the noise residual
Hints:
• Denoising filter can be: Median Filter or Wavelet-Based filter
• Is useful also average uniformly images like photos of a white paper or
a white wall.
• The larger the number of images over, the more we suppress random
noise and the camera reference pattern is accurate.
Calculate the camera
reference pattern

32. 32
A Camera Reference
Pattern that we have
calculate with MatLab.

33. 33
Calculate the camera
reference pattern
Calculate the noise of an
image
Apply a denoise filter like before
Subtract the denoised image with the original one
We obtain the noise of the image
While the PRNU is unique to the sensor, the other artifacts like color
interpolation and JPEG compression are shared among cameras of the same
model or sensor design.
Subtracting the image denoised with the original image allow us to suppress
all other artifcats and avoid false identification rate.

34. Calculate the camera
reference pattern
Calculate the noise of an
image
Find out the correlation
between camera reference
pattern and image noise
34
Finding out the
correlation between
camera reference
pattern and the noise of
our image we can link
the device which takes
that photo with the
photo itself.
We obtain a Threshold of acceptance and a false identification rate

35. 35
The factor K is thus a very useful forensic quantity, responsible for a unique
sensor fingerprint with the following important properties:
• Dimensionality: the fingerprint is stochastic in nature and has a large
information content, which makes it unique to each sensor.
• Universality: all imaging sensors exhibit PRNU.
• Generality: the fingerprint is present in every picture independently
of the camera optics, camera settings, or scene content, with the
exception of completely dark images.

36. 36
• Stability: It is stable in time and under a wide range of
environmental conditions (temperature, humidity, etc.).
• Robustness: it survives lossy compression, filtering, gamma
correction, and many other typical processing procedures.

37. 37
• Destroying the PRNU
• Removing the PRNU
• Forging the PRNU
• Other methods
Remove PRNU Methods:

38. Destroying the PRNU (1)
Adding random noise to the picture
• Changing Least Significant Bit (LSB)
1 0 1 0 0 1 0 1
1 0 1 0 0 1 0 1
1 0 1 0 0 1 0 1
38

39. Destroying the PRNU (2)
Blurring
At least using a 4x4 matrix
Sharpening
To get the details back into the picture
39

40. Example of destroying (1)
Original Blur4x4 + Sharp3x3
40

41. Example of destroying (2)
Original Blur5x5 + Sharp5x5
41

42. 42
Removing the PRNU (1)
A dark frame (B) is a single picture which was taken with the shutter closed
or the lens cap still on. This dark frame consists of the fixed-pattern noise
(FPN); noise from the sensor itself, like dead or hot pixels.
Subtracting this dark frame from the original input picture (I) will reduce the
inherent noise in the picture, and thus, improving the picture's quality.

43. 43
Removing the PRNU (2)
FlatField (FF) correction is done by taken multiple pictures (N) that are taken
of a, mostly white, at surface. After averaging all the pixels with their
neighbours (depicted in the second equation), which essentially blurs the
FlatField, the pixels can be compared to all their neighbours to see their
individual deviation. These small deviations are then used to correct the
entire picture, getting rid of further sensor defects present in the picture.
Finding the right (n) can be tricky as it partially depends on the strength of
the PRNU.

44. Example of removing
Original PRNU removed
44

45. 45
Forging the PRNU
First the original PRNU pattern is removed just like in the removing formula,
secondly a new pattern of the target camera is added as the second part of the
forge formula.

46. Example of Forging
PRNU Forged
46
Original

47. 47
Other Methods
Rotating a picture by a few degrees would also rotate the pattern
and thus make matching not possible.
Flipping also flips the pattern so this also works, however, this
might be easily noticeable if there is any text or well-known scenery in
the picture.
Scaling the picture, due to the new and dierent size of the
picture it cannot be compared to any original picture from possibly the
same camera.

48. 48

49. 49

50. 50
Matching original pictures with multiple cameras.
Camera four is the original.

51. 51
Matching pictures with PRNU noise pattern destroyed,
using a blur 4x4 and a sharp 3x3 matrix.
Camera four is the original.

52. 52
Matching pictures with PRNU noise pattern destroyed,
using a blur 5x5 and a sharp 3x3 matrix.
Camera four is the original.

53. 53
Matching pictures with PRNU noise pattern destroyed,
using a blur 5x5 and a sharp 5x5 matrix.
Camera four is the original.

54. 54
Matching pictures with PRNU noise pattern removed,
using 10 at-elds.
Camera four is the original.

55. 55
Matching pictures with PRNU noise pattern removed,
using 30 at-elds.
Camera four is the original.

56. 56
removed the PRNU noise pattern with 30 ateld pictures from the
original camera and then forge the PRNU noise pattern with both a
dierent camera and the original camera.

57. 57
Forging the PRNU noise pattern, using 30 flatfield to remove and to forge.

58. 58
The answer is a genuine: yes.
Consequences
Every method that increases anonymity can, and mostly will, be abused for
illegal or controversial activities.
It becomes even more interesting when we look at forging a pattern where we
impose that a picture is taken with a different digital camera.
The validity of evidence in that case decreases even more.

59. 59
Jessica Fridrich
a professor of electrical and computer
engineering at Binghamton University
(SUNY).
She received her Ph.D. in systems science
from Binghamton University in 1995 and
her M.S. in applied mathematics from
Czech Technical University in Prague in
1987.
Her main interests are in steganography, steganalysis, digital
watermarking and digital image forensics.
Since 1995, she received 18 research grants; most for projects on
data embedding and steganalysis that lead to more than 80 papers
and seven U.S. patents. She is a Member of the IEEE and ACM.
• Digital Image Forensics

60. 60
CCD and CMOS
• http://en.wikipedia.org/
• http://www.castfvg.it/notiziar/1998/ccd.htm
• http://www.microscopyu.com/articles/digitalimaging/ccdintro.html
• http://www.sensorcleaning.com/whatisasensor.php
Digital Image Forensics and PRNU
• Source Digital Camcorder Identification Using Sensor Photo Response Non-Uniformity
• Imaging Sensor Noise as Digital X-Ray for Revealing Forgeries
• Digital Camera Identification from Sensor
• Determining Image Origin and Integrity Using Sensor Noise
• http://thelede.blogs.nytimes.com/2008/07/10/in-an-iranian-image-a-missile-too-many/

61. 61
Thanks for your attention