T.Q. Pham, S.W. Perry and P.A. Fletcher, DICTA 2009. Digital Image Computing: Techniques and Applications, Melbourne, Australia, 2009.

1. Paper fingerprinting using alpha-masked image matching
Tuan Q. Pham† Stuart W. Perry Peter A. Fletcher
Canon Information Systems Research Australia (CiSRA)
1 Thomas Holt drive, North Ryde, NSW 2113, Australia.
†
corresponding author: tuan.pham@cisra.canon.com.au
Abstract arrangement of fibres and no two regions on the same sheet
of paper have the same arrangement. The randomness of this
In this paper, we examine the problem of authenticating arrangement can be exploited to create a unique signature for
paper media using the unique fibre structure of each piece the sheet of paper. A recent article described the creation of
of paper (the so-called ”paper fingerprint”). In particular, unique signatures for a variety of common objects such as
we look at methods to authenticate paper media when text paper, coated cardboard packaging and matt-finish plastic
has been printed over the authentication zone. We show how cards using the diffuse reflection from a laser focussed
alpha-masked correlation [8] can be applied to this problem onto a small region of the object [3]. A unique signature
and develop a modification to alpha-masked correlation could be obtained even when the object had been soaked
that is more closely matched to the requirements of this in water and then dried, baked in an oven, crumpled and
problem and produces an improvement in performance. We creased, or written over with a ballpoint pen or thick marker.
also investigate two methods of pixel inpainting to remove An object could be authenticated by cross-correlating the
printed text or marks from the authentication zone and allow signature obtained from the object with those in a database
ordinary correlation to be performed. We show that these of signatures of authentic objects.
methods can perform as well as alpha-masked correlation. The basis of this idea has been around for a while. It can
Finally two methods of improving the robustness to forgery be traced back as least as far as US patents filed in 1981
are investigated. [9] and 1984 [10]. In these patents the inventor discloses
the idea of using information about inherent irreproducible
1. Introduction randomness within an object imaged by an optical system to
form a unique signature for the object. More recently it has
When a reasonable facsimile of an object can be created been noted that a coherent light source is not required for
at a cost less than the value of the object, the object a unique signature to be computed for paper [16]. Indeed,
may be become a target for forgery. The concept of an the earlier work [9], [10] does not assume a coherent light
object’s ”value” can be defined in many different ways. source. By simply imaging the paper surface illuminated
Paper currency is an example of an object with a cost to by incoherent light at high resolution, sufficient information
manufacture that is often small compared to the value society about the random structure of the paper surface can be
places on the object. Even plain paper documents whose obtained to create a highly unique signature for the paper
cost is almost negligible to produce can have a very large [16]. The authors have found that the requirements for
value due to the importance of the information printed on the imaging a piece of common office paper to obtain a highly
document. In recent times the availability of cheap printing unique signature are easily met by many common consumer
technology has further reduced the cost of manufacture document scanners. In addition, experiments we have per-
for paper objects, increasing the temptation of forgery and formed indicate that the signatures obtained are robust to
improving the quality of forgeries. damage to the paper such as light staining, wrinkling, and
There have been many different strategies employed to wetting followed by drying. The use of a desktop scanner
protect objects from forgery. One strategy is to make the for this application provides a number of benefits including
equipment required for manufacture of the authentic object removing the requirement for specialised equipment to read
so expensive that forgery is discouraged. For example, the the signatures. In this work the signature we use is simply a
addition of holograms, special paper and/or inks or finely greyscale 8-bit 256 by 256 pixel image of the paper surface
printed details. captured by the scanner at 600dpi. In [3] and [16], cross-
Another approach that has gained interest recently is correlation was used to match an obtained signature with a
security based on the inherent randomness present in many previously stored signature, and this approach can be used to
objects. An ordinary piece of office paper when viewed at match the above image signatures from a desktop scanner.
the micrometer scale is a highly random arrangement of However, simply cross-correlating an image of the surface
fibres and filler material. Every sheet of paper has a different of the paper does not provide a signature authentication

2. system that is robust to marks, writing or printing across 2.1. Fitch’s alpha-masked correlation
the signature zone. Such marks can obscure the signature of
interest in the cross-correlation operation and increase the Suppose the two ROIs from images f1 and f2 are related
likelihood of an authentic object being declared unauthentic. by a translation (x0 , y0 ) and unknown noise n:
There are many applications of this technology where it
would be useful to allow the user to print additional in- f2 (x − x0 , y − y0 ) = f1 (x, y) + n(x, y),
formation onto the document after the signature is collected Fitch et al. [8] define a weighted error function:
and still have the object be recognised as authentic. In [2], a
method was described where the image of a signature zone E(x0 , y0 ) = (n(x, y))2 α1 (x, y)α2 (x − x0 , y − y0 )
was filtered before comparing the signature with that in a x,y
2
database using cross-correlation. The filtering was intended = f1 (x, y) − f2 (x − x0 , y − y0 )
to remove the effect of any marks within the signature x,y
zone, such as damage or printed text. This method has a
α1 (x, y)α2 (x − x0 , y − y0 ) (1)
disadvantage that the filtered pixels are still included in the
cross-correlation calculations and hence the filtered pixels which can be written as a summation of three correlations:
degrade the accuracy of the signature matching. In fact the 2 2
method presented in [2] would only be effective when the E(x0 , y0 ) = α1 f2 ⊗ α2 − 2α1 f1 ⊗ α2 f2 + α1 ⊗ α2 f2
area of damage or printed text was small compared to the where (f ⊗ g)(x0 , y0 ) = x,y f (x, y)g(x − x0 , y − y0 )
size of the signature zone. Unfortunately, [2] contains almost denotes the correlation between f and g. Once the error
no implementation information about the method presented function is calculated, the translation between the two im-
and no actual numerical results. For this reason, a direct ages can be determined by finding the coordinates (x0 , y0 )
comparison between this method and the presented approach at which E(x0 , y0 ) is minimum.
is not possible. To further simplify the representation of the error func-
m
In this paper we present signature matching algorithms tion, let us make the following definition: Cmn = α1 f1 ⊗
n
that are robust to marks, handwriting or text printed over the α2 f2 . With this notation, the error function can be expressed
signature region either before the initial reference signature as:
is collected, or subsequently. We do this by using several E = C20 − 2C11 + C02
modified signature matching algorithms which are based on
cross-correlation, but are tailored to exclude pixels deter- A problem with the above error function E is that the sum
mined to correspond to printed text from the calculation. in (1) contains a different number of terms for each (x0 , y0 ),
depending on how often α1 (x, y) and α2 (x − x0 , y − y0 ) are
In section 2 we review the alpha-masked correlation of
zero at the same time. Thus, [8] normalizes the error function
Fitch, et al. [8], and present a modification to the alpha-
by the area of overlap between α1 and α2 :
masked correlation algorithm to better account for back-
ground illumination variation. Experiments are performed 1
Eα = (C20 − 2C11 + C02 ) (2)
to show how alpha-masked correlation can be applied to the C00
problem of paper authentication. In section 3, alternative
algorithms to solve this problem using pixel inpainting are 2.2. Correlation vs matching: peaks vs troughs
presented and experiments performed. Section 4 presents
two methods to improve the robustness to forgery of the Image matching as described above is presented as a
proposed methods. Section 5 presents the conclusions. least squares estimation problem, in which the best match
between two images is regarded as the position at which
the squared, normalized differences between two images is
2. Correlation-based alpha-masked image at a minimum, with a perfect match achieving a value of
matching zero. This is at variance with the view of image matching
by correlation, in which the best match between two images
is regarded as the position with the maximum correlation
Alpha-masked image matching is a technique to match value. In the case of image matching, finding the best match
sub-regions of two images. The Regions Of Interest (ROI), is equivalent to finding the minimum value in an image. In
defined by alpha masks α1 and α2 , allow registration of the case of correlation, finding the best match is equivalent
non-rectangular objects. The alpha mask 0 < α(x, y) < 1 to finding the maximum value in an image.
can be seen as a weighting map, where a weight of zero If one attempts to estimate the statistical significance of
at (x, y) means that the pixel at (x, y) should be removed a match, then a maximum correlation value is easier to
from consideration. When pixels have differing α-values, it interpret than a least squares error, as the magnitude of
means that some are considered more important than others. the correlation peak can be compared against an assumed

3. (a) paper 1 before printing (b) paper 2 before printing
distribution of non-matching correlation values, allowing
statistical inferences to be drawn from the data. For example, 3.33 (3.46)
a normal distribution could be used, as the correlation image
contains both positive and negative values, unlike the match
image, which contains only positive values.
A simplistic method is used to convert the least-squares 3.89
error value returned from image matching to something (2.91)
similar to a magnitude value from a correlation process. 23.16 (3.06) 25.64 (3.31)
The least-squares image is simply negated, normalized to
remove any DC offset, and divided by the root-mean-square
4.08
value of the image. This results in an matching image with (3.36)
a standard deviation of 1, and as long as the width of
the correlation peak is small compared to the size of the
correlation image, the peak strength can be analyzed in terms
of the number of standard deviations away from the mean
value. The correlation image around the peak may also be 2.02 (6.22)
processed using quadratic interpolation to estimate a sub-
(c) paper 1 after printing (d) paper 2 after printing
pixel accurate peak position and peak magnitude.
Figure 1. Paper matching using alpha-masked correla-
2.3. Normalized correlation of background sub- tion versus (cross correlation).
tracted images
Image matching using the alpha-masked correlation for- peak at the matching offset rather than a trough like Eα .
mula (2) is not robust to different gains and offsets between As a result, a strong peak in EN signifies a good match
the images being matched due to the mean squared differ- between the two input images.
ence in (1). To make the algorithm robust to different gains
and offsets, the input images should be normalized to have 2.4. Paper matching using alpha-masked correla-
zero mean and unit standard deviation over the masked area tion
before correlation. This is easily done for images of paper
because blank paper usually has a flat intensity distribution. To demonstrate that a paper can be uniquely identified
The offset is removed first by subtracting from f1 and f2 even after substantial marking, we perform a simple paper
their background intensities (average intensity of the non- matching experiment. The experiment involves two different
printed area). The gain is then normalized by the standard pieces of paper scanned before and after text printing. The
deviation over the same background area. 600dpi paper scans are subjected to alpha-masked corre-
For more complicated images that do not have a flat lation and cross-correlation. We show that while different
¯
background, a background image f can be estimated using sheets of paper with the same printed text could be mis-
normalized convolution [15]: classified as matching by standard correlation, alpha-masked
¯ (αf ) ∗ k
f= (3)
correlation correctly classifies them as non-matching.
α∗k Figure 1a and 1b show two 256 × 256 scans of two blank
where k is a blur kernel with large support such as Gaussian sheets of paper. The printed versions of these sheets of
blur of σ = 5 pixels and ∗ is the convolution operator. paper are in Figure 1c and 1d. The intensities of these 8-bit
After background subtraction, the terms C20 /C00 and images are linear stretched between [200 255] so that the
C02 /C00 in (2) are simply variance estimates of the pa- paper texture is more visible in Figure 1. There is a small
per background. Although these estimates do vary due to translation between images 1a and 1c and 1b and 1d due
different areas of overlap C00 , the variation is small if to the feeding mechanism of the scanner. Also due to the
the overlapping area of the two masks does not change same positioning error, the text in Figure 1c and 1d appear
substantially over all x0 , y0 offsets. As a result, C20 /C00 slightly shifted, even though the same text was printed. The
and C02 /C00 can safely be removed from the error function alpha masks are constructed by thresholding the images
(2), leaving a single term in the normalized correlation error at intensity 160 followed by two iterations of 8-connected
function: binary erosion.
α1 f1 ⊗ α2 f2 C11 We performed alpha-masked correlation and cross-
EN = = . (4)
α1 ⊗ α2 C00 correlation between every image pair and the correlation
Compared to Eα in (2), EN requires only half the number peak strengths defined in section 2.2 are displayed in be-
of correlations to compute. EN also produces a correlation tween the paper images in Figure 1 (the cross-correlation

4. peak strengths are displayed within brackets). Zoomed-in Laplacian image, is stored as the base level of the pyramid,
versions of the correlation images are also displayed below while the low-pass image l0 is losslessly downsampled
the peak strength numbers. The contrast of these correlation (requires band-limitedness). The downsampled image f1 is
images has been enhanced to visualize the correlation peaks again subjected to low- and high-pass decomposition. This
more effectively. process is repeated recursively until a desired number of
The most striking difference between alpha-masked cor- pyramid levels are obtained (3 levels as shown in Figure 2).
relation and cross-correlation occurs between image pairs Because the Laplacian pyramid is generated using lossless
from the same paper before and after printing. While alpha- operations, the original image can be reconstructed perfectly
masked correlation produces high peaks around 25 for from the Laplacian pyramid as shown in Figure 2b. Although
matching papers, cross-correlation peaks remain low around the Laplacian pyramid requires 33% more storage than a sin-
3. The correlation images below these numbers (in bold in gle image, it allows separate manipulation of different band-
Figure 1) also confirm very sharp peaks when using alpha- pass images. This is especially useful in texture generation
masked correlation. The cross-correlation peaks, on the other or infilling, where the texture is different at different scales.
hand, are barely visible. Cross-correlation peak values of
3.06 and 3.31 between the matching pairs are no better than 3.2. Alpha-masked Laplacian pyramid
the peak values of non-matching pairs.
Another interesting result is the correlation between the The concept of Laplacian pyramid can be extended to
two different pieces of paper after printing. While alpha- images with uncertain data. The data uncertainty is specified
masked correlation correctly gives a small correlation of by an alpha mask 0 ≤ α ≤ 1, where zero corresponds to
2.02, cross-correlation produces a visible peak at an appre- invalid data and one for valid data. Rather than the low-
ciable strength of 6.22 (in italic in Figure 1). This visible pass operator BLUR in Figure 2a, Normalised Convolution
peak is a result of correlation of the printed text, not the (NC) [15] is used to smooth out an alpha-masked image by
texture of the papers. Cross-correlation is therefore not a kernel k:
robust to printing. (α0 f0 )∗k
l0 = h0 = α0 (f0 −l0 ) α1 = {α0 ∗ k} ↓ (5)
α0 ∗ k
3. Alpha-masked image matching using in- where f0 and α0 are the input image and its alpha-mask,
painting l0 and h0 are the low- and high-pass decompositions, α1 is
a generated alpha-mask for the downsampled image f1 , ∗
Image inpainting is a technique to fill missing pixels of and ↓ denote a convolution and a downsampling operator
an image with plausible intensities. The missing pixels may respectively.
be lost during transmission or they may cover unwanted As can be seen in Figure 3, the normalised convolution
objects to be erased from the image. In a typical solution operation imports valid intensities into the masked areas. It
[1], the image is decomposed into texture and structure does so in a similar fashion to image morphology where the
components. Texture synthesis is then used to fill in the masked regions are successively eroded and filled with local
texture component, whereas anisotropic diffusion is used mean intensities. Note that in (5) the high-pass image hi
to transport intensities along iso-contours into the missing has to be multiplied with the mask αi of the same pyramid
regions of the structure component. level to invalidate the influence of the masked pixels. The
This paper does not introduce a new inpainting solution. Laplacian pyramid generated this way is compatible with a
Rather, a smooth infilling of masked images followed by normal Laplacian pyramid. The difference happens in the
cross-correlation is presented as an alternative to alpha- reconstructed image, where invalid pixels are automatically
masked correlation. Laplacian-based smooth inpainting [5] filled with mean intensities from surrounding valid pixels.
minimizes sudden intensity changes at mask boundaries.
3.2.1. Implementation details. To avoid the need for a
Spectrum distortions are therefore minimized. The inpainted
lossless subsample operation during pyramid construction,
images can then be matched reasonably well even when the
the high-pass image hi is computed slightly differently from
masking area is large.
(5):
hi = αi (fi − {fi+1 } ↑) (6)
3.1. Laplacian pyramid
where ↑ denotes an upsampling operator. In this paper,
A Laplacian pyramid [4] is an over-complete representa- we use a upsampling/downsampling factor of two. The
tion of an image using a set of band-pass images at succes- blur kernel is a separable Burt-Adelson [4] 5-tap filter
sively reduced dimensions. The decomposition of an input [0.05 0.25 0.4 0.25 0.05] (which approximates a Gaussian
image into its Laplacian pyramid is illustrated in Figure 2a. filter of σ = 1). The Laplacian images are successively
The input image f0 is decomposed into a high-pass image h0 generated until the alpha mask αi no longer contains any
and a low-pass image l0 . The high-pass image h0 , a.k.a. the zero pixels.

5. (a) Laplacian pyramid construction from an image (b) Image reconstruction from a Laplacian pyramid
Figure 2. Laplacian pyramid as an over-complete representation of an image.
BLUR
SUBSAMPLE
α 0 α 1
SUBSAMPLE
NC
NC SUBSAMPLE
f2
f1 l1
_
+
(a) Lena with mask (b) Inpainted result of (a)
f0 l0 _
+
h1
Alpha-masked
Laplacian pyramid
h0
Figure 3. α-masked Laplacian pyramid construction.
(c) Barbara with mask (d) Inpainted result of (c)
Figure 4. Smooth inpainting of natural images.
3.2.2. Smooth inpainting results. The results of smooth
image inpainting for two natural images can be seen in
Figure 4. Compared to the images with average intensity
correlation sometimes produces better peak detection than
infilling on the left, the inpainted images on the right look
alpha-masked correlation. Infilling using average local in-
much more pleasing. The text masks are almost invisible.
tensity performs equally well as normalized correlation with
Because image inpainting transports neighboring intensities
background subtraction. This is understandable since both
into the masked areas, low-frequency details are recon-
infilling and background subtraction essentially fills the
structed very well. The missing of information is only
masks with local average intensities.
detectable at textured areas like the fur of Lena’s hat or
the stripe pattern of Barbara’s clothes.
3.3.1. Inpainted paper matching. Two inpainting results
for a scan of a printed paper using mean intensity infill-
3.3. Alpha-masked image matching using inpaint- ing and smooth inpainting are shown in Figure 5. The
ing 256×256 central region of Figure 5a is matched against the
blank paper signatures in Figure 1a-b. Due to optical dot
The alpha-masked images can be inpainted before match- gain [11], the intensities around printed texts are slightly
ing to avoid extra correlations in the alpha-masked and darker than the background. Smooth inpainting in Figure
normalized correlation. By matching two scans of the same 5b transports these dark intensities into the masked area,
paper before and after text printing, we show that inpainted whereas mean infilling in Figure 5a uses the correct average

6. 50
alpha−masked correlation [8]
45 normalized correlation (section 2.3)
mean−filled correlation (section 3.3)
40
fill = 0.77 inpainting correlation (section 3.3)
35 correlation of non−matching pairs
match strength
30
fill = 0.58 25
20
(a) mean intensity infilling (b) smooth Laplacian inpainting
15
fill = 0.45
Figure 5. Paper 1 with printed text after inpainting.
…
10
5
alpha-masked normalized mean-filled inpainting 0
match 23.16 22.32 22.31 14.17 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
non-match 4.08 2.92 2.94 2.68 fill = 0.05 mask fill factor
ratio 5.67 7.63 7.59 5.28 (a)mask (b) match strength versus mask fill factor
Table 1. Matching a printed paper (Fig. 1c) with two
Figure 6. Correlation results under fill-factor stress test.
fingerprints (match = Fig. 1a, non-match = Fig. 1b).
containing Japanese test was scanned twice on an EPSON
10000XL scanner. Japanese text was chosen for this experi-
background intensity. This gives rise to a lower fingerprint ment because compared to English text, the different spatial
match strength for the smooth inpainted image compared to frequency characteristics of the three Japanese character sets
that of the mean-filled image (first row of Table 1). However, (kanji, hiragana, and katakana) would be likely to produce
both infilling methods produce substantially higher match a wider spread of mask fill factors on which to test the
strengths than the correlation peak strengths of non-matching proposed method. To mitigate the effects of fixed pattern
papers. Infilling is therefore an effective yet inexpensive scanner noise on the result, each scan was performed at a
alpha-masked image matching method. different position on the scanner platen. Both scans were
collected as 8bit greyscale images with a resolution of
3.3.2. Robustness against mask fill factor. One big advan- 600dpi. The scans were roughly aligned by eye; however
tage of alpha-masked image matching over other signature no attempt at electronic correction of residual rotational
verification techniques such as [2] is the ability to handle a misalignment between the images was made. Each scan was
large area of alteration from the original signature. In Figure divided into 240 256×256 pixel image patches and each
6, we investigate the performance of alpha-masked image patch in the first scan was matched against the corresponding
matching at different levels of the fill factor of the mask. patch in the second scan using alpha-mask correlation [8].
The masks in Figure 6a are successively eroded to reduce The match strength and mask fill factor for each patch is
their fill factors (defined as ratio of the valid area in white shown in Figure 7 below. It should be noted that each point
over the area of the whole mask). This mask is used with on Figure 7 represents a different part of the document, and
the printed image in Figure 1c to match against two blank two iterations of 8-connected binary erosion of the printed
signatures in Figure 1a-b. The resulting match strengths are content was used to generate different fill-factors.
plotted against fill factors in Figure 6b as continuous lines This experiment was performed using a C implementation
for matching pair and dashed lines for non-matching pair. of the normalised alpha-masked correlation method on a
It is interesting to see that alpha-masked image matching computer running Microsoft Windows XP, Service Pack 3.
still produces a high matching strength after more than 80% The computer had two Intel Xeon 5060 Dual Core CPUs
of the original image has been masked out (continuous lines running at 3.20GHz, with 3.25 GBytes of RAM. The 240
in Figure 6). Normalized correlation and mean-filled correla- match operations took a total of 161.48 seconds, giving
tion performs equally well, both are much better than alpha- an average time of 0.67 seconds per match operation. No
masked correlation and inpainted correlation. The perfor- special optimisation of the code to take advantage of the
mance of alpha-masked correlation, while being preferable multiple CPUs or multiple cores was performed.
for large fill factors (as shown in Table 1), degrades quickly
as the fill factor decreases. All four methods in Figure 4. Improving the robustness of paper finger-
6b also produces consistently low correlation between non- printing
matching paper pairs (dashed lines in Figure 6).
An additional experiment was performed to test the Being a pure image-based method, the paper verification
proposed method’s robustness to mask fill factor. A page technique using correlation is susceptible to attack if a forger

7. (a) paper scanned at 0º (b) paper scanned at 180º
25
alpha−masked correlation [8]
20
fill = 0.94
match strength
15
fill = 0.86
+ _
10
+ +
fill = 0.73
…
5
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
mask fill factor (e) (f)
fill = 0.16
(a)ROI (b) match strength versus mask fill factor (c) diffuse reflectance (d) specular reflectance
Figure 7. Paper matching on a Japanese document. Figure 8. Diffuse and specular components of paper.
manages to print a desired paper signature onto a fake propose to match the two reflectance components sepa-
medium. While we believe that it is difficult to reproduce rately to improve robustness against forgery. The specular
paper textures at different resolutions, we want to improve reflectance image could also be turned into a surface depth
the robustness of our paper fingerprinting technique. This map using shape from shading [13]. While it is possible
section presents two such methods to address the robustness to forge a 3D surface using molds or very fine topology
to forgery problem. creation tools, the forgery needs to have a matching diffuse
reflectance at precisely the same alignment with the 3D
4.1. Multiple orientation paper fingerprinting topology. This makes it more difficult for a document to be
forged at a cost of one extra scan of the candidate document.
The reflection of light from most turbid materials consists
of two major components: the surface and the subsurface 4.2. Double-sided paper fingerprinting
reflectance. Surface reflection, a.k.a. specular reflection, is
the mirror-like reflection of light at the paper surface. Due to Another way to improve the robustness of paper finger-
the roughness of paper, this surface reflection is more diffuse printing is to verify its signatures on both sides. During a
than the specular reflection from a smooth surface (see registering phase, the signatures for both sides of a document
Figure 8f). Subsurface reflection, a.k.a. diffuse reflection, are collected. These signatures are stored in a database to-
is the reflection of light from within the paper after light gether with the displacement between the signatures. When
penetrates the surface and scatters within the substrate (see an object is presented for authentication, the signatures on
Figure 8e). both sides of the document are scanned. These candidate
The diffuse and specular reflectance components can be signatures must match the signatures in the database and
separated from an image of a paper using photometric stere- the alignment of the signatures in the database must also
ography [14]. As illustrated in Figure 8a-b, two scans from be the same before a match is declared. This uses the fact
opposite orientations (i.e. 180◦ difference in illumination that the rear side fingerprint can be detected when the front
angle) obtained by the same scanner setting are required. side of the paper is scanned and visa-versa (see Figure 9).
Because shadows in one image correspond to specular reflec- This allows accurate alignment determination between the
tion in the other image, the specular reflectance is roughly front and back signatures. With the double-side approach,
cancelled when the two aligned images are added together. a forger is required to match signatures on both sides of
The diffuse reflectance is derived in Figure 8c as the sum of the document and match the relative positions between
Figure 8a and 8b. The specular reflectance in Figure 8d is the signatures on both sides. This greatly complicates the
the difference of Figure 8a and 8b. As expected, the specular forgery process while still using only an inexpensive scanner
image looks shiny, whereas the diffuse image looks dull and during verification.
contains pores from the filler material within the paper. A
very similar diffuse reflectance image is obtained from two 5. Conclusion
aligned scans at 90◦ and 270◦ orientation.
Different from [6], which bundles the diffuse and specular In this paper we have looked at the problem of matching
reflectance into a feature vector for matching purposes, we paper media in the presence of printed text, or other mark-

8. published and unpublished patent applications [7], [12].
References
[1] M. Bertalmio, L. Vese, G. Sapiro, and S. Osher. Simultaneous
structure and texture image inpainting. IEEE Trans. on IP,
(a) front side at 0◦ (b) back side at 0◦ (c) correlation of a&b 12(8):882-889, 2003.
Figure 9. Matching different sides of paper at 1200dpi. [2] Y. Boutant, D. Labelle, and H. Seux. Method for recognition
and tracking of fibrous media and applications of such a
method, particularly in the computer field. US patent applica-
tion 2005/0257064, filed in Apr. 2005.
ings, on either one or both of the paper media to be matched.
We examined Fitch’s formulation of alpha-masked correla- [3] J. D. R. Buchanan, R. P. Cowburn, A.-V. Jausovec, D. Petit,
tion [8], and adapted this method to the problem of interest. P. Seem, G. Xiong, D. Atkinson, K. Fenton, D. A. Allwood,
and M. T. Bryan. Forgery: ’fingerprinting’ documents and
In particular, Fitch’s method is modified to better handle packaging. Nature, 436:475, 2005.
the problem non-uniform background illumination and is re-
normalised. An experiment was performed that showed that [4] P. J. Burt and E. H. Adelson. The Laplacian pyramid as
alpha-mask correlation is effective at matching paper with a compact image code. IEEE Trans. on Communications,
printed text present, whilst ignoring correlations amongst the 32(4):532-540, 1983.
printed text that might otherwise create false matches. Our [5] P. J. Burt and E. H. Adelson. Method for compensating for
modified version of alpha-mask correlation was shown to void-defects in images. US patent 4698843, RCA Corporation,
out-perform the basic alpha-mask correlation in some cases. Oct. 1987.
As alternatives to the alpha-masked correlation approach, we
investigate two methods of pixel inpainting; mean intensity [6] W. Clarkson, T. Weyrich, A. Finkelstein, N. Heninger, J. A.
Halderman, and E.W. Felten. Fingerprinting blank paper using
in-filling of pixels identified as text and an approach based commodity scanners. In Proc. IEEE Symposium on Security
on the Laplacian pyramid decomposition of the paper image. and Privacy, 2009.
After in-painting text or markings on the paper image,
ordinary cross-correlation may then be used to compute a [7] S. E. Ecob, S. W. Perry, and S. A. Rudkin. Authenticating
match strength for the paper media. We performed a number partially transparent medium. Australian patent application
AU2007254655, Canon Kabushiki Kaisha, filed in Dec. 2007.
of experiments and found that Laplacian pyramid inpainted
images followed by ordinary cross-correlation perform no [8] A. J. Fitch, A. Kadyrov, W. J. Christmas, and J. Kittler. Fast
worse than alpha-masked correlation in this problem. In robust correlation. IEEE Trans. on IP, 14(8):1063-1073, 2005.
fact, the Laplacian pyramid inpainted augmented method
performed better in some cases. The mean-filled pixel in- [9] R. N. Goldman. Non-counterfeitable document system. US
patent 4423415, Light Signatures Inc., Jun. 1981.
painting method performed even better than the Laplacian
method, equalling the performance of our modified version [10] R. N. Goldman. Verification system for document substance
of alpha-masked correlation. and content. US patent 4568936, Light Signatures, Dec. 1984.
In addition we investigated methods to further improve the
security of paper authentication. The first method involved [11] S. Gustavson. Dot gain in color halftones. Ph.D. disertation
no. 492, Link¨ ping University, Sweden, 1997.
o
scanning the paper from two different directions and de-
composing the scans into a specular and diffuse component. [12] S. J. Hardy, K. G. Larkin, S. Farrar, P. A. Fletcher, and T. Q.
The specular component is mostly a function of the 3D Pham. Alpha-masked image matching. US patent application
surface structure of the paper, while the diffuse component 2009/0003601, Canon Kabushiki Kaisha, filed in Jun. 2008.
is mostly a function of the internal structure of the paper.
[13] B. K. P. Horn and M. J. Brooks (eds). Shape from shading.
By separating these components, two pieces of paper can MIT press, 1989.
be matched on the internal and surface structure separately,
improving the robustness to forgery. The second method [14] P.-A. Johansson and P. Hansson. Method of determining an
made use of the fact that for many ordinary office papers, illuminated surface. US patent 6757065, STFI, 2004.
the fingerprint collected on one side of the paper can be
[15] H. Knutsson and C.-F. Westin. Normalized and differential
detected in the image collected on the other side of the paper. convolution: methods for interpolation and filtering of incom-
By collecting signatures from both sides of the paper, and plete and uncertain data. In Proc. of CVPR, 1993.
measuring the through paper displacement of the signatures,
a greater robustness to forgery can be achieved. [16] H. Sugino, K. Ito, and T. Shimizu. Document verification
system. US patent 6948657, Fuji Xerox Co. Ltd., Apr. 2003.
Aspects of this work are the subject of a number of