Image Processing

1. ATMOSPHERIC TURBULENCE DEGRADED IMAGE RESTORATION USING BACK
PROPAGATION NEURAL NETWORK
A
DISSERTATION
Presented
In partial fulfillment of the requirement for the award of degree of
MASTER OF TECHNOLOGY
IN
INFORMATION TECHNOLOGY
Submitted by
Azad Singh
(0901IT13MT03)
Under the supervision of
Rajeev Kumar Singh
(Assistant Professor)
Department of Computer Science & Engineering and Information Technology
Madhav Institute of Technology & Science, Gwalior (MP) - 474005
Session 2013-2015

2. Contents
• Introduction
 Image restoration
Atmospheric blur
Artificial neural network
• Literature Survey
• Problem Statement
• Proposed methodology
• Result analysis
• Conclusions and future work
• References
• Publications

3. Introduction
• Image restoration is a technique used to recover image from degraded image. Image
may be distorted due to blur and noise; blur can occur due to atmospheric
turbulence, motion of objects and camera mis-focus[1].
• Image Restoration is an area that also deals with improving the appearance of an
image.
• The degradation model of imagine system can be define as,
𝑔 𝑘 = 𝐼 𝑘⦻ℎ + 𝑛 𝑘 (1)
Where, 𝑔 𝑘 is the degraded image, in which ⦻ denotes the two- dimensional linear
convolution operation,𝐼 𝑘 is the original image and h is point spread function 𝑛 𝑘 is
the additive noise.

4. Continue…
Image Restoration Techniques:
i. Deterministic method:-In deterministic method [2] the prior knowledge about
degradation of image so that we employ the method easily and get restored image.
In deterministic method we restore image using filters.
(1) Wiener filter[3]
(2) Median filter[4]
(3) Inverse filter[5]
ii. Stochastic method: - The stochastic technique [6] uses the probability theory to
form most likely result, given data. In stochastic method does not known prior
knowledge of degradation source. So in this method firstly estimate the point spread
function of blurred image. It is also called as blind de-convolution.

5. Continue…
• Atmospheric blur is the distortion of image due to long time exposure, fog, wind
speed and due to randomly change in refractive index of air through which light
travels.
• The point spread function of atmospheric turbulence image can be described
reasonably well by a Gaussian function[7].
ℎ 𝑥, 𝑦, 𝜎 𝐺 = 𝐶 𝑒𝑥𝑝
𝑥2+𝑦2
2𝜎2
𝐺
(2)
Here, 𝜎 𝐺 determines the amount of spread of the blur, and the constant 𝐶 is to be
chosen so that above equation is satisfied.
• The estimation of point spread function of atmospheric turbulence image is very
challenging without knowing prior knowledge of clear image[8]. So, it is very
difficult to restore image using point spread function (PSF).

6. Example of images
• Here see images which is affected due to atmospheric blur given below:
Figure 1: (a) image is blur (b) image is blur due to
due to fog temperature i.e., mirage

7. Continue…
• The effects to atmospheric turbulence can be measured by calculating the
scintillation index[9]. This index is related to the mean and standard deviation of
the intensity distribution of image.
• The atmospheric turbulence can be categorized in two categories:
 In rigid bodies restoration, we take single frame image which is degraded
through turbulence and restoration techniques will be applied in degraded image,
which is known as single image deconvolution [10].
 In non-rigid bodies, we take degraded videos and apply image registration
technique to restore the object frame of video sequences, so uses the different
registration algorithm for image registration.

8. Artificial Neural Network
• Artificial Neural Network [ANN] is an information paradigm which is inspired by
our biological nervous system, like brain and processing system. In which artificial
neurons are interconnected to each other through weights [11].
• It can be trained through the supervised learning process. An ANN is composed of a
large number of highly interconnected processing elements neurons working in
union to solve specific problem.
• The simple neural network is a two layers or multilayer feed-forward and feedback
network with 𝑀 input and 𝑁 output units. Each input unit is connected to each of
the output units, and each connection is associated with weight, which are denoted
as 𝑤1, 𝑤2 … 𝑤 𝑁 respectively or the strength of the connection

9. Continue…
.
X 1 Z 1 Y 1
X i-n Z i-n Y i-n
X n Z n Y n
Figure 2: Architecture of feed-forward network
A trained network can be thought as an “expert” in a particular category of
information, if it has given to analyze. This is architecture of simple feed
forward network
Input layer Hidden layer Output layer

10. Literature Survey
• Dalong Li and Steven Simske [12] proposed kurtosis minimization method for
restoration. Atmospheric turbulence is caused by the random fluctuations of the
refraction index of the medium. It can lead to blurring in images acquired from a
long distance away. Since the degradation is often not completely known, the
problem is viewed as blind image deconvolution or blur identification.
• Their work has observed that blurring increases kurtosis and introduced a new blur
identification method based on kurtosis minimization[13].

11. Continue…
• Luxin Yan, Mingzhi Jin, Houzhang Fang, Hai Liu, and Tianxu Zhang [14]
proposed Atmospheric turbulence affects imaging systems by virtue of wave
propagation through a medium with a non-uniform index of refraction. It can lead
to blurring in images acquired from a long distance away. In this letter, it is
observed that blurring increases the second-order central moment (SOCM)of
images, and we introduce a new parametric blur identification method by
minimizing SOCM.
• The method applies to finite-support images, in which the scene consists of a finite-
extent object against a uniformly black, gray, or white background. The SOCM
method has been validated by direct comparisons with other methods on simulated
and real degraded images.
• Dalong Li, Russell M. Mersereau, and Steven Simske [15] uses PCA revealed a
connection between blind image deconvolution and principal components analysis
(PCA).

12. Continue…
• Although PCA is derived from blur models that do not contain additive noise, it can
be justified on both theoretical and experimental grounds that the PCA-based
restoration algorithm is actually robust to the presence of white noise.
• The algorithm is applied to the restoration of atmospheric turbulence-degraded
imagery and compared to wiener filter [14] on both real and simulated atmospheric
turbulence blurred images. It is shown that the PCA-based blind image
deconvolution runs faster and is more robust to noise. PCA is applied on only shift
invariant blur image and do not consider spatially varying blur image.
• PCA based image deconvolution algorithm is fast, robust to noise and provide
better result in multichannel image.

13. Continue…
• Jin-Bao Wang, Ning He, Lu-Lu Zhang, and Ke Lu [15] provide fog removal
techniques using physical model and dark channel used for atmospheric scattering
and optical reflecting imaging for estimation of atmospheric light in the dehazed
image to get proper result.
• It is mostly for non-sky patches, as at least one color channel has low intensity at
some pixel, it can be happen due to shadows of objects [16]. As we know
atmospheric images are usually full of shadows, so due to fog image is more
contrast than its image without fog.
• After apply the normalize operation we get restored image and improve the
visibility through CLAHE (Contrast Limited Adaptive Histogram Equalisation).
• This technique is useful for removing haze but provide better result in presence of
different noise and other blur present in foggy image.

14. Continue…
• Anantrasirirchai, Achi,G.Kingsbury and David R. Bull revealed complex
wavelet [17] for image fusion for restore the image. A new approach that
overcomes the problem of ineffective occur due to conventional fusion methods for
large distortion and also time consuming.
• The conventional fusion methods require a large number of frames to select lucky
regions. Image registration using the phase of a DTCWT coefficient is robust to
noise and temporal intensity variations thereby providing an efficient tool for
removing distorting ripples. After fusion, the effect of haze is reduced using locally-
adaptive histogram equalization.

15. Continue…
• Zhu and Milanfar [18] explored the technique to restore atmospheric turbulence
image and reduces the space and time-varying problem to shift invariant one. Thus
method divides each frame into overlapping patches. These patches can be viewed
in small region containing space-invariant blur. So blur can be deblurred using
multi-frame blind deconvolution algorithm. In which calculate the PSF (Point
Spread Function) for each patches for deconvolution.

16. Problem Statement
The atmospheric turbulence image restoration becomes more beneficial for
numerous vision applications. The following gaps concluded using literature
survey:-
1. In the past years, many researchers have provided techniques for restoration of fog
or turbulence independently but not both and the restored image was not perfectly
retrieved by traditional filter.
2. The existing methods have neglected to reduce the noise issue, which is presented
in the output images.
3. The problem of the uneven illuminate in image is also neglected.

17. Proposed methodology
• In our proposed methodology, we create new feed-forward network which have 20
hidden layers and one output layer. The activation function is associated with each
layer of hidden layer and output layer of network. At the hidden layer tangent
function and output layer uniform sigmoidal function is used as shown in equation 3
and 4. The training of network is done by back propagation. The levenberg-
Marquardt algorithm uses in back propagation and MSE for performance measure,
perform number of iteration and validation check to get better result.
• Unipolar sigmoid: 𝑓 𝑥 =
1
1+𝑒−𝜆𝑥 (3)
• Tan hyperbolic: 𝑓 𝑥 =
𝑒 𝜆𝑥−𝑒−𝜆𝑥
𝑒 𝜆𝑥+𝑒−𝜆𝑥 (4)

18. Algorithm
The following steps are performed given below:
1. Take input of original image 𝐼(𝑥, 𝑦).
2. Normalize size of image 𝐼(𝑥, 𝑦) into 𝑛 𝘹 𝑛 blocks.
3. Normalize image 𝐼(𝑥, 𝑦) by dividing with 255.
4. Take input blurred image 𝐵(𝑥, 𝑦) and perform Step 2 and Step 3.
5. Reshape input and blurred image from 2-D to 1-D.
6. Create a new feed-forward neural network 20 hidden layer and one output layer.
The tangent sigmoidal and uniform activation function is associated with each
hidden and output layer respectively.
7. Randomly initializes weight of network to train network by back propagation
algorithm and set parameter for stop the training.

19. Continue…
8. Apply blurred image to trained network to get restored image.
9. Reshape image data from 1-D to 2-D.
10. Finally get restored image.

20. Start
Original image
Atmospheric
degraded images
Training neural
network
Simulate network
Restored Image
End Figure 3: The flow chart diagram and gui of proposed method

21. Yes
No
Start
Pre-processing
of image
Convert image
into 1D
Initialize weights
of network
Input blurred
image
Compute error
Error <=
acceptabl
e error
Weight update
Stop training and
save weights
Figure 4 : Flow chart of BPNN
Back

22. Continue…
Now the training phase of network using back propagation algorithm discuss
in figure 5, in which parameter is set for training, we have set 300 iteration
and 90 validation check. In this algorithm, the weights are initializing with some
small random values
Figure 5: Training process of proposed methodology

23. Result analysis
• The analysis of result obtain by proposed methodology is done by comparing the
result of other existing methods. On the basis of two parameters, MSE and PSNR.
• In the restoration, the imperceptibility or the quality of the image is measured by
using Peak Signal to Noise Ratio (PSNR). PSNR is used to calculate the similarity
in the original image and the blurred image. The PSNR is calculated by using two
images one is the original image and other is the restored image. The higher PSNR
means better the quality of the restored image. The basic formula of PSNR is given
below:
𝑃𝑆𝑁𝑅 = 10𝑙𝑜𝑔10
2552
𝑀𝑆𝐸
(5)
Where, 𝑀𝑆𝐸 the mean square error between two images.

24. Continue…
Mean Square Error
• The MSE is used to measures average squared disparities between the original
image and the restored image.
𝑀𝑆𝐸 =
1
𝑀×𝑁 𝑖=0
𝑀−1
𝑗=0
𝑁−1
𝑥𝑖,𝑗 − 𝑥𝑖,𝑗
2
(6)
Where, 𝑥𝑖,𝑗 and 𝑥𝑖,𝑗are the gray scale values of original and restored image
𝑀 × 𝑁 is the size of image
• Firstly the MSE (Mean Square Error) will be calculated then the PSNR value is
calculated. Therefore, higher value of PSNR denotes less distortion.

25. Restored Images
Example of Satellite and moon image:
Figure 6 (a) original image (b) blurred satellite
image
(c) Restored image using proposed
methodology
Figure 7 (a) original image (b) blurred moon (c) Restored image using proposed
image methodology

26. Continue…
Example of foggy image:
Figure 8 (a) original image (b) blurred foggy (c) Restored image using proposed
image methodology

27. Comparison of result
S.No Name of Image PSNR Value
of blurred
image
PSNR value
of restored
image using
kurtosis[10]
PSNR value
of restored
image using
PCA[13]
PSNR value
of restored
image using
proposed
methodology
1 Foggy image 58.81 59.54 63.54 69.47
2 Moon image 69.20 71.44 72.43 74.72
3 Satellite image 76.0207 78.89 79.69 82.2344
This table shows result compare with the PSNR value of restored image using
our proposed methodology and other existing techniques

28. Continue…
Figure 9: The bar graph of comparison PSNR value of restored image.
This figure shows graph comparison of PSNR value between Kurtosis, PCA and
Proposed method. These methods are used to restore image and calculate PSNR
value.

29. Continue…
S.No Name of Image PSNR
Value of
blurred
image
PSNR value
of restored
image using
kurtosis[10]
PSNR value
of restored
image using
PCA[13]
PSNR value
of restored
image using
proposed
methodology
1 Foggy image 56.91 58.89 61.24 67.57
2 Moon image 60.47 66.84 70.32 73.0934
3 Satellite image 59.578 65.81 71.79 77.154
This table shows result compare with the PSNR value of restored noisy
image using our proposed methodology and other existing techniques

30. Continue…
This graph comparison of restored noisy image PSNR value between
Kurtosis, PCA and Proposed method. These methods are used to restore
image which containing impulse noise and calculate PSNR value.
Figure 10: The bar graph of comparison PSNR value of restored noisy image.

31. Conclusion
• Atmospheric restoration technique is depth field in which a large scope to do
research work and have to increase efficiency of existing algorithm. The proposed
method is restoring atmospheric degraded image using neural network. This method
is designed according our analysis on image in low contrast, defogging techniques
improvement and also robust in presence of noise.
• The results obtained by our methodology in terms of PSNR values are acceptable
and better than other existing techniques. This method is robust towards noise
present in atmospheric degraded image.

32. Future Scope
Some new directions of research in the field of image restoration are not yet
fully explored.
1. Median filter can be used in pre-processing of image.
2. First order features extracted from image and it can be used to assign
weights of network for training purpose.

33. References
[1] M.R. Banham and A. K. Katsaggelos, “Digital Image Restoration”, IEEE Signal Processing Magazine, vol. 14,no.2, pp. 24-41,
1997.
[2] R.E. Hufnagel and N.R. Stanley, “ Modulation transfer function associated with image through turbulence media,” J. opt. Soc.
Amer: A, Opt. Image Sci., vol. 54, no. 1, pp. 52-61, 1964.
[3] M. K. Hu, “Visual pattern recognition by moment invariants,” IRE Trans. Inf. Theory, vol. 8, no. 2, pp. 179–187, Feb. 1962.
[4] Wu H.R. and Chen T. “Adaptive Impulse Detection Using Center Weighted Median Filters” IEEE signal processing letters,
Vol. 8, No. 1, January 2001.
[5] D. Li, S. Simske , and R.M. Mersereau, “ Blind Image deconvolution using constrained variance maximization,” in proc.
Asilomar Conf. Signals, Syst., Comput., 2004, pp. 1762-1765.
[6] R. Gregory, “A technique of minimizing the effects of atmospheric disturbance on photographic telescopes,” IEEE transaction
Nature, vol. 203,1964.
[7] J.L. Davidson and F. Hummer, “Morphological neural networks: An introduction with applications,” IEEE Systems Signal
Processing, Vol., 12, No.2, pp. 177-210, 1993.
[8] David L Donoho, “De-noising by soft thresholding”, IEEE Transactions on Information Theory, Vol. 41, No. 3, pp. 613–627,
May 2002.
[9] O. Rockinger, “Image sequence fusion using a shift-invariant wavelet transforms”, In Proceedings of the IEEE International
Conference on Image Processing, volume 3, pages 288–291, 1997.
[10] P. Baldi, J. Hornik, “Neural networks and principal component analysis: learning from examples without local minima”,
Neural Networks 2 (1), pp. 53-58, 1989.
[11] Y.T.Zhou, R. Chellappa and B.K. Jenkins, “Image restoration using a neural network,” IEEE Trans. Acoust., Speech, Signal
Processing, Vol., ASSP-36, pp. 1141-1151, July 1988.

34. [12] Dalong Li and Steven Simske, “Atmospheric Turbulence Degraded Image by Kurtosis Minimization,” IEEE Geoscience and
Remote Sensing Letters, Dec. 2008.
[13] Luxin Yan, Mingzhi Jin, Houzhang Fang, Hai Liu, and Tianxu Zhang, “Atmospheric-Turbulence-Degraded Astronomical
Image Restoration by Minimizing Second-Order Central moment,” IEEE Geoscience And Remote Sensing Letters, Vol. 9,
No. 4, pp. 672-676, July 2012.
[14] Dalong Li, Russell M. Mersereau, and Steven Simske, “Atmospheric Turbulence-Degraded Image Restoration Using
Principal Component Analysis,” IEEE Geoscience And Remote Sensing Letters, Vol. 4, No. 3, pp. 340-344, July 2012.
[15] Jin-Bao Wang, Ning He, Lu-Lu Zhang, and Ke Lu, “Single Image dehazing with a physical model and dark channel prior,”
Elsevier, neuro-computing Aug. 2014.
[16] K. Gibson and T. Nguyen, “Fast single image fog removal using the adaptive wiener filter,” in Proc. 20th IEEE ICIP, Sep.
2013, pp. 714–718.
[17] Nathaneera Anantrasirirchai, Alin Achim, Nick G.Kingsbury and David R. Bull, “Atmospheric Turbulence Mitigation Using
Complex Wavelet-Based Fusion,” IEEE Trans. Image processing, Vol. 22, no. 6, June 2013.
[18] X. Zhu and P. Milanfar, “Removing Atmospheric Turbulence via Space-Invariant Deconvolution,” in IEEE Transaction on
Pattern Analysis and Machine Intelligence, vol.35, no.1, January 2013.

35. Publications
• Azad Singh, Rajeev Kumar Singh, “A Survey: Image Restoration Techniques on
the basis of blurs”, AICTE Sponsored National Conference on Advances in
Information and Communication Technology (NCAICT), 2015.
• Azad Singh, Rajeev Kumar Singh, “Atmospheric Turbulence Degraded Image
Restoration Using Back Propagation Neural Network”, International Journal of
Signal Processing, Image Processing and Pattern Recognition, Volume 8, Issue 12,
pp. 181-194, ISSN: 2005-4254, 2015. (Published)
• Azad Singh, Rajeev Kumar Singh, “A Survey on Restoration Techniques of
Atmospheric Turbulence Blur Image”, in proceeding of IEEE, International
Conference on Innovations in Computer Science and Engineering, June 2015.

36. Thank you