This document discusses fooling an automatic image quality estimator (BIQA) using adversarial attacks. It presents adapting the definition of an adversarial attack since BIQA predicts a quality score rather than class. It also covers challenges in generating adversarial samples as integer images and quantizing samples before JPEG compression to simulate artifacts. Results show success rates for different quality targets and selection rates by evaluators.

1. Fooling an Automatic
Image Quality Estimator
Benoit Bonnet
benoit.bonnet@inria.fr
Teddy Furon
teddy.furon@inria.fr
Patrick Bas
patrick.bas@centralelille.fr
Univ. Rennes, Inria, CNRS, IRISA
MediaEval 2020

2. Introduction
• Task: Pixel Privacy: Quality Camouflage for Social
Images
• BIQA: Deep Neural Network for Image Quality
Assessment
MediaEval 2020 2

3. Introduction
• Task: Pixel Privacy: Quality Camouflage for Social
Images
• BIQA: Deep Neural Network for Image Quality
Assessment
MediaEval 2020 2
Sensitive to adversarial attacks !

4. What is an Adversarial
Attack ?
• An Attack produces an Adversarial Sample
Original image
3
“baseball”
+ =
Adversarial Sample
“golf ball”
Perturbation
(crafted by the attack)
MediaEval 2020

5. What is an Adversarial
Attack ?
• An Attack produces an Adversarial Sample
• Adversarial Sample = Original Image + Perturbation
• Perturbation:
- Mostly imperceptible for a human
- but enough to fool a classifier
3
MediaEval 2020

6. Our Scenario
• White-box: the parameters of BIQA are known to us
• BIQA does not predict a class but a score
4
MediaEval 2020

7. Our Scenario
• White-box: the parameters of BIQA are known to us
• BIQA does not predict a class but a score
4
Adapting the definition of the adversarial attack:
MediaEval 2020

8. Our Scenario
• White-box: the parameters of BIQA are known to us
• BIQA does not predict a class but a score
4
Adapting the definition of the adversarial attack:
adversarial sample
misclassified
minimizing distortion
MediaEval 2020

9. Our Scenario
• White-box: the parameters of BIQA are known to us
• BIQA does not predict a class but a score
4
Adapting the definition of the adversarial attack:
To:
score below target sa
MediaEval 2020

10. From a Sample to an Image
• An image is preprocessed to be fed as an input
• BIQA: Preprocessed(Image) = ((Image/255.0)-0.5)/0.5
• Image = 3 dimensional array of 0 to 255 integer values
• Preprocessed(Image) = 3 dimensional of seemingly
[-1,1] continuous values
5
MediaEval 2020

11. From a Sample to an Image
• An image is preprocessed to be fed as an input
• BIQA: Preprocessed(Image) = ((Image/255.0)-0.5)/0.5
• Image = 3 dimensional array of 0 to 255 integer values
• Preprocessed(Image) = 3 dimensional of seemingly
[-1,1] continuous values
MediaEval 2020
Problem: Attacks are performed in the preprocessed domain
= reverting preprocessing does not return integer values
5

12. Quantizing a sample
• Using existing method: “What if Adversarial Samples
were Digital Images” Bonnet et al. 2020
• Final constraint: Images are evaluated on their
JPEG(QF=90) counterpart
MediaEval 2020 5
Adaptation of the quantization method to
the DCT domain

13. Results overview
MediaEval 2020
Target scores
%age of images successfully
scoring under sa
PPNG = rate for submitted images
PJPEG = rate for same images with jpeg
compression simulation
Times selected by
the jury
6

14. JPEG artifacts
• When the image is mainly low frequencies, JPEG
artifacts may appear:
MediaEval 2020 7

15. Conclusion
• Interesting task: extend the scope of adversarial attacks
• Adapt previous works of quantization to JPEG
(quantization in the DCT domain)
8
MediaEval 2020

16. Conclusion
• Interesting task: extend the scope of adversarial attacks
• Adapt previous works of quantization to JPEG
(quantization in the DCT domain)
• However hard to work in a gray-box setup (no
knowledge of the JPEG compression used)
• Saving DCT coeff. directly in a JPEG images for better
results
8
MediaEval 2020