In this paper, we propose a fast re-coloring algorithm to improve the accessibility for the color vision impaired. Compared to people with normal color vision, people with color vision impairment have difficulty in distinguishing between certain combinations of colors. This may hinder visual communication owing to the increasing use of colors in recent years. To address this problem, we re-map the hue components in the HSV color space based on the statistics of local characteristics of the original color image. We enhance the color contrast through generalized histogram equalization. A control parameter is provided for various users to specify the degree of enhancement to meet their needs. Experimental results are illustrated to demonstrate the effectiveness and efficiency of the proposed re-coloring algorithm.

1. Enhancing Color Representation for the Color
Vision Impaired
Jia-Bin Huang1, Sih-Ying Wu2, and Chu-Song Chen1
jbhuang@iis.sinica.edu.tw
1 Institute of Information Science
Academia Sinica
2 Department of Electronics Engineering
National Chiao Tung University
October 18, 2008
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

2. Outline
1 Introduction
Color Vision Deficiency (CVD)
Related Works
2 The Proposed Algorithm
Main Idea
Generalized Histogram Equalization
Controlling the Enhancement Degree
3 Results and Discussion
Visual Results
Interactive Interface
Applications
4 Conclusion
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

3. Outline
1 Introduction
Color Vision Deficiency (CVD)
Related Works
2 The Proposed Algorithm
Main Idea
Generalized Histogram Equalization
Controlling the Enhancement Degree
3 Results and Discussion
Visual Results
Interactive Interface
Applications
4 Conclusion
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

4. Color Vision
Normal vision
Three different fundamental photoreceptor cells (cone cells)
Peak responses lie in long (L), middle (M), and short (S)
wavelength regions
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

5. Classification of CVD
Anomalous trichromacy
One of the three cone cells is abnormal
Dichromacy
One of the three cone cells is absent
Monochromacy
All three cone cells are absent
Table: Major genetic color deficiencies.
Type Name Cause of defect
Protanomaly L-cone defect
Anomalous trichromacy Deuteranomaly M-cone defect
Tritanomaly S-cone defect
Protanopia L-cone absent
Dichromacy Deuteranopia M-cone absent
Tritanopia S-cone absent
Monochromacy Rod Monochromacy no functioning cones
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

6. How People with CVD perceive Colors?
Anomalous trichromacy
Protanomaly Deuteranomaly Tritanomaly
Dichromacy
Protanopia Deuteranopia Tritanopia
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

7. Related Works
1 Guideline or assistant tool for designers to avoid
ambiguous color combinations
Provide guideline (Chisholm et al. 01)
Use a restricted CVD palette (Rigden et al. 99, Vienot99 et
al. 99)
Verify color schemes (Walraven et al.97, Jenny07 et al. 07)
2 (Semi-)automatically reproduce colors that are suitable for
CVD viewers
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

8. Related Works
1 Guideline or assistant tool for designers to avoid
ambiguous color combinations
Provide guideline (Chisholm et al. 01)
Use a restricted CVD palette (Rigden et al. 99, Vienot99 et
al. 99)
Verify color schemes (Walraven et al.97, Jenny07 et al. 07)
2 (Semi-)automatically reproduce colors that are suitable for
CVD viewers
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

9. Related Works
1 Guideline or assistant tool for designers to avoid
ambiguous color combinations
Provide guideline (Chisholm et al. 01)
Use a restricted CVD palette (Rigden et al. 99, Vienot99 et
al. 99)
Verify color schemes (Walraven et al.97, Jenny07 et al. 07)
2 (Semi-)automatically reproduce colors that are suitable for
CVD viewers
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

10. Typical Re-coloring Approaches
Common procedures
Select key colors using color quantization
Find the optimal mapping of key colors using optimization
procedure
Obtain the re-colored image through interpolation
(Ichikawa et al. 04, Wakita et al. 05, Rasche et al. 05,
JeRerson et al. 06, Huang et al. 07)
Drawbacks:
Slow. Require a few minutes for an image
Problem of gamut mapping, illumination inconsistency.
Unnatural color for the colorblind
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

11. Typical Re-coloring Approaches
Common procedures
Select key colors using color quantization
Find the optimal mapping of key colors using optimization
procedure
Obtain the re-colored image through interpolation
(Ichikawa et al. 04, Wakita et al. 05, Rasche et al. 05,
JeRerson et al. 06, Huang et al. 07)
Drawbacks:
Slow. Require a few minutes for an image
Problem of gamut mapping, illumination inconsistency.
Unnatural color for the colorblind
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

12. Typical Re-coloring Approaches
Common procedures
Select key colors using color quantization
Find the optimal mapping of key colors using optimization
procedure
Obtain the re-colored image through interpolation
(Ichikawa et al. 04, Wakita et al. 05, Rasche et al. 05,
JeRerson et al. 06, Huang et al. 07)
Drawbacks:
Slow. Require a few minutes for an image
Problem of gamut mapping, illumination inconsistency.
Unnatural color for the colorblind
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

13. Typical Re-coloring Approaches
Common procedures
Select key colors using color quantization
Find the optimal mapping of key colors using optimization
procedure
Obtain the re-colored image through interpolation
(Ichikawa et al. 04, Wakita et al. 05, Rasche et al. 05,
JeRerson et al. 06, Huang et al. 07)
Drawbacks:
Slow. Require a few minutes for an image
Problem of gamut mapping, illumination inconsistency.
Unnatural color for the colorblind
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

14. Outline
1 Introduction
Color Vision Deficiency (CVD)
Related Works
2 The Proposed Algorithm
Main Idea
Generalized Histogram Equalization
Controlling the Enhancement Degree
3 Results and Discussion
Visual Results
Interactive Interface
Applications
4 Conclusion
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

15. Main Idea
Main Idea
Maintain luminance and saturation consistency.
=>Work in the HSV color space, and leave value and
saturation component unchanged
Preserve the natural order of hue.
=>Apply a hue transfer function to hue channel
(non-decreasing)
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

16. Contrast Enhancement using Histogram Equalization
(HE)
One of the most well-known technique for image
enhancement
Assign larger dynamic range for intensity values having
larger number of occurrences
g
T (g) = gmin + (gmax − gmin) hist(g) dg
gmin
T (g): Intensity mapping function
hist(g): Normalized histogram (probability distribution of
the grey levels in the image)
gmax, gmin: Maximum and minimum intensity values
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

17. Histogram Generalization
Original Histogram
Histogram generation
Masking-and-accumulating using 1 × 1 block
Generalized Histogram
Generalized histogram
Extend 1 × 1 block to N × N to encode spatial information
Replace sampling by measurement(feature extraction)
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

18. Three Local Measurement
1 Hue value at point (x, y)
α(x, y) = hue(x, y)
2 Local hue difference
β(x, y) = max{hue(i, j)} − min{hue(i, j)}, (i, j) ∈ N(x, y)
i,j i,j
3 Color confusability due to CVD
γ(x, y) = (||(C(x, y) − C(i, j))||−
(i,j)∈N(x,y)
||(Sim(C(x, y)) − Sim(C(i, j)))||)2
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

19. Three Local Measurement
1 Hue value at point (x, y)
α(x, y) = hue(x, y)
2 Local hue difference
β(x, y) = max{hue(i, j)} − min{hue(i, j)}, (i, j) ∈ N(x, y)
i,j i,j
3 Color confusability due to CVD
γ(x, y) = (||(C(x, y) − C(i, j))||−
(i,j)∈N(x,y)
||(Sim(C(x, y)) − Sim(C(i, j)))||)2
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

20. Three Local Measurement
1 Hue value at point (x, y)
α(x, y) = hue(x, y)
2 Local hue difference
β(x, y) = max{hue(i, j)} − min{hue(i, j)}, (i, j) ∈ N(x, y)
i,j i,j
3 Color confusability due to CVD
γ(x, y) = (||(C(x, y) − C(i, j))||−
(i,j)∈N(x,y)
||(Sim(C(x, y)) − Sim(C(i, j)))||)2
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

21. Color Confusability Map
γ(x, y) = (||(C(x, y) − C(i, j))||−
(i,j)∈N(x,y)
||(Sim(C(x, y)) − Sim(C(i, j)))||)2
1 The value of γ(x, y) in logarithmic scale
(a) (b) (c)
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

22. Combining Three Measurement
α: hue value
β: local hue difference
γ: local color confusability
Conventional histogram accumulating function
δ(h − α)
Generalized histogram accumulating function
h−α
S(h|α, β, γ) = γ × Rect( )
β
1, if − 0.5 ≤ x ≤ 0.5,
Rect(x) ≡
0, otherwise.
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

23. Generalized Histogram Equalization
By scanning over the whole image, the generalized
histogram can be obtained:
GH(h) = S(h|α(x, y), β(x, y), γ(x, y))dxdy
Construct the hue transfer function
h
T (h) = hmin + (hmax − hmin) GH(h)dh
hmin
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

24. Controlling the Enhancement Degree
Add a magnitude mapping function
M(x) = xp
The hue transfer function
h
hmin M(GH(h))dh
T (h) = hmin + (hmax − hmin) hmax
hmin M(GH(h))dh
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

25. Outline
1 Introduction
Color Vision Deficiency (CVD)
Related Works
2 The Proposed Algorithm
Main Idea
Generalized Histogram Equalization
Controlling the Enhancement Degree
3 Results and Discussion
Visual Results
Interactive Interface
Applications
4 Conclusion
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

26. Visual Results
Ishihara test chart
(a) (c) (e) (g)
(b) (d) (f) (h)
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

27. Comparison with Rasche et al’s Approach
Image Simulation Our results Rasche et al’s results
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

28. Comparison with Jeferson et al.’s Approach (1/2)
Image Our results Jeferson et al.’s results
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

29. Comparison with Jeferson et al.’s Approach (2/2)
Image Our results Jeferson et al.’s results
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

30. Controlling the Enhancement Degree
(a) (c)p = 0.2 (e)p = 0.6 (g)p = 1.0
(b)p = 0 (d)p = 0.4 (f)p = 0.8 (h)p = 1.2
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

31. Controlling the Enhancement Degree
(a) (c)p = 0.2 (e)p = 0.6 (g)p = 1.0
(b)p = 0 (d)p = 0.4 (f)p = 0.8 (h)p = 1.2
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

32. Applications
Improve web accessibility for the colorblind
Color adaptation in multimedia content (e.g. images,
videos etc.)
Assistive technologies for designers
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

33. Outline
1 Introduction
Color Vision Deficiency (CVD)
Related Works
2 The Proposed Algorithm
Main Idea
Generalized Histogram Equalization
Controlling the Enhancement Degree
3 Results and Discussion
Visual Results
Interactive Interface
Applications
4 Conclusion
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

34. Conclusion and Future Work
Conclusion
Fast re-color algorithm for CVD viewers
Re-color image using local contrast information
Interactive interface
Future work
Subjective experiments
Re-coloring through optimization
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired

35. The End
Thank you
Jia-Bin Huang et al. (IIS, Academia Sinica) Enhancing Color Representation for the Color Vision Impaired