5. 5
Learned from the Grandmaster
Curiosity
Open minded
New problems
New techniques
Passion
Love, focus - sustained
Scholarship
Unaffected by noise, hype
Uncompromised integrity
Never too old to learn
8. From Classification to Description
Recognizing Realistic Actions from Videos "in the Wild"
UCF-11 to UCF-101
(CVPR 2009)
Similarity btw Videos Cross-Domain Learning
Visual Event Recognition in Videos by
Learning from Web Data
(CVPR2010 Best Student Paper)
Heterogeneous Feature Machine For Visual Recognition
(ICCV 2009)
9. Image Captioning with Semantic Attention
• Motivations
– Real-world Usability
• Help visually impaired people, learning-impaired
– Improving Image Understanding
• Classification, Objection detection
– Image Retrieval
1. a young girl inhales with the intent of blowing
out a candle
2. girl blowing out the candle on an ice cream
1. A shot from behind home
plate of children playing
baseball
2. A group of children playing
baseball in the rain
3. Group of baseball players
playing on a wet field
11. Key Elements
• Additional textual information
– Leverage noisy titles, tags or captions (Web)
– Leverage visually similar nearest neighbor images
12. Key Elements
• Additional textual information
– Leverage noisy titles, tags or captions (Web)
– Leverage visually similar nearest neighbor images
– Incorporate success of low-level tasks
• Visual attribute detection
13. Attention Model on Attributes
• Instead of using the same set of attributes at every
step
• At each step, select the attributes (attention)
m mtmt kKwatt ),(
)softmax VK(wT
tt
))],,(;([),( 11 tttttt hKwattxfhxfh
14. Overall Framework
• Training with a bilinear/bilateral attention model
ht
pt
xt
v
{Ai}
Yt~
RNN
Image
CNN
AttrDet 1
AttrDet 2
AttrDet 3
AttrDet N
t = 0
Word
17. Examples
a skate boarder is
doing trick on his skate
board.
a gloved hand opens to
reveal a golden ring.
a sport car is swinging on
the race playground
the vehicle is moving fast
into the tunnel
22. • Social interactions and social activities
• Public health surveillance
• Web sentiment analysis and trend prediction
• Cyber terrorism, extremism, and activism
• Fads and infectious ideas
• Marketing intelligence analytics
• Traffic and human mobility patterns
• Human and environment
• Social unrest, protest and riot
Understanding the Pulse of Society
30. 30
Forever Young
30
Forever young,
I want to be forever young.
Do you really want to live forever?
Forever, and ever
Forever young,
I want to be forever young.
Do you really want to research forever?
Forever, and ever
Image Processing, Computer Vision, Multimedia, Social Media, Big Data, …
(A younger version of an old song)
……
Let’s Celebrate the Forever Young Huang Academic Tree!