PyData NYC by Akira Shibata

Putting Together
World's Best Data Processing Research
with Python
Copyright 2014 Shiroyagi Corporation. All rights reserved.
Akira Shibata, PhD
Shiroyagi Corporation

Who am I
Akira Shibata, PhD.
TW: @punkphysicist
CEO, Shiroyagi Corporation (shiroyagi.co.jp)
Kamelio: Personalised News Curation
Kamect: Contents Discovery Platform
2004 - 2010:
Data Scientist @ NYU
Statistical data modelling @ LHC, CERN
2010 - 2013
Boston Consulting Group
Copyright 2014 Shiroyagi Corporation. All rights reserved. 2

Statistical modelling of Physics data
Confirmatory:
Highly theory driven model building

Telling discovery from noise
The model tells you the expected uncertainty

Kamelio
“Deep Learning”
“Internet of
Things”
“Medical IT”
“Global Strategy”
Collects news through >3M
topics to chose from

３
“Cats”
“Anime”
“Cats reaction to sighting
dogs for the first time”

Python puts all our tools together
Image in Detect
regions
Object
recog. Scoring Cropping
0 1 2 3 4
Matlab
+Scipy
C++
+Libraries
Numpy PIL
IPython and Python script

Our approach is
heavily influenced by
Berkeley Vision and
Learning Center
Acknowledgement

Detect
regions
0 1 2 3 4

Region detection: Telling where to look at
How do we find regions to feed into object recognition?
Default strategy was to look at the center
1

Exhaustive windows -> segmentation
Search over position,
scale, aspect ratio
Grouping parts of
image at different scales
Exhaustive search far too time inefficient
for use with Deep Learning
1

1 Region detection: in practice
Install Malab and Selective Search algorithm
from author
Run matlab as subprocess
pid = subprocess.Popen(shlex.split(mc), stdout=open('/dev/null',
'w'), cwd=script_dirname)
matlab -nojvm -r "try; selective_search({‘image_file.jpg’},
‘output.mat'); catch; exit; end; exit”
1
2
3
Import output using scipy.io
all_boxes = list(scipy.io.loadmat(‘output.mat')['all_boxes'][0])
subtractor = np.array((1, 1, 0, 0))[np.newaxis, :]
all_boxes = [boxes - subtractor for boxes in all_boxes]

1 Region detection: proposals generated
~200 proposals generated per image

Object
recog.
0 1 2 3 4

Object recognition
Deep blue beat Kasparov at chess in 1997…
2

2 Deep Learning: Damn good at it

2 Convoluted Neural Network
…

Caffe: open R-CNN framework under rapid dev.
C++/CUDA with Python wrapper
2

Pre-trained models published
We used 200-category object recog. model
developed for 2013 ImageNet Challenge
2

2 Object recognition: in practice
Install a bunch of libraries and Caffe
CUDA, Boost, OpenCV, BLAS…
Import wrapper and configure
MODEL_FILE=‘models/bvlc_…_ilsvrc13/deploy.prototxt’
PRETRAINED_FILE = ‘models/…/bvlc_…_ilsvrc13.caffemodel’
MEAN_FILE = 'caffe/imagenet/ilsvrc_2012_mean.npy'
detector = caffe.Detector(MODEL_FILE, PRETRAINED_FILE,
mean=np.load(MEAN_FILE), raw_scale=255, channel_swap=[2,1,0])
1
2
3
Pass found regions for object detection
self.detect_windows(zip(image_fnames, windows_list))

2 Object recognition: Result
Obj Score
0 domestic cat 1.03649377823
1 domestic cat 0.0617411136627
2 domestic cat -0.097744345665
3 domestic cat -0.738470971584
4 chair -0.988844156265
5 skunk -0.999914288521
6 tv or monitor -1.00460898876
7 rubber eraser -1.01068615913
8 chair -1.04896986485
9 rubber eraser -1.09035253525
10 band aid -1.09691572189
Takes minutes to detect all windows

2 Object recognition: Result
Obj Score
0 person 0.126184225082
1 person 0.0311727523804
2 person -0.0777613520622
3 neck brace -0.39757412672
4 person -0.415030777454
5 drum -0.421649754047
6 neck brace -0.481261610985
7 tie -0.649109125137
8 neck brace -0.719438135624
9 face powder -0.789100408554
10 face powder -0.838757038116

Scoring
0 1 2 3 4

3 Scoring
1 For every pixel, sum up score from all detections
for
i
in
xrange(len(detec0ons)):
arr[ymin:ymax,
xmin:xmax]
+=
math.exp(score)

Score heatmap
We used 200-cat object recognition model
developed for 2013 ImageNet Challenge
3

Cropping
0 1 2 3 4

4 Cropping
Generate all possible crop areas
while
y+hws
<=
h:
while
x+hws
<=
w:
window_locs
=
np.vstack((window_locs,
[x,
y,
x+hws,
y+hws]))
Find the crop that encloses the highest point of
interest in the centre
for
i,
window_loc
in
enumerate(window_locs):
x1,
y1,
x2,
y2
=
window_loc
if
max_val
!=
np.max(arr_con[y1:y2,
x1:x2]):
scores[i]=np.nan
else:
scores[i]
=
((x1+x2)/2.-‐xp)**2+
((y1+y2)/2.-‐yp)**2
1
2
3
Crop and save!
img_pil
=
Image.open(fn)
crop_area=map(lambda
x:
int(x),
window_locs[scores.argmax()])
img_crop
=
img_pil.crop(crop_area)

4 Finally

Future improvements
Fast face/human
detection
Aspect detection:
square or rectangle?
Object weighting
Magnification
Unseen object

PyData NYC by Akira Shibata

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie PyData NYC by Akira Shibata

Ähnlich wie PyData NYC by Akira Shibata (20)

Mehr von Akira Shibata

Mehr von Akira Shibata (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

PyData NYC by Akira Shibata