CMPE 258, Image object deteion and ad suggestion nased on image provided. Application is hosted on AWS and Web Application is built using Django Application.ImageAI, Keras,TEnsorFlow was used to make the object model.Yolov3, Retinanet,YoloTiny were some of the models used.

1. AdWorld :
Ad’s Based on Image
Chetan Kulkarni
San Jose State University
Deep Learning

2. Introduction
• With the world getting completely online, Advertisers must
innovate to increase the sales of product.
• The idea at hand was using the continuous data (Image) and
analyzing the image and producing ad’s

3. Dasetset Collection
1. Used Googles OpenImagesV6 Dataset
2. Leveraged openimages PyPi API to download dataset
required for the object detection.
3. Date Composition was image data and its specific
annotated data for that object

4. Data Example
Annotation in Pascal Format Image
Every image will have its annotation xml file , these both will be used as data for transfer learning

5. Data Example
Annotation in Pascal Format Image

6. Object Detection : Techniques
• In this project I have tried to use various Pretrained models to
perform Object Detection and on that Generating Ad’s.
• RetinaNet
• Yolov3
• YoloV3 tiny
• Transfer Learning on Yolov3

7. RetinaNet object Detection
Model built on COCO dataset, and supported following image detection:

8. RetinaNet object Detection

9. RetinaNet object Detection

10. Now Need to Generate Ad’s
Object
Object Category
Advertisement
The object will be detected by models
Admin will make Object Category,
Each object will be mapped to
one or more categories
Ad’s will be made to target
specific category

11. Django Application • TV is a object
• TV is connected to utilities
category
• Subwoofer Ad is targeted to
utilities category
• So whenever we have TV,
we show all the Ad’s of
utilities category

12. • Model Detected TV in the
image
• We have Advertisements of
Subwoofer and couch
connected to TV
• Hence The Ad’s

13. Transfer Learning
• Using Existing Yolov3 pretrained model. Tried to transfer learn for 3
specific imageset of table, clothing and bottle.
• Couldn’t Train model on basic google collab GPU’s
• Tried Google Collab PRO with High Ram and High Gpu
• It took 3000s for one epoch
• Because of limited resource couldn’t train for more than 24 hours
• Hit The Accuracy of 50% for these custom object detection

14. Transfer Learning
Transfer Learning: 3000s per epoch

15. Tensorboard

16. Tensorboard

17. Extending the detection model to Videos
• After the Image detection and Ad Generation on Images.
• Tried to Generate Ad’s on the Videos
• It Takes lot of processing and time.
• Splitting Videos into 20 Frames per image, and running image
detection model on them
• And on overall Image detected, Performing Ad Generation

18. Quick Video Output

19. Application is hosted on AWS

20. Conclusion
• Using Pretrained Models and Transfer learning we can take
image detection to next level
• Application of this image and object detection can be used in
different places like traffic
• Personalized Ads can be Generated based on specific user and
Ad’s can be served real time with live video feed in Mall’s