LectureSIght (http://lecturesight.org) is an open source tracking solution for use in lecture capture systems. LectureSight makes use of real-time video analysis to direct a Sony VISCA, Axis VAPIX or ONVIF PTZ camera.

1. ||
Opencast Community Summit
University of Cologne, March 14-16
LectureSight in action
Benjamin Wulff
Fraunhofer IAIS
Stephen Marquard and Corné Oosthuizen
University of Cape Town

2. ||
LectureSight is an open source real-time video tracking solution for PTZ cameras.
http://lecturesight.org (wiki, code, issue tracker, mailing list)
To run LectureSight, you need:
• a Linux Capture Agent with a GPU
• an overview camera (for example a webcam)
• a PTZ camera which supports VISCA (Sony), VAPIX (Axis) or ONVIF
16/03/2016 2
About LectureSight

3. ||
LectureSight:
Overview image
> detect changes
> detect objects
> direct camera (track objects)
> steer the camera
> send commands to
PTZ camera (IP / serial)
Capture Agent:
Record the PTZ camera image (IP / SDI / HDMI)
16/03/2016 3
How does LectureSight work?

4. ||
 Our installations
 LS in action…
 What could possibly go wrong?
 Production monitoring
 Future plans
16/03/2016 4
LectureSight at the University of Cape Town

5. ||
 Why? Blackboards, body language, gestures
 Planned for up to 20 venues
 Running in production in 4 venues (another 10 pending)
 Hardware:
 PTZ cameras: Axis V5915 (VAPIX), Vaddio ClearView HD-USB (VISCA)
 Overview cameras: Logitech C930 webcam (USB), Raspberry Pi Cameras (IP)
 USB active extension cables (up to 20m), ATEN UCE-260 USB 2.0 over CAT5 (up to 60m)
 Ubuntu 12.04 Capture Agents running Galicaster 1.4.x with Nvidia GPUs (NV310+)
 LectureSight scheduling is integrated with the Opencast / Galicaster calendar:
tracking starts and stops for scheduled recordings.
16/03/2016 5
LectureSight at UCT: Installations

6. || 16/03/2016 6

7. ||
 (Videos)
16/03/2016 7
LectureSight in action

8. ||
 False positives in object detection
 Lack of object persistence
 Inability to distinguish presenter from background movement
projection on screen)
 Inappropriate camera operator strategies (“always follow” not always right)
16/03/2016 8
What could possibly go wrong?

9. ||
Presenter is
tracked, but
another object
is incorrectly
detected, and
the camera
follows the
phantom object
for a while.
16/03/2016 9
False
positives

10. ||
When the presenter
stops moving, the
object times out after a
while.
A secondary object can
start being tracked
instead of the camera
remaining on the
primary target.
16/03/2016 10
Object
persistence

11. ||
Typically too much camera movement from:
rapid movement of a single presenter
too many targets (start/end of lecture)
Rather zoom out (“if in doubt, go wide”) or stop tracking.
16/03/2016 11
Inappropriate camera operator strategies

12. ||
 Has LectureSight started up correctly?
 Is it tracking, or idle (waiting for an
event)?
 telnet to your LS instance on port 2501
(osgi.shell.telnet.ip=x.x.x.x)
 Get status with “lb”, “scr:list”, and
“scheduler:status”
 Script and graph
 To do: snapshot / dashboard of overview
camera images
https://opencast.jira.com/projects/LS/issues/LS-153
16/03/2016 12
Production monitoring

13. ||
University of Osnabrück
University of Manchester
University of Stuttgart
16/03/2016 13
LectureSight is also at …

14. ||
 Improve object tracking (face detection, object persistence, fewer false positives)
 More capable camera operator (pan/tilt/zoom, wider range of strategies)
 Instrumentation / statistics to give insight into tracking performance and quality
 Improved GUI and easier calibration
16/03/2016 14
UCT’s Road Map / Wish List for LectureSight