Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

FOSDEM 2016 - Creating rich WebRTC Applications with Kurento

1.655 Aufrufe

Veröffentlicht am

WebRTC is a disruptive media technology bringing real-time multimedia communications to HTML5 standards. WebRTC is currently available for billions of users as a built-in feature of common browsers such as Chrome and Firefox. This makes possible the emergence of a truly open and interoperable technology competing with proprietary conferencing solutions and enabling developers to create specific-purpose WWW peer-to-peer real-time media applications in a simple and seamless manner.

However, WebRTC developers commonly require more than plain peer-to-peer video conferencing. For this, we introduce Kurento (http://www.kurento.org), a Free Open Source Software (FOSS) initiative build on top of GStreamer providing developers a set of high level abstract APIs making possible the creation of HTML5 multimedia-enabled web application. Kurento pushes GStreamer to the limit building a media server with interesting features such as media recording, media mixing for group communications, media adaption and transcoding, media augmentation, integration with computer vision capabilities, etc.
In the talk we will introduce what’s Kurento and how WWW developers can take advantage of it showing the following aspects of the framework:
- First, introducing Kurento Java and JavaScript APIs and we explain how they interact with the signaling plane in an application server. This allows WebRTC application developers to create their application logic using popular technologies such as Java EE or Node.js.
- Second, by presenting a unique media plane written on top of GStreamer. GStreamer is based on the concept of media pipelines, which can be seen as chains of media elements performing operations to a media flow in real-time. Currently there are more than 1000 media elements written for GStreamer implementing many different capabilities such as codecs (e.g. H.264, H.263, VP8, etc.), recorders and players (for storing/recovering media from files), blenders (for augmenting media), filters (i.e. face blurring, face recognition, etc.) and others. Hence, Kurento enables to inject WebRTC streams into a chain of such elements and perform, in real time, the operations the developer wishes.
- Third, introducing and abstraction of all the complexities of signaling, media control and media management through a powerful server-side API that can be used by average developers who do not require particular expertise about multimedia protocols or formats.

Veröffentlicht in: Internet
  • Als Erste(r) kommentieren

FOSDEM 2016 - Creating rich WebRTC Applications with Kurento

  1. 1. Creating rich WebRTC Applications with Kurento Luis Lopez lulop@kurento.org
  2. 2. Who I am http://www.kurento.org - lulop@kurento.org 2 Associate Professor Escuela Técnica Superior de Ingenieros de Telecomunicación Universidad Rey Juan Carlos (south Madrid) Researcher in the area of RTC • http://www.nubomedia.eu • http://www.fiware.org FOSS enthusiast • Kurento project lead: http://www.kurento.org • NUBOMEDIA Community lead: http://www.nubomedia.eu
  3. 3. Real-Time media Communications http://www.kurento.org - lulop@kurento.org 3 Capture Encode Cipher Transport Transport Decipher Decode RenderNetwork The media plane The signaling plane I wan to call you, do you accept? Yes, do it in this way
  4. 4. WebRTC to the rescue. Before WebRTC First wave of WebRTC technologies Begin End • APIs • Standards • FOSS Development experience when working with real-time media Common WebRTC application (p2p communications) WebRTC video stream http://www.kurento.org - lulop@kurento.org 4
  5. 5. WebRTC infrastructures http://www.kurento.org - lulop@kurento.org 5 Peer-to-Peer WebRTC Application (without media infrastructure) WebRTC video stream WebRTC Application based on media infrastructure media infrastructure
  6. 6. WWW VS (Web)RTC applications http://www.kurento.org - lulop@kurento.org 6 Control Multimedia Application logic (developers’ code) Multimedia Capabilities Recording Transcoding Routing Mixing Analyzing Etc. Media Traffic RTC Media APIs Multimedia Clients Events Application Signaling Control WWW Application logic (developers’ code) Database Capabilities Data storage Data recovery Data querying Data processing Etc. DD.BB. API WWW Clients Events Application Signaling
  7. 7. http://www.kurento.org - lulop@kurento.org 7
  8. 8. http://www.kurento.org - lulop@kurento.org 8 Control Multimedia Application logic (developers’ code) Multimedia Capabilities Recording Transcoding Routing Mixing Analyzing Etc. RTC Media APIs Events
  9. 9. Kurento: a WebRTC infrastructure an its APIs http://www.kurento.org - lulop@kurento.org 9 Control Multimedia Application logic (developers’ code) Recording Transcoding Routing Mixing Analyzing Adapting Media Traffic RTC Media APIs Multimedia Clients Events Application Signaling Computer vision Augmented reality Blending Etc. Multimedia Capabilities Kurento Media Server (KMS) Kurento Client API Kurento Room API Kurento Tree API
  10. 10. Cooking Kurento http://www.kurento.org - lulop@kurento.org 10 10
  11. 11. The Kurento FOSS Community http://www.kurento.org - lulop@kurento.org 11 Community support WebRTC worldwide reference International awards More than 300 companies Very active mailing list First result in Google
  12. 12. The Kurento FOSS project http://www.kurento.org - lulop@kurento.org 12 • http://www.kurento.org – Main web site • https://www.twitter.com/kurentoms – Main social channel • https://groups.google.com/forum/#!forum/kurento – Main mailing list • https://github.com/kurento – Main repository • https://www.youtube.com/channel/UCFtGhWYqahVlzMgGNtEmKug – Kurento Youtube channel
  13. 13. Developing with KMS: The Kurento Client API http://www.kurento.org - lulop@kurento.org 13 SinkSRC Sink SRC SRCSink Sink Media Element • Provides a specific media functionality › Send/receive media › Process media › Transform media • Exchange media through › Sources › Sinks Media pipeline • Chain of media elements implementing the desired media logic. • The Media API provides the capability of creating media pipelines by joining media elements of the toolbox Media Element Sink SRC
  14. 14. A modular API • Modular in the sense of modularity* – Isolation • Internal states of a module don’t affect the rest – Abstracion • Internal states are hidden behind an interface – Composability • Interface must enable module recombination and assembly – Reusability • If a module has it, you can use it – Extensibility • If a module does not have it, you can add it. http://www.kurento.org - lulop@kurento.org 14 * Baldwin, Carliss Young, and Kim B. Clark. Design rules: The power of modularity. Vol. 1. MIT press, 2000
  15. 15. The paradigm of modularity http://www.kurento.org - lulop@kurento.org 15
  16. 16. The Kurento Client API in one word http://www.kurento.org - lulop@kurento.org 16 sourceElement.connect(sinkElement) connect
  17. 17. http://www.kurento.org - lulop@kurento.org 17 Kurento hello world
  18. 18. //MediaPipeline is a holder of media elements MediaPipeline pipeline = kurento.createMediaPipeline(); //Create your media elements WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); //Connect your media elements webRtcEndpoint.connect(webRtcEndpoint); http://www.kurento.org - lulop@kurento.org 18 Control Multimedia Application logic (developers’ code) Recording Transcoding Routing Mixing Analyzing Adapting Kurento Media APIs Multimedia Clients Events Application Signaling Computer vision Augmented reality Blending Etc. Multimedia Capabilities //Where to render the local and remote streams var options = { localVideo : videoInput, remoteVideo : videoOutput, onicecandidate : onIceCandidate } //Start media capture and communications webRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv( options, function(error) { if (error) return console.error(error); webRtcPeer.generateOffer(onOffer); }); Application Server Code (Java) Client Server Code Application source for Spring (Java EE) Application source for Node.js (JavaScript)
  19. 19. Recording http://www.kurento.org - lulop@kurento.org SinkSRC Sink //MediaPipeline is a holder of media elements MediaPipeline pipeline = kurento.createMediaPipeline(); //Create your media elements WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); RecorderEndpoint recorderEndpoint = new RecorderEndpoint.Builder(pipeline).build( “file:///myfolder/myfile.mp4”); //Connect your media elements webRtcEndpoint.connect(webRtcEndpoint); webRtcEndpoint.connect(recorderEndpoint); 19 Application source for Spring (Java EE) Application source for Node.js (JavaScript)
  20. 20. Playing http://www.kurento.org - lulop@kurento.org 20 SinkSRC SRC //MediaPipeline is a holder of media elements MediaPipeline pipeline = kurento.createMediaPipeline(); //Create your media elements WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); PlayerEndpoint playerEndpoint = new PlayerEndpoint.Buildr(pipeline).build( “file:///myfolder/myfile.mp4”); //RSTP and HTTP URIs also supported //Connect your media elements playerEndpoint.connect(webRtcEndpoint); Application source for Spring (Java EE) Application source for Node.js (JavaScript)
  21. 21. Interoperating http://www.kurento.org - lulop@kurento.org 21 Media Pipeline WebRTC audio streaming SinkSRC SRCSink Media Pipeline SinkSRC SRCSink WebRTC video streaming RTP audio streaming RTP video streaming Transparent transcoding!!
  22. 22. Some Media processing modules • Augmented reality – MarkerArModule – MarkerlessArModule • Computer vision – FaceDetector – NoseDetector – EyeDetector – CrowdDetector – PointerDetector – MotionDetector – VirtualFence – Etc. http://www.kurento.org - lulop@kurento.org 22 Sink SRC Sink SRC Sink SRC Sink SRC Sink SRC
  23. 23. Using media processing modules http://www.kurento.org - lulop@kurento.org 23 SinkSRC Sink SRC CrowdEvents SRC Sink SRC CrowdDetector RtspPlayer SinkSRC See demo See demo
  24. 24. Implementing group communications: just connect http://www.kurento.org - lulop@kurento.org 24 Media Pipeline WebRTC streaming SinkSRC WebRTC streaming SRCSinkSRCSink WebRTC streaming WebRtcEndpoint 1 WebRtcEndpoint 3 WebRtcEndpoint 2 SRCSink WebRTC streaming WebRtcEndpoint 4 User 1 User 2 User 3 User 4
  25. 25. Group communications: RTP topologies http://www.kurento.org - lulop@kurento.org 25 SinkSRC SRC Sink SinkSRC SRCSink • Media Mixing Mixer – Composite • Media Switching Mixer – Connect primitive – Natural topology • SFU – WebRtcSfu • Simulcast
  26. 26. Creating your own modules http://www.kurento.org - lulop@kurento.org 26 Kurento IDL Compiler JSON Module IDL Description Java Java- Script cpphpp Media API code Toolchain Auto-generated code Module Implementation cmake Interface Developer (1) (2) (3)
  27. 27. Going up the API stack: the room API http://www.kurento.org - lulop@kurento.org 27 createRoom deleteRoom joinRoom leaveRoom publisMedia unpublishMedia onParticipantJoined onParticipantLeft onMediaPublished onMediaUnpublished onMessage See demo
  28. 28. Going up the API stack: the tree API http://www.kurento.org - lulop@kurento.org 28 createTree deleteTree joinAsPresenter joinAsViewer leave
  29. 29. What’s next http://www.kurento.org - lulop@kurento.org 29
  30. 30. Where to start http://www.kurento.org - lulop@kurento.org 30 • http://www.kurento.org/documentation – Installation and administration guide – Java tutorials for Spring and Java EE developers – JavaScript tutorials for Node.js developers – JSON-RPC protocol documentation • https://github.com/kurento – Main repository
  31. 31. Thanks http://www.kurento.org - lulop@kurento.org 31 Luis Lopez lulop@kurento.org

×