Essential features:<\/h2>\n\n\n\n\n- Low latency \u2013 in its superior form, WebRTC uses UDP sockets for media transfer. This guarantees the lowest possible latencies \u2013 even less than 500ms.<\/li>\n\n\n\n
- No additional plugins. Since it is built into browsers \u2013 we don\u2019t need any additional software to run WebRTC solutions.<\/li>\n\n\n\n
- As part of the HTML5 specification \u2013 WebRTC is currently supported by most of the modern browsers<\/a>: Edge 15+, Firefox 22+, Chrome 23+, Safari 11+, Opera18+ etc.<\/li>\n\n\n\n
- WebRTC connection requires a secure HTTPS protocol by default.<\/li>\n<\/ul>\n\n\n\n
How does it work?<\/h2>\n\n\n\n
While for the end-user, WebRTC based solutions are most often seamlessly integrated into a webpage, thus making it particularly easy to use, its development side is more complex.<\/p>\n\n\n\n
WebRTC APIs<\/h3>\n\n\n\n
Let’s start with what browsers provide us with in terms of setting up a proper WebRTC connection. We do get a set of JavaScript APIs:<\/p>\n\n\n\n
\n- MediaStream<\/em>: this API is responsible for gaining access to webcams and microphones of the peer\u2019s device.<\/li>\n\n\n\n
- RTCPeerConnection<\/em>: the main component of WebRTC technology. It provides codec handling, peer-to-peer communication capabilities, security, bandwidth management. It allows to set up the connection (not without intermediaries though).<\/li>\n\n\n\n
- RTCDataChannel<\/em>: designed to provide the possibility for bi-directional data transfer.<\/li>\n<\/ul>\n\n\n\n
To set up a simple peer-to-peer connection, we would only need to use the first two APIs. The third one, RTCDataChannel, is useful only when we want to exchange additional (apart from the media) data between peers.<\/p>\n\n\n\n
Signaling<\/h3>\n\n\n\n
Now that we know what browsers provide us with out of the box, let’s get into details of what we have to provide ourselves:<\/p>\n\n\n\n
Due to its browser-based peer-to-peer nature, WebRTC requires an additional intermediate negotiator \u2013 called Signaling. The good news is that we can use any technology we like to implement it, e.g., REST or WebSockets. Take a look at the diagram:<\/p>\n\n\n\n![\"Diagram](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/Przechwytywanie.png\")
<\/a>Fig. 1 Signaling<\/figcaption><\/figure>\n\n\n\nAs you can see, signaling is something that stands next to the main audio\/video communication flow, but in fact, it is equally important. Its main purpose is to send the data required to initiate the connection. What exact data need to be sent by signaling? Let\u2019s start with what could be named metadata in the world of WebRTC communication:<\/p>\n\n\n\n
SDP – Session Description Protocol<\/h3>\n\n\n\n
This special set of data contains all the information about the type of media a peer is ready to both transmit and receive. Also includes compression codecs the endpoint is capable of decoding and IP addresses prepared to receive the incoming media stream (although this section is not mandatory because there is another mechanism for this purpose \u2013 ICE).<\/p>\n\n\n\n
ICE – Internet Connectivity Establishment<\/h3>\n\n\n\n
This might be the trickiest part of the whole WebRTC concept. To understand why \u2013 take a look at some facts: peers will most likely be located behind some NAT (Network Address Translation) and\/or firewall. This means that they won\u2019t have public IP addresses themselves but will rather be available through some translated paths defined by the router (NAT) or that opening a direct connection simply won\u2019t be possible (firewall).<\/p>\n\n\n\n
When it comes to WebRTC, it is crucial to know under what IP addresses peers are available. So, in order to bind the connection, each peer has to first make use of the so-called ICE protocol, to find out their publicly available addresses, as well as the shortest path for media to travel between endpoints. ICE protocol achieves this with the help of two other technologies: STUN\/TURN servers. Let\u2019s take a look at the diagram to analyze the STUN scenario:<\/p>\n\n\n\n![\"Diagram](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/2-1.png\")
<\/a>Fig. 2 STUN server<\/figcaption><\/figure>\n\n\n\nSTUN<\/strong> server is used to allow peers to learn their public NAT IP address and port. Once the peer has those details, it can send them to the other peer through the signaling server. This scenario is more desirable because in the end media transfer will happen in a peer-to-peer manner. However, sometimes STUN server is not enough. For some network topologies, we\u2019d need the other solution based on the TURN server:<\/p>\n\n\n\n![\"Diagram](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/3-1.png\")
<\/a>Fig. 3 TURN server<\/figcaption><\/figure>\n\n\n\nTURN<\/strong> is an intermediary server that allows peers to send and receive media. We can imagine that this scenario is less desirable, as it would introduce additional latency and the TURN server itself would have to handle a lot of traffic. This might however be necessary e.g., in topologies behind symmetric NAT.<\/p>\n\n\n\nWebRTC in practice<\/h2>\n\n\n\n
We now have all the pieces required to set up at least a simple connection between two peers. Let\u2019s build the solution.<\/p>\n\n\n\n
Preconditions:<\/h3>\n\n\n\n\n- client (peer) side code written in JavaScript,<\/li>\n\n\n\n
- signaling server \u2013 based on WebSockets, Java\/Kotlin,<\/li>\n\n\n\n
- STUN\/TURN server – free open source COTURN project,<\/a><\/li>\n\n\n\n
- Whenever there is a field \u201cpublicIP\u201d in the code \u2013 it is a placeholder for your public IP address. If you don\u2019t have one, you can use your local LAN address, but then your solution will be limited to the local LAN network.<\/li>\n<\/ul>\n\n\n\n
Client-side code<\/h3>\n\n\n\n
First, let\u2019s define some constants:<\/p>\n\n\n\n![\"part](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/1-2.png\")
<\/a><\/figure>\n\n\n\nWe have a signaling server object, media constraints, STUN server url, RTCPeerConnection<\/em> and two HTML video elements<\/em>.<\/p>\n\n\n\nNow, we will make use of one of the WebRTC APIs provided by the browser. Handling the following events should be enough in our case:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/2-3.png\")
<\/a><\/figure>\n\n\n\n\n- onicecandidate<\/em> event will be called whenever ICE protocol finds an ICE candidate. Our handler will automatically send it to the other peer through the signaling server.<\/li>\n\n\n\n
- ontrack<\/em> is called when we receive a stream from the remote peer.<\/li>\n\n\n\n
- onnegotiationended <\/em>this event would be called when there is a local media device detected and acquired through the WebRTC media API.<\/li>\n<\/ul>\n\n\n\n
The code for handling signaling messages looks as follows:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/3-2.png\")
<\/a><\/figure>\n\n\n\nWe distinguish two types of messages: description<\/em> (for the SDP description that would come from the remote peer) and candidate<\/em> for ICE Candidate objects.<\/p>\n\n\n\nAnd now the last piece of the client JavaScript code:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/4-1.png\")
<\/a><\/figure>\n\n\n\nWe\u2019ve attached a handler that makes use of the WebRTC media API. This function should acquire a local camera device and initiate the connection procedure (through the onnegotiationended<\/em> event).<\/p>\n\n\n\nOur HTML markup is super simple:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/5-1.png\")
<\/a><\/figure>\n\n\n\n<\/div>\n\n\n\nSignaling code<\/h3>\n\n\n\n
Ok, now we will take a look at how to implement a simple WebSocket-based signaling server. First, the configuration class:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/6-1.png\")
<\/a><\/figure>\n\n\n\nAnd the handler bean implementation:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/7-1.png\")
<\/a><\/figure>\n\n\n\nThe handler is prepared to handle two WebSocket sessions. Any message that comes from session one is being forwarded to session two and messages from session two are being sent to session one. Methods for registering\/unregistering sessions have been omitted for readability.<\/p>\n\n\n\n
SSL<\/h3>\n\n\n\n
We can\u2019t forget about the SSL certificates, as the WebRTC wouldn\u2019t work for an unsecured connection. For the signaling part, for demonstration purposes it is enough to generate a self-signed certificate and store it in a .jks file. Having this, we only need to set the following properties in application.properties file:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/8-1.png\")
<\/a><\/figure>\n\n\n\nFor the client part, we need to choose a web server with HTTPS capabilities. Dockerized Nginx seems to be a perfect choice. Here is the Dockerfile:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/9-1.png\")
<\/a><\/figure>\n\n\n\nAs you can see, there is a COPY command for copying the SSL certificate and private key to the Nginx internal structures. Nginx expects SSL files (localhost.crt, localhost.key) defined in the following configuration:<\/p>\n\n\n\n![\"Code](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2022\/03\/10-1.png\")
<\/a><\/figure>\n\n\n\nNow we can finally build the image and run the container:<\/p>\n\n\n\n
docker build -t nginx .<\/em><\/p>\n\n\n\ndocker run -p 1234:443 nginx<\/em><\/p>\n\n\n\nWorking solution<\/h2>\n\n\n\n
Below we have an example of the solution in practice. I\u2019ve set up two laptops next to each other. The camera of one of them points to the stopwatch and the second one is connected through our WebRTC app.<\/p>\n\n\n\n
To add some spice, i.e. to test the solution in a more real live environment, the two machines are connected to separate internet networks, and one of them is additionally proxied through the VPN server located in Portugal, which gives us roughly 2800km distance. The example is shown as a couple-seconds video, with a pause in-between, to show the actual delay:<\/p>\n\n\n\n
How does it work?<\/h2>\n\n\n\n
While for the end-user, WebRTC based solutions are most often seamlessly integrated into a webpage, thus making it particularly easy to use, its development side is more complex.<\/p>\n\n\n\n
WebRTC APIs<\/h3>\n\n\n\n
Let’s start with what browsers provide us with in terms of setting up a proper WebRTC connection. We do get a set of JavaScript APIs:<\/p>\n\n\n\n
- \n
- MediaStream<\/em>: this API is responsible for gaining access to webcams and microphones of the peer\u2019s device.<\/li>\n\n\n\n
- RTCPeerConnection<\/em>: the main component of WebRTC technology. It provides codec handling, peer-to-peer communication capabilities, security, bandwidth management. It allows to set up the connection (not without intermediaries though).<\/li>\n\n\n\n
- RTCDataChannel<\/em>: designed to provide the possibility for bi-directional data transfer.<\/li>\n<\/ul>\n\n\n\n
To set up a simple peer-to-peer connection, we would only need to use the first two APIs. The third one, RTCDataChannel, is useful only when we want to exchange additional (apart from the media) data between peers.<\/p>\n\n\n\n
Signaling<\/h3>\n\n\n\n
Now that we know what browsers provide us with out of the box, let’s get into details of what we have to provide ourselves:<\/p>\n\n\n\n
Due to its browser-based peer-to-peer nature, WebRTC requires an additional intermediate negotiator \u2013 called Signaling. The good news is that we can use any technology we like to implement it, e.g., REST or WebSockets. Take a look at the diagram:<\/p>\n\n\n\n
<\/a>Fig. 1 Signaling<\/figcaption><\/figure>\n\n\n\n As you can see, signaling is something that stands next to the main audio\/video communication flow, but in fact, it is equally important. Its main purpose is to send the data required to initiate the connection. What exact data need to be sent by signaling? Let\u2019s start with what could be named metadata in the world of WebRTC communication:<\/p>\n\n\n\n
SDP – Session Description Protocol<\/h3>\n\n\n\n
This special set of data contains all the information about the type of media a peer is ready to both transmit and receive. Also includes compression codecs the endpoint is capable of decoding and IP addresses prepared to receive the incoming media stream (although this section is not mandatory because there is another mechanism for this purpose \u2013 ICE).<\/p>\n\n\n\n
ICE – Internet Connectivity Establishment<\/h3>\n\n\n\n
This might be the trickiest part of the whole WebRTC concept. To understand why \u2013 take a look at some facts: peers will most likely be located behind some NAT (Network Address Translation) and\/or firewall. This means that they won\u2019t have public IP addresses themselves but will rather be available through some translated paths defined by the router (NAT) or that opening a direct connection simply won\u2019t be possible (firewall).<\/p>\n\n\n\n
When it comes to WebRTC, it is crucial to know under what IP addresses peers are available. So, in order to bind the connection, each peer has to first make use of the so-called ICE protocol, to find out their publicly available addresses, as well as the shortest path for media to travel between endpoints. ICE protocol achieves this with the help of two other technologies: STUN\/TURN servers. Let\u2019s take a look at the diagram to analyze the STUN scenario:<\/p>\n\n\n\n
<\/a>Fig. 2 STUN server<\/figcaption><\/figure>\n\n\n\n STUN<\/strong> server is used to allow peers to learn their public NAT IP address and port. Once the peer has those details, it can send them to the other peer through the signaling server. This scenario is more desirable because in the end media transfer will happen in a peer-to-peer manner. However, sometimes STUN server is not enough. For some network topologies, we\u2019d need the other solution based on the TURN server:<\/p>\n\n\n\n
<\/a>Fig. 3 TURN server<\/figcaption><\/figure>\n\n\n\n TURN<\/strong> is an intermediary server that allows peers to send and receive media. We can imagine that this scenario is less desirable, as it would introduce additional latency and the TURN server itself would have to handle a lot of traffic. This might however be necessary e.g., in topologies behind symmetric NAT.<\/p>\n\n\n\n
WebRTC in practice<\/h2>\n\n\n\n
We now have all the pieces required to set up at least a simple connection between two peers. Let\u2019s build the solution.<\/p>\n\n\n\n
Preconditions:<\/h3>\n\n\n\n
- \n
- client (peer) side code written in JavaScript,<\/li>\n\n\n\n
- signaling server \u2013 based on WebSockets, Java\/Kotlin,<\/li>\n\n\n\n
- STUN\/TURN server – free open source COTURN project,<\/a><\/li>\n\n\n\n
- Whenever there is a field \u201cpublicIP\u201d in the code \u2013 it is a placeholder for your public IP address. If you don\u2019t have one, you can use your local LAN address, but then your solution will be limited to the local LAN network.<\/li>\n<\/ul>\n\n\n\n
Client-side code<\/h3>\n\n\n\n
First, let\u2019s define some constants:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\nWe have a signaling server object, media constraints, STUN server url, RTCPeerConnection<\/em> and two HTML video elements<\/em>.<\/p>\n\n\n\n
Now, we will make use of one of the WebRTC APIs provided by the browser. Handling the following events should be enough in our case:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\n- \n
- onicecandidate<\/em> event will be called whenever ICE protocol finds an ICE candidate. Our handler will automatically send it to the other peer through the signaling server.<\/li>\n\n\n\n
- ontrack<\/em> is called when we receive a stream from the remote peer.<\/li>\n\n\n\n
- onnegotiationended <\/em>this event would be called when there is a local media device detected and acquired through the WebRTC media API.<\/li>\n<\/ul>\n\n\n\n
The code for handling signaling messages looks as follows:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\nWe distinguish two types of messages: description<\/em> (for the SDP description that would come from the remote peer) and candidate<\/em> for ICE Candidate objects.<\/p>\n\n\n\n
And now the last piece of the client JavaScript code:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\nWe\u2019ve attached a handler that makes use of the WebRTC media API. This function should acquire a local camera device and initiate the connection procedure (through the onnegotiationended<\/em> event).<\/p>\n\n\n\n
Our HTML markup is super simple:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\n<\/div>\n\n\n\nSignaling code<\/h3>\n\n\n\n
Ok, now we will take a look at how to implement a simple WebSocket-based signaling server. First, the configuration class:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\nAnd the handler bean implementation:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\nThe handler is prepared to handle two WebSocket sessions. Any message that comes from session one is being forwarded to session two and messages from session two are being sent to session one. Methods for registering\/unregistering sessions have been omitted for readability.<\/p>\n\n\n\n
SSL<\/h3>\n\n\n\n
We can\u2019t forget about the SSL certificates, as the WebRTC wouldn\u2019t work for an unsecured connection. For the signaling part, for demonstration purposes it is enough to generate a self-signed certificate and store it in a .jks file. Having this, we only need to set the following properties in application.properties file:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\nFor the client part, we need to choose a web server with HTTPS capabilities. Dockerized Nginx seems to be a perfect choice. Here is the Dockerfile:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\nAs you can see, there is a COPY command for copying the SSL certificate and private key to the Nginx internal structures. Nginx expects SSL files (localhost.crt, localhost.key) defined in the following configuration:<\/p>\n\n\n\n
<\/a><\/figure>\n\n\n\nNow we can finally build the image and run the container:<\/p>\n\n\n\n
docker build -t nginx .<\/em><\/p>\n\n\n\n
docker run -p 1234:443 nginx<\/em><\/p>\n\n\n\n
Working solution<\/h2>\n\n\n\n
Below we have an example of the solution in practice. I\u2019ve set up two laptops next to each other. The camera of one of them points to the stopwatch and the second one is connected through our WebRTC app.<\/p>\n\n\n\n
To add some spice, i.e. to test the solution in a more real live environment, the two machines are connected to separate internet networks, and one of them is additionally proxied through the VPN server located in Portugal, which gives us roughly 2800km distance. The example is shown as a couple-seconds video, with a pause in-between, to show the actual delay:<\/p>\n\n\n\n
- ontrack<\/em> is called when we receive a stream from the remote peer.<\/li>\n\n\n\n
- Whenever there is a field \u201cpublicIP\u201d in the code \u2013 it is a placeholder for your public IP address. If you don\u2019t have one, you can use your local LAN address, but then your solution will be limited to the local LAN network.<\/li>\n<\/ul>\n\n\n\n
- RTCPeerConnection<\/em>: the main component of WebRTC technology. It provides codec handling, peer-to-peer communication capabilities, security, bandwidth management. It allows to set up the connection (not without intermediaries though).<\/li>\n\n\n\n