I was looking for a simple free software solution which would allow me to have a video call with someone else (I don't care about sound since I've already got that working through Asterisk) and I ended up writing a Gstreamer-based poor man's videoconf solution because I wasn't satisfied with the other options I considered.

Empathy

Empathy was my first choice since it seems to be the preferred GNOME communication software nowadays.

While the quality of the video was excellent, the latency between New Zealand and Canada was unbearable: a full 6 seconds. I suspect that this is due to the fact that it runs everything through the Google Talk STUN server and I couldn't find how to force it to go directly from one host to the other.

Ekiga

Ekiga was my second choice since I had used it succesfully in the past.

It was not too bad latency-wise, but the quality of the video was not as good as Empathy (it was smaller and choppier). Also, given that it was running over SIP, it was interfering with my VoIP phone.

Direct peer-to-peer streaming

Given that I wasn't gonna use the voice features of these video-conference tools, I figured that there must be an easy way to just stream video from one peer to the other. That's when I thought of looking into Gstreamer (apt-get install gstreamer0.10-tools on Debian/Ubuntu).

To stream video from my webcam onto port 5000, I ran:

gst-launch v4l2src device=/dev/video0 ! videorate ! video/x-raw-yuv,width=640,height=480,framerate=6/1 ! jpegenc quality=30 ! multipartmux ! tcpserversink port=5000

which is the best I could do within 85 kbps (100-120 kbps is about the maximum reliable synchronous bandwidth I get between New Zealand and Canada):

  • resolution of 640x480
  • 6 frames per second
  • jpeg quality of 30%

On the other computer, I simply ran this to connect and display the remote stream:

gst-launch tcpclientsrc host=stream.example.com port=5000 ! multipartdemux ! jpegdec ! autovideosink

Then I swapped the roles around to also stream video the other way around. That's it: two-way peer-to-peer video link!

Small tweaks to the Gstreamer pipeline

There are quite a few plugins that can be used within Gstreamer pipelines.

If you have problems with autovideosink refusing to load (I did on one of the two computers), you can also install the gstreamer0.10-sdl package and replace autovideosink with sdlvideosink:

gst-launch tcpclientsrc host=example.com port=5000 ! multipartdemux ! jpegdec ! sdlvideosink

Another change I had to make on one of the machines was to flip the image coming out of the webcam (which insists on giving me a mirror image instead of acting like a real camera):

gst-launch v4l2src device=/dev/video0 ! videorate ! video/x-raw-yuv,width=640,height=480,framerate=6/1 ! videoflip method=horizontal-flip ! jpegenc quality=30 ! multipartmux ! tcpserversink port=5000

Possible improvements

I got down to about 1-2 seconds of latency, which isn't bad considering the processing to be done and the distance bits have to travel, but I would love to further reduce this.

Using jpegenc was a lot better than theoraenc which added an extra 3-4 seconds of latency. Is there a better codec I should be using?

Another thing I thought of trying was to switch from TCP to UDP. I'm currently using tcpserversink and tcpclientsrc but since I don't care about having a few dropped frames, maybe I should look into the udp and rtp plugins. It seems like it might help but it also seems to be quite a bit more complicated and I have yet to find an easy way to make use of the RTP stack in Gstreamer.

Please feel free leave a comment if you can suggest ways of improving my quick 'n dirty solution.

Some small notes on your empathy section:

  • You don't mention which protocol you using, given you mention google talk i assume you
    re using xmpp

  • On xmpp we only ever use TURN/proxy servers (Note, not STUN, that's not what STUN is for) when we can't make a direct peer-to-peer connection. In all other cases the connect will be peer to peer

  • So if you verified that it did indeed go through the google proxy server instead of direct, the actual question is why? firewalls and/or nasty NATs are the usual answer, running an upnp igd service on your routers can solve most of those issues

  • 6 seconds sound like a lot more then what a proxy server would cause. A common issue is the video encoder buffering too much, do check which video encoder empathy ended up using. If it's x264, try setting tune=zerolatency in /usr/share/empathy/element-properties

As a general remark on improving quick and dirty solutions.. Don't start them, figure out why the existing tools don't work for you, don't re-invent the wheel again.. :)

Comment by Sjoerd Simons

Thanks for dropping by Sjoerd. I was indeed hoping to be able to use Empathy (xmpp protocol) and to avoid writing a custom script.

I saw that the tcp/udp traffic was going through the proxy server (I can almost swear it said "STUN") in the Empathy debugging window. I tried to get it to stop using that but I couldn't find any documentation as to which ports need to be opened/forwarded through NAT routers.

I think it was using Theora but I'll try that x264 setting.

Comment by François

STUN is one of the many tools used, it does a lot of nice tasks, just not the forwarding itself :).

For ports to open/forward, you can't do such a thing, we allocate ports dynamically as needed (forwarding ports is alwyas just a workaround not a solution). In general if you don't explictely forbid UDP traffic things work depending on your nats behaviour. Enabling upnp igd is usually the solution for nasty nats as we can then just request them for ports dynamically..

The best way to ensure whether you're sending direclty p2p is using tcpdump/wireshark and see where the majority of the packets go.. The empathy debug logs can be confusing if you don't know what you're looking for.

Comment by Sjoerd Simons
Hi François, I'm wondering if you every figured out how to use x264. I've been using theora, but a 6 second latency is not very useable. Thanks.
Comment by Myles Megyesi

@Myles

I didn't try h264 because I didn't really believe that it would be 5-6 times more responsive than Theora. But I could be wrong. Have you had any luck with it?

Comment by François

I tried this and it works fine. Could the client be VLCPlayer or any other player. I tried VLCPlayer with http://locahost but does not work.

Would this be possible or do you have any ideas on how the client can be a media player ( windows or linux )

Thanks

Comment by Anonymous