I was looking for a simple free software solution which would allow me to have a video call with someone else (I don't care about sound since I've already got that working through Asterisk) and I ended up writing a Gstreamer-based poor man's videoconf solution because I wasn't satisfied with the other options I considered.
Empathy
Empathy was my first choice since it seems to be the preferred GNOME communication software nowadays.
While the quality of the video was excellent, the latency between New Zealand and Canada was unbearable: a full 6 seconds. I suspect that this is due to the fact that it runs everything through the Google Talk STUN server and I couldn't find how to force it to go directly from one host to the other.
Ekiga
Ekiga was my second choice since I had used it succesfully in the past.
It was not too bad latency-wise, but the quality of the video was not as good as Empathy (it was smaller and choppier). Also, given that it was running over SIP, it was interfering with my VoIP phone.
Direct peer-to-peer streaming
Given that I wasn't gonna use the voice features of these video-conference tools, I figured that there must be an easy way to just stream video from one peer to the other. That's when I thought of looking into Gstreamer (apt-get install gstreamer0.10-tools on Debian/Ubuntu).
To stream video from my webcam onto port 5000, I ran:
gst-launch v4l2src device=/dev/video0 ! videorate ! video/x-raw-yuv,width=640,height=480,framerate=6/1 ! jpegenc quality=30 ! multipartmux ! tcpserversink port=5000
which is the best I could do within 85 kbps (100-120 kbps is about the maximum reliable synchronous bandwidth I get between New Zealand and Canada):
- resolution of 640x480
- 6 frames per second
- jpeg quality of 30%
On the other computer, I simply ran this to connect and display the remote stream:
gst-launch tcpclientsrc host=stream.example.com port=5000 ! multipartdemux ! jpegdec ! autovideosink
Then I swapped the roles around to also stream video the other way around. That's it: two-way peer-to-peer video link!
Small tweaks to the Gstreamer pipeline
There are quite a few plugins that can be used within Gstreamer pipelines.
If you have problems with autovideosink refusing to load (I did on one of the two computers), you can also install the gstreamer0.10-sdl package and replace autovideosink with sdlvideosink:
gst-launch tcpclientsrc host=example.com port=5000 ! multipartdemux ! jpegdec ! sdlvideosink
Another change I had to make on one of the machines was to flip the image coming out of the webcam (which insists on giving me a mirror image instead of acting like a real camera):
gst-launch v4l2src device=/dev/video0 ! videorate ! video/x-raw-yuv,width=640,height=480,framerate=6/1 ! videoflip method=horizontal-flip ! jpegenc quality=30 ! multipartmux ! tcpserversink port=5000
Possible improvements
I got down to about 1-2 seconds of latency, which isn't bad considering the processing to be done and the distance bits have to travel, but I would love to further reduce this.
Using jpegenc was a lot better than theoraenc which added an extra 3-4 seconds of latency. Is there a better codec I should be using?
Another thing I thought of trying was to switch from TCP to UDP. I'm currently using tcpserversink and tcpclientsrc but since I don't care about having a few dropped frames, maybe I should look into the udp and rtp plugins. It seems like it might help but it also seems to be quite a bit more complicated and I have yet to find an easy way to make use of the RTP stack in Gstreamer.
Please feel free leave a comment if you can suggest ways of improving my quick 'n dirty solution.
Some small notes on your empathy section:
You don't mention which protocol you using, given you mention google talk i assume you
re using xmpp
On xmpp we only ever use TURN/proxy servers (Note, not STUN, that's not what STUN is for) when we can't make a direct peer-to-peer connection. In all other cases the connect will be peer to peer
So if you verified that it did indeed go through the google proxy server instead of direct, the actual question is why? firewalls and/or nasty NATs are the usual answer, running an upnp igd service on your routers can solve most of those issues
6 seconds sound like a lot more then what a proxy server would cause. A common issue is the video encoder buffering too much, do check which video encoder empathy ended up using. If it's x264, try setting tune=zerolatency in /usr/share/empathy/element-properties
As a general remark on improving quick and dirty solutions.. Don't start them, figure out why the existing tools don't work for you, don't re-invent the wheel again..
Thanks for dropping by Sjoerd. I was indeed hoping to be able to use Empathy (xmpp protocol) and to avoid writing a custom script.
I saw that the tcp/udp traffic was going through the proxy server (I can almost swear it said "STUN") in the Empathy debugging window. I tried to get it to stop using that but I couldn't find any documentation as to which ports need to be opened/forwarded through NAT routers.
I think it was using Theora but I'll try that x264 setting.
STUN is one of the many tools used, it does a lot of nice tasks, just not the forwarding itself :).
For ports to open/forward, you can't do such a thing, we allocate ports dynamically as needed (forwarding ports is alwyas just a workaround not a solution). In general if you don't explictely forbid UDP traffic things work depending on your nats behaviour. Enabling upnp igd is usually the solution for nasty nats as we can then just request them for ports dynamically..
The best way to ensure whether you're sending direclty p2p is using tcpdump/wireshark and see where the majority of the packets go.. The empathy debug logs can be confusing if you don't know what you're looking for.
@Myles
I didn't try h264 because I didn't really believe that it would be 5-6 times more responsive than Theora. But I could be wrong. Have you had any luck with it?
I tried this and it works fine. Could the client be VLCPlayer or any other player. I tried VLCPlayer with http://locahost but does not work.
Would this be possible or do you have any ideas on how the client can be a media player ( windows or linux )
Thanks