An avian carrier's blog – VoIP

As an experienced programmer, I participate in many Free Software projects when time permits. I am committed to a few projects, and I frequently submit patches to random projects that I happen to bump into. I also understand the dynamics of free software: when a bug stands in my way, I often fix it myself rather than waiting for another contributor (who may have her own priorities and agenda) to fix it. Same when I badly need a feature.

In this post, I will compare the submission process of two changes I made to free software recently:

• a new watchdog driver for the Linux kernel;
• a fix for a critical flow in SIP message handling in the Asterisk telephony system.

Linux device driver

I first posted my new device driver code as a patch (a difference between the actual Linux source code and the modified one) on the linux-kernel mailing-list. Shortly after that, some people publicly answered my mail and offered remarks and criticisms about my changes. Most of the advices were well targeted and I modified my patch accordingly. Some of the remarks were a bit off because people commenting the code hadn’t read the device datasheet and were confused by some names used therein and mirrored into the driver; I explained the situation and why I would not act upon those remarks. One point about a possible concurrent access was discussed and resolved after a few technical exchanges. I then posted a modified patch for everyone to comment on. This later patch was then acked (i.e., blessed) by a major developer.

Various parts of the Linux kernel are maintained by different people. The device I was addressing was a watchdog (a piece of hardware that forcibly reboots your computer if the operating system fails to say “I’m still alive” on a regular basis), so the watchdog subsystem maintainer took responsability and integrated it into his own development tree, so that people willing to test this new driver could do so easily. After some time, while the new driver had shown not visible disturbance of the rest of the kernel, it was pulled by Linus Torvalds into the main Linux kernel tree and was released as part of Linux 2.6.19.

Note that when the watchdog subsystem maintainer integrated my new driver into his tree, he was already quite confident that the driver was clean as it had been carefully read and commented on by several other developers. The integration within his tree rather than into the main Linux kernel ensured that all the watchdog drivers can play nicely together.

Asterisk flaw in the SIP engine

Free Telecom is the second most important ADSL provider in France. They provide a triple-play service over ADSL: IP, telephony and television. The telephony service can be accessed either using an analog phone connected to their ADSL modem or using a SIP connection to their server. On the server side, Free Telecom chose to use a solution by Cirpack, made from boxes able to handle several thousands of simultaneous SIP sessions.

When the Cirpack server was upgraded at the beginning of December, all Asterisk boxes using Free Telecom as their SIP provider immediately stopped working: the voice was not going through anymore. This problem was signaled onto a forum by an Asterisk user a few hours after the upgrade and promptly analyzed by a Cirpack engineer: it appeared to be a flaw in Asterisk SIP handling. The engineer rolled back the Free Telecom server to the previous revision and sent me a mail with the description of the problem. Why me? Because we know each other as we studied together, and he knew I was using Asterisk to connect to the Free Telecom SIP server and that I was likely to quickly investigate and fix the problem.

A few hours later, I produced two short fixes for Asterisk and was able to test them against a Cirpack server running the new firmware. Everything went fine and the problem was fixed. I posted the patches to the Asterisk bug tracking system and, less than four hours later, added full debugging information with and without the patches at the request of a manager so that it was clear what the problem was and how the patch fixed it.

I also sent several mails on the Asterisk developers mailing-list to underline the importance of the flaw. As long as the flaw is not fixed, any upgrade made by a VoIP provider may break all its Asterisk clients without any easy workaround. To describe the flaw shortly, an unpatched Asterisk doesn’t understand perfectly valid SIP headers and interprets them in a totally wrong way, causing the subsequent traffic to be sent to the wrong place.

Asterisk 1.4.0 was released 19 days after I explained this critical flaw and posted the patches to correct it. Not only were the patches not included in the release, but as far as I can tell no peer review has occurred on the patches. The only request made by a manager was that some developers, who have not yet answered, test the patch.

Also, at some point, this very same manager added a relationship between this problem and another one without any comment to explain this alleged relationship. As far as I can tell, the two bugs are totally unrelated and I fail to see any relationship between them except that they address two problems in SIP message processing, although one is about SIP headers syntax and the other one about the SIP engine internal state machine.

At this point, it is worth noting that I do not feel bad about Asterisk because my patches were not included in the latest release; what I criticize here is what I consider a lack of feedback on user-contributed fixes and a lack of interaction between developers.

Comparing the two processes

Proposed changes to the Linux kernel are posted on a public mailing-list as plain-text, where anyone is free to comment on them. The plain-text format makes it easy to intersperse the relevant code portion with the comments. One or several structured discussions follow, each one addressing one aspect of the proposed patch. New versions of the patch may then be proposed and discussed until the patch is finally blessed (acked) by one or more fellow developers. Note that this process happens in an email client, without any compilation taking place at this stage. Technical flaws may be found by code reading and discussion rather than by testing whether the code seems to trigger a bug or not. Also, if the code would benefit from extra documentation, such documentation will be requested publicly by other developers.

Proposed changes to Asterisk are posted onto the Asterisk bug tracking system maintained by Digium (the original authors and the current maintainers of Asterisk). A disclaimer also needs to be filled by contributors, as Digium wants to be able to make a proprietary version of Asterisk, while others may only distribute it as a GPL software. I have the impression that the patches are not peer reviewed: the use of a bug tracking system doesn’t ease such a code review process, compared to a mailing-list as in the Linux kernel patches case. I am also under the impression that patches are tested rather than being read first. If enough developers report that the patch hasn’t visibly broken their system, the patch may eventually be integrated.

Also, parts of Asterisk sometimes undergo major rewritings without any attempt to explain what has been changed exactly. For the Linux kernel, it would be unacceptable: a serie of incremental patches would be required to be submitted on the mailing-list, with a step-by-step justification of why things need to be changed. When incremental patches are not doable, because changes depend on each other, separate patches that need to be applied at the same time will still be required so that individual changes are reviewable by other developers.

As you may have guessed at this stage, I much prefer the Linux kernel way of doing it. The peer review system exposes proposed changes to several pairs of hackers eyes. The patches and the subsequent discussions also teach potential contributors what they need to send and how they need to present it. This iterative process not only generates better code but also shows good practices to other programmers.

I would really like other large software projects, such as Asterisk, to adopt it to increase the code quality and the developers interaction.

Comme je l'avais expliqué dans Asterisk - build your own PBX, la prise téléphonique de ma Freebox était connectée à mon PC par une interface analogique de type FXO. Sur le PC, qui tourne sous GNU/Linux, l'autocommutateur libre Asterisk gère mes communications et mes services. Tout fonctionnait correctement, même si la reconnaissance du raccroché du correspondant était parfois (mais très rarement) un peu aléatoire.

Aujourd'hui, Free a ouvert l'accès en SIP à son service de téléphonie. Cela signifie que j'ai pu connecter Asterisk au service téléphonique de Free (appelé freephonie) en IP, sans passer par la ligne de téléphone analogique. Cela a instantanément supprimé l'écho qui survenait en début de conversation (avant le réglage automatique de l'annulateur), et les détections d'état pendant la communication sont parfaites.

Pour aider ceux qui voudraient faire la même chose, voici un extrait de mon fichier sip.conf :

[general]
defaultexpirey=1800
dtmfmode=auto
qualify=yes

register => NuméroDeTéléphoneFreebox:MotDePasseSIPFree@freephonie.net

[freephonie_outbound]
type=peer
allow=all
host=freephonie.net
secret=MotDePasseSIPFree
fromuser=NuméroDeTéléphoneFreebox
username=NuméroDeTéléphoneFreebox
qualify=yes
fromdomain=freephonie.net

[freephonie.net]
type=peer
context=fromfree
host=freephonie.net
qualify=yes
allow=all
deny=0.0.0.0/0.0.0.0
permit=212.27.52.5/255.255.255.255

Quelques remarques :

• Vous obtiendrez votre mot de passe SIP Free dans l'interface de gestion de votre compte sur http://adsl.free.fr/.

• Il est possible que, dans le futur, je doive changer le numéro IP du serveur de Free ou en autoriser plusieurs. En attendant, cela limite les possibilités d'appels intempestifs.

• Il faut augmenter l'expiration à 1800 secondes. Asterisk ne semble pas comprendre le serveur SIP de Free lorsque celui-ci lui indique, et il tente de s'enregistrer avec la durée d'expiration par défaut qui est de 120 secondes.

• Le contexte freephonie_outbound est celui utilisé pour les appels sortants, freephonie.net celui pour les appels entrants. Dans mon cas, les appels entrants sont aiguillés vers le contexte fromfree, extension s. Le contexte doit être défini dans le fichier extensions.conf.

• L'ordre de déclaration des deux entrées SIP est important, la dernière correspondant à un host donné l'emportant lors d'un appel entrant.

Several people asked me to describe my home PBX installation, based on the Asterisk Open Source PBX in order to build their own.

Infrastructure

My home network is made of several devices; only those related to VoIP (voice over IP) are represented here:

My home phone line comes from France Telecom, the historical PSTN (Phone Switched Telephony Network) provider in France. When it arrives in France Telecom building (not represented here), my line is splitted in two parts: lower frequencies go to France Telecom equipment for voice processing while higher frequencies are distributed to equipment belonging to Free Telecom, my ADSL provider.

Free Telecom is an innovative player in the ADSL world. They distribute more than IP over ADSL. They have built their own ADSL modem, called the Freebox, which is lended to the clients. The Freebox can not only transport IP over ADSL, it also delivers an independent phone line (which itself uses VoIP internally on the Free Telecom network) as well as television. Not everyone can get television, it depends on the equipment installed by Free Telecom in France Telecom premises and on the line quality. As far as I am concerned, I get around 6Mb/s of download bandwidth and 400kb/s of upload bandwidth.

On the Free Telecom phone line, one can call any French landline number for free with illimited time. Also, the incoming number attributed to you by Free Telecom (you can even choose it) can be called from any France Telecom landline at a local rate, from anywhere in France.

This is all great, but two phone lines (France Telecom and Free Telecom) mean two physical telephones. Two voice mail systems. That’s why I installed Asterisk, a free software PBX system released under the GNU General Public License on one of my computer running the free Debian GNU/Linux operating system.

Hardware

I bought two DIGITX100P cards from DigitNetworks to interface my Asterisk PBX with the physical phone lines. Although the Free Telecom line uses VoIP while in the Free Telecom network, it can only be accessed through a regular phone plug if you use the Freebox. I have seen one report of someone using it directly with a MGCP client with a regular ADSL modem, but in this case you loose the Freebox other benefits such as TV.

Also, I bought two Grandstream Budgetone 100 phones. Those are IP phones; they connect to Asterisk using the SIP protocol. At this time, I only use one of them, I have yet to bring the network into the bedroom to use the other one (although I am not sure this is a good idea to have the network in the bedroom at all).

That’s about everything you need to build your own phone system. If you do not want to buy IP phones, you can either look for FXS interfaces allowing you to use regular phones or use softphones, computer programs that let you phone using a mike and headphones.

Placing an outgoing call

The goal of this system was that anyone could use it while staying at my place. I didn’t want people to do anything special to be able to place a phone call.

If someone dials the phone located on my desk, the phone will transmit the dialed number to Asterisk. Asterisk will first look whether it can reach that phone number using VoIP:

• if the phone number is registered in the ENUM database, the published route will be used; for example, if someone uses this system with any of my phone numbers, he will automatically gets redirected to my Asterisk PBX without using any phone operator
• several providers are tried, as some of them offer free call terminations in some countries or for some categories of numbers; for example, if I call a US toll-free number, I get several routes that I can use for free. That means that even from France, I can call US toll-free numbers without paying a dime. Moreover, the callee will not pay anything more than if the call had been placed from within the USA
• if I really have to, one of my phone lines will be used, depending on their availability and on the number dialed; Free Telecom is often the preferred choice, although at some times in the day it is cheaper to use France Telecom to call French cell phones

The following picture shows (in green) a call placed on the IP phone at my place going out on the France Telecom line.

The next one shows a call for which a VoIP route has been located and is being used. The call may reach a real PSTN phone at some other places, or may reach an IP phone if the target user uses VoIP.

Incoming calls

Incoming calls can arrive in three different ways: the France Telecom line, the Free Telecom line or using VoIP. My firewall and router box (a lovely Soekris Net4801 disk- and fan-less PC running the free sofware OpenBSD operating system from a compact flash memory card) has been instructed to redirect incoming VoIP packets to my Asterisk PBX. So at the end, incoming calls are handled at the same place and will enter the same processing loop (with different parameters such as an identification of the incoming line).

The Soekris uses OpenBSD packet filter to provide QoS (Quality of Service) over my Internet connection. VoIP packets will get out first as long as there are no more than four simultaneous conversations. After that, the bandwidth will be shared with other applications to prevent a DoS (Denial of Service) attach on my VoIP server.

Depending on the caller ID, different actions may be taken. First of all, if the caller ID is unknown and the call came from the France Telecom line (the only one present in directories), the caller will be presented with a IVR (Interactive Voice Response) system. He will hear some messages asking him to press a certain key on his phone keyboard if he is not a telemarketer. If he confirms, he still needs to type my birth date to get through. Otherwise, Asterisk hangs up. This may sound harsh, but this has not been a problem so far (at least noone told me it was one), and I’ve screened probably many telemarketers phone calls.

Then Asterisk tries to present the call to me. In order to do that, it tries to reach simultaneously:

• my IP phone at home
• a VoIP softclient on my laptop, running Lunar Linux
• any SIP or IAX softphone I could have registered from anywhere

The following pictures illustrates the case where a call comes from the PSTN through the Free Telecom phone line, goes to Asterisk, and goes out using VoIP to a phone registered from a remote location. The red lines show failures to either contact the client or get it to go off-hook.

Asterisk goodies

Asterisk by being a free software is easily expandable and benefits from a very active developpers community. The software is already excellent but gets better every day.

One feature I use a lot is AGI (Asterisk Gateway Interface), a communication mechanism allowing one to use any programming language to extend Asterisk. A simple textual protocol is used to exchange meaningful information between Asterisk and the module. For example, my whole dial plan logic has been coded using the Python programming language. My DISA (Direct Inwards System Access) is also coded in Python; it allows me to call home, authenticate myself, and do just as if I were there. For example, if I need to call abroad from my cell phone, it is much cheaper for me to call home and redial from there (especially if you consider that I often find free providers for the countries I call much such as the USA).

Asterisk also comes with integrated applications such as a powerful voice mail system (although you can write your own using AGI), DISA, IVR, text-to-speech and so on. Do not hesitate: build your own PBX system; you don’t have to buy any equipment to do that, you can first try to make a full VoIP system using free soft phones. Just do it. But be careful: you may be hooked very fast.

(you can also have a look at this blog entry)

Update: on 2006-05-16, Free Telecom opened their SIP server. It means that it is no longer necessary to use a FXO card with them. Log into your Free ADSL account and configure your SIP account.