An avian carrier's blog – Android

J'ai été agréablement surpris de remarquer aujourd'hui, notamment après mes plaintes contre l'interface volontairement restreinte offerte sur le web, une application pour Android appelée Mon Suivi La Poste développée par « La Poste mobile ». Curieux, et destinataire de nombreux colis en transit en cette période de fêtes, j'ai installé et essayé l'application. Or, systématiquement, j'ai obtenu l'erreur suivante, y compris lorsque j'étais convaincu d'être parfaitement connecté :

La qualité des réseaux environnants est momentanément insuffisante pour vous permettre d'accéder à nos services mobiles. Nous vous invitons à reessayer ultérieurement. L'équipe de La Poste.

Curieux, et en application de l'alinéa III de l'article L122-6-1 du Code de la Propriété Intellectuelle, j'ai cherché à connaître les appels faits par mon mobile sur demande de l'application afin de voir quel composant, dans mon système CyanogenMod, faisait échouer ces requêtes. Il s'avère que l'appel fait ressemble à cela (j'ai remplacé le numéro de suivi réel sur 13 caractères par « NUMERO_DE_SUIVI » :

GET /outilsuivi/web/suiviInterMetiers.php?key=d112dc5c716d443af02b13bf708f73985e7ee943&method=xml&code=NUMERO_DE_SUIVI
  HTTP/1.1
User-Agent: Dalvik/1.5.2 (Linux; U; Android 2.3.1)
Host: www.laposte.fr
Connection: Keep-Alive
Accept-Encoding: gzip

Notons que la requête est faite sans chiffrement (utilisation de HTTP et pas de HTTPS) ce qui permet à tous les intermédiaires de l'examiner (opérateur mobile, fournisseur du WiFi). Un simple analyseur de protocole comme Wireshark permet d'examiner la requête ci-dessus ainsi que la réponse associée :

HTTP/1.1 200 OK
Date: Sat, 17 Dec 2011 11:07:13 GMT
Server: Apache
Cache-Control: no-cache, must-revalidate
Expires: Sat, 26 Jul 1997 05:00:00 GMT
Content-Type: text/xml
Content-length: 497
Connection: Keep-Alive
Set-Cookie: lb_outilsuivi_new_pf=balancer.route2; path=/;

<?xml version='1.0' ?>
<response>
 <status><![CDATA[1]]></status>
 <code><![CDATA[NUMERO_DE_SUIVI]]></code>
 <client><![CDATA[Particulier]]></client>
 <date><![CDATA[16/12/2011]]></date>
 <message><![CDATA[Votre colis est arrivé sur son site de distribution]]></message>
 <gamme><![CDATA[4]]></gamme>
 <base_label><![CDATA[Coliposte]]></base_label>
 <link><![CDATA[http://www.coliposte.net/particulier/suivi_particulier.jsp?colispart=NUMERO_DE_SUIVI]]></link>
 <error><![CDATA[]]></error>
</response>

Tout à l'air correct de mon côté, même si l'application considère qu'il y a une erreur :

• L'URL de base est http://www.laposte.fr/outilsuivi/web/suiviInterMetiers.php avec des paramètres supplémentaires :

  • code est le numéro de suivi du colis, sur 13 caractères.
  • method, qui contient ici xml, est la méthode de codage du résultat à utiliser, on peut penser que json donnerait un résultat en JSON plutôt qu'en XML. D'après ce qu'on peut voir sur le site web, image doit renvoyer une image.
  • key est probablement un identifiant de l'application Android. Mon navigateur utilisait initialement un User-Agent non-standard, il semblerait que Dalvik doive être présent pour que la requête fonctionne.

• La réponse contient un certain nombre de champs permettant le suivi du colis :

  • status contient « 1 » si tout va bien est est vide en cas d'erreur (numéro de colis inconnu par exemple).
  • code reprend le numéro de suivi transmis.
  • client contient « Particulier » dans mon cas, je ne connais pas les autres valeurs possibles, mais cela semble logique.
  • date contient la date de l'événement décrit en cas de succès, est vide sinon.
  • message contient le message décrivant l'événement ou celui décrivant l'erreur.
  • gamme contient « 4 » pour moi dans tous les cas.
  • base_label contient « Coliposte » pour moi en cas de succès, est vide sinon.
  • link contient le lien vers le suivi sur le site web de Coliposte en cas de succès, est vide sinon.
  • error est vide en cas de succès, et contient une chaîne en anglais telle que « error_invalid_code » en cas de numéro de suivi inconnu ou « error_nb_chars » si un numéro de suivi ne comportant pas 13 caractères est utilisé. La liste des codes d'erreur et leur explication en français est donnée dans ce fichier Javascript fourni par La Poste.

Au vu de ces constatations, je dois en déduire que c'est l'application « Mon Suivi La Poste » qui est fautive en ne comprenant pas la réponse, pourtant apparemment correcte, renvoyée par le serveur.

People programming for the Android platform know that you are not supposed to perform lengthy operation in the user interface task. You must create a background task, which will interact with your user interface task through a Handler, or you will use a pre-built AsyncTask to ease the interaction.

However, if you chose to use Scala instead of Java to build your Android application, you can easily take advantage of its functional nature to build much more exciting constructs to handle this mandatory parallelism. For example, while developing a piece of code to retrieve the state of my city shared bike stations from the network, I wrote a little piece of utility code based on two functions each taking an expression to run: onBG executes something in a background task, and its companion onUI executes something back in the user interface task. Also, when calling onUI, the result of the given expression can later be used from the background task; if needed, it will block until this result is available.

Here is a short example showing the loading of the bike stations list from the network, then the loading of the state of the first five stations. A progress dialog, part of the user interface, indicates the current state of the background operation. Compare to what you would have needed to write in Java.

    // Execute lengthy loading from the network in a background (BG) task as to not
    // block the user interface (UI) task.
    this.onBG {
      // Create a progress dialog on UI task, and return a pointer to it which, if
      // used, will block as long as needed until the expression has terminated.
      // Otherwise, do not block.
      val progressDialog = this.onUI(ProgressDialog.show(this, "Loading stations",
                                                         "Loading list of stations"))
      try {
        // Load a list of stations from the network (long operation) and keep only
        // five of them.
        val stations = Station.loadFromNetwork.take(5)

        // Change the progress dialog so that now it can be used to indicate
        // the progress while station information is loaded from the network.
        // This has to be done on the UI task. Note that this will wait until
        // the progress dialog creation has terminated if needed, as we use
        // the result of the previous "onUI" call.
        this.onUI {
          progressDialog.setIndeterminate(false)
          progressDialog.setMax(stations.size)
        }

        // For every station…
        for ((station, index) <- stations.zipWithIndex) {
          // …update the progress dialog on the UI task…
          this.onUI {
            progressDialog.setProgress(index)
            progressDialog.setMessage("Loading station details\n" + station.name)
          }
          // …and load its availability from the network.
          station.reloadAvailability
        }
      } finally {
        // Whatever the completion is (regular or exception), dismiss the
        // progress dialog.
        this.onUI(progressDialog.dismiss)
      }
    }

Why did I write those two functions? Just because, in Scala, I can.

Don't wait: try Scala today.

Every year since 2003, Alexis Polti and myself run a course named "ROSE" (Robotique et Systèmes Embarqués, Robotics and Embedded Systems) for future engineers at Télécom ParisTech. During this 120 hour curriculum, students have to design and buid embedded systems, including designing their own electronic boards and programming them. Classical courses are limited to the minimum (real-time operating systems, signal integrity), and students must learn by themselves all the other topics while the two teachers offer lots of assistance (we are physically with the students most of the time to answer their questions).

As every year, the 2011 occurrence introduced some changes (hopefully for the better), that I now want to analyze.

Afterthoughts

Git vs. Hg

Until last year, we were using Mercurial as our revision control system because we thought it was simpler to use than Git for the students although the teachers both used it. We decided to try Git with the Gitolite backend tool that we already used for research projects. The outcome was unexpectedly successful: every project used lots of branches for their development, merging and rebasing at will.

The presence of Clément Moussu, a student who had previously done an internship at Gostai where Git is used intensively (they even use "git notes" that almost noone knows about), has been a tremendous help, and has been acknowledged during the debriefing session by other students. He and three other students explained Git to the others, and spoke about the best practices right from the beginning. So we plan to keep using Git as our preferred revision control system.

Linux-based boards

For the first time, we accepted that some projects use Linux-based boards in addition to the micro-controllers boards they had to design. The mix of those Linux-based boards (one Armadeus APF27, one BeagleBoard xM and one Gumstix Overo FE COM) allowed them to use high-level languages (Python), libraries (OpenCV, 0MQ) and cloud-based processing capabilities (Google Appengine) very easily. We plan to keep this possibility as well, but we need to ensure that every project needs to build additional micro-controller based boards as we want our students to really know how to design a board from scratch.

Best programming practices

This is something we did not do: ensure that our students know the best programming practices. Next year, we plan to do a live-coding session where we will collectively try to write the best possible code. The teacher will write, compile and run the code as suggested by the students, and explain how the code may be improved and what needs to be done to guarantee reliability and ease of maintenance. Tricky exercises will be proposed, to ensure that students need to know when volatile needs to be used, and when it is not needed. Also, lockless algorithms, depending on the underlying hardware, will be used whenever possible. The effect of inlining some functions (and when not to inline and let the compiler work it out) will be studied intensively, and methods to avoid any code duplication will be taught.

Some students naturally know how to write good code, but some don't and just write code that works but is unmaintainable. Instead of having them fix the code afterward, we will make sure that they write proper code from day one. So next year, the students will learn this skill at the beginning of the class rather than along its course.

The projects

If you are curious to see what has been done, here are links to the various projects done by the students in 2011 during this 2.5 month course:

• Casper: a talking and listening robot shaped like an elephant trunk
• Copterix: a helicopter with eight engines
• IRL: a nightclub laser that displays your tweets and let you control via Twitter the color of the club as well as other equipment such as the smoke machine
• MB Led: a very nice set of blocks letting you play games by moving them around
• Rosewheel: a Segway clone, remotely controlled using an Android phone
• TSV Safe Express: control a model railroad layout using cheap components (unfortunately, the web site is almost empty)

In a misinformed article about Android libc and the Linux kernel, Florian Mueller seems to attempt to create buzz about an alleged Linux kernel copyright infrigement. Even if Linux kernel copyright holders decided to complain (which is not the case as far as I know), the article is full of mistakes and approximations.

In view of all of that, I think the only viable option will be for Google to recognize its error with Bionic and to replace it as soon as possible with glibc (GNU C library). That library is licensed under the LGPL ("Lesser GPL"), which has the effect that applications can access the Linux kernel without necessarily being subjected to copyleft if certain criteria are fulfilled.

The GPLv2 licence used in the Linux kernel does not allow to reuse parts of the covered software and distribute it under the LGPL license.

Using glibc is the industry-standard approach, and it is the approach used by those in the open source world who are trying to "play by the rules."

Come on, many embedded Linux systems use the more compact µClibc. Aren't they playing by the rules?

In fact, Google's decision to forego glibc is one of the reasons Android is considered a Linux fork rather than a true Linux implementation.

Wrong. Several distributions use other C libraries, including the eglibc which is a fork of the glibc. Is Debian a Linux fork because it is now using eglibc instead of glibc? Certainly not!

Android is sometimes considered a Linux fork because some of the features it needs related to the device wakeup when a call arrives for example have disturbed quite a lot of device drivers code, and the Linux mainstream maintainers do not feel comfortable in including those changes in the default kernel.

Florian Mueller, while trying to make some noise, completely dismisses the fact that Google wrote its own libc (based on a preexisting one using a BSD license) to get better performances. Accessing kernel structures directly instead of trying to build a set of insulation layers allows them to use the Linux kernel more efficiently. As stated on the GNU project web site, The GNU C library is primarily designed to be a portable and high performance C library. It follows all relevant standards (ISO C 99, POSIX.1c, POSIX.1j, POSIX.1d, Unix98, Single Unix Specification). This is just not needed in a restricted capability device where you do not necessarily need to implement or even comply with all the existing standards.

Even if copyright holders chose to complain, the solution proposed by Florian Mueller is completely misguided, unrealistic and useless. Even if I somewhat did it by writing this post, please do not feed the troll.

Geocaching is a worldwide outdoor treasure hunt game in which you have to locate and find hidden tokens by using a GPS receiver. Not only is this game a good way of discovering new places, it may also contain real brain teasers if you have to solve puzzles to find the right coordinates. I have been enjoying this activity since nearly 8 months now, and have already discovered more than 100 geocaches in 11 different countries.

Unfortunately, the biggest geocaches database is located on the geocaching.com web site. This site does not offer any API letting you access your own data, let alone information on geocaches. If you happen to be a paying member (as I do), you can generate what they call pocket queries: those are database runs, triggered manually or at scheduled times, listing basic geocaches information for a limited area or around a given path. Those pocket queries must then be transferred to your GPS device or your smartphone. When you want to log the fact that you have discovered a cache and let a comment to the geocache owner, you have to once again log onto the web site and do it manually. The terms of use prohibit you from using any robot to automate those tasks.

A SOAP API exists, but its access is reserved to trusted partners (read: paying partners who themselves charge a recurring fee to their users). An official Android application has been developped by Groundspeak, the owner of geocaching.com, but it is priced much higher than typical Android applications and has much less features than c:geo, a wonderful rogue application which scrapes the geocaching.com web pages on your behalf. Also, since it parses HTML page in order to extract the needed information, c:geo needs to be updated each time the web page presentation gets changed.

This excellent post from Scot Hacker's Foobar Blog hits the nail on the head, and what was written in 2008 is still valid in 2011:

This blew my mind. The culture of the site is so web 1.0 that a basic question about interoperability was met with distrust. Not only is geocaching.com lacking the technology it needs to enter the web 2.0 world, it’s lacking the culture needed to support it. In 2008, interoperability between sites needs to be encouraged, not discouraged. Sad that geocaching.com’s traditional closed-ness has created this kind of culture.

[...]

The irony is that geocaching.com relies so heavily on the open APIs provided by Google and other mapping services, but provides no open-ness back to the web in return. Imagine using geocaching.com without the map mashups integration – it would be nearly impossible. One would think that the folks at geocaching.com would see their own mashups as an example of the great ideas that bloom when datasets and APIs are open and shared.

But the world of geocaching may be changing soon: Garmin, the well-known GPS receiver maker, recently announced the launch of opencaching.com. I hope that many geocachers will register their caches and their finds on this site, and that many applications will take advantage of their publicly usable API. On Android, Cache Me seems to be the first application using opencaching.com data. I hope that c:geo will soon be able to use those liberated bits as well.

I have been intending to write this post for some time now. I do not necessarily like "top Firefox extensions"-like posts, but I sometimes stumble upon a gem which I could not live without after trying it. Here is a list of Mozilla Firefox extensions I install on every computer I use regularly.

Vimperator

Vimperator adds vim-like key bindings to Firefox. My Firefox (always running in full-screen mode) does not have any more toolbar consuming precious screen space. Quickmarks let me bookmark my favorite sites and go there with three key presses, either in the current tab or in a new one. Also, I seldomly need to use the mouse, as I can highlight hyperlinks and jump there immediately. Of course, Vimperator is scriptable, comes with its own plugins written in Javascript and let you search the web very easily.

For example, :open rfc1149 (or o rfc1149) will search for rfc1149 on Google while :open wikipedia rfc1149 will do the same thing in Wikipedia. :tab addons will open the Firefox extensions page in a new tab. gt will go to the next tab. b mail will jump to the first tab with mail in its title.

I hope that 2011 will bring us an even better Vimperator 3.

Password Hasher

Password hasher lets you remember a single master password and still use a different password on every site you have to register with. Considering that even the most reputable sites sometimes leak password databases, it keeps you safe by not reusing the same password on different sites.

Certificate Patrol

Certificate Patrol warns you when the certificate of a trusted web site change, and tells you if you should look twice before using the site. For example, the use of a new certificate authority may reveal that you are currently the target of a man-in-the-middle attack. Most of the time, such changes are innocuous, but if one day you notice that the allegedly new Google HTTPS certificate is signed by a company in a totalitarian country you'll be happy to have Certificate Patrol warn you.

Dafizilla ViewSourceWith

Stéphane Bortzmeyer recommended this extension to me almost four years ago (I was previously using the "It's All Text!" extension) and I will never go back. Launching GNU Emacs on any text field where I have to edit long text is much more comfortable than using Firefox limited editing capabilities.

Shareaholic

Shareaholic lets you share any web page to multiple places (Google Reader, Facebook, Twitter, etc.) and does so by directly using the native third-party sites capabilities. It means that you do not to create a new account on a new web site to use this service.

Lazarus: Form Recovery

Did you ever need to fill a lengthy form and have the web site clear it completely because one field was wrong or missing? Did you ever close Firefox by mistake while in the middle of submitting a multiple-pages form? If this is the case, you should install Lazarus, which brings your text back. Lazarus saves your form content securely using Firefox security manager (you did define a master password, didn't you?).

FoxToPhone

If you happen to have a phone running Android 2.2 or newer, this extension based on ChromeToPhone lets you send links, maps, images or text directly from your browser to your phone. The phone must have the Google Chrome to Phone application installed.

I'll follow Dave Ray and will try to say something nice about a bunch of programming languages I use or have used seriously:

• Ada – The only language I would trust my life to.

• C – It gets things done easily in a controlled space when resources are scarce. I use it in many embedded situations, often with FreeRTOS.

• C++ – Its templating system with specialization beats everything I know. When I worked on Urbi at Gostai, I had a lot of pleasure using it.

• Erlang – The language to use to develop distributable parallel applications. I wrote many programs for research projects with it.

• Factor – One of the languages I feel the most comfortable with. I really like the reverse polish notation and the powerful combinators. I use it for many personal and teaching projects.

• Forth – Forth is one of the languages that I have been liking since the first time I heard about it. Its conciseness, simplicity, grammar and ease of implementation beats almost everything when it comes to size on very small embedded systems. I used it to write a Forth compiler targetting the Microchip PIC16Fxxx microcontrollers family.

• Haskell – I started using it when I had to send patches for Darcs. I really love monads, and I also love explaining them in class. My window manager configuration is also written in Haskell.

• J – It is unbeatable if you have RSI and need to type as little characters as possible for a task that can be applied to a whole array. I use it mostly to solve Project Euler problems.

• Java – Well, everyone knows it so it may be used to explain a simple concept. Is that nice enough?

• Javascript – Javascript lets us do things in the browser I would not have imagined five years ago. For example, this web page is static but includes Twitter updates and comments, thanks to Javascript. On the server side, I use it within a CouchDB database where I store a whole web application; it dynamically generates iCalendar views for multiple people from data gathered at TVrage.com using their XML API.

• Python – I can hack anything in a few minutes and still be able to read it later. I wrote a Forth compiler for the Microchip PIC18Fxxx microcontrollers family with it.

• Ruby – Feels like Python, only more functional and cleaner. I would use it more if I had not been bitten by threading unstability on Sparc64Linux in the past (for the will-spam-for-food.eu.org service we ran with Pierre Beyssac and Thomas Quinot). Ruby helps me run this blog.

• Scala – There comes a useful, powerful and pleasant to use language targetting the Java virtual machine. I used it to write my HarassMe Android application.

I probably forgot some languages in the list. However, if I use them, I am sure I can tell something nice about them.