In a previous post, I’ve described my first proposal for localization support in jetpack addons. I’ve decided to change locale files format for YAML instead of JSON. During MozCamp event, folks helped me identifying some pitfalls with JSON:

• No multiline string support. Firefox parser allows multiline but it is not officialy supported! So that it will disallow third party tools to work properly.
• No easy way to add comments. It is mandatory for localizers to have context description in comments next to keys to translate. As there is no way to add comments in JSON, it will end up complexifying a lot locale format.

Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# You can add comments with `#`...
Hello %s: Bonjour %s         # almost ...
hello_key: Bonjour %s        # wherever you want!

# For multiline, you need to indent your string with spaces
multiline:
  "Bonjour
   %s"

# Plural forms.
# we use a nested object with attributes that depends on the target language
# in english, we only have 'one' (for 1) and 'other' (for everything but 1)
# in french, it is the same except that 'one' match 0 and 1
# in some language like Polish, there is 4 forms and 6 in arabic
#
# So that having a structured format like YAML,
# help us writing these translations!
pluralString:
  one: "%s telechargement"
  other: "%s telechargements"

# I need to enclode these strings with `"` because of %. See note after.

1
2
3
4
5
6
7
8
9
10
11
12

// Get a reference to `_` gettext method with:
const _ = require("l10n").get;

// These three forms end up returning the same string.
// We can still use a locale string in code, or use a key.
// And multiline string gets its `\n` removed. (there is a way to keep them)
_("Hello %s", "alex") == _("hello_key", "alex") == _("multiline", "alex")

// Example of non-naive l10n feature, plurals:
_("pluralString", 0) == "0 telechargement"
_("pluralString", 1) == "1 telechargement"
_("pluralString", 10) == "10 telechargements"

Advantages of YAML

• Multiline strings are supported nicely / easy to read. You do not need to add a final \ on all lines. As mulitiline is easier, localizers can use them more often and it will surely improve readability of locale files!
• Structured data format. we can use this power whenever it is needed. For example, when we need to implement complex l10n features like plural forms or any feature that goes beyond simple 1-1 localization. The cool thing if we compare to JSON is that even if we define structures, we keep a really simple format with no noise (like {, }, “, …).

As nothing comes without any issues, here is what I’ve found around YAML:

• This format is not a Web standard. I don’t think it makes much sense to avoid using it because of that. We are clearly missing a standardized format for localization in the web world.
• You may hit some issues when you do not enclose your strings with " or '. For example, you can’t start a string with %, nor having a : in middle of your string without enclosing it.
• Even if YAML is not a web standard, it has been formaly specified. And unfortunately, a handy feature becomes a pitfall for our purpose! Some strings are automatically converted. Yes, True, False, … are automatically converted to a boolean value. We can work around this in multiple ways, either by documenting it, or modifying the parser. The same solution apply here, you need to enclose your string with quotes.

Again, feedback is welcomed on this group thread and you can follow this work in bug 691782.

I’m going to describe the first proposal of localization support for Jetpack. This approach uses gettext pattern and json files for locales. It is the first step of multiple iterations. This one only allows retrieving localized string in javascript code. We are going to give ways to translate files, mainly HTML files, through another iteration. And we are about to offer an online tool to ease addon localization (like babelzilla website).

Let’s start by looking at a concrete example, then I’ll justify our different choices.

1
2
3
4

{
  "Hello %s": "Bonjour %s",
  "hello_user": "Bonjour %s"
}

1
2
3
4
5
6
7
8
9

// Retrieve a dynamic reference to `_` gettext method with:
const _ = require("l10n").get;

// Then print to the console a localized string:
console.log(_("Hello %s", "alex"));
// => Prints "Bonjour alex" in french.

// Or, if we don't want to use localized string in addon code:
console.log(_("hello_user", "alex"));

Why gettext?

1. It gives a way to automatically fetch localizable strings or ids from source code by searching for _( ) pattern.
2. It allows to use either strings or IDs as value to translate. It is obviously better to use IDs. Because locales will broke each time addon developer fix a typo in the main language hard coded in the code.

But we should not forget that the high level APIs is trying to simplify addon development. So that it has to be really easy to translate a simple addon that has only 2 JS files and less than 50 lines of code! And the simple fact to mandatory require a locale file for the default language appears like a big burden for such small addon.

Having said that, I’m really happy that gettext approach doesn’t discourage, nor makes it harder to use IDs, and so, if an addon developer build a big addon or just want to take more time to use better pratice, he still can do it, easily!

Why JSON for locales?

We could have used properties files, like XUL addons. But this format has some limitations that are not compatible with gettext pattern. Keys can’t contain spaces and are limited to ASCII or something alike, so that we can’t put text in a key.

So instead of using yet another specific format, I’m suggesting here to use JSON. JSON is really easy to parse and generate from both client and server side, and I’m convinced that it is simple enough to be edited with a text editor. On top of that we can build a small web application to ease localization.

In my very first proposal, I used a complex JSON object with nested attributes. But it ends up complexifying the whole story without real advantage. So I’m suggesting now to use the most simple JSON file we can require: one big object with keys being strings or id to translate and values being translated strings. Then we will be able to use JSON features to implement complex localization features, like plurals handling. So that values may be an array of plurals forms.

The big picture

Everything starts with one addon developer or one of its contributor. If one of them want to make the addon localizable, they have to use this new localization module.

1

const _ = require("l10n").get;

There is already multiple choices that has been made here:

• _ is not a magic global. We need to explicitely require it. This choice will simplify compatibility with other CommonJS environnements, like NodeJS.
• The name of the module itself is l10n instead of localization in order to ease the use of it.
• This module expose _ function on get attribute in order to be able to expose another methods. I’m quite confident we will need some functions for plurals or files localization.

Then, they need to use _ on localizable strings:

1
2
3
4
5

var cm = require("context-menu");
cm.Item({
  label: _("My Menu Item"),
  context: cm.URLContext("*.mozilla.org")
});

Now, they have two choices:

• use a string written in their prefered language, like here. So that they don’t have to create a locale file.
• use an ID. Instead of _("My Menu Item"), we will use: _("contextMenuLabel"). But it forces to create a localization file in order to map contextMenuLabel to My Menu Item.

Then, either a developer or a localizer can generate or modify locales files. Each jetpack package can have its own locale folder. This folder contains one JSON file per supported language. Here is how looks like a jetpack addon:

* my-addon/
  * package.json   # manifest file with addon name, description, version, ...
  * data/          # folder for all static files
    * images, 
    * html files, 
    * ...
  * lib /          # folder that contains all JS modules:
    * main.js      # main module to execute on startup
    * my-module.js # custom module that may use localization module
    * ...
  * locale/       # our main interest!
    * en-US.json
    * fr-FR.json
    * en-GB.json
    * ...

The next iteration will add a new feature to our command line tool, that is going to generate or update a locale file for a given language by fetching localization strings from source code. For example, the following command will generate my-addon/locale/fr-FR.json file:

1

$ cfx fetch-locales fr-FR

1
2
3

{
  "My Menu Item": "My Menu Item"
}

Finally, we need to replace right side values with the localized strings:

1
2
3

{
  "My Menu Item": "Mon menu"
}

And build the final addon XPI file with:

1

$ cfx xpi

Any kind of feedback would be highly appreciated on this group thread.

If you want to follow this work, subscribe to bug 691782.

Here is the very first release of Jetpack runner. This firefox extension built on top of the Addon SDK is a personal project that aim to ease development of firefox extension using the SDK. It is a great exhibit of SDK capabilities as we can now develop such tool using the SDK itself! For now, to create an addon you need to go thought a python application that only has a command line interface: This is painfull to install and even more annoying to use on Windows as there is no really decent command line interface. Finally, if we compare to chrome extensions, we only need chrome to build an addon! This leads me to build a Firefox extension, that can be really easy to install in Firefox and allow to build really cool interfaces to create, run and test your addons.

Jetpack runner features:

• Download and install SDK automatically
• Create addon from templates
• Run an addon
• Execute unit-tests
• Generate firefox extension XPI file or xulrunner application package
• You can run these either in current firefox instance or run them in a new one
• We can execute a package as a firefox extension or as a xulrunner application

Jetpack runner first steps:

On extension installation, a tab opens automatically on "jetpack:" url, the main jetpack runner interface. That allow to download and install a precise SDK release: Then it displays a list of packages provided by addon SDK. "addon-sdk" is the main package to play with. After clicking on "Create addon" button, you would easily create a new one by filling obvious form and selecting a template addon: And then, you end up on your newly created addon package page, where you can run it, execute unit tests or download as a firefox extension XPI file:

Jetpack runner!!!

Last but not least, here is a link to install it or to checkout the source.

Firefox Extension:
jetpack-runner-0.1.1.xpi

Source code:
Github project

http://arretetoi.la/

Cette application web va vous permettre de récupérer une liste de restaurants le long d’un trajet en voiture. D’abord, vous indiquez votre lieu de départ, d’arrivée ainsi que l’heure à laquelle vous comptez partir. Tout ceci pour obtenir une liste de restaurants chaudement recommandés par Dismoiou.fr. Jusque là, rien de révolutionnaire! Mais la particularité de ce service est d’afficher des adresses uniquement autour des villes que vous allez croiser autour des horaires de repas. Ainsi, si vous faites Paris-Marseille en partant à 8h, le site vous proposera de déjeuner près de lyon, car vous devrez y être autour de midi!

Arretetoi.la est un service à la fois géolocalisé mais surtout horodaté!

Les site marche aussi bien sur ordinateur que sur téléphone. La version sur ordinateur vous permettra de préparer un parcours et d’estimer où vous pourriez manger. Tandis que la version mobile vous permettra de mettre à jour l’estimation en fonction de votre progression. En effet, votre téléphone pourra transmettre votre position exacte et mettre à jour la zone de déjeuner/diner.
Voici un aperçu de la version ordinateur:

Indication des lieux de départ/arrivée ainsi que l’heure de départ Votre trajet horodaté Lieu où vous serez à midi ou 20h, avec les restaurants recommandés

La version mobile:

Indication de votre lieu d’arrivée, le reste se remplit automatiquement Liste des restaurants recommandés autour de midi ou 20h Affichage du restaurant sélectionné sur une carte avec votre trajet

Enfin, notez que vous pouvez ajouter cette application à votre écran d’accueil sur iphone et android. Ainsi que sur ordinateur avec le navigateur chrome via le chrome web store.

Another day, another library!
This new library brings some usefull functions and one of them is just awesome :)
Start with these usefull functions:

• return the distance between two points,
• retrieve the a point at a given distance between two another,
• compute the square box with a given size around a point

And the awesome function allows you to find the point at a given duration on a direction computed by google maps API. For example, you may retrieve the precive point where you are going to be after 1hour driving on the road from Paris to Berlin!
For more information, I suggest you to check at the README available on github: https://github.com/ochameau/google-map-api-path-tools

As usual, source code is available under LGPL Licence.

As I’m hacking around Google Maps API, I tried to write some libraries around it in order to ease futher developments around this very cool API! The first library brings auto-suggest in html inputs, so users can see a dynamic list of cities name when they start typing in a form :)

Here is the source code of it, licenced under LGPL:
https://github.com/ochameau/google-map-api-suggest

And what users may expect:

Let’s change the subject: this time no more talks about memory but always on UIWebView component. When we use this component for something else than just displaying webpages, like building UI with HTML, Javascript, … We often want to call Javascript functions from objective C and the opposite.

Call Javascript function from Objective-C:

The first move is easily done with the following piece of code:

  // In your Javascript files:
    function myJavascriptFunction () {
    
      // Do whatever your want!
    
    }
  
  // -----------------------------------
  
  // And in your Objective-C code:
  // Call Javascript function from Objective-C:
    [webview stringByEvaluatingJavaScriptFromString:@"myJavascriptFunction()"];

Call Objective-C function from Javascript:

But calling objective-c from a Javascript function is not easy as Iphone SDK doesn’t offer any native way to do this! So we have to use any king of hack to do this …
The most known, used and buggy practice is to register a UIWebViewDelegate on your web view and « catch-and-immediatly-cancel » a location change done in javascript.

(a very extremely plenty much advised practice!)

  // In Objective-C
  - someFunctionOnInit {
    
    webView = [[UIWebView alloc] init];
    // Register the UIWebViewDelegate in order to shouldStartLoadWithRequest to be called (next function)
    webView.delegate = self;  
    
  }
  
  // This function is called on all location change :
  - (BOOL)webView:(UIWebView *)webView2 
          shouldStartLoadWithRequest:(NSURLRequest *)request 
          navigationType:(UIWebViewNavigationType)navigationType {
    
    // Intercept custom location change, URL begins with "js-call:"
    if ([[[request URL] absoluteString] hasPrefix:@"js-call:"]) {
      
      // Extract the selector name from the URL
      NSArray *components = [requestString componentsSeparatedByString:@":"];
      NSString *function = [components objectAtIndex:1];
      
      // Call the given selector
      [self performSelector:NSSelectorFromString(functionName)];
      
      // Cancel the location change
      return NO;
    }
    
    // Accept this location change
    return YES;
    
  }
  
  - (void)myObjectiveCFunction {
    
    // Do whatever you want!
   
  }

  // -----------------------------------
  
  // Now in your javascript simply do this to call your objective-c function:
  // /!\ But for those who just read title and code, take care, this is a buggy practice /!\\n  window.location = "js-call:myObjectiveCFunction";

What’s wrong with UIWebViewDelegate, shouldStartLoadWithRequest and location change ?

There is weird but apprehensible bugs with this practice:
a lot of javascript/html stuff get broken when we cancel a location change:

• All setInterval and setTimeout immediatly stop on location change
• Every innerHTML won’t work after a canceled location change!
• You may get other really weird bugs, really hard to diagnose …

Sample application highlighting these bugs

Key files of this example:

• MyWebview.m: Objective-c part, that inherit from UIWebView. Set the UIWebViewDelegate and catch requests in shouldStartLoadWithRequest selector.
• NativeBridge.js: Tiny javascript library in order to change the location and offer a way to send arguments and receive a response.
• webview-script.js: Test case script, that highlight these bugs.

In webview-script.js: InnerHTML stop working whereas textContent continues to …

  document.getElementById("count").innerHTML = i;
  document.getElementById("count2").textContent = i;

But we can’t charge Apple on this bug. I mean we try to load another location in the document we are working on! The webview component may start doing stuff before the delegate call, which cancel the load …
We have to find alternative way to communicate with the native code!

Better way to call Objective-C

The only thing we have to change is in Javascript code. Instead of changing the document location, we create an IFrame and set its location to a value that trigger the shouldStartLoadWithRequest method.
And voilà!

  var iframe = document.createElement("IFRAME");
  iframe.setAttribute("src", "js-frame:myObjectiveCFunction";
  document.documentElement.appendChild(iframe);
  iframe.parentNode.removeChild(iframe);
  iframe = null;

Here is another sample application, with exactly the same structures and test file.
But this time you are going to see innerHTML and setTimeout working! Again, this demo contains a library (NativeBridge.js) that allow to send arguments to native code and get back a result in javascript asynchronously, with a callback function.

Best practice example!

Free Objective-C<->Javascript library

Finally I provide the communication library under LGPL licence so it can ease your work on iphone platform! As I know that it’s really not easy ;-)

• MyWebView.m: ObjectiveC part,
• NativeBridge.js: Javascript side.

The code is full of comment, so you may easily use and tweak it!

Github repo

Continue on iphone with leaks around UIWebView! Here is another big one with no apparent solution. When we try to release a UIWebView component, very few memory is freed. So any application using this object to display webpages is going to crash quickly with Low memory exception :(

I think this memory usage graph gives an idea on how big is this new king of leak:

1. Create a UIWebView
2. Load http://www.google.com/
3. Release the webview (UIWebView dealloc is correctly called!)
   Look how so few memory is freed :/

Test Application

The leak is all but tiny! Before the loading of the webpage, the application was using 630kB of memory, and after the release of the UIWebview, 1150kB! So we have a 500KB leak in order to simply display the home of Google.com!

This time, I didn’t manage to find any hack to solve this bug.
So if you have any tips to fix this, don’t hesitate to post a comment!

I’ve tried a lot of differents hacks to free more memory (or use less), like:

  // Setup the webview to disable some fancy effect on phone number, but doesn't change anything on memory released ...
  webview.dataDetectorTypes = UIDataDetectorTypeNone;
  webview.allowsInlineMediaPlayback = NO;

or

  // Remove and disable all URL Cache, but doesn't seem to affect the memory
  [[NSURLCache sharedURLCache] removeAllCachedResponses];
  [[NSURLCache sharedURLCache] setDiskCapacity:0];
  [[NSURLCache sharedURLCache] setMemoryCapacity:0];

or

  // Remove all credential on release, but memory used doesn't move!
  NSURLCredentialStorage *credentialsStorage = [NSURLCredentialStorage sharedCredentialStorage];
  NSDictionary *allCredentials = [credentialsStorage allCredentials];
  for (NSURLProtectionSpace *protectionSpace in allCredentials) {
    NSDictionary *credentials = [credentialsStorage credentialsForProtectionSpace:protectionSpace];
    for (NSString *credentialKey in credentials) {
      [credentialsStorage removeCredential:[credentials objectForKey:credentialKey] forProtectionSpace:protectionSpace];
    }
  }

or

  // Cleanup the HTML document by removing all content
  // This time, this hack free some additional memory on some websites, mainly big ones with a lot of content
  [webview stringByEvaluatingJavaScriptFromString:@"document.body.innerHTML='';"];

But I never reach complete release of memory used by the web component :(

My first blog post on iphone subject reveal a big memory bug when using UIWebView component. This is the (only one) component to display some HTML content in an iphone interface. UIWebView object has a lot of differents issues and I’m going to highlight the biggest of them. Actually, all XMLHttpRequests used in javascript code are fully leaking!!! I mean when you do a request that retrieve 100ko of data, your memory used grow up for 100ko! This bug is not always active, but almost always. In fact the trigger to enable it is to simply open one link in your UIWebView. For example, clicking on a <a> link.

But let’s look at a memory usage graph while we execute this simple test application:

1. Create the UIWebView object
2. Load a local HTML test file
3. Execute 3 XMLHttpRequest to google.com, notice how the memory is freed three times after each request!
4. Trigger the leak by opening a page that redirect back to our test file
5. Execute the same 3 XMLHttpRequest and look how much memory is used and totally leaked :/
6. We clean the HTML document with document.body.innerHTML=”; (sometimes free some memory, when we have a lot of DOM objects)
7. release the UIWebView (almost no memory freed, next post is going to analyse that)

Test Application

So, to sum up, usually, when you execute this Javascript in a UIWebView:

  var xmlhttp = new XMLHttpRequest();
  xmlhttp.onreadystatechange = function() {
    if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
      // Do whatever you want with the result
      
    }
  };
  xmlhttp.open("GET", "http://your.domain/your.request/...", true);
  xmlhttp.send();

Your are going to have a big memory usage and leak a lot of data!

But there is a hack to solve this problem: revert what is done when you open a link.
In fact, the key property which leads to this leak is the WebKitCacheModelPreferenceKey application setting. And when you open a link in a UIWebView, this property is automatically set to the value "1". So, the solution is to set it back to 0 everytime you open a link. You may easily do this by adding a UIWebViewDelegate to your UIWebView :

- (void)webViewDidFinishLoad:(UIWebView *)webView {
  [[NSUserDefaults standardUserDefaults] setInteger:0 forKey:@"WebKitCacheModelPreferenceKey"];
}

So are you going to have much less crash due to "Low Memory" :)

The amazing power of web technologies like CSS, HTML and SVG comes when you mix them all together! The arrival of the new mozilla specific CSS property -moz-element totally unleashed the power of CSS animation/transition when it comes to doing graphical effects. And I’m pretty sure that we may go even more futher by using others standards like HTML and SVG …
In order to show you how simple and powerfull these technologies mix can be, I’ve add two extra features on top of this kaleidoscope:

• The first one, use a video instead of an image: Video Kaleidoscope
• The second one allow you to specify a custom image by select or drag’n drop: Custom image

I find these two features quite powerfull, and what’s totally awesome is how simple they are: only 3 lines of HTML in the first case, tens lines of javascript in the second one!

Kaleidoscope demo

Need Firefox 4 Beta 6 or a nightly!

Finally, you may look at this tutorial that explain point by point how to achieve a Kaleidoscope by mixing -moz-element+SVG+CSS animations and transform:

Kaleidoscope tutorial