Techno Barje

Addon SDK 1.11 - the page-mod release

2012-09-19T07:00:00.000Z

page-mod API is the most commonly used API in jetpack. It allows to execute Javascript piece of code against any given website. It is very similar to greasemonkey and userscripts.

In Addon SDK version 1.11, which is due for October, 30th, we will bring various subtle but very important fixes, features and improvements to this API. In the meantime we will start releasing beta versions on tuesday (09/25) with 1.11b1.

Here is an overview of these changes:

You will now be able to execute page-mod scripts to already opened tab, by using the new attachTo option. [bug 708190]
With the same attachTo option, you can execute page-mod scripts only on top-level tab documents, and so avoid being applied to iframes. The following blogpost goes into detail about this new option. [bug 684047]
page-mod now ignores non-tab documents like: panel, widget, sidebar, hidden document living in firefox's hidden window, ... [bug 777632]
Your addon will be more efficient as we removed some costly workaround: the Javascript proxies layer between your content script and the page. We are now relying directly on C++ wrappers, also known as Xraywrappers. We are expecting a major improvement in term of memory and CPU usage. As this change depends on modifications made in Firefox, it will only be enabled on Firefox 17 and greater. [bug 786976]
Content scripts are now correctly frozen when you go back and forth in tab history. Before that, your content script was still alive and could throw some unexpected exception or modify an unexpected document. [bug 766088]
Random fixes: window.top and window.parent will be correct for iframes [bug 784431].
Last but not least and still at risk for 1.11 release. You will be able to extend priviledges of your content script to extra domains. So that your script will now be able to execute some action on your own domain in addition to the current page domain, without facing cross domain limitations. This rely on some improvements being made to Firefox and will only be enabled on Firefox 17+. [bug 786681]

It is realy exciting to see our most used API receiving so many improvements and I hope that we fixed most of the long-living issues you may have faced with page-mod!!

We would really like to get your feedback on these changes. If you find anything wrong, please, file bugs here and do not hesitate to come discuss with our team in the mailing-list

Jetpack localization using YAML format

2011-11-17T08:00:00.000Z

In a previous post, I've described my first proposal for localization support in jetpack addons. I've decided to change locale files format for YAML instead of JSON. During MozCamp event, folks helped me identifying some pitfalls with JSON:

No multiline string support. Firefox parser allows multiline but it is not officialy supported! So that it will disallow third party tools to work properly.
No easy way to add comments. It is mandatory for localizers to have context description in comments next to keys to translate. As there is no way to add comments in JSON, it will end up complexifying a lot locale format.

Example


# You can add comments with `#`...
Hello %s: Bonjour %s         # almost ...
hello_key: Bonjour %s        # wherever you want!

# For multiline, you need to indent your string with spaces
multiline:
  "Bonjour
   %s"

# Plural forms.
# we use a nested object with attributes that depends on the target language
# in english, we only have 'one' (for 1) and 'other' (for everything but 1)
# in french, it is the same except that 'one' match 0 and 1
# in some language like Polish, there is 4 forms and 6 in arabic
#
# So that having a structured format like YAML,
# help us writing these translations!
pluralString:
  one: "%s telechargement"
  other: "%s telechargements"

# I need to enclode these strings with `"` because of %. See note after.


// Get a reference to `_` gettext method with:
const _ = require("l10n").get;

// These three forms end up returning the same string.
// We can still use a locale string in code, or use a key.
// And multiline string gets its `\n` removed. (there is a way to keep them)
_("Hello %s", "alex") == _("hello_key", "alex") == _("multiline", "alex")

// Example of non-naive l10n feature, plurals:
_("pluralString", 0) == "0 telechargement"
_("pluralString", 1) == "1 telechargement"
_("pluralString", 10) == "10 telechargements"

Advantages of YAML

Multiline strings are supported nicely / easy to read. You do not need to add a final \ on all lines. As mulitiline is easier, localizers can use them more often and it will surely improve readability of locale files!
Structured data format. we can use this power whenever it is needed. For example, when we need to implement complex l10n features like plural forms or any feature that goes beyond simple 1-1 localization. The cool thing if we compare to JSON is that even if we define structures, we keep a really simple format with no noise (like {, }, ", ...).

As nothing comes without any issues, here is what I've found around YAML:

This format is not a Web standard. I don't think it makes much sense to avoid using it because of that. We are clearly missing a standardized format for localization in the web world.
You may hit some issues when you do not enclose your strings with " or '. For example, you can't start a string with %, nor having a : in middle of your string without enclosing it.
Even if YAML is not a web standard, it has been formaly specified. And unfortunately, a handy feature becomes a pitfall for our purpose! Some strings are automatically converted. Yes, True, False, ... are automatically converted to a boolean value. We can work around this in multiple ways, either by documenting it, or modifying the parser. The same solution apply here, you need to enclose your string with quotes.

Again, feedback is welcomed on this group thread and you can follow this work in bug 691782.

Jetpack localization

2011-10-31T07:00:00.000Z

I'm going to describe the first proposal of localization support for Jetpack. This approach uses gettext pattern and json files for locales. It is the first step of multiple iterations. This one only allows retrieving localized string in javascript code. We are going to give ways to translate files, mainly HTML files, through another iteration. And we are about to offer an online tool to ease addon localization (like babelzilla website).

Let's start by looking at a concrete example, then I'll justify our different choices.

{
  "Hello %s": "Bonjour %s",
  "hello_user": "Bonjour %s"
}

// Retrieve a dynamic reference to `_` gettext method with:
const _ = require("l10n").get;

// Then print to the console a localized string:
console.log(_("Hello %s", "alex"));
// => Prints "Bonjour alex" in french.

// Or, if we don't want to use localized string in addon code:
console.log(_("hello_user", "alex"));

Why gettext?

It gives a way to automatically fetch localizable strings or ids from source code by searching for _( ) pattern.
It allows to use either strings or IDs as value to translate. It is obviously better to use IDs. Because locales will broke each time addon developer fix a typo in the main language hard coded in the code.

But we should not forget that the high level APIs is trying to simplify addon development. So that it has to be really easy to translate a simple addon that has only 2 JS files and less than 50 lines of code! And the simple fact to mandatory require a locale file for the default language appears like a big burden for such small addon.

Having said that, I'm really happy that gettext approach doesn't discourage, nor makes it harder to use IDs, and so, if an addon developer build a big addon or just want to take more time to use better pratice, he still can do it, easily!

Why JSON for locales?

We could have used properties files, like XUL addons. But this format has some limitations that are not compatible with gettext pattern. Keys can't contain spaces and are limited to ASCII or something alike, so that we can't put text in a key.

So instead of using yet another specific format, I'm suggesting here to use JSON. JSON is really easy to parse and generate from both client and server side, and I'm convinced that it is simple enough to be edited with a text editor. On top of that we can build a small web application to ease localization.

In my very first proposal, I used a complex JSON object with nested attributes. But it ends up complexifying the whole story without real advantage. So I'm suggesting now to use the most simple JSON file we can require: one big object with keys being strings or id to translate and values being translated strings. Then we will be able to use JSON features to implement complex localization features, like plurals handling. So that values may be an array of plurals forms.

The big picture

Everything starts with one addon developer or one of its contributor. If one of them want to make the addon localizable, they have to use this new localization module.

const _ = require("l10n").get;

There is already multiple choices that has been made here:

_ is not a magic global. We need to explicitely require it. This choice will simplify compatibility with other CommonJS environnements, like NodeJS.
The name of the module itself is l10n instead of localization in order to ease the use of it.
This module expose _ function on get attribute in order to be able to expose another methods. I'm quite confident we will need some functions for plurals or files localization.

Then, they need to use _ on localizable strings:

var cm = require("context-menu");
cm.Item({
  label: _("My Menu Item"),
  context: cm.URLContext("*.mozilla.org")
});

Now, they have two choices:

use a string written in their prefered language, like here. So that they don't have to create a locale file.
use an ID. Instead of _("My Menu Item"), we will use: _("contextMenuLabel"). But it forces to create a localization file in order to map contextMenuLabel to My Menu Item.

Then, either a developer or a localizer can generate or modify locales files. Each jetpack package can have its own locale folder. This folder contains one JSON file per supported language. Here is how looks like a jetpack addon:

* my-addon/
  * package.json   # manifest file with addon name, description, version, ...
  * data/          # folder for all static files
    * images, 
    * html files, 
    * ...
  * lib /          # folder that contains all JS modules:
    * main.js      # main module to execute on startup
    * my-module.js # custom module that may use localization module
    * ...
  * locale/       # our main interest!
    * en-US.json
    * fr-FR.json
    * en-GB.json
    * ...

The next iteration will add a new feature to our command line tool, that is going to generate or update a locale file for a given language by fetching localization strings from source code. For example, the following command will generate my-addon/locale/fr-FR.json file:

$ cfx fetch-locales fr-FR

{
  "My Menu Item": "My Menu Item"
}

Finally, we need to replace right side values with the localized strings:

{
  "My Menu Item": "Mon menu"
}

And build the final addon XPI file with:

    $ cfx xpi

Any kind of feedback would be highly appreciated on this group thread.

If you want to follow this work, subscribe to bug 691782.