Techno Barje

Debug pure javascript leaks

2013-03-09T00:00:00+00:00

Mozilla ecosystem already has plenty of built-in features, scripts and addons to debug memory usage. But most of them are focused on the internals of the C++ codebase. These tools are very verbose and expose very-very few Javascript metadata. So that you have to start learning tons of internal C++ classes before being able to undersstand that your Javascript objects are actually visible in these tools output!

For now, when chasing Addon SDK memory leaks, I was just looking at overall memory usage and tried to read and re-read our codebase until I finally find the leak by seeing it in the code… But that practice may come to an end! We should have Javascript-oriented memory debugging tool. With a clear picture of which objects are still allocated at a given point in time. Without any C++ aspect. With an output that any confirmed Javascript developer can easily read and understand without knowing much about how Mozilla engine works.

With that in mind, I started looking at the object CC/GC graph. This graph contains a view of all objects allocated dynamically by the Garbage collector. All Javascript objects end up being in this graph. But also way more other C++ objects that we will have to translate into a meaningfull Javascript paradigm to the developer.

Then I realized that an XPCOM component already expose the whole CC graph: nsICycleCollectorListener. But again, with very few Javascript information other than “this is a Javascript object”, or, “this is a javascript function”. Not much more. It ends up being quite frustrating as most of the information is there, we just miss few pinches of Javascript metadata. Like:

what are the attributes of this object?
in which document it lives?
in which script it has been allocated?
in which line?
in which function?
what other Javascript objects refers this one?
what is the function name/source?
…

Finally, because of -or- thanks to the extra motivation given by padenot and vingtetun, I ended up doing crazy hacks to fetch this few information directly from Javascript: Call the jsapi library by using jsctypes with the object addresses given by the nsICycleCollectorListener interface. The benefit is that this experiment can run on any firefox release build (i.e. no need for a custom Firefox build). Using only JS also allows to experiment faster by avoiding the compiling phase. But that should definitely be kept as an experiment as I would not consider this as a safe practice!!

The result of this is:

a jsctypes C++ mangling library,
a work-in-progress addon to identify DOM node leaks involving multiple compartments/documents, and,
two (1, 2) bug fixes.

You can install this addon, it should work on Windows and Linux with FF20+. You can easily see bug 839280’s leaks on today’s Aurora (FF21) by opening firefox with this addon, then open and close the devtool inspector panel (CTRL+MAJ+I) and finally run the memory script by pressing ALT+SHIFT+D shortcut. Wait a bit, the addon is processing the whole CC graph and will freeze your firefox instance. And then open a folder with a log file that displays various information about potential cross compartment leaks.

Let me show you addon’s output for this leak. The code involved in this leak is the following button’s click listener: /browser/devtools/shared/DeveloperToolbar.jsm

button.addEventListener("click", function() {
  requisition.update(buttonSpec.typed);
  //if (requisition.getStatus() == Status.VALID) {
    requisition.exec();
  /*
  }
  else {
    console.error(‘incomplete commands not yet supported’);
  }
  */
}, false);

The script will print this in the log file:

############################################################################
DOM Listener leak.

>>> Leaked listener ctypes.uint64_t.ptr(ctypes.UInt64("0x128a16c0")) - JS Object (Function)
Function source:
function () {
"use strict";

          requisition.update(buttonSpec.typed);
          //if (requisition.getStatus() == Status.VALID) {
            requisition.exec();
          /*
          }
          else {
            console.error(‘incomplete commands not yet supported’);
          }
          */
        }

>>> DOM Event target holding the listener ctypes.uint64_t.ptr(ctypes.UInt64("0x12a95f60"))
FragmentOrElement (XUL) toolbarbutton id=’command-button-responsive’ class=’command-button’ chrome://browser/content/devtools/framework/toolbox.xul


############################################################################
Scope variable leak.

>>> Function keeping ‘button’ scope variable alive ctypes.uint64_t.ptr(ctypes.UInt64("0xf9a1640")) - JS Object (Function)
Function source:
function () {
"use strict";

          requisition.update(buttonSpec.typed);
          //if (requisition.getStatus() == Status.VALID) {
            requisition.exec();
          /*
          }
          else {
            console.error(‘incomplete commands not yet supported’);
          }
          */
        }

It immediatly tells you that you may leak something via this anonymous function. may leak, and not do leak, as it is always hard to tell which references are expected to be removed or not, but at least, it tells you that this reference still exist and may keep your compartment/document/global alive.

To make it short, the script first search for FragmentOrElement objects in the CC and search for all objects from the same compartment. Then I focused my work on cross compartment leaks so that I looked for edges going from and to these objects. Finally I analysed each of these objects having references from and to other compartments and tried to translate C++ object patterns into a meaningfull sentence for the Javascript paradigm.

Now What?

I’d like to get feedback from people used to debug leaks (no matter the language) and also discuss with people used to gecko internals like nsXPCWrappedJS, JS Object (Call), … in order to know if assumptions I made here are correct. So that I can continue translating new potential C++ object patterns into meaningfull Javascript usecases.

How to write a new WebAPI in Firefox Desktop, mobile, OS - part 1 ?

2013-02-14T00:00:00+00:00

Mozilla teams recently wrote tons of new API in a very short period of time, mostly for Firefox OS, but not only. As Firefox Desktop, Firefox Mobile and Firefox OS are based on the same source code, some of these API can easily be enabled on Desktop and mobile.

Writing a new API can be seen as both complicated and simple. Depending on the one you want to write, you don’t necessary need to write anything else than Javascript code (for example the settings API). That makes such task much more accessible and easier to prototype as you do not enter in compile/run development cycles, nor have to build firefox before even trying to experiment. But there is a significant number of mozilla specific knowledges to have before being able to write your API code.

The aim of this article is to write down a simple API example from ground and try to explain all necessary things you need to know before writing an API with the same level of expertise than what did Firefox OS engineers.

The example API: « CommonJS require »

Let’s say we would like to expose to websites a require() method that act like the nodejs/commonjs method with the same name. This function allows you to load javascript files exposing a precise interface, without polluting your current javascript scope.

So given the following javascript file:

http://blog.techno-barje.fr/public/webapi/module.js

// All properties set to `exports` variable will be returned to the requirer
exports.hello = function () {
  return "World";
};

Any webpage will be able to use its hello function like this:

the web

var module = navigator.webapi.require("http://blog.techno-barje.fr/public/webapi/module.js");
alert(module.hello()); // >> Display "World"

Simpliest implementation possible

In this first example I stripped various advanced features in order to ease jumping into Firefox internal code. I bundled this example as a Firefox addon so that you can easily see it running and also hack it. You can download it here. Once installed, you will have to relaunch Firefox, open any webpage, then open a Web console and finally execute the navigator.webapi.require code I just gave.

Now let’s see what’s inside. This .xpi file is just a zip file so you can open it and see three files:

install.rdf:

A really boring file describing our addon. The only two important fields in this file are: <em:bootstrap>false</em:bootstrap> and <em:unpack>true</em:unpack> required when you need to register a XPCOM file. More info here.

chrome.manifest:

# Those two lines allow to register the Javascript xpcom component defined in
# `web-api.js`
component {20bf1550-64b8-11e2-bcfd-0800200c9a77} web-api.js
contract @mozilla.org/webapi-example;1 {20bf1550-64b8-11e2-bcfd-0800200c9a77}

# That line registers the xpcom component in the "JavaScript-navigator-property"
# category which add it to the list of components that inject a new property in
# `navigator` web pages global object. The second argument defines the name of
# the property we would like to set.
category JavaScript-navigator-property webapi @mozilla.org/webapi-example;1

web-api.js:

And last but not least. The Javascript XPCOM file. XPCOM is a component object model overused in Mozilla codebase. More info here Let’s analyse its content by pieces:

function WebAPI() {}

WebAPI.prototype = {
  // Define the XPCOM component id, that has to match the one given in
  // chrome.manifest file
  classID: Components.ID("{20bf1550-64b8-11e2-bcfd-0800200c9a77}"),

  // Mandatory XPCOM method, that defines which interfaces
  // an object exposes.
  // * nsIDOMGlobalPropertyInitializer:
  // https://developer.mozilla.org/fr/docs/XPCOM_Interface_Reference/nsIDOMGlobalPropertyInitializer
  // This interface is related to the XPCOM category "JavaScript-navigator-property"
  // declared in the chrome.manifest file and this interface defines the `init`
  // method that is called when a webpage try to access navigator.webapi property.
  QueryInterface: XPCOMUtils.generateQI([
    Ci.nsIDOMGlobalPropertyInitializer
  ]),

  // nsIDOMGlobalPropertyInitializer:init
  init: function init(win) {
    // The `init` method can return an object that will be the one exposed as
    // `navigator.webapi`. This object will be created for each web document.
    return {
      require: function (url) {
        return require(win, url);
      },
      // Special internal attribute used to define which property the website
      // will be able to access. Only attribute whose name is specified here
      // are going to be accessible by the page.
      // https://wiki.mozilla.org/XPConnect_Chrome_Object_Wrappers
      __exposedProps__: {
        require: 'r'
      }
    };
  }
};

// Last XPCOM thingy that allows to expose our WebAPI Component to the system
// More info here: https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/XPCOMUtils.jsm
const NSGetFactory = XPCOMUtils.generateNSGetFactory([WebAPI]);

I let you discover the implementation of the require method, but it will be your job to implement such method. Now, you have the very minimal set where you can tweak the returned value of the init method and expose your own API to webpages.

Now note that this is a very minimal example. I’ll try to continue blogging about that and eventually talk about:

interfaces definition,
custom event implementation,
other XPCOM categories (in order to inject on other object than navigator),
how to implement a cross process API (mandatory for Firefox OS),
prototyping via the Addon SDK,
…

Firefox OS Bootstrap: How to Build It on a VM

2012-10-27T00:00:00+00:00

During my on-boarding on Firefox OS team I kept a draft of all necessary stuff that need to be done in order to build the project and flash it to the phone. I’m pretty sure it can help people on-boarding the project by having a single page that would allow anyone to start working on Firefox OS. I highly suggest you to take a look at MDN Firefox OS documentation if you visit this page later on, as this blogpost will most likely be outdated in some weeks.

Environnement

Use a Virtual Machine

I’m suggesting everyone to use a VM. It allows you to use exactly same environment, in order to maximize your chances to succeed building Firefox OS! Using another OS, another linux distro or even another Ubuntu version will introduce differences in dependencies versions and can easily give you errors no one but you are facing :(

You can use VMware Player which is free and available here, or any other VM software you are confortable with that has decent USB support (required to flash the phone).

Use Ubuntu 11.10

For the same reason than the VM, I suggest you to use the recommended linux distro and version You can download this Ubuntu 11.10 x64 ISO image and create a VM out of it (It is super easy with VMware, it almost does everything for you). The only important things are to set a large enough virtual drive, 30GB is a safe minimum, and enough memory, 4GB is a safe minimum.

Now open a terminal and launch all following commands in order to install all necessary dependencies.

Install dependencies

Install build dependencies:

sudo apt-get install build-essential bison flex lib32ncurses5-dev lib32z1-dev lib32z1-dev ia32-libs libx11-dev libgl1-mesa-dev gawk make curl bzip2  g++-multilib libc6-dev-i386 autoconf2.13 ccache git
sudo apt-get build-dep firefox

Java JDK 6 needed for adb

# The following PPA allows you to easily install the JDK through apt-get
sudo add-apt-repository ppa:ferramroberto/java
sudo apt-get update
sudo apt-get install sun-java6-jdk

Android SDK in order install adb

# Your first need to install 32 bit libs as we are using 64bit OS
# otherwise, you will have following error while running adb:
# $ adb: No such file or directory
sudo apt-get install ia32-libs

# There is no particular reason to use this SDK version
# It was the current version when I’ve installed it
wget http://dl.google.com/android/android-sdk_r20.0.3-linux.tgz
tar zxvf android-sdk_r20.0.3-linux.tgz
cd android-sdk-linux/
# The following command installs only "platform-tools" package which
# contains adb and fastboot
./tools/android update sdk –no-ui –filter 1,platform-tool

# Register adb in your PATH
echo "PATH=`pwd`/platform-tools:\$PATH" >> ~/.bashrc

# Execute in a new bash instance in order to gain from this new PATH
bash

Tweak udev in order to recognize your phone

If you do not do that at all, or not properly, $ adb devices will print this:

???????????? no permissions

You need to put the following content into /etc/udev/rules.d/51-android.rules

cat <<EOF | sudo tee -a /etc/udev/rules.d/51-android.rules
SUBSYSTEM=="usb", ATTRS{idVendor}=="19d2", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", MODE="0666"
EOF
sudo restart udev

Here I register only internal Mozilla phones otoro and unagi IDs. You may want to add lines for other phones. See this webpage for other vendor IDs.

Checkout all necessary projects

Checkout B2G repository

git clone https://github.com/mozilla-b2g/B2G.git

Take a minute to configure git, otherwise next steps will keep bugging you asking for your name and email.

cat > ~/.gitconfig <<EOF
[user]
  name = My name
  email = me@mail.com
[color]
  ui = auto
EOF

Connect your phone and ensure it is visible from your VM.

In order to do so run adb devices, you should see non-empty list of devices.

$ adb devices
List of devices attached
full_unagi       device

If you see no permissions message, checkout udev step.
Note that you have to setup your Virtual machine software to connect the USB port to the VM. In VMware player, click on: Player menu > Removable devices > "…something…" Android > Connect (Disconnnect from host).

Checkout all dependencies necessary for your particular phone

Before running the following command, ensure that your phone is connected. Note that you have to run this command with your phone still being on Android OS and ICS version. If your phone is already on B2G, you will have to retrieve the backup-otoro or backup-unagi folder automatically created when running the following command.
If your device is on an Android version older than ICS, you will have to flash it first to ICS. For both of these issues, ask in #b2g for help. This step will take a while, as it will download tons of big projects: android, gong, kernel, mozilla-central, gaia,… More than 4GB of git repositories, so be patient.

cd B2G/
# Run ./config –help for the list of supported phones.
./config.sh unagi

Install Qualcomm Areno graphic driver

Only if you are aiming to build Firefox OS for otoro or unagi phones, you will have to manually download Qualcomm areno armv7 graphic driver, available here.
Unfortunately, you will have to register to this website in order to be able to download this file. Once downloaded, put this Adreno200-AU_LINUX_ANDROID_ICS_CHOCO_CS.04.00.03.06.001.zip into your B2G directory.

Build Firefox OS

If ./config.sh went fine, you can now build Firefox OS!

./build.sh

Here is the possible error you might see:

arm-linux-androideabi-g++: Internal error: Killed (program cc1plus)

You are most likely lacking of memory. 4GB is a safe minimum.
KeyedVector.h:193:31: error: indexOfKey was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]

Your gcc version is too recent. Try using gcc 4.6.x version.

Flash the phone

If ./build.sh went fine, you can now flash your phone:

  ./flash.sh

Note that I have to unplug replug the device in order to make it work in the VM. When running ./flash.sh, the unagi phone switch to a blue screen, then ./flash.sh script is stuck on < waiting device > message. If I unplug and plug in back, it immediately starts flashing. Be carefull if you have to do the same, ensure that ./flash.sh doesn’t start flashing when you unplug it!

If ./flash.sh failed by saying that the image is too large, It might mean that you have to root your phone first. Again, ask in #b2g for help.

Addon SDK 1.11 - the page-mod release

2012-09-19T00:00:00+00:00

page-mod API is the most commonly used API in jetpack. It allows to execute Javascript piece of code against any given website. It is very similar to greasemonkey and userscripts.

In Addon SDK version 1.11, which is due for October, 30th, we will bring various subtle but very important fixes, features and improvements to this API. In the meantime we will start releasing beta versions on tuesday (09/25) with 1.11b1.

Here is an overview of these changes:

You will now be able to execute page-mod scripts to already opened tab, by using the new attachTo option. [bug 708190]
With the same attachTo option, you can execute page-mod scripts only on top-level tab documents, and so avoid being applied to iframes. The following blogpost goes into detail about this new option. [bug 684047]
page-mod now ignores non-tab documents like: panel, widget, sidebar, hidden document living in firefox’s hidden window, … [bug 777632]
Your addon will be more efficient as we removed some costly workaround: the Javascript proxies layer between your content script and the page. We are now relying directly on C++ wrappers, also known as Xraywrappers. We are expecting a major improvement in term of memory and CPU usage. As this change depends on modifications made in Firefox, it will only be enabled on Firefox 17 and greater. [bug 786976]
Content scripts are now correctly frozen when you go back and forth in tab history. Before that, your content script was still alive and could throw some unexpected exception or modify an unexpected document. [bug 766088]
Random fixes: window.top and window.parent will be correct for iframes [bug 784431].
Last but not least and still at risk for 1.11 release. You will be able to extend priviledges of your content script to extra domains. So that your script will now be able to execute some action on your own domain in addition to the current page domain, without facing cross domain limitations. This rely on some improvements being made to Firefox and will only be enabled on Firefox 17+. [bug 786681]

It is realy exciting to see our most used API receiving so many improvements and I hope that we fixed most of the long-living issues you may have faced with page-mod!!

We would really like to get your feedback on these changes. If you find anything wrong, please, file bugs here and do not hesitate to come discuss with our team in the mailing-list

Jetpack localization using YAML format

2011-11-17T00:00:00+00:00

In a previous post, I’ve described my first proposal for localization support in jetpack addons. I’ve decided to change locale files format for YAML instead of JSON. During MozCamp event, folks helped me identifying some pitfalls with JSON:

No multiline string support. Firefox parser allows multiline but it is not officialy supported! So that it will disallow third party tools to work properly.
No easy way to add comments. It is mandatory for localizers to have context description in comments next to keys to translate. As there is no way to add comments in JSON, it will end up complexifying a lot locale format.

Example

French locale file in YAML format

# You can add comments with `#`…
Hello %s: Bonjour %s         # almost …
hello_key: Bonjour %s        # wherever you want!

# For multiline, you need to indent your string with spaces
multiline:
  "Bonjour
   %s"

# Plural forms.
# we use a nested object with attributes that depends on the target language
# in english, we only have 'one' (for 1) and 'other' (for everything but 1)
# in french, it is the same except that 'one' match 0 and 1
# in some language like Polish, there is 4 forms and 6 in arabic
#
# So that having a structured format like YAML,
# help us writing these translations!
pluralString:
  one: "%s telechargement"
  other: "%s telechargements"

# I need to enclode these strings with `"` because of %. See note after.

Addon code

// Get a reference to `_` gettext method with:
const _ = require("l10n").get;

// These three forms end up returning the same string.
// We can still use a locale string in code, or use a key.
// And multiline string gets its `\n` removed. (there is a way to keep them)
_("Hello %s", "alex") == _("hello_key", "alex") == _("multiline", "alex")

// Example of non-naive l10n feature, plurals:
_("pluralString", 0) == "0 telechargement"
_("pluralString", 1) == "1 telechargement"
_("pluralString", 10) == "10 telechargements"

Advantages of YAML

Multiline strings are supported nicely / easy to read. You do not need to add a final \ on all lines. As mulitiline is easier, localizers can use them more often and it will surely improve readability of locale files!
Structured data format. we can use this power whenever it is needed. For example, when we need to implement complex l10n features like plural forms or any feature that goes beyond simple 1-1 localization. The cool thing if we compare to JSON is that even if we define structures, we keep a really simple format with no noise (like {, }, “, …).

As nothing comes without any issues, here is what I’ve found around YAML:

This format is not a Web standard. I don’t think it makes much sense to avoid using it because of that. We are clearly missing a standardized format for localization in the web world.
You may hit some issues when you do not enclose your strings with " or ’. For example, you can’t start a string with %, nor having a : in middle of your string without enclosing it.
Even if YAML is not a web standard, it has been formaly specified. And unfortunately, a handy feature becomes a pitfall for our purpose! Some strings are automatically converted. Yes, True, False, … are automatically converted to a boolean value. We can work around this in multiple ways, either by documenting it, or modifying the parser. The same solution apply here, you need to enclose your string with quotes.

Again, feedback is welcomed on this group thread and you can follow this work in bug 691782.

Jetpack localization

2011-10-31T00:00:00+00:00

I’m going to describe the first proposal of localization support for Jetpack. This approach uses gettext pattern and json files for locales. It is the first step of multiple iterations. This one only allows retrieving localized string in javascript code. We are going to give ways to translate files, mainly HTML files, through another iteration. And we are about to offer an online tool to ease addon localization (like babelzilla website).

Let’s start by looking at a concrete example, then I’ll justify our different choices.

French locale file

{
  "Hello %s": "Bonjour %s",
  "hello_user": "Bonjour %s"
}

Addon code

// Retrieve a dynamic reference to `_` gettext method with:
const _ = require("l10n").get;

// Then print to the console a localized string:
console.log(_("Hello %s", "alex"));
// => Prints "Bonjour alex" in french.

// Or, if we don't want to use localized string in addon code:
console.log(_("hello_user", "alex"));

Why gettext?

It gives a way to automatically fetch localizable strings or ids from source code by searching for _( ) pattern.
It allows to use either strings or IDs as value to translate. It is obviously better to use IDs. Because locales will broke each time addon developer fix a typo in the main language hard coded in the code.

But we should not forget that the high level APIs is trying to simplify addon development. So that it has to be really easy to translate a simple addon that has only 2 JS files and less than 50 lines of code! And the simple fact to mandatory require a locale file for the default language appears like a big burden for such small addon.

Having said that, I’m really happy that gettext approach doesn’t discourage, nor makes it harder to use IDs, and so, if an addon developer build a big addon or just want to take more time to use better pratice, he still can do it, easily!

Why JSON for locales?

We could have used properties files, like XUL addons. But this format has some limitations that are not compatible with gettext pattern. Keys can’t contain spaces and are limited to ASCII or something alike, so that we can’t put text in a key.

So instead of using yet another specific format, I’m suggesting here to use JSON. JSON is really easy to parse and generate from both client and server side, and I’m convinced that it is simple enough to be edited with a text editor. On top of that we can build a small web application to ease localization.

In my very first proposal, I used a complex JSON object with nested attributes. But it ends up complexifying the whole story without real advantage. So I’m suggesting now to use the most simple JSON file we can require: one big object with keys being strings or id to translate and values being translated strings. Then we will be able to use JSON features to implement complex localization features, like plurals handling. So that values may be an array of plurals forms.

The big picture

Everything starts with one addon developer or one of its contributor. If one of them want to make the addon localizable, they have to use this new localization module.

const _ = require("l10n").get;

There is already multiple choices that has been made here:

_ is not a magic global. We need to explicitely require it. This choice will simplify compatibility with other CommonJS environnements, like NodeJS.
The name of the module itself is l10n instead of localization in order to ease the use of it.
This module expose _ function on get attribute in order to be able to expose another methods. I’m quite confident we will need some functions for plurals or files localization.

Then, they need to use _ on localizable strings:

var cm = require("context-menu");
cm.Item({
  label: _("My Menu Item"),
  context: cm.URLContext("*.mozilla.org")
});

Now, they have two choices:

use a string written in their prefered language, like here. So that they don’t have to create a locale file.
use an ID. Instead of _("My Menu Item"), we will use: _("contextMenuLabel"). But it forces to create a localization file in order to map contextMenuLabel to My Menu Item.

Then, either a developer or a localizer can generate or modify locales files. Each jetpack package can have its own locale folder. This folder contains one JSON file per supported language. Here is how looks like a jetpack addon:

* my-addon/
  * package.json   # manifest file with addon name, description, version, …
  * data/          # folder for all static files
    * images, 
    * html files, 
    * …
  * lib /          # folder that contains all JS modules:
    * main.js      # main module to execute on startup
    * my-module.js # custom module that may use localization module
    * …
  * locale/       # our main interest!
    * en-US.json
    * fr-FR.json
    * en-GB.json
    * …

The next iteration will add a new feature to our command line tool, that is going to generate or update a locale file for a given language by fetching localization strings from source code. For example, the following command will generate my-addon/locale/fr-FR.json file:

$ cfx fetch-locales fr-FR

my-addon/locale/fr-FR.json

{
  "My Menu Item": "My Menu Item"
}

Finally, we need to replace right side values with the localized strings:

{
  "My Menu Item": "Mon menu"
}

And build the final addon XPI file with:

$ cfx xpi

Any kind of feedback would be highly appreciated on this group thread.

If you want to follow this work, subscribe to bug 691782.

Jetpack runner

2011-03-31T00:00:00+00:00

Here is the very first release of Jetpack runner. This firefox extension built on top of the Addon SDK is a personal project that aim to ease development of firefox extension using the SDK. It is a great exhibit of SDK capabilities as we can now develop such tool using the SDK itself! For now, to create an addon you need to go thought a python application that only has a command line interface: This is painfull to install and even more annoying to use on Windows as there is no really decent command line interface. Finally, if we compare to chrome extensions, we only need chrome to build an addon! This leads me to build a Firefox extension, that can be really easy to install in Firefox and allow to build really cool interfaces to create, run and test your addons.

Jetpack runner features:

Download and install SDK automatically
Create addon from templates
Run an addon
Execute unit-tests
Generate firefox extension XPI file or xulrunner application package
You can run these either in current firefox instance or run them in a new one
We can execute a package as a firefox extension or as a xulrunner application

Jetpack runner first steps:

On extension installation, a tab opens automatically on "jetpack:" url, the main jetpack runner interface. That allow to download and install a precise SDK release: Then it displays a list of packages provided by addon SDK. "addon-sdk" is the main package to play with. After clicking on "Create addon" button, you would easily create a new one by filling obvious form and selecting a template addon: And then, you end up on your newly created addon package page, where you can run it, execute unit tests or download as a firefox extension XPI file:

Jetpack runner!!!

Last but not least, here is a link to install it or to checkout the source.

Firefox Extension:
jetpack-runner-0.1.1.xpi

Source code:
Github project

Arretetoi.la - la géolocalisation horodaté!

2011-01-08T00:00:00+00:00

http://arretetoi.la/

Cette application web va vous permettre de récupérer une liste de restaurants le long d’un trajet en voiture. D’abord, vous indiquez votre lieu de départ, d’arrivée ainsi que l’heure à laquelle vous comptez partir. Tout ceci pour obtenir une liste de restaurants chaudement recommandés par Dismoiou.fr. Jusque là, rien de révolutionnaire! Mais la particularité de ce service est d’afficher des adresses uniquement autour des villes que vous allez croiser autour des horaires de repas. Ainsi, si vous faites Paris-Marseille en partant à 8h, le site vous proposera de déjeuner près de lyon, car vous devrez y être autour de midi!

Arretetoi.la est un service à la fois géolocalisé mais surtout horodaté!

Les site marche aussi bien sur ordinateur que sur téléphone. La version sur ordinateur vous permettra de préparer un parcours et d’estimer où vous pourriez manger. Tandis que la version mobile vous permettra de mettre à jour l’estimation en fonction de votre progression. En effet, votre téléphone pourra transmettre votre position exacte et mettre à jour la zone de déjeuner/diner.
Voici un aperçu de la version ordinateur:

Indication des lieux de départ/arrivée ainsi que l’heure de départ Votre trajet horodaté Lieu où vous serez à midi ou 20h, avec les restaurants recommandés

La version mobile:

Indication de votre lieu d’arrivée, le reste se remplit automatiquement Liste des restaurants recommandés autour de midi ou 20h Affichage du restaurant sélectionné sur une carte avec votre trajet

Enfin, notez que vous pouvez ajouter cette application à votre écran d’accueil sur iphone et android. Ainsi que sur ordinateur avec le navigateur chrome via le chrome web store.

Google maps hacks part2 - cool function to power up route, directions

2010-11-04T00:00:00+00:00

Another day, another library!
This new library brings some usefull functions and one of them is just awesome :)
Start with these usefull functions:

return the distance between two points,
retrieve the a point at a given distance between two another,
compute the square box with a given size around a point

And the awesome function allows you to find the point at a given duration on a direction computed by google maps API. For example, you may retrieve the precive point where you are going to be after 1hour driving on the road from Paris to Berlin!
For more information, I suggest you to check at the README available on github: https://github.com/ochameau/google-map-api-path-tools

As usual, source code is available under LGPL Licence.

Google maps hacks part1 - auto-suggest location in

2010-11-04T00:00:00+00:00

As I’m hacking around Google Maps API, I tried to write some libraries around it in order to ease futher developments around this very cool API! The first library brings auto-suggest in html inputs, so users can see a dynamic list of cities name when they start typing in a form :)

Here is the source code of it, licenced under LGPL:
https://github.com/ochameau/google-map-api-suggest

And what users may expect:

UIWebView secrets - part3 - How to properly call ObjectiveC from Javascript

2010-10-06T00:00:00+00:00

Let’s change the subject: this time no more talks about memory but always on UIWebView component. When we use this component for something else than just displaying webpages, like building UI with HTML, Javascript, … We often want to call Javascript functions from objective C and the opposite.

Call Javascript function from Objective-C:

The first move is easily done with the following piece of code:

  // In your Javascript files:
    function myJavascriptFunction () {
    
      // Do whatever your want!
    
    }
  
  // ———————————–
  
  // And in your Objective-C code:
  // Call Javascript function from Objective-C:
    [webview stringByEvaluatingJavaScriptFromString:@"myJavascriptFunction()"];

Call Objective-C function from Javascript:

But calling objective-c from a Javascript function is not easy as Iphone SDK doesn’t offer any native way to do this! So we have to use any king of hack to do this …
The most known, used and buggy practice is to register a UIWebViewDelegate on your web view and « catch-and-immediatly-cancel » a location change done in javascript.

(a very extremely plenty much advised practice!)

  // In Objective-C
  - someFunctionOnInit {
    
    webView = [[UIWebView alloc] init];
    // Register the UIWebViewDelegate in order to shouldStartLoadWithRequest to be called (next function)
    webView.delegate = self;  
    
  }
  
  // This function is called on all location change :
  - (BOOL)webView:(UIWebView *)webView2 
          shouldStartLoadWithRequest:(NSURLRequest *)request 
          navigationType:(UIWebViewNavigationType)navigationType {
    
    // Intercept custom location change, URL begins with "js-call:"
    if ([[[request URL] absoluteString] hasPrefix:@"js-call:"]) {
      
      // Extract the selector name from the URL
      NSArray *components = [requestString componentsSeparatedByString:@":"];
      NSString *function = [components objectAtIndex:1];
      
      // Call the given selector
      [self performSelector:NSSelectorFromString(functionName)];
      
      // Cancel the location change
      return NO;
    }
    
    // Accept this location change
    return YES;
    
  }
  
  - (void)myObjectiveCFunction {
    
    // Do whatever you want!
   
  }

  // ———————————–
  
  // Now in your javascript simply do this to call your objective-c function:
  // /!\ But for those who just read title and code, take care, this is a buggy practice /!\n  window.location = "js-call:myObjectiveCFunction";

What’s wrong with UIWebViewDelegate, shouldStartLoadWithRequest and location change ?

There is weird but apprehensible bugs with this practice:
a lot of javascript/html stuff get broken when we cancel a location change:

All setInterval and setTimeout immediatly stop on location change
Every innerHTML won’t work after a canceled location change!
You may get other really weird bugs, really hard to diagnose …

Sample application highlighting these bugs

Key files of this example:

MyWebview.m: Objective-c part, that inherit from UIWebView. Set the UIWebViewDelegate and catch requests in shouldStartLoadWithRequest selector.
NativeBridge.js: Tiny javascript library in order to change the location and offer a way to send arguments and receive a response.
webview-script.js: Test case script, that highlight these bugs.

In webview-script.js: InnerHTML stop working whereas textContent continues to …

  document.getElementById("count").innerHTML = i;
  document.getElementById("count2").textContent = i;

But we can’t charge Apple on this bug. I mean we try to load another location in the document we are working on! The webview component may start doing stuff before the delegate call, which cancel the load …
We have to find alternative way to communicate with the native code!

Better way to call Objective-C

The only thing we have to change is in Javascript code. Instead of changing the document location, we create an IFrame and set its location to a value that trigger the shouldStartLoadWithRequest method.
And voilà!

  var iframe = document.createElement("IFRAME");
  iframe.setAttribute("src", "js-frame:myObjectiveCFunction";
  document.documentElement.appendChild(iframe);
  iframe.parentNode.removeChild(iframe);
  iframe = null;

Here is another sample application, with exactly the same structures and test file.
But this time you are going to see innerHTML and setTimeout working! Again, this demo contains a library (NativeBridge.js) that allow to send arguments to native code and get back a result in javascript asynchronously, with a callback function.

Best practice example!

Free Objective-C<->Javascript library

Finally I provide the communication library under LGPL licence so it can ease your work on iphone platform! As I know that it’s really not easy ;-)

MyWebView.m: ObjectiveC part,
NativeBridge.js: Javascript side.

The code is full of comment, so you may easily use and tweak it!

Github repo

UIWebView secrets - part2 - leaks on release

2010-10-04T00:00:00+00:00

Continue on iphone with leaks around UIWebView! Here is another big one with no apparent solution. When we try to release a UIWebView component, very few memory is freed. So any application using this object to display webpages is going to crash quickly with Low memory exception :(

I think this memory usage graph gives an idea on how big is this new king of leak:

Create a UIWebView
Load http://www.google.com/
Release the webview (UIWebView dealloc is correctly called!)
Look how so few memory is freed :/

Test Application

The leak is all but tiny! Before the loading of the webpage, the application was using 630kB of memory, and after the release of the UIWebview, 1150kB! So we have a 500KB leak in order to simply display the home of Google.com!

This time, I didn’t manage to find any hack to solve this bug.
So if you have any tips to fix this, don’t hesitate to post a comment!

I’ve tried a lot of differents hacks to free more memory (or use less), like:

  // Setup the webview to disable some fancy effect on phone number, but doesn’t change anything on memory released …
  webview.dataDetectorTypes = UIDataDetectorTypeNone;
  webview.allowsInlineMediaPlayback = NO;

  // Remove and disable all URL Cache, but doesn’t seem to affect the memory
  [[NSURLCache sharedURLCache] removeAllCachedResponses];
  [[NSURLCache sharedURLCache] setDiskCapacity:0];
  [[NSURLCache sharedURLCache] setMemoryCapacity:0];

  // Remove all credential on release, but memory used doesn’t move!
  NSURLCredentialStorage *credentialsStorage = [NSURLCredentialStorage sharedCredentialStorage];
  NSDictionary *allCredentials = [credentialsStorage allCredentials];
  for (NSURLProtectionSpace *protectionSpace in allCredentials) {
    NSDictionary *credentials = [credentialsStorage credentialsForProtectionSpace:protectionSpace];
    for (NSString *credentialKey in credentials) {
      [credentialsStorage removeCredential:[credentials objectForKey:credentialKey] forProtectionSpace:protectionSpace];
    }
  }

  // Cleanup the HTML document by removing all content
  // This time, this hack free some additional memory on some websites, mainly big ones with a lot of content
  [webview stringByEvaluatingJavaScriptFromString:@"document.body.innerHTML=”;"];

But I never reach complete release of memory used by the web component :(

UIWebView secrets - part1 - memory leaks on xmlhttprequest

2010-10-04T00:00:00+00:00

My first blog post on iphone subject reveal a big memory bug when using UIWebView component. This is the (only one) component to display some HTML content in an iphone interface. UIWebView object has a lot of differents issues and I’m going to highlight the biggest of them. Actually, all XMLHttpRequests used in javascript code are fully leaking!!! I mean when you do a request that retrieve 100ko of data, your memory used grow up for 100ko! This bug is not always active, but almost always. In fact the trigger to enable it is to simply open one link in your UIWebView. For example, clicking on a <a> link.

But let’s look at a memory usage graph while we execute this simple test application:

Create the UIWebView object
Load a local HTML test file
Execute 3 XMLHttpRequest to google.com, notice how the memory is freed three times after each request!
Trigger the leak by opening a page that redirect back to our test file
Execute the same 3 XMLHttpRequest and look how much memory is used and totally leaked :/
We clean the HTML document with document.body.innerHTML=”; (sometimes free some memory, when we have a lot of DOM objects)
release the UIWebView (almost no memory freed, next post is going to analyse that)

Test Application

So, to sum up, usually, when you execute this Javascript in a UIWebView:

  var xmlhttp = new XMLHttpRequest();
  xmlhttp.onreadystatechange = function() {
    if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
      // Do whatever you want with the result
      
    }
  };
  xmlhttp.open("GET", "http://your.domain/your.request/…", true);
  xmlhttp.send();

Your are going to have a big memory usage and leak a lot of data!

But there is a hack to solve this problem: revert what is done when you open a link.
In fact, the key property which leads to this leak is the WebKitCacheModelPreferenceKey application setting. And when you open a link in a UIWebView, this property is automatically set to the value "1". So, the solution is to set it back to 0 everytime you open a link. You may easily do this by adding a UIWebViewDelegate to your UIWebView :

- (void)webViewDidFinishLoad:(UIWebView *)webView {
  [[NSUserDefaults standardUserDefaults] setInteger:0 forKey:@"WebKitCacheModelPreferenceKey"];
}

So are you going to have much less crash due to "Low Memory" :)

CSS animation+CSS transform+SVG powered by Firefox = Kaleidoscope

2010-09-27T00:00:00+00:00

The amazing power of web technologies like CSS, HTML and SVG comes when you mix them all together! The arrival of the new mozilla specific CSS property -moz-element totally unleashed the power of CSS animation/transition when it comes to doing graphical effects. And I’m pretty sure that we may go even more futher by using others standards like HTML and SVG …
In order to show you how simple and powerfull these technologies mix can be, I’ve add two extra features on top of this kaleidoscope:

The first one, use a video instead of an image: Video Kaleidoscope
The second one allow you to specify a custom image by select or drag’n drop: Custom image

I find these two features quite powerfull, and what’s totally awesome is how simple they are: only 3 lines of HTML in the first case, tens lines of javascript in the second one!

Kaleidoscope demo

Need Firefox 4 Beta 6 or a nightly!

Finally, you may look at this tutorial that explain point by point how to achieve a Kaleidoscope by mixing -moz-element+SVG+CSS animations and transform:

Kaleidoscope tutorial

jsctypes + win32api + jetpack = jetintray

2010-08-27T00:00:00+00:00

JSCtypes! What a powerfull tool, that allows to call native libraries with our simple Javascript.
Jetpack! What a powerfull tool, that allows to build reliably javascript applications, with unittests, memory profiling, web IDE, …
And WinAPI … a giant C library still in production in 2010 that allows to do very various things on Windows platform.

Mix all that and you get:

JetInTray

A jetpack API for adding Tray icons on windows via jsctypes and on linux with a binary xpcom component (I didn’t had time to work on a jsctypes version).
You may checkout this jetpack package directly from github.
Or if you want to learn jsctypes I suggest you to look at files in lib directory and to read my two previous posts on jsctypes.

I explained on the first one how to start playing with jsctypes, how to create C-structures and call functions.
Then I showed in the second post, how to create a JS callback passed to the native library as a function pointer.

That said, I wanted to highlight some underground hacks around win32api! In WinAPI, there is no addEventListener/setEventCallback/addActionListener/… In fact, there is the well known WndProc messages function, that receives absolutely all event of the application!! (Yes for real!) We define this function as a static function named WndProc. But in Jsctypes case, that’s impossible to define a static function, we can only create function pointers. That’s where comes the not so known hack which allow to register dynamically such event listener.

First we have to define our listener function following the WinAPI datatypes

Components.utils.import("resource://gre/modules/ctypes.jsm");
var libs = {};
libs.user32 = ctypes.open("user32.dll");

// Define the function pointer type
var WindowProcType = 
  ctypes.FunctionType(ctypes.stdcall_abi, ctypes.int,
    [ctypes.voidptr_t, ctypes.int32_t, ctypes.int32_t, ctypes.int32_t]).ptr;

// Bind a usefull API function
var DefWindowProc = libs.user32.declare("DefWindowProcA", ctypes.winapi_abi, ctypes.int,
    ctypes.voidptr_t, ctypes.int32_t, ctypes.int32_t, ctypes.int32_t);

// Set our javascript callback
function windowProcJSCallback(hWnd, uMsg, wParam, lParam) {
  
  // … do something smart with this event!
  
  // You HAVE TO call this api function when you don’t known how to handle an event
  // or your apply is going to crash or do nothing
  return DefWindowProc(hWnd, uMsg, wParam, lParam);
}

// Retrieve a C function pointer for our Javascript callback
var WindowProcPointer = WindowProcType(windowProcJSCallback);

Then we may fill a WNDCLASS structure with our fresh function pointer. This structure is used to create a new window class that use it own WndProc (not the default static function). See msdn doc for more information.

var WNDCLASS = 
  ctypes.StructType("WNDCLASS",
    [
      { style  : ctypes.uint32_t },
      { lpfnWndProc  : WindowProcType }, // here is our function pointer!
      { cbClsExtra  : ctypes.int32_t },
      { cbWndExtra  : ctypes.int32_t },
      { hInstance  : ctypes.voidptr_t },
      { hIcon  : ctypes.voidptr_t },
      { hCursor  : ctypes.voidptr_t },
      { hbrBackground  : ctypes.voidptr_t },
      { lpszMenuName  : ctypes.char.ptr },
      { lpszClassName  : ctypes.char.ptr }
    ]);
var wndclass = WNDCLASS();
wndclass.lpszClassName = ctypes.char.array()("class-custom-wndproc");
wndclass.lpfnWndProc = WindowProcType(windowProcCallback);   // <—- here it is!
RegisterClass(wndclass.address());

After that we may create a hidden window that is created only to catch events.

var CreateWindowEx = 
  libs.user32.declare( "CreateWindowExA", ctypes.winapi_abi, ctypes.voidptr_t,
      ctypes.long,
      ctypes.char.ptr,
      ctypes.char.ptr,
      ctypes.int,
      ctypes.int,
      ctypes.int,
      ctypes.int,
      ctypes.int,
      ctypes.voidptr_t,
      ctypes.voidptr_t,
      ctypes.voidptr_t,
      ctypes.voidptr_t
    );
var HWND_MESSAGE = -3; // This is the code for message-only window
                      // http://msdn.microsoft.com/en-us/library/ms632599%28VS.85%29.aspx#message_only
var win = CreateWindowEx(
    0, wndclass.lpszClassName,
    ctypes.char.array()("messages-only-window"),
    0, 0, 0, 0, 0,
    ctypes.voidptr_t(HWND_MESSAGE), null, null, null);

Finally, we only have to bind this window to any component which dispatch messages/events in order to receive them in our windowProcJSCallback callback. That’s it!

jsctypes unleashed

2010-08-24T00:00:00+00:00

As bugs 573066 and 585175 are fixed and available in last Firefox nightlies, we can now use JSCtypes at full power!

Thanks to dwitte for quick fixes!

That means :

Complex C-struct usage,
The possibility to define a JS callback seen by C library as a function pointer, and,
Full Win32 API (also called MFC) supports!

Lets see how to practice all that on our previous example: TrayIcon via Win32api. We were able to just display an icon in the previous blogpost. Now we are able to intercept events from win32api thanks to ctypes.FunctionType. First we define a plain old javascript function like this one:

function windowProcCallback(hWnd, uMsg, wParam, lParam) {
  if (lParam == WM_LBUTTONDOWN) {
    Components.utils.reportError("Left click!");
    /* 0 means that we handle this event */
    return 0; 
  }
  else if (lParam == WM_RBUTTONDOWN) {
    Components.utils.reportError("Right click!");
    return 0;
  }
  /* Mandatory use default win32 procedure! */
  return DefWindowProc(hWnd, uMsg, wParam, lParam);
};

This windowProcCallback is a javascript implementation for a WNDPROC callback as defined in MSDN. WNDPROC is a key part of Win32Api. These functions receive all kind of events. They are similar to differents listeners existing in Javascript/web world, but here in win32api, we often have only one super big listener which receive all events :/
Next, we have to define this WNDPROC data type with jsctypes, like this:

var WindowProcType =
  ctypes.FunctionType(ctypes.stdcall_abi, ctypes.int,
    [ctypes.voidptr_t, ctypes.int32_t, ctypes.int32_t, ctypes.int32_t]).ptr;

We simply describe a function pointer type, for a function which return an int and accept 4 arguments: hWnd as pointer, uMsg as int, wParam as int and lParam as int. Then, in our case, we give this function pointer via a structure. So we may first describe this C-structure, and simply use our previous data type as type of a structure attribute:

var WNDCLASS =
  ctypes.StructType("WNDCLASS",
    [
      { style : ctypes.uint32_t },
      { lpfnWndProc : WindowProcType}, // <– Here is the function pointer attribute
      { cbClsExtra : ctypes.int32_t },
      { cbWndExtra : ctypes.int32_t },
      { hInstance : ctypes.voidptr_t },
      { hIcon : ctypes.voidptr_t },
      { hCursor : ctypes.voidptr_t },
      { hbrBackground : ctypes.voidptr_t },
      { lpszMenuName : ctypes.char.ptr },
      { lpszClassName : ctypes.char.ptr }
    ]);

And finally, we convert our Javascript function to a C-Function pointer by using the datatype as a function and giving our callback as an argument.

var wndclass = WNDCLASS();
wndclass.lpszClassName = ctypes.char.array()("class-trayicon");
wndclass.lpfnWndProc = WindowProcType(windowProcCallback);   // <—- here it is!
RegisterClass(wndclass.address());

All this hard work to be able to detect clicks on our tray icon! I’ve built a full example file right here (with a lot of comments). And here is one hack that allow you to test it remotly in your Javascript Console. You just have to copy an icon in c:\default.ico. Here is a sample ico file.

  var x=new XMLHttpRequest(); x.open("GET","http://blog.techno-barje.fr/public/demo/jsctypes/example-jsctypes-full-power.txt",false); x.send(null); window.parent.eval(x.responseText);

Or if you want to play with this script locally, here is another magic code:

  var x=new XMLHttpRequest(); x.open("GET","file://C:/Users/YourUsername/Downloads/example-jsctypes-full-power.txt",false); x.send(null); window.parent.eval(x.responseText);

JSctypes round two

2010-08-06T00:00:00+00:00

Jsctypes has been introduced in Firefox 3.6 with simple C function call and only simple types: int, char, string, … But the next iteration of jsctypes that is coming in Firefox 4 is going to allow full C binding, with support of C structures and the ability to define a javascript function and give it to C library as a function pointer.

No more compilation, no more mozilla sdk download, nor XPCOM stuff, just plain javascript and only a tiny part of function and datatype declaration before doing a native binding!

But let the code talk! Here is an example that display a tray icon on windows. You can copy and paste this code in your Javascript Console in Firefox 4 beta, just do not forget to change the icon path defined in the loadimage function call.

  /* simply load "ctypes" object */
  Components.utils.import("resource://gre/modules/ctypes.jsm");
  
  /* Load libraries that we are going to use */
  var libuser32 = ctypes.open("user32.dll");
  var libshell32 = ctypes.open("shell32.dll");

  /* Here is the tedious work of declaring functions arguments types and struct attributes types */
  /* In fact it’s quite easy, you just have to find which precise type are using your native functions/struct */
  /* but it may be hard to known, for example in windows API, which precise type is behing their "HANDLE" type … */
  /* I recommend you to find and look at python ctype binding source code because they already had done this work */
    
  /*
  HANDLE WINAPI LoadImage(
    __in_opt  HINSTANCE hinst,
    __in      LPCTSTR lpszName,
    __in      UINT uType,
    __in      int cxDesired,
    __in      int cyDesired,
    __in      UINT fuLoad
  );
  */
  var loadimage = libuser32.declare("LoadImageA",
    ctypes.stdcall_abi,
    ctypes.int,
    ctypes.int,
    ctypes.char.ptr,
    ctypes.int,
    ctypes.int,
    ctypes.int,
    ctypes.int);
  const LR_LOADFROMFILE = 16;
  const IMAGE_ICON = 1;
  
  var notificationdata = ctypes.StructType("NOTIFICATIONDATA",
                                [{ cbSize  : ctypes.int          },
                                 { hWnd    : ctypes.int          },
                                 { uID     : ctypes.int          },
                                 { uFlags  : ctypes.int          },
                                 { uCallbackMessage : ctypes.int },
                                 { hIcon        : ctypes.int     },
                                 { szTip        : ctypes.char.array(64) },
                                 { dwState      : ctypes.int     },
                                 { dwStateMask  : ctypes.int     },
                                 { szInfo       : ctypes.char.array(256) },
                                 { uTimeoutOrVersion : ctypes.int },
                                 { szInfoTitle  : ctypes.char.array(64) },
                                 { dwInfoFlags  : ctypes.int },
                                 { guidItem     : ctypes.int },
                                 { hBalloonIcon : ctypes.int }
                                ]);
  const NIF_ICON = 0x00000002;
  
  /*
  BOOL Shell_NotifyIcon(
    __in  DWORD dwMessage,
    __in  PNOTIFYICONDATA lpdata
  );
  */
  var notifyicon = libshell32.declare("Shell_NotifyIcon",
                                    ctypes.stdcall_abi,
                                    ctypes.bool,
                                    ctypes.int,
                                    notificationdata.ptr);
  const NIM_ADD = 0x00000000;
  

  /* And now, the "real" code that is calling C functions */

  /* load our ico file */
  var hIcon = loadimage(0, "c:\default.ico", IMAGE_ICON, 16, 16, LR_LOADFROMFILE);
  
  /* create a C struct that is defining a notification in tray */
  var icon = notificationdata();
  icon.cbSize = notificationdata.size;
  icon.uFlags = NIF_ICON;
  icon.szTip = "My Tray Icon";
  icon.hIcon = hIcon;
  
  /* Display this notification! */
  notifyicon(NIM_ADD, icon.address());

We will be able to go futher and define a function callback to handle click events on the trayicon, but there is currently a bug which cause some crashes when using ctypes.FunctionType on windows. (ctypes.FunctionType allow to transform a custom Javascript function to a C function pointer)
Here is related bugs, which are still in process:

The first leads to crashes with FunctionType, and the second may lead to lib.declare with unfindable symbols errors when using ctypes.stdcall_abi.

Freemonkeys

2010-02-12T00:00:00+00:00

Freemonkeys

Yet another mozilla powered project! This time, a graphical unit tests editor/executer which enables you to spellcast an army of monkeys always happy to work hard on your projects and find bugs for free!
Here is what they can do:

Launch any Mozilla application: Firefox, Thunderbird and any xulrunner app,
Use an empty profile for each test execution, or an existing one,
Speak fluently "assert" language: isTrue, isFalse, isDefined, equals, … etc,
Report you in real time test execution directly in your test source code,
They are always ready to work. You don’t need to restart Freemonkeys on each test execution, nor on your application reboot. Freemonkeys is an independant xulrunner app, which launch yours and then controls it remotly with a network connexion,
Spot easily any window, any tab and any DOM element with usefull distinctive parameters: XPath, windows attributes, zIndex order, … etc,
Offer a way to facilitate node identification by simply clicking on it, and seeing in real time what are selected node parameters,
They are able to write down some debug messages, inspect javascript objects with DOM Inspector, take screenshots of any elements,
Ease you some kind of tests, by providing you a simple HTTP webserver in order to simulate some websites answers,
They are not narrow-minded to synchronous tests but offers an assert library and some usefull API embracing asynchronous execution of your code!

Now let’s highlight some of these cool features …

Node selection

Here I was overing the tip of the day image. Freemonkeys spot it by highlighting it in red and show me all parameters which are going to be used to find this node later:
You just have to click on it to get back to test editor and have all javascript code needed to get a reference to this node, something like this:

var top = windows.getRegistered("firefox-window", "topmost");
var tab = top.tabs.current;
var element = elements.xpath(tab, "id(‘frame’)/tbody[1]/tr[5]/td[1]/table[1]/tbody[1]/tr[1]/td[1]/table[1]/tbody[1]/tr[1]/td[1]/img[1]");

Elements screenshots

Simply write:

element.screenshot();

And get a screenshot directly in the test editor:

Live test execution reporting

Your monkeys report each assert status in the test editor, allowing you to keep the focus on test writing and not losing time by switching from your app to your terminal, then to your code editor, your terminal and your app … etc, etc.

HTTP API

// Get a reference to a firefox tab
var top = windows.getRegistered("firefox-window", "topmost");
var tab = top.tabs.current;

// Start an HTTP server on port 81
http.start("81");

// A successfull test
// Create an assert objet which is going to wait for a request on root path of our http server
var test = http.assertGetRequest("/");
// Open this page in the tab
tab.open("http://localhost:81/");
// Now wait for this request
test.wait();

// The same test but with a non-existant page on our local server, so a failing test!
var test = http.assertGetRequest("/");
tab.open("http://localhost:81/foo");
test.wait();

Asynchronous tests

// A usefull function which allow you to block test execution for an amount of time in ms
wait.during(1000);

// The simpliest asynchronous test
wait.forTrue(function () {
  return true;
});

// Another, which is going to pass after 3s, with this setTimeout
var v=true;
wait.setTimeout(function() {
  v=false;
},3000);
wait.forFalse(function () {
  return v;
});

// Finally, a test which will pass when the test function is going to be called ten times
// (wait for the anonymous function returns 10)
var i=0;
wait.forEquals(function () {
  return i++;;
}, 10);

How to get it ?

Source code is available on github: http://github.com/ochameau/freemonkeys (LGPL licence)
If you are on windows:

Download this package: freemonkeys-0.1-win.zip
Extract this zip file somewhere
Launch freemonkeys.exe

And for linux and mac:

Download this one: freemonkeys-0.1.zip
Extract it somewhere
If you don’t have xulrunner, download it from here

Finally, launch Freemonkeys with this command:

/path/to/your/xulrunner/dir/xulrunner /path/to/freemonkeys/application.ini

Session per tab in Firefox

2009-12-05T00:00:00+00:00

The brand new version of Yoono, Yoono7 brings a long-awaited Firefox feature: the possibility to sign-in on the same website with multiple accounts in different tabs or windows.

Althought this new feature is a big technical challenge, for the end user, it’s really simple and non-intrusive! This feature only add one simple "profile" selector on the left of the URL bar : This is the default status of the selector, it simply says that the current firefox behavior is "as before", using the same global session for all tabs.
Let’s look it when we enable session per tab on different profiles :

When you start the browser, open a bookmark, do a search or enter manually an URL, your request is loaded with the default global session (ie "as before"). But when you select a profile with this selector, the current tab is reloaded in order to use one specific totally independant session. Futhermore, if this website open a new tab or window, or if you click on a link, the new tab, window or webpage is going to use the same specific session.

But let’s see how to use this feature from the beginning.

First, we create one session linked to a personnal account. In this example I take gmail, but it can be any website : twitter, facebook, flickr, … whatever! To do so we click on the profile selector and get menu on the left and we click on "+ new profiles" and get the right’s one.
We are redirected to the homepage URL, where we must sign-in for this personnal account: poirot.alex.
And we are now signed in for this "Personnal gmail" session :
Then, we do the same for one professional account: yoono.test.
Later, we can reopen one of these sessions directly to the homepage with the profile selector and get automatically signed in. The "switch to" link doesn’t go to the homepage and only reload the current tab with the selected session (very usefull for Facebook connect, sharing, …).

So we can open as many different account in multiple tabs or windows!

Important note:

The current version of Yoono7 doesn’t allow custom profile creation. The only profiles you can use is the ones automatically created for each account registered in the yoono’s sidebar. But watch for the next minor releases of Yoono7, we are going to ship all this very soon!

Another demo of this feature in video:

Mozilla memory profiling, part 2: a working tool

2009-11-26T00:00:00+00:00

Here is another part of my work on memory analysis in Mozilla :

another-profiler_techno-barje.fr.xpi

This new version of "Another memory profiler" is now an effective tool, which display a lot of information about all objects living in your Firefox instance! By all I mean not only website javascript objects, but all objects used by Firefox in its windows, sidebars, extensions, all tabs, iframes, etc. The previous version allowed you only to select one component : a DOM window(website, sub-iframe or any XUL window), a XPCOM service or a JS Module :

Component selection

Now you can get a report about currently living objects : the ones that are still used because there is at least one reference to each of them. This report first display all javascript files involved in your component :

File selection

By selecting one file, you are seeing the number of living object sorted by there instantiation line :

Living objects information

Finally, this tool display objects counts for each line sorted by there type. But Javascript is not a strongly typed language, so it’s not really easy to sort its objects by a type! That’s why there are several way to describe a JS object :

Define a JS object by its attributes, like Atul Varma is doing in its current work,
By its JS Prototype name, very usefull "typing" when you are using Prototype and build Object-Oriented JS,
We are facing some specialized objects like all DOM objects : HTMLFormElement, HTMLDivElement, …
And finally all native types, like Array, String, Date, RegExp, … etc.

Finally, let’s see how to make this extension work :

First It contains a binary component which is only built for Firefox 3.5 and 3.6 for Windows and Linux-32.
Secondly The memory profiling component is a patched version of the Mozilla Jetpack’s one, so take care to disable Jetpack, before testing this!
Then In order to get the maximum information about your living JS object, I strongly encourage you to set these two prefs to false :
```
javascript.options.jit.content = false
javascript.options.jit.chrome = false
```
(That’s because Tracemonkey optimise loops and seems to embed empty stack frame information on these loop’s execution …)
That being said, you just have to install this extension another-profiler_techno-barje.fr.xpi, go to your Tools menu and click on "Open another memory profiler".

Come back for the next post for some more explanation on displayed results with simple scripts examples.