Mozilla Foundation(makers of Firefox,Thunderbird) using Hadoop & HBase.

This month’s Bay area HBase user group meet up was held at Mozilla Foundation in Mountain View. (Mozilla foundation is known for there open source products Firefox, Thunderbird, Bugzilla etc).

Andrew Purtel presented interesting talk on installing HBase using Amazon EC2 cluster. Mozilla’s metrics engineer Daniel talked about how  HBase and Hadoop are used at Mozilla corporation.

Daniel Einspanjer who is Metrics Software Engineer at Mozilla Corporation mentioned Mozilla is using Hadoop and HBase for generating usage and crash report metrics. According to Daniel there are more than 350 million Firefox users who reports around 1 million crash reports every day. These crash reports are valuable information for developers, which can be used for debugging purposes. Also these metrics includes information about plugin usage which is valuable for plugin developers.

Mozilla foundation wants to process these crash report data so that they can share important usage and debug information with developers and users. Currently Mozilla is using around 10-15% of crash reports received daily.

By processing these crash reports Mozilla will be able to gain information such as

  1. Total number of Firefox users.
  2. Add-on update pings.
  3. Average add-ons used by per user.

  • Source : http://blog.mozilla.com/data/tag/hadoop/
  • Mozilla aims to process more than 15 % crash reports. To achieve this goal they are  building a system to store crashes data for long time and process these data using Hadoop and HBase. Currently Mozilla is running production cluster with Hadoop 0.20 on 20 nodes to process crash report data.

Advertisements