Mozilla Firefox: Development Process

Status: Draft

This document describes the process by which changes to Firefox source code become a general Firefox release. Firefox has hundreds of millions of users, so some care is in order.

Firefox uses a schedule-driven process, where releases take place at regular intervals. That means each release happens regardless of whether a given feature is ready, and releases are not delayed to wait for a feature to stabilize. The goal of the process is to provide regular improvements to users without disrupting longer term work.

Powers of Ten

All changes to Firefox source code are initially integrated in the mozilla-central Mercurial repository. At scheduled intervals, the changes are imported from mozilla-central to release channels with wider audiences, such as the beta channel. Eventually, the changes appear in the general Firefox release. To get your patch into the mozilla-central repository, see How to Submit a Patch. Larger features and projects are usually initially developed in other repositories which track mozilla-central.

The four channels available are: mozilla-central (or "nightly"), mozilla-aurora, mozilla-beta, and Firefox (release). The nightly channel gets new features as soon as they are ready, but it has the lowest stability of the four channels. The UI might change each day, and websites might not work at times. The mozilla-aurora channel gets new features at regular intervals, but some of them might be disabled if it looks like they need more work. The beta channel receives only new features that are slated for the next Firefox release.

Each release channel is backed by a Mercurial repository containing a distinct copy of the Firefox source code.As a set of changes progresses through the repositories, features that aren't quite ready are disabled. New features are rarely directly added to the mozilla-aurora or mozilla-beta channels. Features that end up disabled or miss the scheduled transition to the experimental channel can be pulled again the next time the schedule permits. This policy does allow for features that take a long time to develop. It's just that they'll be present only on mozilla-central until they're ready. The number of users on each channel grows by roughly 10x as the changes make their way through the release process, and features on each channel require an accompanying improvement in stability.

Again, new features should generally only be added to the release in the first phase, with 100K users, because the rest of the time in the schedule is spent stabilizing those changes. Additionally, some features might only make it part of the way through the process for some period of time. For example, one could imagine a feature with a server-side component being available only on the experimental channel in order to test the scalability of companion server-side infrastructure.

Releases Over Time

Since the four stages of development each require some time, they can be staggered. This arrangement allows for continuous new feature development on mozilla-central, while the other channels are devoted to stabilizing features ready for a wider audience.

In the diagram above, the blue bars represent one set of changes progressing through each channel. Simultaneously, work continues on the mozilla-central repository. Some features might need three cycles (18 weeks) on mozilla-central before they're ready. Since mozilla-central will be pulled into the mozilla-aurora channel at week 6 and week 12, such features will need to have a mechanism allowing them to be easily disabled. Essentially, it needs to be easy to get the code on mozilla-central into a shippable state by disabling new code.

The mozilla-central, mozilla-aurora, and mozilla-beta channels receive frequent updates so changes to the program can be validated before appearing in the general release. The release channel receives updates from the beta channel periodically.

Moving work from one channel to another

New code migrates from one repository to another on a fixed schedule. For example, in the diagram above, new code from mozilla-central arrives in week 6, 12, and 18. It might seem expedient to land new features from mozilla-central as soon as they are "ready", but such a pattern would introduce too many variables into stability measurements. The one exception to this schedule is that unscheduled releases may be required for critical security and stability issues.

The first few days after code arrives on a channel are spent turning features off, backing out patches that cause stability problems, etc. There will be many judgement calls concerning whether it's possible to save a feature with a few spot fixes. The reason this work happens after new code appears on a channel is so that work can continue uninterrupted in the other repositories.

If a feature needs to be turned off on a release channel, it's the developer's responsibility to make a patch to do so. If a feature can't be disabled without turning off or backing out other things, then those other things will also get turned off or backed out.

The Lifecycle of One Release

A single release takes about 16 weeks of time. This section outlines some key release management dates in that timeline. The diagram below describes where quality assurance, localization, security, extension compatibility, and security updates fit in.

Week 1-5

Feature development on mozilla-central.

The feature development is tracked by a core of the localization community, giving feedback about l12y early in the development process.

Week 6

The code on mozilla-central is pulled to the mozilla-aurora channel.

Extension developers can rely on the APIs they see here, with the exception of new APIs that might end up disabled. The front end XUL might change for stability reasons, but not in major ways. Again, XUL related to new features may not be available if those features end up getting disabled.

en-US strings are frozen.

Week 7-10

Release management, QA, and engineering work to stabilize or disable new features.

The majority of localization teams finish their work on the product.

Week 11

Release management makes a Go/No Go decision on whether the mozilla-aurora channel can be pulled.

If the decision is made to Go, a repository pull from mozilla-aurora to mozilla-beta is done.

Only minor changes to localizations are still taken.

Week 12-13

Release management, QA, and engineering work to stabilize any issues that arise as the code is exposed to the mozilla-beta channel's wider audience.

Week 14

Release management makes a Go/No Go to make release the contents of the mozilla-beta channel to the general audience.

Week 16

If there's a release to do, it happens during this week.

Localizations that are in beta are not pulled into the release channel.

Acknowledgements: Thanks to the following contributors for their input and ideas. Listed in alphabetical order by first name:

Alexander Limi, Anthony LaForge, Asa Dotzler, Axel Hecht, Benjamin Smedberg, Brendan Eich, Cheng Wang, Christian Legnitto, Damon Sicore, Daniel Veditz, David Anderson, David Ascher, David Mandelin, Ehsan Akhgari, Erica Jostedt, Jay Sullivan, John O'Duinn, Johnathan Nightingale, Johnny Stenback, Juan Becerra, Justin Wood, L. David Baron, Laura Mesa, Lucas Adamski, Mark Finkle, Matt Evans, Mayumi Matsuno, Michael Morgan, Mike Beltzner, Mike Shaver, Mitchell Baker, Nick Thomas, Rob Sayre, Robert Kaiser, Robert O'Callahan, Robert Strong, Sheila Mooney, Stuart Parmenter