Mozilla Marketing Engineering & Operations blog

Kuma Report, December 2017

, by John Whitlock

Here’s what happened in December in Kuma, the engine of MDN Web Docs:

Here’s the plan for January:

Done in December

Purged 162 KumaScript Macros

We moved the KumaScript macros to GitHub in November 2016, and added a new macro dashboard. This gave us a clearer view of macros across MDN, and highlighted that there were still many macros that were unused or lightly used. Reducing the total macro count is important as we change the way we localize the output and add automated tests to prevent bugs.

We scheduled some time to remove these old macros at our Austin work week, when multiple people could quickly double-check macro usage and merge the 86 Macro Massacre pull requests. Thanks to Florian Scholz, Ryan Johnson, and wbamberg, we’ve removed 162 old macros, or 25% of the total at the start of the month.

macro-massacre

Increased Availability of MDN

We made some additional changes to keep MDN available and to reduce alerts. Josh Mize added rate limiting to several public endpoints, including the homepage and wiki documents (PR 4591). The limits should be high enough for all regular visitors, and only high-traffic scrapers should be blocked.

I adjusted our liveness tests, but kept the database query for now (PR 4579). We added new thresholds for liveness and readiness in November, and these appear to be working well.

We continue to get alerts about MDN during high-traffic spikes. We’ll continue to work on availability in 2018.

Improved Kuma Deployments

Ryan Johnson worked to make our Jenkins-based tests more reliable. For example, Jenkins now confirms that MySQL is ready before running tests that use the database (PR 4581). This helped find an issue with the database being reused, and we’re doing a better job of cleaning up after tests (PR 4599).

Ryan continued developing branch-based deployments, making them more reliable (PR 4587) and expanding to production deployments (PR 4588). We can now deploy to staging and production by merging to stage-push and prod-push for Kuma as well as KumaScript, and we can monitor the deployment with bot notifications in #mdndev. This makes pushes easier and more reliable, and gets us closer to an automated deployment pipeline.

Added Browser Compatibility Data

Daniel D. Beck continued to convert CSS compatibility data from the wiki to the repository, and wrote 35 of the 57 PRs merged in December. Thanks to Daniel for doing the conversion work, and thanks to Jean-Yves Perrier for many reviews and merges over the holiday break.

Stephanie Hobson continued to refine the design of the new compatibility tables, including an icon for the Samsung Internet Browser and an updated Firefox icon (Kuma PR 4605). Florian Scholz added a legend, to explain the notation (KumaScript PR 437). We’re getting closer to shipping these to all users. Please give any feedback at Beta Testing New Compatibility Table on Discourse.

Said Goodbye to Stephanie Hobson

Stephanie Hobson is moving to the bedrock team in January, where she’ll help maintain and improve www.mozilla.org. Schalk Neethling will take over as the primary front-end developer for MDN Web Docs.

Over the past 3½ years, Stephanie has had a huge impact on MDN. She shared her expertise on accessibility, multi-language support, readable HTML tables and all things Google Analytics. She advocated for the users during the spam mitigations and Persona shutdown. She’d argue for design changes from a web developer’s perspective, and back it up with surveys and interviews.

She’s also a talented developer, authoring over 400 PRs. She’s responsible for a lot of the changes on MDN in 2017:

mdn-one-year

Schalk has been working on MDN for most of 2017. He’s been focused on the interactive examples project that fully shipped in December. He’s also been reviewing front-end PRs, and his feedback and suggestion have improved the front-end code for months. In December, Stephanie and Schalk worked closely to make a smooth transition, which included getting all the JavaScript to pass eslint tests (PR 4596 and PR 4597).

We look forward to seeing what Stephanie will do on bedrock, and we look forward to Schalk’s work and fresh perspective on MDN Web Docs.

Shipped Tweaks and Fixes

There were 209 PRs merged in December (which was supposed to be a light month):

Several of these were from first-time contributors:

Other significant PRs:

Planned for January

We’re contining on existing projects like BCD in January, and starting some larger projects that will start to ship in February.

Prepare for a CDN

We’ve exhausted the easy solutions for increasing availability on MDN. We believe the next step is to put developer.mozilla.org behind a Content Distribution Network, or CDN. Once we have everything setup, most requests won’t even hit the Kuma engine, but instead will be handled by caching servers around the world. We expect it to take 1 - 2 months before we can get the majority of requests served by the CDN.

A first step is to reduce the page variants sent to anonymous users, so that the CDN edge servers can handle most requests. Schalk Neethling has been removing waffle flags or migrating them to switches over many PRs, such as PR 4561.

In January, Ryan Johnson will start adding the caching headers needed for the CDN to store and serve the pages without contacting Kuma.

We believe a CDN will reduce downtime and alerts from increased traffic. More importantly, we expect it will speed up MDN Web Docs for visitors outside the US.

Ship More Interactive Examples

We launched the interactive example editor on a dozen pilot pages, and the analytics look good. Just before the holiday break, we decided we can ship the interactive example editor to any MDN page. You can see it on CSS background-size, Javascript Array.slice(), and more.

background-size array-slice

We have many more interactive examples ready to publish, including many JavaScript examples by Mark Boas. We’ll roll these and more out to MDN. We’ll also start on HTML interactive examples, and we’re planning to ship them in February. Follow mdn/interactive-examples to see the progress and learn how to help.

Update Django to 1.11

MDN Web Docs is built on top of Django. We’re currently using Django 1.8, first released in 2015. It is a Long-Term Release (LTS) that will be supported with security updates until at least April 2018. Django 1.11, released in 2017, is the new LTS release, and will be supported until at least April 2020. In January, we’ll focus on updating our code and third-party libraries so that we can quickly make the transition to 1.11.

For now, our plan is to stay on Django 1.11 until April 2019, when Django 2.2, the next LTS release, is shipped. Django 2 requires Python 3, and it may take a lot of effort to update Kuma and switch to third-party libraries that support Python 3. We’ll make a lot of progress during the 1.11 transition, and we’ll monitor our Django 2 and Python 3 compatibility in 2018.

Plan for 2018

We have a lot of things we have to do in Q1 2018, such as the CDN and Django 1.11 update. We postponed a detailed plan for 2018, and instead will spend some of Q1 discussing goals and priorities. During our discussions in December, a few themes came up.

For the MDN Web Docs product, the 2018 theme is Reach. We want to reach more web developers with MDN Web Docs data, and earn a key place in developers’ workflows. Sometimes this means making developer.mozilla.org the best place to find the information, and sometimes it means delivering the data where the developer works. We’re using interviews and surveys to learn more and design the best experience for web developers.

For the technology side, the 2018 theme is Simplicity. There are many seldom-used Kuma features that require a history lesson to explain. These make it more complicated to maintain and improve the web site. We’d like to retire some of these features, simplify others, and make it easier to work on the code and data. We have ideas around zone redirects, asset pipelines, and translations, and we hope to implement these in 2018.

One thing that has gotten more complex in 2017 is code contribution. We’re implementing new features like browser-compat-data and interactive-examples as their own projects. Kuma is usually not the best place to contribute, and it can be challenging to discover where to contribute. We’re thinking through ways to improve this in 2018, and to steer contributor’s effort and enthusiasm where it will have the biggest impact.

Related Posts