Kuma Report, December 2017
- Purged 162 KumaScript Macros
- Increased availability of MDN
- Improved Kuma deployments
- Added browser compatibility data
- Said goodbye to Stephanie Hobson
- Shipped tweaks and fixes by merging 209 pull requests, including 13 pull requests from 11 new contributors.
Here’s the plan for January:
Done in December
We moved the KumaScript macros to GitHub in November 2016, and added a new macro dashboard. This gave us a clearer view of macros across MDN, and highlighted that there were still many macros that were unused or lightly used. Reducing the total macro count is important as we change the way we localize the output and add automated tests to prevent bugs.
We scheduled some time to remove these old macros at our Austin work week, when multiple people could quickly double-check macro usage and merge the 86 Macro Massacre pull requests. Thanks to Florian Scholz, Ryan Johnson, and wbamberg, we’ve removed 162 old macros, or 25% of the total at the start of the month.
We made some additional changes to keep MDN available and to reduce alerts. Josh Mize added rate limiting to several public endpoints, including the homepage and wiki documents (PR 4591). The limits should be high enough for all regular visitors, and only high-traffic scrapers should be blocked.
I adjusted our liveness tests, but kept the database query for now (PR 4579). We added new thresholds for liveness and readiness in November, and these appear to be working well.
We continue to get alerts about MDN during high-traffic spikes. We’ll continue to work on availability in 2018.
Ryan Johnson worked to make our Jenkins-based tests more reliable. For example, Jenkins now confirms that MySQL is ready before running tests that use the database (PR 4581). This helped find an issue with the database being reused, and we’re doing a better job of cleaning up after tests (PR 4599).
Ryan continued developing branch-based deployments, making them more reliable
and expanding to production deployments
We can now deploy to staging and production by merging to
prod-push for Kuma as well as KumaScript, and
we can monitor the deployment with bot notifications in
This makes pushes easier and more reliable, and gets us closer to
an automated deployment pipeline.
Daniel D. Beck continued to convert CSS compatibility data from the wiki to the repository, and wrote 35 of the 57 PRs merged in December. Thanks to Daniel for doing the conversion work, and thanks to Jean-Yves Perrier for many reviews and merges over the holiday break.
Stephanie Hobson continued to refine the design of the new compatibility tables, including an icon for the Samsung Internet Browser and an updated Firefox icon (Kuma PR 4605). Florian Scholz added a legend, to explain the notation (KumaScript PR 437). We’re getting closer to shipping these to all users. Please give any feedback at Beta Testing New Compatibility Table on Discourse.
Over the past 3½ years, Stephanie has had a huge impact on MDN. She shared her expertise on accessibility, multi-language support, readable HTML tables and all things Google Analytics. She advocated for the users during the spam mitigations and Persona shutdown. She’d argue for design changes from a web developer’s perspective, and back it up with surveys and interviews.
She’s also a talented developer, authoring over 400 PRs. She’s responsible for a lot of the changes on MDN in 2017:
We look forward to seeing what Stephanie will do on bedrock, and we look forward to Schalk’s work and fresh perspective on MDN Web Docs.
There were 209 PRs merged in December (which was supposed to be a light month):
- 98 mdn/kumascript PRs
- 57 mdn/browser-compat-data PRs
- 32 mozilla/kuma PRs
- 13 mdn/interactive-examples PRs
- 9 mozmeao/infra PRs
Several of these were from first-time contributors:
- Add Worklets and related specifications (KumaScript PR 511), from Jedipedia.
- spec2 macro: Make ESDraft a Draft instead of a Living Specification. (KumaScript PR 522), from Mathias Schäfer.
- IE11 partially supports
new Map(iterable)(BCD PR 691), from Nathan Hunzaker.
- Firefox 59 allows browserAction set* methods to accept null (BCD PR 710), from Oriol Brufau.
- Add compat for module attribute of script element (BCD PR 717), from Davilink.
- Safari 11 supports Cache-Control: immutable (BCD PR 726), from Rouven Weßling.
- Update data for comma-optional syntax for color (BCD PR 745), from csnardi.
- Update browser compat data for mix-blend-mode CSS property (BCD PR 757), from owaisalhashimi.
- fix bug 1420354 - Use rel=”license” instead of rel=”copyright” (Kuma PR 4603), from Mai Truong.
- Correct spelling of broccoli in array-pop example (Interactive Examples PR 365), from Leonid Kovalev.
- Adding a CSS interactive example for word-break property (PR 368), Adding an interactive example for the opacity CSS property (PR 369), and Clarify a bit the initial-choice attribute (PR 370), from (first-time to interactive examples contributor) SphinxKnight.
Other significant PRs:
- Fix bug 1063560: Change search output with locale=* (Kuma PR 4580), from Deep Bhattacharyya.
- Fix Bug 1420288, only load GA if DNT disabled (Kuma PR 4571), from Schalk Neethling.
Planned for January
We’re contining on existing projects like BCD in January, and starting some larger projects that will start to ship in February.
We’ve exhausted the easy solutions for increasing availability on MDN. We believe the next step is to put developer.mozilla.org behind a Content Distribution Network, or CDN. Once we have everything setup, most requests won’t even hit the Kuma engine, but instead will be handled by caching servers around the world. We expect it to take 1 - 2 months before we can get the majority of requests served by the CDN.
A first step is to reduce the page variants sent to anonymous users, so that the CDN edge servers can handle most requests. Schalk Neethling has been removing waffle flags or migrating them to switches over many PRs, such as PR 4561.
In January, Ryan Johnson will start adding the caching headers needed for the CDN to store and serve the pages without contacting Kuma.
We believe a CDN will reduce downtime and alerts from increased traffic. More importantly, we expect it will speed up MDN Web Docs for visitors outside the US.
MDN Web Docs is built on top of Django. We’re currently using Django 1.8, first released in 2015. It is a Long-Term Release (LTS) that will be supported with security updates until at least April 2018. Django 1.11, released in 2017, is the new LTS release, and will be supported until at least April 2020. In January, we’ll focus on updating our code and third-party libraries so that we can quickly make the transition to 1.11.
For now, our plan is to stay on Django 1.11 until April 2019, when Django 2.2, the next LTS release, is shipped. Django 2 requires Python 3, and it may take a lot of effort to update Kuma and switch to third-party libraries that support Python 3. We’ll make a lot of progress during the 1.11 transition, and we’ll monitor our Django 2 and Python 3 compatibility in 2018.
We have a lot of things we have to do in Q1 2018, such as the CDN and Django 1.11 update. We postponed a detailed plan for 2018, and instead will spend some of Q1 discussing goals and priorities. During our discussions in December, a few themes came up.
For the MDN Web Docs product, the 2018 theme is Reach. We want to reach more web developers with MDN Web Docs data, and earn a key place in developers’ workflows. Sometimes this means making developer.mozilla.org the best place to find the information, and sometimes it means delivering the data where the developer works. We’re using interviews and surveys to learn more and design the best experience for web developers.
For the technology side, the 2018 theme is Simplicity. There are many seldom-used Kuma features that require a history lesson to explain. These make it more complicated to maintain and improve the web site. We’d like to retire some of these features, simplify others, and make it easier to work on the code and data. We have ideas around zone redirects, asset pipelines, and translations, and we hope to implement these in 2018.
One thing that has gotten more complex in 2017 is code contribution. We’re implementing new features like browser-compat-data and interactive-examples as their own projects. Kuma is usually not the best place to contribute, and it can be challenging to discover where to contribute. We’re thinking through ways to improve this in 2018, and to steer contributor’s effort and enthusiasm where it will have the biggest impact.