In Januarywe landed a significant refactoring of the KumaScript codebase and that is going to be the topic of the article because the job comprised several techniques of attention to JavaScript programmers.

Among the joys of undertaking a big refactor like this will be the opportunity to modernize the codebase. . .of loops, the spread (…) operator, and destructuring mission from the refactored code. Because KumaScript functions as a Node-based server, I did not need to worry about browser compatibility or transpilation: ” I had been liberated (like a kid in a candy store!) To use each the latest JavaScript characteristics supported by Node 10.

With this new library set up, the refactor was simple. Convert them to use Promises instead of the extension that is node-fibers and I had to find all the blocking functions available to our own macros. ThenI was able to perform a search-and-replace on the macro files to insert the wait key words before all invocations of those functions. A few of our more complex macros define their own internal capabilities, and when those internal functions used wait , I had to take the additional step of changing those functions to be async. I did get tripped up by one piece of syntax Once I converted an old lineup of blocking code
I have always found it reassuring to know Object.freeze() is there if I need it, but I’ve rarely really needed it. Therefore it was exciting to have a use for this particular function. There was one hitch worth mentioning, yet: after triumphantly using Object.freeze(), I discovered that my efforts to stub out macro API methods like wiki.getPage() were failing silently. By locking the execution environment down tightly, I’d locked out my ability to write evaluations!
And here is the kicker: when rendering macros that do not make HTTP requests (or if the HTTP outcomes are cached) I saw making speeds increase by a factor of 25 (not 25% quicker –25 times quicker!) . And in the exact same time CPU load dropped in half. In manufacturing, the new KumaScript server is measurably faster, but not nearly 25x faster, because, of course, the time required to make asynchronous HTTP requests dominates time required to synchronously leave the template. But achieving a 25x speedup, even made this refactor a gratifying experience!
So, so as to render macros, KumaScript must prepare an object which includes a pair of variables that are page-specific bindings for a API, and also a set of invocation-specific arguments. When refactoring this code, I had two aims:
However there was a catch: some of our macros needed to make HTTP requests to fetch data they needed. A link is rendered by that macro to the MDN documentation for a specified HTML tag. But, it also includes a tooltip (through the title attribute) on the link that includes a Fast summary of the element:

This relatively small shift in our KumaScript code means we no longer need the fibers extension compiled in to our Node binary; it means we don’t require the hirelings package any more; also it usually means that I managed to get rid of a bunch of code that managed the complicated details of communicating between the primary process and the hireling worker processes which were actually rendering macros.

I achieved the first goal by using the JavaScript prototype series and Object.create(). As opposed to defining all 3 layers of the environment on a single object, an object that described the macro API as well as the variables was created by me. I reused this thing inside a page for all macros. When it was time to render an individual macro, I utilized Object.create() to create a new object that inherited the API and per-page bindings, and I then added the macro argument bindings to this new item. This meant that there was much less installation work to perform to be rendered.

  • Should you declare a function async, you’re indicating that it yields a Promise. Until it’s returned and if you return a value that isn’t a Promise, this value will be wrapped in a solved Promise.
  • The await operator creates asynchronous Promises appear to act synchronously. It permits you to write code that is as easy to read and reason about as code.

Let reply = await fetch(url);

Let title = await wiki.getPage(slug).title;
Each of our KumaScript macros is employed as an EJS template, as I mentioned previously. Templates are compiled by the EJS library . And to my delight, the latest version of the library has been updated with an alternative to compile templates to async functions, meaning that await is currently encouraged in EJS.
It has to be the next element of an HTMLElement("html") component.

To this:
Let name = (wait wiki.getPage(slug)).title;
This Object.create() technique was not enough, however, because a macro that comprised the code wiki.getPage = null; will still have the ability to alter its execution environment and affect the output of subsequent macros. So, I took the excess step of phoning Object.freeze() on the model item (and recursively on the objects it references) before I generated objects that inherited out of it.

In web browsers, the fetch() function starts an HTTP request and returns a Promise thing that can resolve to a response object when the HTTP response starts to arrive in the server. Without wait , you would need to telephone the .then() method of the returned Promise, and pass a callback function to get the answer object. But the magic of wait lets us pretend that fetch() actually blocks until the HTTP reply is obtained. There is only one catch:
MDN has been using macros like this since before the Kuma server existed. Before Kuma, we utilized a wiki merchandise that enabled macros to be described in a language they called DekiScript. DekiScript was a JavaScript-based templating language using a particular API for interacting with the wiki. So when we moved into the Kuma serverour documents were filled with macros and we had to implement.
The article Refactoring MDN macros using async, wait, and Object.freeze() appeared first on Mozilla Hacks - the Web programmer blog.
This fibershirelings solution rendered MDN macros but by 2018 it was becoming obsolete. The original design decision that macro writers should not have to understand asynchronous programming using callbacks (or Promises) is still a fantastic decision. However, when Node 8 added support for your newest async and wait keywords, the fibers extension and hirelings library were no longer necessary.

  • I didn't need to reconstruct the entire object for each macro to be rendered.
  • I wished to ensure that macro code couldn't alter the environment and thereby affect the output of future macros.

Async and wait

Until I started viewing failures from the 17, I didn't catch the mistake on that line. In the older KumaScript, wiki.getPage() would obstruct and return the requested data synchronously. From the new KumaScript, wiki.getPage() is declared async which means it yields a Promise. Along with the code above is attempting to get a non-existent title land on this Promise object.
That summary has to come. This means that the execution of the KumaScript macro should fetch the page it's linking to in order to extract a number of its content. What's more, macros in this way are written by technical writers, not software engineers, and thus the choice was made (I suppose by those made the DekiScript macro program ) that things like HTTP fetches will be carried out with blocking functions that returned synchronously, to ensure specialized writers wouldn't have to manage nested callbacks.

To begin with, you should understand that Kuma is your Python-based wiki that forces MDN, and KumaScript is a server that renders macros from MDN documents. If you look at the raw kind of an MDN document (such as the HTML <body> component ) you will see lines like that:


In March of this past year, the MDN Engineering team started the experimentation of publishing a changelog on Mozilla Hacks. After nine months of the changelog structure , we’ve decided it’s time to try out something that we hope will be of interest to the web development community more widely, and more enjoyable for us to write. These articles might not be and they will not contain the type of detail that you would expect from a changelog. They can cover some of the more interesting engineering work we perform to handle and develop the MDN Web Docs website. And if you would like to know precisely what's changed and that has contributed to MDN, then you can always assess that the repos on GitHub.
What this means is the design aim of shielding KumaScript macro writers from the complexity of callbacks can now be done with the await keyword. And this is the insight with which our KumaScript refactor was undertaken by me.
There's one piece of this KumaScript refactor I want to speak about since it highlights some JavaScript techniques that deserve to be better known. As I've written previously, KumaScript uses EJS templates. You pass when you leave an EJS template. In order for it to accomplish that, KumaScript must pass an item to the EJS template making function which binds the name wiki to an object that includes a getPage property whose value is the relevant function.

It was not reason enough to warrant the time spent around the refactor, although updating to contemporary JavaScript was a lot of fun. To comprehend my team allowed me to work on this project, you need to understand how it functions and what KumaScript does. So bear with me while I explain this context, we'll return to the most interesting areas of the refactor.

  • You can only use wait within functions which are themselves announced async. Meantime, await doesn't really make anything obstruct: the underlying performance remains basically asynchronous, and even when we pretend that it is not, we could only do this within some larger asynchronous operation.

The material within the curly braces that are double is a macro invocation. In this case, the macro is defined to render a cross-reference connection to the MDN documentation for your html component. Using macros such as this makes matters simpler for writers and keeps our links and angle-bracket formatting consistent throughout the site.

Var title = wiki.getPage(slug).title;
For Example, consider this line of code:
However, if I was planning to reuse the object that defined the API and per-page variables, I had to be very certain that a macro couldn't change the environment, because that would mean that a bug at 1 macro could alter the output of another macro. Employing Object.create() helped a lot for this: when a macro runs a line of code like wiki = null;, that is only going to influence the environment object created for that one render, not the model object that it inherits from, and thus that the wiki.getPage() purpose will continue to be accessible to the next macro to be rendered. (I need to point out that using Object.create() similar to this may cause some confusion when debugging because a thing made this way will seem like it's empty even though it has inherited properties.)

Node does not encourage blocking network operations, while it brought documents for orders and even if it did, the KumaScript host couldn't only quit responding to incoming requests. The upshot was that KumaScript used the node-fibers binary extension to Node in order to specify methods that blocked while community orders were pending. And additionally, KumaScript embraced the node-hirelings library to manage a pool of child processes. (It was written by the original writer of KumaScript for this purpose). This allowed the KumaScript host to continue to handle incoming requests in parallel because it could farm out the possibly-blocking macro making calls to a pool of hireling child procedures.

Object.freeze() has been a part of JavaScript since 2009, but you might not have ever used it if you're not a library writer. It locks down an object, making all of its properties read-only. Additionally it "seals" the object, meaning that new properties cannot be added and existing properties can not be deleted or configured to make them writable again.

There are 3 layers of this environment which we make available to EJS templates. Most fundamentally, there's the macro API, including wiki.getPage() and a number of associated purposes. The API is shared by all macros. Above this API coating is an env object that provides macros access to page-specific values such as the language and title of the page within which they appear. As soon as an MDN page is submitted by the Kuma server there are typically multiple macros. Ultimately, every individual macro invocation on a webpage may include debates, and these are exposed by binding them to variables $0, $1, etc..

In case this all sounds intriguing, you can have a look at the Environment course in the KumaScript origin code.
Mechanically inserting an await in front of this invocation doesn't change that fact because the await operator has lower precedence than the . Property accessibility operator. In this case, I had to add some extra parentheses to Await the Promise to resolve before accessing the name property:

We couldn't implement them straight therefore KumaScript became a different company, Considering our macros were defined with JavaScript. This was 7 years back in early 2012, when Node itself was on version 0.6. Luckily, a JavaScript-based templating system known as EJS already existed in the time, so the basic tools for creating KumaScript were all in place.