With this new library set up, the refactor was simple. Convert them to use Promises instead of the extension that is node-fibers and I had to find all the blocking functions available to our own macros. ThenI was able to perform a search-and-replace on the macro files to insert the
wait key words before all invocations of those functions. A few of our more complex macros define their own internal capabilities, and when those internal functions used
wait , I had to take the additional step of changing those functions to be
async. I did get tripped up by one piece of syntax Once I converted an old lineup of blocking code
I have always found it reassuring to know
Object.freeze() is there if I need it, but I’ve rarely really needed it. Therefore it was exciting to have a use for this particular function. There was one hitch worth mentioning, yet: after triumphantly using
Object.freeze(), I discovered that my efforts to stub out macro API methods like
wiki.getPage() were failing silently. By locking the execution environment down tightly, I’d locked out my ability to write evaluations!
And here is the kicker: when rendering macros that do not make HTTP requests (or if the HTTP outcomes are cached) I saw making speeds increase by a factor of 25 (not 25% quicker –25 times quicker!) . And in the exact same time CPU load dropped in half. In manufacturing, the new KumaScript server is measurably faster, but not nearly 25x faster, because, of course, the time required to make asynchronous HTTP requests dominates time required to synchronously leave the template. But achieving a 25x speedup, even made this refactor a gratifying experience!
So, so as to render macros, KumaScript must prepare an object which includes a pair of variables that are page-specific bindings for a API, and also a set of invocation-specific arguments. When refactoring this code, I had two aims:
However there was a catch: some of our macros needed to make HTTP requests to fetch data they needed. A link is rendered by that macro to the MDN documentation for a specified HTML tag. But, it also includes a tooltip (through the
title attribute) on the link that includes a Fast summary of the element:
This relatively small shift in our KumaScript code means we no longer need the fibers extension compiled in to our Node binary; it means we don’t require the hirelings package any more; also it usually means that I managed to get rid of a bunch of code that managed the complicated details of communicating between the primary process and the hireling worker processes which were actually rendering macros.
Object.create(). As opposed to defining all 3 layers of the environment on a single object, an object that described the macro API as well as the variables was created by me. I reused this thing inside a page for all macros. When it was time to render an individual macro, I utilized
Object.create() to create a new object that inherited the API and per-page bindings, and I then added the macro argument bindings to this new item. This meant that there was much less installation work to perform to be rendered.
- Should you declare a function
async, you’re indicating that it yields a Promise. Until it’s returned and if you return a value that isn’t a Promise, this value will be wrapped in a solved Promise.
awaitoperator creates asynchronous Promises appear to act synchronously. It permits you to write code that is as easy to read and reason about as code.
Let reply = await fetch(url);
Let title = await wiki.getPage(slug).title;
Each of our KumaScript macros is employed as an EJS template, as I mentioned previously. Templates are compiled by the EJS library . And to my delight, the latest version of the library has been updated with an alternative to compile templates to
async functions, meaning that
await is currently encouraged in EJS.
It has to be the next element of an HTMLElement("html") component.
Let name = (wait wiki.getPage(slug)).title;
Object.create() technique was not enough, however, because a macro that comprised the code
wiki.getPage = null; will still have the ability to alter its execution environment and affect the output of subsequent macros. So, I took the excess step of phoning
Object.freeze() on the model item (and recursively on the objects it references) before I generated objects that inherited out of it.
In web browsers, the
fetch() function starts an HTTP request and returns a Promise thing that can resolve to a response object when the HTTP response starts to arrive in the server. Without
wait , you would need to telephone the
.then() method of the returned Promise, and pass a callback function to get the answer object. But the magic of
wait lets us pretend that
fetch() actually blocks until the HTTP reply is obtained. There is only one catch:
The article Refactoring MDN macros using async, wait, and Object.freeze() appeared first on Mozilla Hacks - the Web programmer blog.
This fibershirelings solution rendered MDN macros but by 2018 it was becoming obsolete. The original design decision that macro writers should not have to understand asynchronous programming using callbacks (or Promises) is still a fantastic decision. However, when Node 8 added support for your newest
wait keywords, the fibers extension and hirelings library were no longer necessary.
- I didn't need to reconstruct the entire object for each macro to be rendered.
- I wished to ensure that macro code couldn't alter the environment and thereby affect the output of future macros.
Async and wait
Until I started viewing failures from the 17, I didn't catch the mistake on that line. In the older KumaScript,
wiki.getPage() would obstruct and return the requested data synchronously. From the new KumaScript,
wiki.getPage() is declared
async which means it yields a Promise. Along with the code above is attempting to get a non-existent
title land on this Promise object.
That summary has to come. This means that the execution of the KumaScript macro should fetch the page it's linking to in order to extract a number of its content. What's more, macros in this way are written by technical writers, not software engineers, and thus the choice was made (I suppose by those made the DekiScript macro program ) that things like HTTP fetches will be carried out with blocking functions that returned synchronously, to ensure specialized writers wouldn't have to manage nested callbacks.
To begin with, you should understand that Kuma is your Python-based wiki that forces MDN, and KumaScript is a server that renders macros from MDN documents. If you look at the raw kind of an MDN document (such as the HTML <
body> component ) you will see lines like that:
In March of this past year, the MDN Engineering team started the experimentation of publishing a changelog on Mozilla Hacks. After nine months of the changelog structure , we’ve decided it’s time to try out something that we hope will be of interest to the web development community more widely, and more enjoyable for us to write. These articles might not be and they will not contain the type of detail that you would expect from a changelog. They can cover some of the more interesting engineering work we perform to handle and develop the MDN Web Docs website. And if you would like to know precisely what's changed and that has contributed to MDN, then you can always assess that the repos on GitHub.
What this means is the design aim of shielding KumaScript macro writers from the complexity of callbacks can now be done with the
await keyword. And this is the insight with which our KumaScript refactor was undertaken by me.
wiki to an object that includes a
getPage property whose value is the relevant function.
- You can only use
waitwithin functions which are themselves announced
awaitdoesn't really make anything obstruct: the underlying performance remains basically asynchronous, and even when we pretend that it is not, we could only do this within some larger asynchronous operation.
The material within the curly braces that are double is a macro invocation. In this case, the macro is defined to render a cross-reference connection to the MDN documentation for your
html component. Using macros such as this makes matters simpler for writers and keeps our links and angle-bracket formatting consistent throughout the site.
Var title = wiki.getPage(slug).title;
For Example, consider this line of code:
However, if I was planning to reuse the object that defined the API and per-page variables, I had to be very certain that a macro couldn't change the environment, because that would mean that a bug at 1 macro could alter the output of another macro. Employing
Object.create() helped a lot for this: when a macro runs a line of code like
wiki = null;, that is only going to influence the environment object created for that one render, not the model object that it inherits from, and thus that the
wiki.getPage() purpose will continue to be accessible to the next macro to be rendered. (I need to point out that using
Object.create() similar to this may cause some confusion when debugging because a thing made this way will seem like it's empty even though it has inherited properties.)
Node does not encourage blocking network operations, while it brought documents for orders and even if it did, the KumaScript host couldn't only quit responding to incoming requests. The upshot was that KumaScript used the node-fibers binary extension to Node in order to specify methods that blocked while community orders were pending. And additionally, KumaScript embraced the node-hirelings library to manage a pool of child processes. (It was written by the original writer of KumaScript for this purpose). This allowed the KumaScript host to continue to handle incoming requests in parallel because it could farm out the possibly-blocking macro making calls to a pool of hireling child procedures.
There are 3 layers of this environment which we make available to EJS templates. Most fundamentally, there's the macro API, including
wiki.getPage() and a number of associated purposes. The API is shared by all macros. Above this API coating is an
env object that provides macros access to page-specific values such as the language and title of the page within which they appear. As soon as an MDN page is submitted by the Kuma server there are typically multiple macros. Ultimately, every individual macro invocation on a webpage may include debates, and these are exposed by binding them to variables
In case this all sounds intriguing, you can have a look at the Environment course in the KumaScript origin code.
Mechanically inserting an
await in front of this invocation doesn't change that fact because the
await operator has lower precedence than the
. Property accessibility operator. In this case, I had to add some extra parentheses to Await the Promise to resolve before accessing the