{"id":906,"date":"2015-11-30T21:54:52","date_gmt":"2015-12-01T05:54:52","guid":{"rendered":"https:\/\/www.goer.org\/?p=906"},"modified":"2015-11-30T21:54:52","modified_gmt":"2015-12-01T05:54:52","slug":"npm-3s-flatter-module-tree-makes-babel-much-much-faster","status":"publish","type":"post","link":"https:\/\/www.goer.org\/Journal\/2015\/11\/npm-3s-flatter-module-tree-makes-babel-much-much-faster.html","title":{"rendered":"npm 3&#8217;s Flatter Module Tree Makes Babel Much, Much Faster"},"content":{"rendered":"<p>Recently, I became frustrated with the slow speed of my JavaScript build. After running some experiments, I discovered that the bottleneck was transpiling ES2015 code to ES5 code via <a href=\"http:\/\/babeljs.io\">Babel<\/a>.<\/p>\n<p>To illustrate this, consider an example project consisting of:<\/p>\n<ul>\n<li><code>node_modules\/<\/code>, containing <code>babel-cli@6.2.0<\/code> and <code>babel-preset-es2015@6.1.18<\/code><\/li>\n<li><code>index.js<\/code>, containing <code>console.log('hello');<\/code><\/li>\n<\/ul>\n<p>What happens if we transpile this 1-line script?<\/p>\n<pre><code>$ time babel --presets es2015 index.js \n'use strict';\n\nconsole.log('hello');\n\n\nreal        0m3.425s\nuser        0m3.194s\nsys         0m0.272s\n<\/code><\/pre>\n<p>Three and a half seconds is a long time to do nothing. In fact, in both this fake project and in my real projects, transpiling was taking well over an order of magnitude longer than bundling and minifying. Why is the transpiling step so slow?<\/p>\n<p>Babel has a handy debug mode that prints out each parsing step and the associated time. Maybe Babel is spinning on some parsing step or something?<\/p>\n<pre><code>$ DEBUG=babel babel --presets es2015 index.js \nbabel [BABEL] index.js: Parse start +0ms\nbabel [BABEL] index.js: Parse stop +7ms\nbabel [BABEL] index.js: Start set AST +2ms\nbabel program.body[0] ExpressionStatement: enter +3ms\nbabel program.body[0] ExpressionStatement: Recursing into... +0ms\nbabel program.body[0].expression CallExpression: enter +1ms\n... (snip) ...\nbabel program.directives[1].directives[0] Directive: Recursing into... +0ms\nbabel program.directives[1].directives[0] Directive: exit +0ms\nbabel [BABEL] index.js: End transform traverse +0ms\nbabel [BABEL] index.js: Generation start +0ms\nbabel [BABEL] index.js: Generation end +4ms\n<\/code><\/pre>\n<p>Nope, Babel is Babelifying reasonably fast. However, I also noticed that the three second delay was occurring between typing the command, and seeing the the first line of debug output appear. This is what we refer to as, &#8220;being hit with the clue bat.&#8221;<\/p>\n<p>So I turned to dtrace and started looking at file access activity, which was an eye-opening experience. Instead of going into the gory details, I&#8217;ll illustrate the problem more succinctly by counting the files under &#8216;node_modules&#8217;:<\/p>\n<pre><code>find node_modules\/ -type f | wc\n    42929   42929 6736163\n<\/code><\/pre>\n<p>If Node has to open some appreciable fraction of these 43K files at startup&#8230; well, I ain&#8217;t no fancy Full Stack Engineer or nothin&#8217;, but that seems like a lot of file I\/O.<\/p>\n<p>Now for me at least, Babel is an indispensable development tool. I would give up minification before Babel. I would give up source maps before Babel. I might even give up vim before Babel. So how to make Babel faster?<\/p>\n<p>One way would be to open fewer files.<\/p>\n<p>I had been using trusty old npm 2, but npm 3 has a rewritten dependency management system, which is designed to produce an &#8220;as flat as possible&#8221; dependency tree. Which potentially means less file duplication.<\/p>\n<p>So let&#8217;s throw away the npm 2 tree and install an npm 3 tree:<\/p>\n<pre><code>$ npm --version\n2.14.3\n$ sudo npm install -g npm \nPassword:\n\/opt\/local\/bin\/npm -&gt; \/opt\/local\/lib\/node_modules\/npm\/bin\/npm-cli.js\nnpm@3.5.0 \/opt\/local\/lib\/node_modules\/npm\n$ rm -rf node_modules\/\n$ npm install babel-cli babel-preset-es2015\n... (snip) ...\n$ find node_modules\/ -type f | wc\n    4495    4495  225968\n<\/code><\/pre>\n<p>And now for the moment of truth:<\/p>\n<pre><code>$ time babel --presets es2015 index.js \n'use strict';\n\nconsole.log('hello');\n\n\nreal        0m0.683s\nuser        0m0.621s\nsys         0m0.079s\n<\/code><\/pre>\n<p>Switching to npm 3 yields a 6x speedup. In my real projects, the <em>total<\/em> speedup for the entire build (including bundling, minification, and source map generation) is more like 3x.<\/p>\n<p>Lessons learned: Computers are pretty fast. Opening enormous numbers of files, not so much.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently, I became frustrated with the slow speed of my JavaScript build. After running some experiments, I discovered that the bottleneck was transpiling ES2015 code to ES5 code via Babel. To illustrate this, consider an example project consisting of: node_modules\/, containing babel-cli@6.2.0 and babel-preset-es2015@6.1.18 index.js, containing console.log(&#8216;hello&#8217;); What happens if we transpile this 1-line script? [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-906","post","type-post","status-publish","format-standard","hentry","category-tech"],"_links":{"self":[{"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/posts\/906","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/comments?post=906"}],"version-history":[{"count":7,"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/posts\/906\/revisions"}],"predecessor-version":[{"id":913,"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/posts\/906\/revisions\/913"}],"wp:attachment":[{"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/media?parent=906"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/categories?post=906"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.goer.org\/Journal\/wp-json\/wp\/v2\/tags?post=906"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}