Magento 2 Javascript Bundling with Magepack

created May 3, 2020, last updated September 3, 2020.

.

Magento 2 PageSpeed (Lighthouse) performance audit results for mobile and desktop are notoriously bad. Imagine you have worked for months on a new Magento 2 eCommerce store, followed best practices for setup and optimisation, the store seems to be running fine but the first time you run a Lighthouse report you see a performance score like this:There are a lot of factors that can affect the Lighthouse performance results for any website but for Magento 2 a big performance killer is the sheer amount of external resources required to render a page whether it be a product page, cms page or category page. Some of these render blocking resources such as Javascript or CSS can cause significant delays in page loading and affect performance. You will see this type of performance problem identified in Lighthouse as “Eliminate render-blocking resources”.

Magento 2 uses the RequireJs javascript module system to load Javascript source code required for each Magento 2 page. If you have a lot of custom features with modules implementing additional Magento 2 Javascript mixins the number of Javascript resources in addition to the core javascript code required by Magento will increase and adversely affect page loading performance. As an example, here is the network console log from a really simple product page from my development site, you can see that there are 194 requests for Javascript resources!

Simple Magento 2 product page loads 194 Javascript files
Simple Magento 2 product page loads 194 Javascript files

There are various ways to try and reduce the performance impact of loading lots of Javascript including using http2 which is great at handling small file requests quickly or minifying the Javascript source to reduce it’s size but the most effective way of optimising Javascript loading is to use bundling.

Javascript bundling is a technique that combines or bundles multiple files in order to reduce the number of HTTP requests that are required to load a page.

Magento 2 has a built in javascript bundler that is extremely ineffective! Users report it creating a huge multi megabyte javascript file that decreases performance instead of improving it. You will actually see the recommendation not to use the built in Magento 2 bundling referenced in Lighthouse reports – “Disable Magento’s built-in JavaScript bundling and minification, and consider using baler instead.”

Baler mentioned here is an AMD (Asynchronous Module Definition) module bundler / preloader for Magento 2 stores. You will find a lot of Magento 2 js bundling guides that recommend using Baler but for the average developer (like me) or Magento 2 merchant the bundling process with Baler can be quite complex and daunting. There is however a new Magento 2 js bundler available that is much easier to use.

MageSuite Magepack

The Magepack from MageSuite is a “Next generation Magento 2 advanced JavaScript bundler” it’s pretty easy to implement and as of version 2.0 the results it achieves are very impressive.

  • Up to 91 points mobile score in Google Lighthouse.
  • Up to 98% reduction in JavaScript file requests.
  • Up to 44% reduction in transferred JavaScript size.
  • Up to 75% reduction in total load time.
  • Works with Magento’s JavaScript minification and merging enabled.
  • Uses custom solution (inspired by Baler)

I installed Magepack on my Magento 2 development site in May 2020 and achieved a 100 desktop performance score with PageSpeed  –

This is a simple product page, using the default Luma theme and I am also using Nginx as a container proxy running the PageSpeed module, so you probably won’t achieve this kind of result on a real world product page but you will see a huge improvement. Check the results yourself here.

Let’s look at how to setup and install MagePack for Magento 2.3.x.

Setup and install Magepack for Magento 2.3.x and 2.4.x

MagePack consists of a NodeJS bundler app and a Magento 2 module. The bundler app runs on Node JS v10 or higher. I’m running MagePack in my Docker Magento 2 php container, it’s running Ubuntu server 18.04 and I’ve tested Magepack with Magento 2.3.3 and 2.3.5. To install Node JS simply run

curl -sL https://deb.nodesource.com/setup_10.x | bash -
apt-get install -y nodejs

Ubuntu will probably need some more dependencies before MagePack will install

apt-get install gconf-service libasound2 libatk1.0-0 libatk-bridge2.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget

Finally to install the NodeJS MagePack app itself run
npm install -g magepack --unsafe-perm=true --allow-root

Installing Magepack NodeJS app and dependencies
Installing Magepack NodeJS app and dependencies

You will see that Magepack pulls down Chromium – it needs a web browser to analyse your Magento 2 site, most of the dependencies installed earlier are required for Chromium.

With Magepack installed, we now need to install the Magepack Magento 2 module

composer require creativestyle/magesuite-magepack

Next, depending on the version of Magento 2 you are running you might need to install some patches.

  • For Magento 2.3.3 and earlier 7 patches are required
  • For Magento 2.3.4 and 2.3.5 1 patch is required
  • For Magento 2.4.0 no patches are required

The most painless way of patching Magento 2 is to use Cweagans/Composer-Patches

composer require cweagans/composer-patches

You will find all the patches you need here : https://github.com/integer-net/magento2-requirejs-bundling

In your Magento 2 installation folder create a patches folder copy the patches into it and edit your Magento 2 composer.json file to include the following composer extra patches config.

If you see the error Evaluation failed: ReferenceError: BASE_URL is not defined running magepack generate with 2.3.5+ or 2.4.x check your CSP configuration, or try disabling the CSP module temporarily – errors reported by CSP may create a magepack error.

2.3.3 composer extra patches config
    "extra": {
            "magento-force": "override",
            "composer-exit-on-patch-failure": true,
            "patches": {
                "magento/magento2-base": {
                    "[Performance] Fix missing shims and phtml files with mage-init directives (https://github.com/magento/magento2/commit/db43c11c6830465b764ede32abb7262258e5f574)": "patches/composer/M233/github-pr-4721-base.diff",
                    "Refactor JavaScript mixins module https://github.com/magento/magento2/pull/25587": "patches/composer/M233/github-pr-25587-base.diff"
                },
                "magento/module-braintree": {
                    "[Performance] Fix missing shims and phtml files with mage-init directives (https://github.com/magento/magento2/commit/db43c11c6830465b764ede32abb7262258e5f574)": "patches/composer/M233/github-pr-4721-braintree.diff"
                },
                "magento/module-catalog": {
                    "[Performance] Fix missing shims and phtml files with mage-init directives (https://github.com/magento/magento2/commit/db43c11c6830465b764ede32abb7262258e5f574)": "patches/composer/M233/github-pr-4721-catalog.diff"
                },
                "magento/module-customer": {
                    "[Performance] Fix missing shims and phtml files with mage-init directives (https://github.com/magento/magento2/commit/db43c11c6830465b764ede32abb7262258e5f574)": "patches/composer/M233/github-pr-4721-customer.diff"
                },
                "magento/module-msrp": {
                    "[Performance] Fix missing shims and phtml files with mage-init directives (https://github.com/magento/magento2/commit/db43c11c6830465b764ede32abb7262258e5f574)": "patches/composer/M233/github-pr-4721-msrp.diff"
                },
                "magento/module-paypal": {
                    "[Performance] Fix missing shims and phtml files with mage-init directives (https://github.com/magento/magento2/commit/db43c11c6830465b764ede32abb7262258e5f574)": "patches/composer/M233/github-pr-4721-paypal.diff"
                },
                "magento/module-theme": {
                    "[Performance] Fix missing shims and phtml files with mage-init directives (https://github.com/magento/magento2/commit/db43c11c6830465b764ede32abb7262258e5f574)": "patches/composer/M233/github-pr-4721-theme.diff",
                    "fix_baler_jquery_cookie": "https://gist.github.com/tdgroot/f95c398c565d9bbb83e0a650cdf67617/raw/69ee2d001ff509d25d1875743e417d914e20fd85/fix_baler_jquery_cookie.patch"
                }
            }
        },
2.3.4, 2.3.5 composer extra patches config
    "extra": {
            "magento-force": "override",
            "composer-exit-on-patch-failure": true,
            "patches": {
                "magento/magento2-base": {
                    "Refactor JavaScript mixins module https://github.com/magento/magento2/pull/25587": "patches/composer/M234/github-pr-25587-base.diff"
                }
            }
        },

Now run composer update Magento 2 will be patched and we are good to go.

Let’s get ready to bundle

Magepack needs to analyse pages from your Magento 2 store to determine the Javascript files your store is using and how they can be bundled. It saves this information in a configuration file called magepack.config.js. The magepack config file is generated by analysing three different type of pages from your Magento 2 store, a cms page i.e. the home page, a category page and a product page. This is done using the magepack generate command and supplying three store urls.

magepack generate --cms-url="http://magento2.gaiterjones.com/" --category-url="http://magento2.gaiterjones.com/en/buy-x-get-y.html" --product-url="http://magento2.gaiterjones.com/en/affirm-water-bottle.html"

Run this command in the root folder of your Magento 2 installation to create the magepack.config.js file. It’s worth noting that you could run this generate command from any system, and just copy the generated config file to your Magento 2 server.

If you take a look at magepack.config.js you will see it contains references to all the javascript required to load Magento pages. Below is an example from a product page.

example product section from magepack.config.js
    name: 'product',
    modules: {
      'Magento_Catalog/js/price-utils': 'Magento_Catalog/js/price-utils',
      'Magento_Catalog/js/price-box': 'Magento_Catalog/js/price-box',
      'Magento_Wishlist/js/add-to-wishlist': 'Magento_Wishlist/js/add-to-wishlist',
      'Magento_Cookie/js/require-cookie': 'Magento_Cookie/js/require-cookie',
      'Magento_Swatches/js/configurable-customer-data': 'Magento_Swatches/js/configurable-customer-data',
      'Magento_Review/js/error-placement': 'Magento_Review/js/error-placement',
      'Magento_Review/js/process-reviews': 'Magento_Review/js/process-reviews',
      'Elgentos_LargeConfigProducts/js/swatch-renderer-mixin': 'Elgentos_LargeConfigProducts/js/swatch-renderer-mixin',
      'text!Magento_Theme/templates/breadcrumbs.html': 'Magento_Theme/templates/breadcrumbs.html',
      'magnifier/magnifier': 'magnifier/magnifier',
      'magnifier/magnify': 'magnifier/magnify',
      'Magento_Catalog/js/gallery': 'Magento_Catalog/js/gallery',
      'Magento_ProductVideo/js/load-player': 'Magento_ProductVideo/js/load-player',
      'Magento_ProductVideo/js/fotorama-add-video-events': 'Magento_ProductVideo/js/fotorama-add-video-events',
      'Magento_Theme/js/model/breadcrumb-list': 'Magento_Theme/js/model/breadcrumb-list',
      'Magento_Theme/js/view/breadcrumbs': 'Magento_Theme/js/view/breadcrumbs',
      'Magento_Theme/js/view/add-home-breadcrumb': 'Magento_Theme/js/view/add-home-breadcrumb',
      'Magento_Catalog/js/product/breadcrumbs': 'Magento_Catalog/js/product/breadcrumbs',
      'jquery/jquery.parsequery': 'jquery/jquery.parsequery',
      'Magento_ConfigurableProduct/js/options-updater': 'Magento_ConfigurableProduct/js/options-updater',
      'Magento_Review/js/validate-review': 'Magento_Review/js/validate-review',
      'Magento_Swatches/js/swatch-renderer': 'Magento_Swatches/js/swatch-renderer',
      'Magento_Catalog/product/view/validation': 'Magento_Catalog/product/view/validation',
      'Magento_Catalog/js/product/view/product-ids': 'Magento_Catalog/js/product/view/product-ids',
      'Magento_Catalog/js/product/view/product-ids-resolver': 'Magento_Catalog/js/product/view/product-ids-resolver',
      'Magento_Catalog/js/catalog-add-to-cart': 'Magento_Catalog/js/catalog-add-to-cart',
      'Magento_Catalog/js/validate-product': 'Magento_Catalog/js/validate-product',
      'Magento_Catalog/js/product/view/provider': 'Magento_Catalog/js/product/view/provider',
      'text!mage/gallery/gallery.html': 'mage/gallery/gallery.html',
      'text!Magento_InstantPurchase/template/confirmation.html': 'Magento_InstantPurchase/template/confirmation.html',
      'Magento_InstantPurchase/js/view/instant-purchase': 'Magento_InstantPurchase/js/view/instant-purchase',
      'Magento_Review/js/view/review': 'Magento_Review/js/view/review',
      'fotorama/fotorama': 'fotorama/fotorama',
      'mage/gallery/gallery': 'mage/gallery/gallery',
      'text!Magento_InstantPurchase/template/instant-purchase.html': 'Magento_InstantPurchase/template/instant-purchase.html'
    }
  },

All that remains now is for us to create the bundle files and deploy them for all our store views and themes. This is simply done with the magepack bundle command which you can execute from the Magento installation root folder. If you are running in development mode, deploy frontend static files first.

magepack bundle

Magepacl bundle command
Magepack bundle command

Finally enable Magepack Javascript bundling in admin :

Stores – Configuration – Advanced – Developer – Javascript Settings

Enable Magepack bundling in admin
Enable Magepack bundling in admin

If you are in production mode these options will be hidden in admin. To enable Magepack Javascript bundling from the command line use

bin/magento config:set dev/js/enable_magepack_js_bundling 1

Note that you should also enable the other Javascript optimisation options here including minfy javascript files and move js code to the bottom of the page – but don’t enable the default bundling!

MagePack Javascript bundling should now be enabled. To check it’s working go to a Magento 2 product page and look at the source code, do a search for “bundle” and you should see the magepack javascript bundles

Now refresh the page and have a look at your network log

After bundling there are only 7 js requests on the product page
After bundling there are only 7 js requests on the product page

Instead of loading 194 Javascript files, the product page now loads 7, Magepack has bundled all the Javascript into two main bundle files.

I guess it’s now time to look at the PageSpeed Lighthouse performance reports for your optimised Magento 2 pages. If you are using the Chrome browser simply run a Lighthouse report from the DevTools page. You can also use Googles PageSpeed insights tool at https://developers.google.com/speed/pagespeed/insights/

This is the improvement I saw in a live production Magento 2 site

Performance results before and after bundling
Performance results before and after bundling

If you don’t see a big improvement remember there are a lot of other factors taken into Lighthouse performance reports. Work through the report and try to find out where you can make further improvements.

Deployment in production

Whenever you flush your sites static files you will need to remember to run magepack bundle again. In production mode you should add this to your deployment process

bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento setup:static-content:deploy en_GB --area adminhtml
bin/magento setup:static-content:deploy en_GB --area frontend --theme MY/Theme -f
magepack bundle
bin/magento cache:clean

Top Tip – if for any reason you want to generate the magepack.config.js bundle config again remember to disable the Magepack module first!

Testing and Troubleshooting

You should test your store thoroughly to make sure there are no Javascript problems caused by the bundling process. Magepack cannot always 100% bundle all the Javascript required by some pages. Check your web browser console for errors. If you find some features of your store are not working, try and identify if the code was included in the magepack.config.js file. Try removing the code from the bundle and test again.

Magepack is pretty new with updates being made regularly, be sure to check out the projects GitHub page for new issues.

References

https://github.com/magesuite/magepack
https://github.com/magesuite/magepack-magento
https://www.integer-net.com/magento-2-javascript-bundling-integer-net-2/
https://gist.github.com/tdgroot/f95c398c565d9bbb83e0a650cdf67617
https://github.com/integer-net/magento2-requirejs-bundling
https://developers.google.com/speed/pagespeed/module

PageSpeed optimisation in Magento 1

Comments

  1. Kris says:

    This is interesting. Will it work though if you have several kinds of product pages, each using a different set of modules?

  2. PAJ says:

    It should work as any scripts that do not get detected by Magepack and bundled will still be loaded by Magento via the usual require process. You would need to identify these and manually add them to the bundle configuration for product pages.

  3. Rafael Corrêa Gomes says:

    Wow, this is a very good method to bundling, it seems a smooth integration, thank you for sharing it!

  4. mage2fanny says:

    Just a note about patching for MagePack Magento 2 Module, it has been merged into Magento 2.4
    So no manual patching required if you’re now on Magento 2.4!

  5. Peter Kincses says:

    Thank you for the post, I found it quite useful! Is there an option to bundle js form multiple stores which run from the same codebase with different urls? ie, x.com, y.com, z.com? We currently run 25 stores from the same codebase, not just different locale’s but different themes and not every store has the same modules enabled, hence js output can also be different.
    I’ve noticed that only one file is generated so if I run magepack generate for x.com and then magepack generate for y.com, y.com will overwrite x.com’s magepack.config.js.

    • PAJ says:

      I think this issue has been raised by others on github. I have used the module with a multi store Magento 2 but the stores were using the same parent theme so there were no major js differences between the themes. I guess what I would do would be to create a magepack config for each store naming them store1, store2 etc. then rename them to the magepack.config.js and deploy for each store theme. It’s a bit clunky but should work…

This site uses Akismet to reduce spam. Learn how your comment data is processed.