How to get a 100/100 on PageSpeed Insights by using the Advanced CSS/JS Aggregation module

You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

Most sites when using the current 7.x-2.31+ version of AdvAgg can achieve a perfect 100/100 score on https://developers.google.com/speed/pagespeed/insights/. Directions on how to achieve it for a fresh of install of AdvAgg below.

Be sure to check the site after every section to make sure the change didn't mess up your site. The changes under AdvAgg Modifier are usually the most problematic but they offer the biggest improvements.

Advanced CSS/JS Aggregation

Go to admin/config/development/performance/advagg

  • Select "Use recommended (optimized) settings"

AdvAgg Compress Javascript

Install AdvAgg Compress Javascript if not enabled and go to
admin/config/development/performance/advagg/js-compress

  • Select JSMin if available; otherwise select JSMin+
  • Select Strip everything (smallest files)
  • Click the batch compress link to process these files

AdvAgg Async Font Loader

Install AdvAgg Async Font Loader if not enabled and go to
admin/config/development/performance/advagg/font

  • Select Local file included in aggregate (version: X.X.X). If this option is not available follow the directions right below the options on how to install it.
  • Check "Use localStorage so the flash of unstyled text (FOUT) only happens once."
  • Check "Set a cookie so the flash of unstyled text (FOUT) only happens once."

AdvAgg Bundler

Install AdvAgg Bundler if not enabled and go to
admin/config/development/performance/advagg/bundler

HTTP/2.0 Settings

  • Under "Target Number Of CSS Bundles Per Page" select 25
  • Under "Target Number Of JS Bundles Per Page" select 25
  • Under "Grouping logic" select "File size"

HTTP/1.1 Settings (default)

  • Under "Target Number Of CSS Bundles Per Page" select 2
  • Under "Target Number Of JS Bundles Per Page" select 5
  • Under "Grouping logic" select "File size"

25 bundles vs 2 and 5 has to do with browser caching. More files equals a better chance of the browser having that combo in its cache, but more files means more connections being established and opened in HTTP 1.1. Use 2 for CSS as this number doesn't wait for any new connections; JS is 5 as most browsers have a concurrent connections limit of 6. This number for bundles was picked after many tests. In HTTP 2.0 there is one streaming connection and the penalty for multiple CSS and JS files is almost non existent; thus optimizing for the browser cache is the best choice; thus 25 should be used for CSS and JS when serving HTTP/2.0.

AdvAgg Relocate

Install AdvAgg Relocate if not enabled and go to
admin/config/development/performance/advagg/relocate

  • Select "Use recommended (optimized) settings"

AdvAgg Modifier

Install AdvAgg Modifier if not enabled and go to
admin/config/development/performance/advagg/mod

  • Select "Use recommended (optimized) settings"
Generating Critical CSS Files

Go to https://www.sitelocity.com/critical-path-css-generator and input as many landing pages as needed for critical css; the top 10 is usually a good start. If you have Google Analytics this will show you how to find your top landing pages https://developers.google.com/analytics/devguides/reporting/core/v3/comm...
or for Piwik https://piwik.org/faq/how-to/faq_160/. If you don't know what page to start with do your site's homepage. You can also us this to generate css https://chrome.google.com/webstore/detail/critical-style-snapshot/gkoeff...

Example filenames and paths below are for the "bootstrap" theme; they all start with sites/all/themes/bootstrap/critical-css/; in your theme create the critical-css/ directory. The next directory is based on the user; anonymous/, all/, or authenticated/ can be used.

The next directory can be urls/ or type/. Looking at urls/; front is a special case for the front page, all other paths use current_path() as the directory and filename with .css added to the end; See https://api.drupal.org/api/drupal/includes%21path.inc/function/current_p...

Looking at type/ this is a special case for node types. Use the machine name and add .css to the end. Any node of this type will then have this critical css file applied and inlined. Below are some examples showing how this works.

valid example file locations for the "front" page:
sites/all/themes/bootstrap/critical-css/anonymous/urls/front.css

valid example file locations for "node/1" current_path() page:
sites/all/themes/bootstrap/critical-css/anonymous/urls/node/1.css

valid example file locations for the node type of "page":
sites/all/themes/bootstrap/critical-css/anonymous/type/page.css

If you want some sort of automated way to generate these css files fourkitchens
has an excellent article on how to set that up:
https://fourword.fourkitchens.com/article/use-grunt-and-advagg-inline-cr...

Comments

AdvAgg

beautifulmind's picture

I have configured my Drupal 8 websites using AdvAgg and they are doing quite good. However, there are aspects like Images and browser caching which still need to be addressed. Even if you do at your end, chances are you may not get 100/100

Regards.

AdvAgg 7.x will handle CSS/JS browser caching

mikeytown2's picture

External 3rd party CSS and JS code can now be kept locally taking care of most browser caching issues (examples being google analytics and facebook pixel). Images can be handled by using image styles and https://www.drupal.org/project/imageapi_optimize.

There are some things on the page that are too hard to do like if you have an embedded google map; getting a 100/100 with that on the page is impossible for the most part. But for the average site 100/100 is now attainable with the 7.x version of AdvAgg.

reinvented's picture

I was able to achieve 100/100 on my website, by following this guide. I got a little hung up, however, on the final Generating Critical CSS Files step, so here's a little elaboration, using this article page as an example.

Generate the Critical CSS

I went to the Critical Path CSS Generator and pasted in the URL https://ruk.ca/content/pei-peak-load-january and generated the critical CSS.

I then removed the opening <style> and closing </style> from the result.

Add Critical CSS to Theme

My site is ruk.ca, my theme is named at_ruk, and the content type of the page I want to affect is article, so I created:

sites/ruk.ca/themes/at_ruk/critical-css/all/type/article.css

and pasted the CSS I generated above into this file.

Verifying The Result

With this CSS file in place, when I View > Source on the page in question, I see that the CSS has been inlined in the head, right under the title:

<style type="text/css" media="all">
...and so on...
</style>

At this point, feeding the URL of this page into Page Speed Insights jumped my Mobile and Desktop scores both to 100/100 (from 57/100 and 93/100 respectively).

Tanked my PageSpeed score

dalehgeist's picture

Sort of. Without AdvAgg I got a 73 on desktop and a 52 on mobile. I installed AdvAgg 7.x-2.30, and configured it as above with the following exceptions:
- Modifications: Move JS to the footer - DISABLED
- Modifications: Deferred JavaScript Execution: Add The defer Tag To All Script Tags - DISABLED

The reason for those exceptions is that they hosed a map widget (JVectorMap) that I need.

Also, I did not generate critical CSS, just because it seemed like a lot of work and I wanted to see what my result was without it.

My result (after marinating for 16 hours): 78 desktop (with a Speed of Slow) and 39 mobile (also with a Speed of Slow).

What PageSpeed says it's hating, a lot, is the render-blocking resources.

I am impressed with your great support and I'd love to love AdvAgg. Any thoughts?

The site is http://onlinedriversed.com

Optimization Suggestions

mikeytown2's picture

Reduce server response time:
- Is your page cache enabled? Go to "admin/config/development/performance" and under "Caching" make sure "Cache pages for anonymous users" is checked.

Eliminate render-blocking JavaScript and CSS in above-the-fold content:
- Defer JS and critical CSS will fix this. The guide above explains what to do.

Leverage browser caching:
- The relocate module has a checkbox for "Move inline fbevents.js loader code to drupal_add_js", make sure it's checked.

Here's what the webpage looks like on a slow mobile connection
http://www.webpagetest.org/video/view.php?id=180119_EK_32be8931a466a0b45... Some takeaways from looking at this:
- Use the AdvAgg Async Font Loader submodule.
- If Eliminate render-blocking was fixed you could cut the start render time in half from 8.7 seconds down to 4 seconds.
- Check the "dns prefetch" and "preconnect" boxes on "admin/config/development/performance/advagg".
- Once you have the front.css file for critical css saved (see guide above) you should create 2 more files front.dns and front.pre.
In front.pre add the following:

sites/default/files/ode_logo.png
sites/default/files/girl-driving.jpg
sites/all/themes/Porto/vendor/fontawesome/fonts/fontawesome-webfont.woff2?v=4.4.0

for front.dns you'll want to run the http://www.webpagetest.org/ test again and add in any domains that have a dns (Blue) , connect (Orange), or ssl (Purple) bar under http://www.webpagetest.org/result/180119_EK_32be8931a466a0b45bcb8f1459ae... like so
http://widget.trustpilot.com/
https://static-v.tawk.to/
https://cdn.jsdelivr.net/
https://va.tawk.to/
https://vs34.tawk.to/

Doing all of these will cut about 5 seconds off of your Document Complete time, cut your start render time in half (4 seconds), and give you a page speed score in the upper 90's if not 100.

Results

dalehgeist's picture

You're a champ. Very much appreciated.

Here's what I did, what I had already done, what I didn't do (and why), and my results:

  • "Make sure 'Cache pages for anonymous users' is checked."
    Didn't do it. I recalled that it was disabled because it had invoked a weird bug. I'll fix the bug for real, then enable it.

  • "Defer JS"
    Can't do it at this time. It hoses a map widget I need. I'll try and sort this out.

  • "Critical CSS"
    Did it.

  • "Move inline fbevents.js loader code to drupal_add_js"
    I had already done this

  • "Use the AdvAgg Async Font Loader submodule."
    Enabled it. (On my test environment, it showed a version for ffo on github. On production, it said version: NULL. Thoughts?)

  • 'Check the "dns prefetch" and "preconnect" boxes'
    I'd already done it.

  • 'create front.dns and front.pre'
    Did it, but wasn't sure where they should go. They're currently in /sites/all/themes/[mytheme]/critical-css/anonymous/urls (along with front.css)

RESULTS: 77 desktop, 36 mobile

Some things are working some are not

mikeytown2's picture

"Critical CSS"
Did it.

Unfortunately it appears to not be working; I do see the file located at sites/all/themes/Porto/critical-css/anonymous/urls/front.css; did you select "All in head, use link rel="preload" (recommended)" and "File Controlled (Recommended).". You could also try copying the front.css file to sites/all/themes/Porto/critical-css/anonymous/urls/node/19.css. One last thing to do is on the admin/config/development/performance/advagg page pick "Normal ~ 60ms" under AdvAgg Cache Settings, as there is a bug with the render cache under certain conditions that I'm working on.

Looking at the site and I can see that the async fonts are working as you can read the text right away in this video http://www.webpagetest.org/video/view.php?id=180119_9A_61107aa5ecb54f1c5.... As for version coming back as NULL that means that the http request from your server to this file https://raw.githubusercontent.com/bramstein/fontfaceobserver/master/pack... that checks the version number is not working. This would also prevent the relocate module from working which seems to make sense from what I can tell. This would most likely be a hosting configuration change.

Check the "dns prefetch" and "preconnect" boxes'
I'd already done it.

This most likely points to an issue with your theme or another module not working 100% correctly with drupal_add_html_head()/drupal_get_html_head(). Looks like it's a paid theme so I'd ask them for support on this issue; see if they use the standard $variables['head'] location for drupal_get_html_head() that is shown in template_process_html().

'create front.dns and front.pre'
Did it, but wasn't sure where they should go. They're currently in /sites/all/themes/[mytheme]/critical-css/anonymous/urls (along with front.css)

Yep this is the correct spot; but it won't work due to issue with drupal_get_html_head as mentioned above.

Thanks

dalehgeist's picture

I truly appreciate your responsiveness and detailed feedback. I will definitely buy you a beverage of your choice if we are ever in the same town at the same time.

I had not selected "All in head, use link rel="preload" (recommended)" and "File Controlled (Recommended)," (I didn't see them) so I've just done that. Without letting it marinate, I'm back up to 46 mobile and 78 desktop.

(Version: NULL for ffo was only there when I first opened the UI. When I went back and checked, it had found the correct version on GitHub.)

I'll try to contact the theme creators and see what they have to say about the location of $variables['head']. (I inherited this site, and, poking around the theme, it looks like they do a few nonstandard things.) The only reference I have found to drupal_get_html_head() in my /sites/all folder (which includes all contrib modules and themes) is in the AdvAgg module. Should I be finding it in the theme's template.php or html.tpl.php file?

I suspect I'm getting killed by not having 'Cache pages for anonymous users' checked, am I right? Unfortunately, the site was engineered in such a way that the nav menu gets swapped out when you select a state (that's right, there are 51 nav menus!). If the page is cached, when you change states, you see two menus. Re-engineering this is a larger effort that I can't undertake yet.

Thanks again!

Ideas

mikeytown2's picture

This might give you some ideas on how to get the page cache working in you case
https://www.drupal.org/project/switchtheme/issues/361832#comment-4204294

blocking css

mikeytown2's picture

Looking at the report and looking at the html source there are a couple more things that need to happen to fix the render blocking issue. These lines of code are hard coded in your theme. Adding them via drupal_add_css() would let advagg know about them and they could then be deferred correctly.

<!-- Call bootstrap.css before $scripts to resolve @import conflict with respond.js -->
<link rel="stylesheet" href="/sites/all/themes/Porto/vendor/bootstrap/bootstrap.css">

<!-- Web Fonts  -->
<link href="//fonts.googleapis.com/css?family=Open+Sans:400,300,600,700,800&subset=latin,latin-ext" type="text/css" rel="stylesheet">
<link href='//fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>

If this is not possible then changing the css resource link to this in the theme; do it for web fonts as well.

<link type="text/css" rel="preload" href=""/sites/all/themes/Porto/vendor/bootstrap/bootstrap.css" as="style" onload="this.rel=&#039;stylesheet&#039;" />

Implemented this, did not fix render blocking

dalehgeist's picture

Like the Subject line says. Still stuck at 41 and 79. :(

There are 4 resources the page is trying to load (three images and a video) that are getting 404s - could that affect it? I'd like to hunt them down in any case.

There's also something really messed up about this install, which might be affecting performance: there are multiple theme folders inside the site's intended theme folder - and the site is using the templates from one of these inner folders.

(I'm working on fixing this.)

The CSS is no longer render blocking

mikeytown2's picture

The JS is now the only part that is render blocking, being able to defer that should make your score jump up quite a bit. If the report isn't complaining about a 404 then it won't affect the score most likely (still get it fixed). Looking at http://www.webpagetest.org/result/180122_EJ_25626c10f94e724b4ee4964fb176... confirms that JS is blocking the rendering; the start render time should drop by about 4 seconds once you get this fixed.

The other thing is to fix the page cache; using a cookie like in the example given is probably the easiest way as cookies can be changed via js and read for the drupal cache.

Final step is to fix the Facebook Pixel Code; add that via drupal_add_js instead of inlined js and advagg can then relocate it to be local.

More info

dalehgeist's picture

I've done some extensive testing on the code that breaks when "Deferred JavaScript Execution: Add The defer Tag To All Script Tags" is enabled.

There were two .js file references plus an inline script that were living in the database (body field of the node). I put the file references in template.php (using drupal_add_js), wrapped in a test for the node ID, and I put the inline script into the theme's script file. To verify that everything was working as intended, I turned on the AdvAgg bypass cookie. It was working - yay.

Then I turned off the AdvAgg bypass cookie, and it broke.

Here's what's interesting: I verified that the theme's script file was aggregated - but the two other file references never got aggregated!

They live right next to some other file references in template.php that did get aggregated. The only difference is this: the ones that got aggregated live in the theme folder; the ones that did not get aggregated live in /sites/all/libraries.

Does this make sense to you?

Confirmed

dalehgeist's picture

That was the problem, all right. I moved the files into the theme folder and they aggregated just fine. Oddly, there is also a CSS file in the /libraries folder that is referenced the same way, and it seems to have gotten aggregated just fine.

I'll push that change, see what it does to PageSpeed, then move on to your other recommendations.

Cheers!

Getting better

dalehgeist's picture

55 and 85.

I'll try to figure out that page cache trick with cookies, and I'll try and hunt down the FB pixel code and add it with drupal_add_js

Good job!

mikeytown2's picture

https://developers.google.com/speed/pagespeed/insights/?url=http%3A%2F%2...

Not being able to cache the page is the last thing holding you back. The "Leverage browser caching" is icing on the cake at this point. Get the page cache fixed and then add in the inline facebook pixel via drupal_add_js() if you want to go the extra mile.

Looking at http://www.webpagetest.org/result/180125_KA_4e9caeab9d0a172cdd25b10a14b7... and the start render time went from 8+ seconds down to 4.5 seconds. That's a huge improvement!

Can you add the following to your front.dns file? This should help with the Document Complete time.

97 and 99

dalehgeist's picture

The caching code was gonna be a challenge for me - would have taken me ages to mod it for my purposes. I found a module called cacheexclude that worked perfectly. It allows you to designate pages that don't get cached - and since there is only one page that everyone must use to switch states, I excluded that page from caching.

Before caching for anonymous users, I was up to 55 and 85. Now, with anonymous caching turned on, I've got 97 & 99. I'm good with that.

Just added the recommended domains to front.dns. You're a wizard!

I'll see if I can locate the source of the FB pixel code and drupalize it.

Chrome User Experience report

mikeytown2's picture

Data from the Chrome User Experience report indicates this page's median FCP (3.4s) and DCL (3.6s) ranks it in the bottom third of all pages.

https://developers.google.com/web/tools/chrome-user-experience-report/ I don't know how long it will take for the FCP & DCL data to be updated; looks like the data set is from November 2017.
FCP = First Contentful Paint (Start render time)
DCL = DOM Content Loaded (Usually when JavaScript is done)

You might be able to view more recent data here https://developers.google.com/web/tools/chrome-user-experience-report/ge.... These tools are all fairly new.

Results

mikeytown2's picture

Before: http://www.webpagetest.org/result/180119_EK_32be8931a466a0b45bcb8f1459ae...

After: http://www.webpagetest.org/result/180126_GH_4e5490b9b46b216a968aa09585ed...

I see some issues with AdvAgg that will hopefully be addressed with the next version; so when it gets released you should see a better Speed Index metric (lower is better) as well as a better Document Complete time (lower is better). For now going from a Start Render time of 8.7s down to a 2.9s is a huge improvement, and it shows with a Google Page Speed score in the upper 90's.

Next

dalehgeist's picture

onlinedriversed.com was my test case. My big site is trafficschoolonline.com.

When I first ran PageSpeed I got a 68m, 83d. After turning on standard Drupal caching, I was up to 85m, 83d. When I moved a bunch of files from templates and blocks into drupal_add_js and drupal_add_css and turned on AdvAgg, that dropped me down near where I started. Then I did the steps in this article, plus some of the things you recommended in your replies, and now I'm up to 91m, 91d.

Here are some questions: Insights says I should "Leverage browser caching" on 8 resources. I had already created a front.dns file and uploaded it to /sites/all/themes/[mytheme]/critical-css/anonymous/urls - is that supposed to help with browser caching?

Insights also says I've got 4 CSS files that are render-blocking - they are all identical AdvAgg files. I did generate critical CSS files per instructions - could it not be seeing them?

As always, grateful for your thoughts.

(PS - there was one external JS file I was loading through drupal_add_js that refused to get aggregated by AdvAgg. Weird. I ended up realizing I didn't need it, so I stopped loading it. Do you know what could cause that?)

The "Your page has 4 blocking

mikeytown2's picture

The "Your page has 4 blocking CSS resources" and "Prioritize visible content" appears to be a failure of https://www.sitelocity.com/critical-path-css-generator not giving the best output. Viewing the source, and the critical css is there; just that only 48% of the page gets rendered with it (without it its 0%). I'll be working on an alternative to this site in the future but for now there's just a couple alternatives.

Chrome extension
https://chrome.google.com/webstore/detail/critical-style-snapshot/gkoeff...

Other websites
https://jonassebastianohlsson.com/criticalpathcssgenerator/
https://criticalcss.com/

The Leverage browser caching part isn't as critical; try to get a better front.css. If needed we can look into this section more, I will say the relocate module should handle some of the issues that it's found here though.

Solved it

dalehgeist's picture

Where we left it: the Chrome plugin worked great - except that, since it grabbed the CSS from my desktop window, it applied the wrong styles to phone and pad, rendering them useless. Oops!

Here's what worked:
- In front.css, create three breakpoints
- Using the device toolbar in Chrome Dev Tools, choose a small screen
- Use the Chrome plugin to get that CSS
- Paste it into the small breakpoint
- Repeat the process with medium and large

This got me back up to 99m / 93 d, which isn't quite where I used to be (I was at 95 d), but it's acceptable for the time being.

Solved it

dalehgeist's picture

Where we left it: the Chrome plugin worked great - except that, since it grabbed the CSS from my desktop window, it applied the wrong styles to phone and pad, rendering them useless. Oops!

Here's what worked:
- In front.css, create three breakpoints
- Using the device toolbar in Chrome Dev Tools, I choose a small screen
- Use the Chrome plugin to get that CSS
- Paste it into the small breakpoint
- Repeat the process with medium and large

This got me back up to 99m / 93 d, which isn't quite where I used to be (I was at 95 d), but it's acceptable for the time being.

No change

dalehgeist's picture

I used the Chrome extension to generate front.css, but as of now, my Insights results are the same.

I also ran across an interesting artifact: AdvAgg has killed my AddThis code. It appears to have been aggregated, but it's not working any more. I'm guessing it has to do with the order in which it loads, but I don't really know how to change that.

Cheers!

http://www.webpagetest.org/vi

mikeytown2's picture

http://www.webpagetest.org/video/compare.php?tests=180129_CJ_35ecf01cbbe... shows that the above the fold background is white with white text; making the background grey might help. Add this to the bottom of the front.css file and try again.

section.fold{background-color:#898077}

Wow

dalehgeist's picture

99m, 95d

And all I did was add that CSS rule. (I also cc all, which I don't think I did after I updated front.css last time.)

You're a damn genius, my friend.

Cheers!

Caution

dalehgeist's picture

I hadn't checked my mobile site until today. Yikes: the front.css that was generated by the Chrome plugin totally messed it up.

This makes sense, of course: the plugin was looking at the window that was open, which was desktop. So folks should use caution when generating their critical CSS this way.

(The other two links you gave were to the same guy's website - one of them didn't work - probably deprecated - and the other one required a fee. Which I would pay if I was doing this a lot.)

So I went back and generated a new one from https://www.sitelocity.com/critical-path-css-generator and I'll just live with imperfect output until there's a better tool.

EDIT: For my solution, see https://groups.drupal.org/node/517292#comment-1156768