Jekyll minification optimization
Jekyll minify intro
So as you can see, I build websites with Jekyll static site generator a lot. The problem with this is the jekyll implementation is usually used on GitHub for internal sites, so I couldn’t find a tool that did some things that I wanted to increase page load speed, clear offsite cache servers, and reduce bandwidth use before actually making a page live on the internet. I have two servers that run oxasploits.com, one under a staging domain, so it makes it easier to manage posts that are semi-live for testing, but are not yet propogated to the main server where they will display on oxasploits.com.
People just don’t like slow websites.
Next I’m going to go over the functions and give quick rundown on the necessecivity and logic behind each one.
Checking then building with Jekyll
So here we are just verifying that the necessary dependancies are installed before we start moving things around and rebuilding code, not knowing if it will even come to frutition.
Then we have our build function below, which just locally builds the site, and checks to make sure it was properly built before continuing.
So one of the primary, and easier things you can do to reduce total page size, is to simply compress your images. Most JPG images are set at a default of around 70 as their compression level, which is great for say, your home image galary, it is balanced between looking good, and saving a little storage space. This however isn’t great for being loaded on the fly from a website… it’s just too much data, especially if your images are higher resolution.
So as you can see, this is my pretty simple compression function. It just takes a user defined, or default compression level (set lower) of 40. The smaller the number the smaller the file, but the worse it looks… so be cautious not to set this variable too low, or your images will look very pixelated, users don’t really appriciate that either. I also resize the image to slightly larger than the viewport in the DOM where the image is displayed in my Jekyll theme to have a maximum width of 630 pixels. We preserve the original images, instead of overwriting, in case something goes wrong… like the quality was sub par.
Inlining of CSS
So the theory of having external CSS files, is it’s just easier to manage, and you do not need to write the same or similar CSS code over and over in each page. However, it’s actually more efficient for the browser to take as much as it can in, in as few files possible. Also CSS can block page loading, so for example the DOM is ready and waiting, and the HTML has been sucessfully loaded by the browser, however it is waiting on a large CSS file to be loaded so that it has an idea of how the page should look, as far as colors, shapes, font faces, etc. The last reason we inline CSS is because the CSS inlining parser can decide what pieces of the then .css file will be used by the browser in each specific spot, so we coul have been loading dormat code that applied to a different page… that’s no good, just slowing things down and making the page bulkier and the process more memory intensive.
So our first objective in this code block is to generate the correct CSS file that would have been applied, by running the .scss through sass to merge the pieces, as they are designed to be dynamic. Then for each HTML file that
jekyll build has genearted, we will open 8 threads, as the process is a little bit process intensive. Each HTML file then has CSS merged itno it, leaving only an HTML file left, with only what CSS is necessary for that page in it. We then, again, loop over each HTML file and move what we originally saved it as, back to what the filename should be. This process is to automatically correct if, say, there was a merge error in one of the CSS or HTML files, and the output was never generated, in that case, the original file still remains.
This function is similar to the inlining function above, and works nearly the same way, except that it’s primary function is to take the HTML and remove uncessary to it’s functionality things, such as whitespace, comments, fragments, empty elements.
This makes the page considerably smaller, and thus less data to transfer, and load, without changing the way the page acts in practice. Many, even very high traffic sites do this, including google.com, because the optimization, while not strictly necessary, does improve load time. It also is a CPU intensive process, so it is done in parallel.
Pushing it to the internet or staging server
So the only, athough somewhat anticlimactic, funtion necessary to the project is to push the contense of
_site/ to our webserver. Here I do this with rsync, though in an earlier version it was scp. The rsync way seems to work better though becuase we can run a checksum on each file, and then when it comes time for it, only clear what parts of the cache that changed, instead of the entire site, hence increasing our cache hit rate.
Clear the Cache and Housekeeping
Since I have a caching server in front of my webserver, and would otherwise need to manually clear the cache for each page I changed, to keep my cache hit rate high, I parse the rsync output from the previous section, and save it to a file called modified.txt, which we then run through jo, an on the fly json generator. Once we have our json output we cna then make an API call to my caching servers at Ezoic. An API key for this is ncessary, and can be specified through a command line argument. Clearing the cache is optional though, as the servers will hit it eventaully by design, but you wouldn’t be gaurenteed an up to date page otherwise.
and then clean up our temporary files…
We then run a cleanup function, that removes all our temporary files used, the html.2 and html.3 files, modified.txt, rsync.txt, as well as cache_clear.txt.
So this little utilitiy has already considerably incrased my page load speed, as well as pushed me higher in the pageranks with a more responsive website. It also saves me a ton of time, and is designed to be easily modifiable to be used with other projects, for example Hugo, different types of cache server, etc.
Hope you have had as much fun reading, as I have writing this. Thank you!!
If you enjoy my work, sponsor or hire me! I work hard keeping oxasploits running!
Thank you so much and happy hacking!