MageHost - Manage Varnish

Disclaimer: MageHost has joined forces with Savvii. Therefore, these articles are only relevant to existing MageHost customers. For more information on this, visit www.savvii.com/en/magehost.  

When we have installed Varnish on your hosting account, you can use these tools to manage Varnish.

Configure Magento 2 for Varnish

We have made a script which configures the basic settings to use Varnish with Magento 2. We recommend to test on a Staging account first.

~/bin/configure_varnish_magento2.sh

Varnish Configuration File

When the Varnish server starts or restarts, it loads the VCL file located at:

~/.config/varnish/default.vcl

If you don't use our script to configure Magento 2, make sure this script contains the right VCL. Inside this default VCL file you can also point to the Magento 1 VCL using an "include".

Restart Varnish

~/bin/restart_varnish.sh

This command will stop and start the Varnish daemon. The cache will become empty, the VCL config will be read again. If there is a fault in your VCL, Varnish will not start after stopping.

Note: You should only need/use this command when you are debugging and it should never be used during high traffic.

Certainly you should not add this command to your crontab. Please talk to us if you need to do it in a cron job.

Reload Varnish

~/bin/reload_varnish.sh

This command will instruct Varnish to re-read the VCL config. The syntax will be checked before reloading. Varnish will keep the cached data.

Purge an URL 

The matching URL will be purged from cache on all domains.

varnishadm ban req.url  '~'  '^/robots\\.txt$'

Purge the homepage (all domains)

The homepage of all domains will be purged.

varnishadm ban req.url  '~'  '^/?$'

Purge a Domain

All pages on a specific domain will be purged.

varnishadm ban req.http.host  '~'  '^(www\\.)?yourdomain\\.com$'

List URLs with most Cache Misses

This will create a list of the URLs that caused the most cache misses the last hour.

varnishtop -p 3600 -i BereqURL

Press Q to exit this tool.   

Investigating a low varnish hitrate

Using varnishstat, you can monitor your site’s hitrate. For high traffic environments, this is preferably around 90% (with a minimum of 80%). Sometimes, implementing new features, might impact the hitrate. Below are the most common issues and how to debug them.

Varnishlog

1.1 Recognizing patterns

As always, logging is key when investigating issues. Varnishlog is an easy-to-use tool and contains vital information. If you have no idea what pages are causing the low hitrate, you can start by filtering by misses.

varnishlog -q 'VCL_CALL eq MISS' -q 'VCL_CALL eq HITMISS'

When you have recognized a pattern (like the same page or the same type of pages appearing a lot), move to step 1.2. When there is no visible pattern, move to step 2.

1.2 Analyzing the patterns

When analyzing patterns, there are 2 common issues:

• The backend setting cookies (noticed the hit for miss filter?)
• HTTP content causing a hash variation The first one is easy to detect. If you see a hit for miss, check the headers of the page for Set-Cookie. The second one, requires analyzing hashes for specific requests. If you want to analyze this in production, the easiest way to do it is to filter the log by path.

varnishlog -q 'ReqURL ~ "$PATH"' 

Below is an example of a HASH

- Hash "https%00"
- Hash "/%00"
- Hash "varnish.m2.magehost.pro%00" 

In the example above, only the URL is part of the hash. For several CMSes, it should be like this by default. However, Magento for example includes vary cookies to segment different languages and currencies. Any discrepancy in this hash (like an URL query from a marketing redirect tool), causes variation in the cache and a lower hitrate.

Bans and purges

In a lot of cases, your CMS or certain plugins are a little too eager when it comes to invalidating cache. Recognizing this issue can be done by logging bans and purges over a longer amount of time. Here are the commands we use to filter the logs.

varnishlog -q 'CLI ~ "^Rd ban"'
varnishncsa | grep PURGE 

If you find the underlying issue, there are some standard solutions you could try. When having a lot of duplicate bans, you can queue your bans, delay them and make them unique. It also helps to check your timing / events. For example, Magento 2 bans a product on each purchase pre version 2.4.4. However, said product only has to be banned when the stock status changes (backorder or out of stock). It can also help to bundle item imports instead of spreading them.

Varnish CLI Tools

You can use these advanced Varnish CLI tools. We have configured them to know where to find your Varnish daemon. Most commands will work without commandline paramaters.

• varnishadm
• varnishhist
• varnishlog
• varnishncsa
• varnishstat
• varnishtop