Manage Varnish

When we have installed Varnish on your hosting account, you can use these tools to manage Varnish.

Configure Magento 2 for Varnish

We have made a script which configures the basic settings to use Varnish with Magento 2. We recommend testing on a Staging account first.

#Run the script from your magento2 root folder. 
~/bin/configure_varnish_magento2.sh

Varnish Configuration File

When the Varnish server starts or restarts, it loads the VCL file located at:

~/.config/varnish/default.vcl

If you don't use our script to configure Magento 2, make sure the file contains the right VCL. Inside this default VCL file you can also point to the Magento 1 VCL using an "include".

Reload Varnish

~/bin/reload_varnish.sh

This command will instruct Varnish to re-read the VCL config. The syntax will be checked before reloading. Varnish will keep the cached data.

Restart Varnish

# Restart via mnage
mnage service:restart varnish
# Restart Script
~/bin/restart_varnish.sh

This command will stop and start the Varnish daemon. The cache will become empty, the VCL config will be read again. If there is a fault in your VCL, Varnish will not start after stopping.

Note: You should only need/use this command when you are debugging and it should never be used during high traffic.

You should also not add this command to your crontab. Please talk to us if you need to do it in a cron job.

Purge a URL

The matching URL will be purged from cache on all domains.

varnishadm ban req.url  '~'  '^/robots\\.txt$'

Purge the homepage (all domains)

The homepage of all domains will be purged.

varnishadm ban req.url  '~'  '^/?$'

Purge a Domain

All pages on a specific domain will be purged.

varnishadm ban req.http.host  '~'  '^(www\\.)?yourdomain\\.com$'

List URLs with most Cache Misses

This will create a list of the URLs that caused the most cache misses the last hour.

varnishtop -p 3600 -i BereqURL

Press Q to quit.

Investigating a low Varnish hit rate

Using Varnishstat, you can monitor your site's hit rate. For high traffic environments, this is preferably around 90% (with a minimum of 80%). Sometimes, implementing new features, might impact the hit rate. Below are the most common issues and how to debug them.

Varnishlog

1.1 Recognizing patterns

As always, logging is key when investigating issues. Varnishlog is an easy-to-use tool and contains vital information. If you have no idea what pages are causing the low hit rate, you can start by filtering by misses.

varnishlog -q 'VCL_CALL eq MISS' -q 'VCL_CALL eq HITMISS'

When you have recognized a pattern (like the same page or the same type of pages appearing a lot), move to step 1.2. When there is no visible pattern, move to step 2.

1.2 Analyzing the patterns

When analyzing patterns, there are 2 common issues:

• The backend setting cookies (noticed the hit for miss filter?)
• HTTP content causing a hash variation. The first one is easy to detect. If you see a hit for miss, check the headers of the page for Set-Cookie. The second one, requires analyzing hashes for specific requests. If you want to analyze this in production, the easiest way to do it is to filter the log by path.

varnishlog -q 'ReqURL ~ "$PATH"' 

Below is an example of a HASH

- Hash "https%00"
- Hash "/%00"
- Hash "varnish.m2.savvii.pro%00" 

In the example above, only the URL is part of the hash. For several CMSes, it should be like this by default. However, Magento for example includes vary cookies to segment different languages and currencies. Any discrepancy in this hash (like a URL query from a marketing redirect tool), causes variation in the cache and a lower hit rate.

Bans and purges

In a lot of cases, your CMS or certain plugins are too eager when it comes to invalidating the cache. Recognizing this issue can be done by logging bans and purges over a longer amount of time. Here are the commands we use to filter the logs.

varnishlog -q 'CLI ~ "^Rd ban"'
varnishncsa | grep PURGE 

If you find the underlying issue, there are some standard solutions you could try. When having a lot of duplicate bans, you can queue your bans, delay them and make them unique. It also helps to check your timing / events. For example, Magento 2 bans a product on each purchase, before version 2.4.4. However, said product only has to be banned when the stock status changes (backorder or out of stock). It can also help to bundle item imports instead of spreading them.

Varnish CLI Tools

You can use our CLI Tool dbug for quick access to live data and varnish stats, check out the dbug page for more information.

We have also configured the following advanced Varnish CLI tools to use your Varnish daemon out of the box.

• varnishadm
• varnishhist
• varnishlog
• varnishncsa
• varnishstat
• varnishtop