Reducing the Size of a Large (50GB) WordPress Blog

This is a technical post. It’s about reducing the storage needed for a large WordPress-based blog in order to save on hosting costs. It talks about offloading a large number (100s of thousands) of images from a shared Hostgator server to Backblaze B2 cloud-based storage.

The Problem Being Solved

This blog’s been around for 15 years and has grown organically to around 50GB on a shared platform. The host (Hostgator, based in the US) turned a blind eye to us being over our package storage limit for many years. Our renewal was coming up though, and they emailed to say we were being shifted from their Hatchling Package to their Business Package, which shifted our hosting costs from around $400 to $800 over three years.

The blog’s not actively being updated at the moment as we’ve other stuff going on in life so our travels are curtailed. We really wanted to keep it alive though, but weren’t so keen on paying for a business plan which, apart from the storage, we really didn’t need.

The Options

We looked around for another host which would provide 50GB+ of storage for a lower cost but didn’t have much luck. We’re not that keen on going through the whole rigmarole of a migration either. The blog’s old and has been through various wordpress versions and had umpteen plug-ins installed and removed over the years. Getting it across to a new host in one piece would probably have been possible, but we just didn’t have the enthusiasm for it.

Firing up chatgpt and explaining the problem, it suggested offloading the images to cloud storage. The basic approach would be:

• Keep using Hostgator as our WordPress host
• Set up storage on a cloud platform like Amazon S3
• Install a WordPress plug-in to carry out the image migration
• Use the plug-in to copy the files from our Hostgator storage to the cloud platform
• The same plug-in would update the database to point each image at the new cloud location
• Once completed, I’d remove the files from Hostgator and ask them to downgrade the package back to Hatchling

We ended up using Backblaze B2 rather than Amazon S3 as Amazon wanted us to do some faffing with DNS entries which I really wasn’t fussed about doing. The result would have been our images still had ourtour.co.uk addresses, but I’m not sure anyone will ever care they’re being served up from elsewhere. Might become an issue in future, dunno.

The Costs

Cloud storage is cheap. Backblaze B2 charge $6 per TB per month. 50GB is roughly £10 over three years at that rate, saving us about £290 in hosting costs. The plug-in we used (Advanced Media Offloader) is free.

Cloud storage providers also charge for ‘egress’ fees (people downloading your files by visiting your site, or you moving them to another stroage region). Backblaze B2 provide free egrees for up to 3 x your storage per month, so around 150GB a month for us. At present we’re seeing around half that on our site (75GB a month), so won’t pay for egree unless we become much more popular! Above the 3 x storage, egress is $0.01 per GB, so if we saw a total of 250GB per month of downloads it would cost $1 per month.

We also opted to keep using Jetpack to back up the site, costing another $70 a year. We tried the free BackWPup plug-in but it repeatedly failed with a PHP memory limit error. Jetpack/Vaultpress has always worked well for us, is easy to use and has saved our site a few times, so we’ll stick with it. The back-up no longer covers the image files of course, so we’ll need to have a think about how we handle this in future.

The Offload Process

The offload process wasn’t as simple as I’d hoped but worked in the end. The Advanced Media Offloader can be used through the Dashboard or through an SSH/terminal connection to the server. The dashboard (web) version only does 50 attachments at a time. We’d over 20,000 so would need to run it manually 400 times. Fortunately there’s a command line version. I found this would offload around 200 to 400 attachments at a time before bombing out.

I ran the offload job though SSH manually for a few days, restarting it whenever the server killed it, but it would stop overnight. In the end I used chatgpt to write a cron job which ran on the hour every hour. This killed any previous offload task and restarted it. With that running 24-7 the task was completed within 24 hours.

The plug-in shifted 242,000 files to Backblaze without any real issues. It errored on a few of them, but the next session picked these up and shifted them. The images are now all stored on Backblaze EU servers (which I think are in the Netherlands). This *might* speed the site up a little, as previously there were all in the US and we weren’t using a CDN (unless Hostgator were doing something automatically).

The Issues

The main issue was the fact the plug-in missed many image references, so hundreds of posts still referenced files on Hostgator. I spotted this early in testing and restarted the offload task to the URLs on Backblaze were predictable, rather than containing any version numbers. This enabled me to use chatgpt to create a couple of fixes:

• It provided .htaccess entries which automatically replaced any request for an image file from ourtour.co.uk to the same image on Backblaze. This is still in place as a fall back. I made sure I’d a .htaccess file backup before editing it.
• It also provided the SQL needed to replaced database references to ourtour.co.uk images with the equivalent URL on Backblaze. I tested this on one post first and ensured I had a database back-up before running for the entire site.

Throughout the process I ensured I had Jetpack (Vaultpress) back-ups in case I needed to revert back, which I needed a couple of times. Whenever I came across an issue I’d upload the HTML, .htaccess entry, SSH command/result to chatgpt and it helped me resolve the issue every time.

The Conclusion

The site is now around 3GB on Hostgator and they’ve shifted us back down to the cheapest plan (which allows up to 10GB). The images all appear to load OK from Backblaze and new images are automatically saved in Backblaze when loaded to the Media Library. I need to ensure the new back-up/restore process works. We also need to see how reliable Backblaze proves to be.

The offload process was worth running for us, although without chatgpt solving lots of small issues it would have been a much more laborious process. In the end we appear to have saved us a few hundred quid and have future-proofed the blog in terms of storage growth.

Cheers, Jay