4 min read
Large websites and performance in Craft CMS
We’ve been using Craft CMS for around 3 years now, and like most new CMS platforms the first projects you put through it are small. Mainly the gauge the capabilities and also longevity of the CMS.
But that time has now passed and we 100% trust Craft CMS capabilities to handle not only small projects but also large ones. For example - Our most recent project had over 10,000+ entries, 400,000+ users and a daily growth of around 20 articles and 100 users per day average.
By passing these types of big projects through the CMS we did come across a few hurdles and we thought it would be great to outline a few things that helped us along the way and also a few minor limitations to the platform.
Our latest project was a redesign of a site that ran previously on a bespoke CMS so one of the requirements was to transfer all of the content, categories and users over to Craft. Luckily, Josh Crawford built a very handy plugin to handle this - Feed Me (Part way through the project he also released v2 which is KILLER!)
For us, importing was a breeze with Feed Me. We exported the content from the old MSSQL database as XML (After many many complex SQL queries!) and imported these to Feed Me - Mapping the old table columns to the fields we’d setup in Craft CMS.
The only thing we had issue with was importing Assets. The site had over 150GB+ of images and Craft did not like indexing these files whatsoever due to the tasks timing out. Overtime we’ll be releasing a plugin that allows indexing of large asset folders. All is not lost if you use Amazon S3 though, A Digital (Matt Shearing) put together a little plugin to do just this - Asset Indexer.
One thing you should always use is the {% cache %} twig tags to stop your server from crushing under the load - Combining this with a CDN (Content Delivery Network) makes Craft CMS sing. We made the mistake of using just the standard {% cache %} tags, and a mediocre CDN, so the site ran pretty slow when we initially put it live (Not to mention around 5,000+ visits within about 1 hour).
With Craft, once an entry is updated or you clear cache manually in the settings, each page then needs to be cached again by the first user that visits that page. This is bad because it means a few users are going to see a slow loading page. What we ended up using was Cache Monster - This saves the cached files automatically after an entry has been saved, meaning users don’t need to trigger this task.
Because the site was also being updated regularly we decide to split the templates up using the {% cache %} tags, and setting time limits e.g. {% cache for 2 hours %} so that comments etc could be pulled through on to the news articles.
We also have started implementing Cache Flag, which allows us to tag ‘parts’ of the cached templates to be cleared at certain trigger points. Really useful.
Craft CMS comes with a native way of doing image transforms e.g. cropping to a particular width and changing the quality. For us though this wasn’t enough when dealing with images that were uploaded via various CMS users at various file sizes and dimensions.
We needed a way of compress the files and make them all unformed and consistent. This is where Imager saved our skin.
Imager allowed us to use features like TinyPNG and JPEGOptim direct on the server and shrink file sizes after they had been resized and cropped. It also works out great when moving the site between environments, allowing you to turn off certain features locally using their own config settings (Stored in /craft/config/imager.php)
You could also use this image compressor for file sizes up to 50mb in both JPEG and PNG format.
We did come across a few things using Craft which we still need to tweak and fix. This mainly was to do with performance of the users over 300,000+ and also the entries within certain sections. Accessing the ‘Users’ bit give us PHP memory issues even at 128M. Brad at P&T told us that they have noticed an issue with Craft 2.x when entries / users are around the 400,000+ mark, but give us a work around for this!
The only other downfall (Which isn’t Craft’s fault) is the database size… With so many entries users and cached templates the database is now around 3.5GB. Obviously it’s quite time consuming to transfer this between environments even using SSH.
The good news is they have also fixed the performance hits in Craft 3! So we can’t wait for this to be released to fix a few of our issues without plugins and hacks. We’re also hoping Craft 3 can help us out on the database size issue especially when moving this between environments.
-
Overall though Craft performed really well and mostly went smooth when putting a very large project through it. It was certainly a bit of a learning curve, but we’re now armed with more techniques to tackle other large projects.
If you have any pointers or questions we’d love to hear them.
I'm Jason, Co-Founder, and Lead Developer at MadeByShape. When i'm not walking my pooch Sully (Or him walking me), you'll find me cycling in the hills, at a gig or starting another DIY project.
