Databases are slow.
I shouldn’t say that, because I’m a DBA, but it’s the sad truth – databases are often the bottleneck for web apps. WordPress is just another web app – it builds pages by querying a MySQL database. If we want WordPress to scale, we need to minimize the number of times it has to hit the database. Reducing the number of plugins helps, but sometimes it’s still not enough.
I used to use the excellent WP-Super-Cache plugin to cache my blog pages. It’s a filesystem-based cache that saves generated versions of your pages. It works great, but you have to be very careful that all of your other plugins work well with it. For example, I tried using a Google AdSense plugin that checked where you were coming from, and if you’d stumbled in via Google, it showed Google ads related to your search query. Unfortunately, once that page got into cache, EVERYBODY saw the ads – whether they’d come from Google or not. Bad. I had to check every plugin I used and determine whether it was impacting my caching, and I ended up pulling a lot of good plugins out just so I could scale to more viewers.
W3 Total Cache has a whole different bag of tricks. It uses memcached, a memory-based cache, and as a result it appears to be much faster than WP-Super-Cache. The downside is that memcached isn’t available on most shared hosting providers. It’s not even usually installed on dedicated virtual servers, so you’ll need to follow a few instructions to get memcached up and running first. MediaTemple users can follow their memcached setup instructions. (This is yet another reason why I love MediaTemple – their documentation is stellar.) Other hosting company users still get an excellent checklist in the W3 Total Cache Installation page built into the plugin.
Since it’s a memory-based cache, memcached also requires more memory than WP-Super-Cache. MediaTemple recently rolled out free memory upgrades to their hosting customers (did I mention that I love MediaTemple?) so I had more than enough available memory to handle it.
Beyond Caching: Options for Content Delivery Networks
When your site hits the front page of Digg or Slashdot, you might want even more scalability. Content Delivery Networks, or CDNs, host your static files (images, css, js) on their servers, thereby reducing yet more load on your web server. CDN companies replicate content across their worldwide servers, thereby making your pages render faster for users on other continents.
This option is probably overkill for 99.9% of the blogs out there, but when you need it, you have no time to set it up. If you’re writing the kinds of content that you hope will end up on Digg’s home page, then you need to set up a CDN (and caching, for that matter) long before you need it. My web site got knocked offline in May when Sherri Shepherd, one of the hosts on The View, linked to it, and I had to scramble to set up better caching and a CDN. My site only really gets hammered once or twice a month, and using a CDN helps me survive those without getting an expensive dedicated server just for my blog.
W3 Total Cache has some support for CDNs that are accessible via FTP. It will automatically upload your static files to your CDN, and update your web page code to the CDN without you having to lift a finger. Unfortunately, Amazon S3 and Amazon CloudFront aren’t directly accessible via FTP, so if I chose to go that route, I’d need to switch CDN’s. For now, I’m using the TanTan Amazon S3 plugin which uploads my post images to S3 and hosts ‘em there. It doesn’t utilize Amazon CloudFront, so it’s not as fast as a real CDN, and it also doesn’t handle any of the CSS/JS/include files that W3 Total Cache will. I’d really love to see W3 Total Cache implement CloudFront capability.
W3 Total Cache Also Shrinks Your Pages
W3 Total Cache can use HTTP compression (like GZIP) to shrink your pages, but it goes beyond that. It will minify your JavaScript and CSS code, stripping out comments, spaces, and other things that don’t affect how the code works. These things reduce the time it takes to download your web pages. Here’s what one of my bigger pages looked like before W3 Total Cache minified it:

Before Minify
After W3 Total Cache:

After Minify
Hubba hubba – almost a 30% drop in size without affecting content. (Yes, I know it’s hilariously ironic to illustrate smaller web pages by using big screenshots. I like images on my blog. Get over it.)
It’s not point-and-click setup, though – you have to tell W3 Total Cache where each of your CSS and JS files are located. In the screenshot here, I’ve entered in the exact locations of all of my CSS and JS files. To find them, I could have tracked them down via view-source, but the easier way is to use a site like the Web Page Analyzer. It points out all of the CSS and JS files that aren’t compressed and gives you the exact location. You can copy/paste those into W3 Total Cache’s configuration, and it automagically handles ‘em. It appears to be transparent to your plugins.
The downside of this manual setup is that you have to revisit it every time you change your theme or add a plugin. W3 Total Cache doesn’t recognize that you’ve added CSS or JS files to your site. I’ll have to set myself a reminder to use a web page analyzer once a month and make sure everything’s still working as designed.
Initially, you have to double-check your particular blog. My CSS and JS files minified okay, but tag cloud broke when my HTML was minified:

W3 Total Cache Minify Bug
It ripped out the spaces between tags, thereby rendering my cloud as long lines. That one was easy to troubleshoot, but to fix it, I had to disable minification on my HTML, which cost me a little performance. Other issues are not as easy to troubleshoot – I pity the fool who tries to view-source a minified web page, because it’s crammed together on a single line. Well, not quite one line.
W3 Total Cache Adds a Footnote To Your Source Code
The plugin credits itself one of two ways. By default, it tacks on a link at the end of your blog page. If you uncheck that option in the Preferences page, W3 Total Cache instead adds a footnote inside your source code for anyone to see if they view your source. Even when you choose to minify your pages, W3 Total Cache won’t minimize its own absurdly long footnote:
“This site’s performance optimized by W3 Total Cache:
W3 Total Cache improves the user experience of your blog by caching
frequent operations, reducing the weight of various files and providing
transparent content delivery network integration. The goal is to improve the
user experience for the readers of your blog without having to change
WordPress, your theme, your plugins or how you produce your content. When fully
utilized, your blog will be able to sustain extremely high traffic spikes
without requiring hardware upgrades or removing features or functionality from
your theme.
Features and benefits include:
– Improved progressive render (non-blocking CSS and JS embedding)
- Reduced HTTP Transactions, DNS lookups, reduced document load time
- Transparent content delivery network (CDN) support with automated media
library import
- Bandwidth savings via Minify and HTTP compression (gzip / deflate) for HTML, CSS
and JS
- Minification (concatenation, white space removal) of inline, external or 3rd
party JS and CSS with scheduled updates
- Optional embedding of JS just above </body>
- Support for caching pages, posts, feeds, database objects, CSS, JS in memory
with APC or memcached or both
- Caching of RSS/Atom Feeds (comments, page and site), URIs with query string
variables (like search result pages), Database queries, Pages, Posts, CSS and JS
- Complete header management including Etags
- Increased web server concurrency and reduced resource consumption, increased scale
Learn more about our WordPress Plugins: http://www.w3-edge.com/wordpress-plugins/“
That content is tacked onto every single page, thereby adding more length to your site. Ugh – that’s the opposite of what I wanted. I would have no problem with adding 2-3 lines summing up the benefits, but this is absurdly overboard.
There’s no option to switch it off in the preferences, either, but I figured out a sneaky trick. In the preferences, you have the option to add their link either to the footer of your web site or to your blogroll. Since I don’t use WordPress’s built-in blogroll feature, I used that, and voila – W3 Total Cache stopped adding that ridiculous footer. Note to the author: either accept donations, or compromise on a fair length of credits in the source code.

W3 Total Cache Preferences
But, ah, those preferences…
The Good News: Lots of Options. The Bad News: Lots of Options.
This plugin’s options and documentation set a high bar for other plugins. I’ve never seen so much tweaking potential in a WordPress plugin. It approaches the level of usability in the very tweakable Atuahualpa theme that I use here on my blog. If a particular feature doesn’t work, like the HTML minify problems I had, you don’t have to throw the baby out with the bathwater – just uncheck that particular part of one feature.
All these preferences and dependencies, though, is an indicator of the targeted audience. If you’re just getting started with caching, you can’t go wrong with WP Super Cache. It’s easy to install, easy to configure, and hard to screw up. If you’re willing to put more time and effort into installing, configuring and testing your caching solution, then W3 Total Cache gives you more choices and flexibility.
W3 Total Cache is still in its infancy, but already it shows great potential. I’ve run across bugs – sometimes comments don’t show up right away, even though the correct number of comments shows in the post description. For example, the post will say “3 comments”, but only 2 will display. I’m going to keep checking it out as new revisions come out, and I hope they implement S3/CloudFront CDN caching in the near future.


Thanks for the coverage! s3/Cloudfront support will be added before v1.0. The plugin comes out of the box with all that you need in a ready to go right away, if you want to minify your HTML or CSS you have to specify what which is reasonable because any other way the solution becomes less reliable. Disk caching will be added by version 1.0 to open this up to shared hosting users.
As for the large HTML comment. I don’t accept donations. According to my testing, what W3TC delivers is not compromised by my large HTML content in any way. If people want to see why a site is so fast, I really want them to understand that there’s a single solution to realizing it and what steps it takes. I’ll work on communicating that in fewer words.
I think you might be able to slim your pages down even more by using PNG images (especially for screenshots) instead of JPEGs. As a test I took a screenshot of this comments section to compare against the last image in your post. It was a 68 KB PNG (versus your 60 KB JPG). Then I ran the PNG through an app called PNGCrusher, which is a simple front-end for a command line tool called OptiPNG that optimizes PNGs. That trimmed it down to 48 KB with no loss in quality (and none of the compression artifacts that your JPEG has).
I tried using PNGs, but Skitch (the tool I use for screen captures) consistently generates larger PNGs than JPGs. I’ll try PNGCrusher though, that looks cool!
PNGGauntlet works well too. I started using that after reading about it on The Coding Horror. For some reason his blog seems to be very slow today so I can’t link it. PNGGauntlet will transform other images to png’s if you wish as well. Of course most complex jpgs are better off remaining jpgs.
One caveat for people with custom JavaScript, you will want to run your JavaScript through jslint before you minify it. There are some syntactical shortcuts that you can take with JavaScript that will cause it to break once you’ve minified it. You can usually trust most 3rd party libraries and plugins, but always be sure. I’ve spent several hours un-linting my JavaScript and found plenty of issues that would have caused a lot of problems if I had minified my code.
Yeah, I’m running into other JavaScript problems that I think might be hosing things up, so I’m pulling W3 Total Cache for now and switching back to WP-Super-Cache until the next rev of W3 Total Cache comes out. I do like it a lot – it just needs to work a little better.
BTW, another release will be available in a few minutes.
Version 0.7.51 just released! http://wordpress.org/extend/plugins/w3-total-cache/
Version 0.7.52 just released!
Version 0.8 is finally here with some exciting improvements:
* Added disk as method for page caching
* Added support for mirror (origin pull) content delivery networks
* Added options to specify minify group policies per template
* Added options for toggling inline CSS and JS minification to improve minify reliability
* Added option to update Media Library attachment hostnames (when migrating domains etc)
* Added “Empty Cache” buttons to respective tabs
* Added additional file download fallback methods for minify
* Improved cookie handling
* Improved header handling
* Improved reliability of Media Library import
* “Don’t cache pages for logged in users” is now the default page cache setting
* Fixed minify bug with RSS feeds
* Fixed minify bug with rewriting of url() URI in CSS
* Addressed more page cache invalidity cases
* Addressed rare occurrence of PHP fatal errors when saving post or comments
* Addressed CSS bug on wp-login.php
* Addressed rare MySQL error when uploading attachments to Media Library
* Modified plugin file/directory structure
* Confirmed compatibility with varnish and squid