April 22, 2008
Lately I've been delving into the WordPress ecosystem, as it seems to be the most popular blogging platform around at the moment. I've set up two blogs with it so far. In the process, I've gotten quite comfortable with the setup, interface, and overall operation of WordPress.
I've been thoroughly impressed with the community around WordPress, and the software itself is remarkably polished. That's not to say that I haven't run into a few egregious bugs in the 2.5 release, but on the whole, the experience has been good bordering on pleasant.
Or at least it was, until I noticed how much CPU time the PHP FastCGI process was using for modest little old blog.stackoverflow.com.
For context, this is running on a Windows Web Server 2008 virtual machine with a single core of a 2.13 GHz Xeon 3210 entirely dedicated to it.
This is an incredibly scary result; blog.stackoverflow.com is getting, at best, a moderate trickle of incoming traffic. It's barely linked anywhere! With that kind of CPU load level, this site would fall over instantaneously if it got remotely popular, or God forbid, anywhere near the front page of a social bookmarking website.
For a bare-bones blog which is doing approximately nothing, this is a completely unacceptable result. It's appalling.
As evidence of what a systemic problem this is, there's an entire cottage industry built around shoehorning better caching behavior into WordPress. Take your pick: WP-Cache, WP-Super-Cache, or Bad Behavior. The caching add-ins don't work very well under IIS because they assume they're running on a *NIX platform, but they can be coerced into working.
Does it work? Does it ever. Here's what CPU usage looks like with basic WP-Cache type functionality enabled:
I'm not alone; just do a web search on WordPress CPU usage or WordPress Digg Effect and you'll find page after page of horror stories, most (all?) of which are solved by the swift and judicious application of the WP-Cache plugins.
It's not like this a new issue. Personally, I think it's absolutely irresponsible that WP-Cache like functionality isn't already built into WordPress. I would not even consider deploying WordPress anywhere without it. And yet, according to a recent podcast, Matt Mullenweg dismisses it out of hand and hand-wavingly alludes to vague TechCrunch server reconfigurations.
A default WordPress install will query the database twenty times every time you refresh the page, even if not one single element on that page has changed. Doesn't that strike you as a bad idea? Maybe even, dare I say it, sloppy programming?
I understand that users may have umpteen thousand WordPress plugins installed, all of which demand to change on every page load. Yes, the easiest path, the path of least resistance, is to mindlessly query the database every time you're building a page. But I cannot accept that a default, bare-bones WordPress install hasn't the first clue how to cache and avoid expensive, redundant trips to the database.
It's frustrating, because caching is a completely solved problem in other programming communities. For example, the .NET framework has had page output caching and page fragment output caching baked into ASP.NET for years.
I sure am glad I started this blog in Movable Type way back in 2004. Their classic static rendering blog engine approach may be derided today, but I shudder to think of the number of times the Coding Horror webserver would have been completely incapacitated over the years by the naive -- no, that's too tame -- brainlessly stupid dynamic rendering approach WordPress uses.
What I just don't understand is why, after all these years, and all these documented problems, WordPress hasn't folded WP-Cache into the core. If you're ever planning to have traffic of any size on a WordPress blog, consider yourselves warned.
Update: Matt Mullenweg kindly responded to this post and offered his recommended MySQL configuration optimizations. I definitely agree that the Query Cache is extremely important to performance, and for some reason it defaulted to off (zero size) on my installation. You may also want to look into innotop and mysqlreport to ensure that all your MySQL caches are functioning at appropriate levels. Also, thanks to a few commenters for letting me know that one of this year's Google Summer of Code projects is integrating caching into the core WordPress code. It is badly needed.
Posted by Jeff Atwood
This is slightly out of topic...so excuse me. Why PHP? Are you leaving Microsoft development because you're partnering with Joel and he doesn't like MS products?
I agree WP-Cache should be included in the default bundle. And I agree that you need it for WP to perform properly. There's really no reason not to download, install and activate it, though. It comes pre-bundled in DreamHost's One-Click Install of WordPress for a reason.
With WP-Cache enabled, WordPress flies like an unladen European swallow. Not only does that cause all WP pages to be cached, but they are served in a RESTful, HTTP-friendly manner too, with Last-Modified headers and everything you need to do conditional GETs.
Although it doesn't improve WordPress' code quality, I'd also recommend you to host it in a Linux environment and preferably with mod_php (not FCGI). Just out of interest; are you running the database and web server instances on the same virtual machine?
"[WordPress 2.5] came out of the box with a broken file uploader for a whole lot of people because of its required dependence on a Flash component."
Yes, it's dependent on Flash, but if you don't have flash it gracefully degrades to the old pre-2.5 file uploader. What's the problem here?
"The problem is, besides the huge task of refactoring itself, it's impossible to change WP into something well structured without breaking *all* plugins and templates, and it's this ecosystem of add-ons that is major part of WP's success in the first place."
If the plugins properly use the hooks as they are supposed to, you should be able to do just about anything to the core backend without breaking them. Reworking the core would only break plugins that try to access things directly. Isn't that the whole _point_ of abstraction???
The more modular and manageable the web-app, the worst the database query efficiency.
L-A-M-P, they were designed to work together
LAMP was a retronym coined for a particularly popular stack of software (each having been developed entirely independently).
Are you hosting with Godaddy? (as your WHOIS lookup suggests)
From their site it looks like they're running Windows 2003 on virtual dedicated, or do you know something I don't?
LAMP was a retronym coined for a particularly popular stack of software (each having been developed entirely independently).
Apache was built to run on Linux, as was mysql. PHP was built to run on Apache. It may be a retronym, but they are all very closely linked.
A. Freaking. Men.
Dive into the code. Prepare to be disappointed. Browse the Trac bug entries. Prepare to be upset.
Consider that Automattic just got $25 million in round B funding. Prepare to understand how dumb stuff like the subprime crisis is possible.
20 database lookups per page view is unforgivable. 20 times PHP had to contact MySQL, send a query, wait for it to be executed, get the results back and parse them into PHP-usable data. I've seen some blogs that had stats of up to 50 some database queries. Again. Totally unacceptable and unforgivable, given what WordPress is supposed to be doing.
If you have a 1,000 users, that means 20,000 database queries being executed unless you have some very tight caching going on there. And... this is considered acceptable out of the box?
In the open source world, if you want something, you *might* have to do it yourself.
Go for it Jeff!
The Roll Your Own solution works fine for a super-basic blog, but honestly, wordpress has a LOT of features baked in. The plugin system, auto-generated RSS feeds, creating "friendly" urls, letting you define the "friendly url" structure... categories, tags, permalinks, pagination, widgets... You'll pretty much need a team and a year or so of 40-hour weeks to get to where wordpress is today.
On the other hand, if you don't care about any of that... A very basic blog can be thrown together in about an hour, given enough familiarity with the language you're working in. I tossed together something that fit the bill for what I needed in Coldfusion in about a half hour (of course, CF is more of a RAD language for web stuff than ASP.NET, so mileage will definitely vary)
Also, you might want to check out some other alternatives: Pivot's also PHP, and I was a fan of it- only switched over to Wordpress because the Admin ui drove me nuts (they're working on that).
"blog.stackoverflow.com is getting, at best, a moderate trickle of incoming traffic"
trickle defined based on your stats? Or just on a hunch? Joel linked to it, and he's got an incredibly popular blog himself.
Also, how does WordPress handle or react to RSS? Could that pounding have anything to do with all the RSS programs your readers use? To say nothing of now all the iTunes instances hitting it for the podcast.
Thanks for the link to WP Super Cache! If you scroll to the end of that page you'll see a *huge* spike of traffic that was handled by laughingsquid's WordPress blog in combination with WP Super Cache.
This plugin goes beyond what WP-Cache does by creating static html files and then serving them with the help of a few mod_rewrite rules. With that installed even a small web host can survive a huge bombardment of hits.
Why is it not included by default? I'm not sure, but a few issues spring to mind:
1. Some people don't have access to mod_rewrite so the static html file functionality won't work.
2. Added bloat. Despite the CPU graphs above, a bare bones WordPress install works perfectly fine for many people. Why make it more complicated?
You should look up the object cache stuff Ryan Boren worked on. There are memcached, APC and Xcache backends for that if memory serves. With that installed your blog can be fully dynamic and not hit the db.
Here's a pretty good overview of it:
I took the path of least resistance, and registered my blog with Wordpress themselves in the end (let them take the strain).
I've seen the same situation develop with several open source projects - as soon as you have different people pulling in different directions, you lose focus. The biggest problem with Wordpress is that there is no one core of developers in charge.
@mark - Drupal ain't no magic bullet. I maintain some relatively large Drupal sites and let me tell you, it has its own scaling and caching issues.
The built-in "Minimum Cache Lifetime" function is totally broken in the latest 5.x and 6.x releases and the Free Tagging functionality works in such a way that it creates several temp tables *on disk* every time you visit a tag page.
Comparing .Net to WordPress is apples to oranges. Heck, comparing .Net to PHP is peaches to strawberries. Once you say '.Net', you are talking about a specific framework which runs in a limited set of environments (i.e., Windows).
OTOH, WordPress is written to run in a very heterogenous set of environments. It can run under Window, Linux, BSD, MacOS. It can run under Apache, IIS, and a couple of other web servers -- just about anything that supports PHP and MySQL. It can run under pretty strict shared hosting setups, where the user might not even be able to modify the directory permissions necessary to allow caching. So I think it's at least a little bit forgivable that caching is left to plugins, for those who need it.
As mentioned above, checking the MySQL query cache (and other tuning) might be another area for investigation. The WP Object Cache is getting a rework, and is supposed to return in WP 2.6. I don't think it will turned on out of the box, but again, for those who need it, the underpinnings will be there.
Out of curiosity, I assume the .Net caching bits you mentioned are similar in nature to PHP's output buffering? So, you have to tell it where in the filesystem to put the cached files, yes? And I assume that this also means that you have to make sure that the web service has permissions to write to that location? Or does .Net make this a magic-bullet that you just fire and forget?
But seriously, there are a lot of layers of optimising you need to make WP run fast.
1. MySQL query and key caching will save you a lot of time. Most times a page is displayed and nothing has changed.
2. PHP opcode caching. There are several. I use xcache.
3. Selectable object cache. It's a little known fact that Wordpress does some half-arsed internal caches of intermediate objects; an even less-known fact that you can get it to use PHP opcode caches or memcached to hold those intermediate objects. This combination can really zip up the PHP side of things.
4. Reduce the number of plugins running if you can.
5. Don't put your blogroll, archives or author listings on the front page. Move them onto a separate page. Each of those listings performs a nasty old bunch of queries. Not single queries that reduce to a single result set; oh no, dozens of queries.
I agree that Wordpress is an evolutionary dead end. But unfortunately it currently does what I need it to do. I have Stockholm syndrome.
Email me -- email@example.com -- if you have any other questions or queries.
Apache was built to run on Linux
No, it wasn't.
as was mysql
mysql was originally built on Solaris.
PHP was built to run on Apache
No, it wasn't.
but they are all very closely linked
Actually, no they aren't. PHP for years was run predominately as CGI, which is basically as unlinked from the webserver as possible, and to this day has its own world of configuration separate from the web server.
Unbelievably, there are people around there who value more the dynamic rendering approach of WP than, say, the Perl driven static philosophy of WP. If the cache stuff was built-in, I'd deactivate it for a number of well thought reasons. Unbelievably, there are also people who seem to handle slashdotting/digg/whatever on WordPress and a cache plugin. Go figure.
Damn, you're 100% right. Still a great platform though :P
Why is everyone knocking PHP when the problem is the db requests coming out of PHP code and not the code itself... ?
Why else would caching help?
Maybe I misunderstood something?
Dare I say it - it's a characteristic fault of the open-source ecosystem: "I'll fix it when _I_ realize it's important." (Curiously similar in this respect to Big Software.)
And CPU-hogging isn't the only hidden failing of WordPress. I'm moderately familiar with (X)HTML/CSS, yet I'm continually baffled by the uncommented, undocumented, messy and highly individual CSS coding of WordPress theme authors. In stark contrast, TypePad is wonderfully slick and comes with decent support - TypePad's tech people may be a nonchalant, superficial, and lazy, but they'll deliver the answers if you goose them hard enough. And it's worth the money. For blogging newbies who plan to tweak their themes, WordPress is the worst possible choice; TypePad is the only way to go.
I'm too far down to expect a reply (:/) but I'm curious why you didn't use a .NET blog (like blogengine.net).
That just seems more likely considering the source.
You're right, there is a fundamental problem with WP making 20 db calls; however, caching would help because WP wouldn't have to go out to the db to display the article, it could grab it from the cache - therefore utilizing less resources.
"Unbelievably, there are people around there who value more the dynamic rendering approach of WP than, say, the Perl driven static philosophy of WP. If the cache stuff was built-in, I'd deactivate it for a number of well thought reasons. Unbelievably, there are also people who seem to handle slashdotting/digg/whatever on WordPress and a cache plugin. Go figure."
Why do you need dynamic publishing for most blogs? If it gives you warm fuzzies or you have a real reason for doing so, that's fine, but seriously, why do you need dynamic publishing? The very fact that you need a cache plugin to handle a link from slashdot or digg is a bad sign about the nature of dynamic publishing.
@Dougal: It's actually along the lines of the "magic bullet". You just add a declaration at the top of the page and the framework does the rest (google for "asp.net page cache" if you are interested).
I agree that WP-Cache should be standard install. I've been feeling that way for a couple years now, ever since my own blog's performance problems demanded that I install and configure it.
And in arguing the merits of WordPress vs. other services, it often comes up "But you just have to install WP-Cache." Yes, but you do just have to install it. And that sucks.
So it's making 20 database calls? That's all I/O. Why is the CPU doing so much?
Why not just turn on output caching from the IIS7 Output Cache module? You can even turn on kernal mode caching. Should take just a few minutes.
It's clear to see that the open-source movement combined with the added code reuse capabilities of OOP are working towards reducing development effort. "Write your own blog software" is getting as many votes as "Install Linux, PROBLEM SOLVED".
What if he installed Linux, customized the kernel so that it was highly optimized for hosting a blog, wrote his own web scripting language (with caching built-in), then wrote blog software using that language? It might take a few years, but then his blog would be *optimal*.
Or maybe WP could fix their product so that he can click a button and have a blog ready for him, instead of wasting months reinventing a wheel that comes in 31 flavors already?
"So it's making 20 database calls? That's all I/O. Why is the CPU doing so much? "
No, it's not. First of all, MySQL has to execute the statements, which includes finding and possibly modifying data. Second, PHP has to wait for MySQL to finish and parse the results into a format usable by the PHP script.
Jeff, is it possible that you put "moderate trickle of incoming traffic" into some numbers, so we can understand how severe is the problem?
I recommend using a PHP op-code cache, like xcache. In my experience, it can cut CPU usage by 75%.
I also recommend switching to Drupal ;)
Will every one saying "don't use Windows/IIS use Linux" JUST FUCK OFF!!!
Might have been posted here already, but I'd highly, highly recommend Drupal. I'm 3 months into learning how to use it and Wordpress (AND MovableType) pale in comparison. So much so that I'm re-doing my Wordpress-based blog in Drupal at the moment :)
I'm developing a bigger project based on wordpress, so I'm alarmed reading this excellent post. I'll try these plugins and hope they'll help me out.
blog articles themselves may be static, but blog comments are not. therefore a db is needed. and i'm pretty sure jeff would want a comment section on his blogs.
i had to switch hosting company to get wordpress to run smoothly. i'm now on a linux server. yes, windows server *can* run php, but there are limitations.
I'm a huge fan of caching, I even use it for dynamic sites because it drops your serverload to almost nothing. For example, I recently build a site for company with loads of active trackers. I needed around 11 DB calls (including making the connection) but because of the nature of the site MySQL's query cache was almost useless and the 5-10 visits per second during office hours quickly brought the server to it's knees. I enabled Smarty's (http://smarty.php.net/) caching mechanism with a cache validity of 2 seconds, and now it's working fine. The temporary files are written to and read from a ramdisk until the new server arrives with a much faster I/O setup, and the CPU load is now around 20-30%. Because of the low cache validity of only 2 seconds the information is still almost realtime, and my customer is very happy with it.
congrats you are now result 4 on google searches for wp cpu usage!
@Donny, thanks for posting the link to BlogEngine.NET - it looks like an interesting WordPress alternative. I'd much rather work in a .NET environment than php.
I'm currently running WordPress 2.3 on IIS, and it took quite a bit of fiddling to get WP-Cache working. If anyone is interested, I posted the steps required to get it working at http://www.fanrastic.com/archives/2007/06/06/wp-cache-up-and-running-on-iis/
"Even for sophisticated blogs, you probably want to write the changes to a database then update the static files as necessary. You don't want the brunt of your web-requests hitting application code unless it is required."
it depends on the nature of the blog site, and how the author intends to run it. For me, i like blogsites that show comments instantly, because on a lot of popular blogsites(very much like jeff's) there are always conversations going on, in within commenters.
again, i think it really depends on the nature of the blog site.
Well, since you are using IIS7 why dont you just use output caching, its provided by iis and can be setup through the iis console, or through the web.config file. iis.net should have some screencasts on it.
Its pretty slick, you can even set it up so that it creates a cache for every combination of query string variables.
I work for a web hosting company's tech support.
Why the hell does wordpress store all the posts in the DB anyway? It's not a forum, and it's completely unneccessary. The whole thing could be rewritten not to use SQL, it would run better and use 1/3 the cpu/memory/resources.
Many web applications do these excessive queries. The caching system is most important but other things in PHP such as OPT Code caching will increase performance an average of 30%.
Make sure your WordPress database isn't collecting spam comments. Don't forget to back up your database as well because my hosting company hosed my site and I had to use search engine cached pages to restore my old posts. I'm using SQLyog to export a SQL Dump.
While backing up the MySQL database I saw that the comments table had a lot of records. I discovered that by entering spam keywords to filter comments I was essentially hiding them from the usual moderation procedure. Now I have to run a query to delete comments marked as spam.
Quite a few MovableType users in shared hosting complain that though the static page does boost the performance, the process of staticization consumes too much memory and CPU usage, and got killed.
That is kind of ironic.
Thankx captaion of "coding horror", i've enabled my wp-cache. Cheers and keep up the good work.
I'm extremely amused by the linux fanboys on here. I can guarantee that 99% of them have never used IIS, let alone IIS7.
IIS7 is currently the fastest application server to run PHP on, how fast? Try 800 times faster:
That said, PHP sucks, mySQL sucks, and Wordpress sucks. All three are a beautiful catastrophe of sloppy coding that scale by throwing obscene amounts of hardware at them.
I'm sorry. Did you say *Windows* and *PHP* and *lighttpd*??
*Windows*?? For hosting?
On other notes, I do agree that IIS can be made to run PHP nicely. However, surely WP blogs are one example of where caching can be *far* more aggressive than other situations due to almost complete lack of dynamic content bar comments, which are exceptionally simple to implement.
Did you get into computers so that you can sit in the corner and wank over windows verses linux or php verses .net?
How does this graph show anything?
Where are the metrics?
What are you measuring?
What is your bias?
What are your baselines?
Are there any affecting/impacting parameters?
Simple scientific methods would make you a laughing stock of the science community if you presented this shitard piece of work with little to no supporting documentation (geez reminds me of the typical consultants I clean up after)
Here's a graph of performance before on an IIS install with php:
and here it is after I rebooted:
here it is after I installed colinux on my windows box with php in a VM:
About a year ago I did some heavy customization of word press and all I can say is....
WHAT A ROLLING PILE OF CRAP!!!
I'm sorry, MAYBE it has a good user interface (not really, but it get's the job done some of time..)
What really drove me nuts was the code.
Wordpress' coding framwork... isn't a framework. A bunch of files randomly sewn and included together that somehow manages to produce a page. UHG!
The sad bit is, IS one of the best things I've seen so far.
If I had to depoly a blog tho, I'd use Joomla. It's WAYYYY more than you need, but there are awsome plugins.. and your blog can grow if you need it to.
Another thing I've done is manually edit my themes. A lot of the database calls are from PHP code that can just be written into the theme. For example, why not just edit the title of your blog in the theme. Not only do you avoid a DB call, you also avoid some PHP code!
Thanks a ton Jeff, I had no clue that wordpress didn't have a solid cache. I have wp-cache running on all the blogs I manage now.
This thread is so frustrating because Jeff is getting flamed but not for the right reasons. WTF does windows vs linux have to do with anything?
What really annoyed me was:
"It's frustrating, because caching is a completely solved problem in other programming communities. For example, the .NET framework has had page output caching and page fragment output caching baked into ASP.NET for years."
So right there you went from "Wordpress sucks" to "PHP sucks and ASP.NET is much better". Now I'm not a fan of PHP but that's a giant non sequitur if ever I saw one.
All those people suggesting Jeff install linux are missing the point btw, wordpress scales badly on any platform. The only reason it's so popular is because it managed to get some nice themes going and has a large addon community. Performance wasn't on the developer's minds. So now everyone seems to be stuck with it.
Wordpress is utter garbage. The only part that doesn't suck is the installer. Moveable Type is greatly superior.
Jeff, if you really do read all the comments you get, my condolences.
The following isn't meant to absolve the Wordpress team of responsibility, and is coming from someone who (in part) makes his living with PHP, usually in a LAMP style environment, and usually cleaning up someone else's mess (because it's more lucrative than making a mess of your own).
It's not just Wordpress. There's a lot about PHP, both as a platform and as a culture, that results in rank abuse of the database.
First and foremost, the early history of PHP is linked closely with the business of shared web hosting. When you're writing an application where that's your target and you need to write something down, you have two choices. The file system or a database. Since using the filesystem comes with a set of security and configuration headaches, the database quickly evolved into the go-to place to store, and later serve, information.
For a lot of PHP programmers without the formal CS or programming background, this is all they've known. For a lot of PHP programmers **with** a formal CS or programming background, they face an uphill battle maintaining legacy applications while at the same time facing the never ending onslaught of new feature requests/demands. Do you rewrite something that works most of the time and can be made to work when traffic spikes come along? Now you're in holy war territory (http://www.joelonsoftware.com/articles/fog0000000069.html).
Wordpress is a legacy application. It's been around since 2003, and is the official fork or the b2\cafelog weblog software which was started in 2001 (http://cafelog.com/index.php). It is, at it's core, a **PHP3** application, with a pile of evolutionary OOP thrown into the mix over the years. It's easy to say "Hey, wtf, where's caching?", but the team placed its priorities elsewhere, and it's hard to argue with their success.
To the larger issue of PHPs red-haired step-child status in the developer community, it's important to remember it brought two major "value propositions" (ugh word) to the table.
The first is horribly cold hearted, mean, shortsighted, something no one likes to hear, and is 100% true. In a market where a professional .NET, Java, or perl/python developer will cost a company $80,000/year, they can find a PHP developer willing to work for $50,000 a year, and get the same immediate results. Yeah, they're probably setting themselves up for real pain when it comes time to scale and there'll be code that will haunt future developers for years to come and OMG security but American business (in general) is horribly focused on the short-term.
The second value proposition is a bit more heartening, although a lot of people will disagree. Like all the big shifts in programming, PHP enabled a new class of people to use code and develop software that solved a problem they were having, problems that the CS-community wasn't (again, in general) interested in. PHP and those early, hacky perl scripts (Damn you Matt Wright!) laid the foundation for the social-web as it exists today.
Someone asked about handling RSS traffic. I recommend Feedburner for this. They will cache your feeds and do other neat things if you want them.
"it depends on the nature of the blog site, and how the author intends to run it. For me, i like blogsites that show comments instantly, because on a lot of popular blogsites(very much like jeff's) there are always conversations going on, in within commenters."
You don't have to cache the whole page. An alternative would be to cache everything except the comments.
"IIS7 is currently the fastest application server to run PHP on, how fast? Try 800 times faster"
That was because IIS can cache requests in memory, so that it never involves PHP in the first case... So what you are saying is that if IIS can output HTML directly from memory, it is faster than executing a PHP script...
Using a memory cache is not something new, and there exists solutions for that on any platform. That it took Microsoft so many years to implement it themselves, is not somthing to brag about.
Ah, yes, this happened to me; I responded by moving to MovableType and eventually Blogger; spam was getting to be a big problem, and I didn't want to deal with it myself.
The funny thing is, Wordpress used to be far worse, speedwise. Until quite recently every page load involved lots of MySQL cache-busting queries...
Alexander: "You'll pretty much need a team and a year or so of 40-hour weeks to get to where wordpress is today."
A single decent engineer could do it in about 2 weeks with sufficient caffeine, and the code wouldn't suck nearly as much. WP is such a disgrace it deserves its own thedailywtf section.
wait until you your site gets hacked, there are so many security holes in this thing is not even funny
Cris: "I'd blame PHP. Its a horrendous language."
PHP is fine. Bad developers are the problem. Yahoo!, Facebook, Flickr and others have made it abundantly clean there's nothing wrong with PHP as long as a real software engineer is writing the code.
Yes... wordpress always was one of the most CPU loading CMS. But it useful. And it do wordpress popular.
Has anyone ever conducted a comprehensive comparison study of the resources used by the various popular CMS: Wordpress, Drupal, Serendipity, MovableType, Joomla, Textpattern, DotNetBlogEngine, DotNetNuke. If Wordpress runs very inefficiently out of the box, how much worse does Wordpress run relative to other options?
Bad coding in WordPress? You're kidding ... :D
That's why I'm so happy I switched to a Windows environment so I could discover, and use, BlogEngine.NET ( http://dotnetblogengine.net/ ).
someone: "blog articles themselves may be static, but blog comments are not. therefore a db is needed."
Not true. You could process your form and append the comments to a text file in your file system. Not that you'd want to, but the db is not an optimal content delivery method in a lot of cases. It's great for storage, but for reading there's some overhead you can sometimes do without. The reason people jump to the database is because that's all they know, not because it's the best solution. That's why there are so many caching mechanisms--it just ain't all that.
Thanks for this - I will be following your advice immediately!
What percentage of hosted Wordpress sites get Dugg?
What percentage of hosted Wordpress sites get 1500 visitors/day?
What percentage of hosted Wordpress sites use more than 5 plugins? (or ANY plugins?)
The power of Wordpress (and the value) is found in a CMS that is easy on the novice, is easily extensible, and empowers people to communicate in ways beyond their technical skill.
So hey -- let's slobber all over ourselves while gnashing at the project, because it doesn't elegantly scale for those one-in-a-thousand sites that EVER gets a DiggStorm. Let's have a party over the grave of a platform that can't be all things to everyone.
Is the coding somewhat spaghetti? Yes.
Do I wish it were more consistent? Yes.
Now let's talk about a couple of possible solutions you HAVEN'T mentioned yet.
First, there are many unnecessary calls that come about in the themes. Blog title, RSS feeds, and other information that should not change should be tweaked. Once you decide on a theme, replace the PHP with direct links and text. It's not that hard. Also, the "queries and time" in the footer is just more unnecessary DB calls. Ditch them.
Ideally, there would be an admin panel option that would replace these specific things in the header and footer. Actually do a find-and-replace of those specific pieces.
Second, I've seen a noticeable improvement when using one of the AJAX plugins. No need to call header-footer-sidebar every time to click on a different post.
Third, blogrolls. Either stash them and tend them externally, or ideally get rid of them. They serve little purpose these days. Unless someone comes up with a better way to cache them.
Wordpress is a Ford Taurus: suitable for most people, and it CAN be tricked out for NASCAR if you know what you're doing... but let's not bash it because it's not a Formula-1 racer out of the box. Most people can't afford it, and it's too much horsepower to carry the groceries.
Is Apache on Windows any better than IIS for Wordpress?
One of the main problems with PHP is that it doesn't maintain a cached version of the page by default, whereas other languages like Java, C# and Python do. So each time a user makes a request, not only does it make those 20 database calls, but it also recompiles the PHP script, which is extremely inefficient. By installing APC or e-accelerator, you can make it so that PHP stores the compiled version of that script in memory, and only recompiles it when the code actually changes.
However, note that these opcode caches are relatively unstable, and sometimes will segfault (at least with Apache, I don't know about IIS). I personally have a script that checks the Apache logs every minute to see if it has segfaulted, and it then restarts Apache if so.
Another thing that will make a HUGE improvement is enabling MySQL query caching. It stores the results of all SELECT queries until the underlying data changes, so that the results are pulled using an O(1) hash operation, rather than actually re-running the query.
Also, Zend Optimizer can optimize the PHP scripts in a way similar to how a compiler does, so that helps it run a little faster.
I find that using Zend Optimizer, an opcode cache and MySQL query caching leads to huge reductions in CPU usage (depending on your site, can be on the order of 10x). For each request, it removes the compilation of the script and the execution of the common SELECT queries from the process, which is where the bulk of the time is spent.
For further reading about this, I highly recommend the following two books:
More from the masses:
Why not use BlogEngingeDotNet? All C#. All MS.
FROM: another satisifed user of BE.NET.
I have to say that this scares me as i've recently created a blog myself with wordpress.
Interestingly enough, Matt DID think about including Wp-Cache in the CORE wordpress code at one stage, as written here:
Anyway, I never got bothered by the CPU limit as much as by the MEMORY limits... but caching solves them both.
Of course writing better queries and database structure would help a lot too. WordPress is very popular as it is EASY to understand (it grew more complex, but started VERY basic and easy years ago) and that initial simplicity, together with the APIs, made the plugin ecosystem/community possible. Therefore it became popular. But I won't call it really "polished". Who cares, anyway. I have been using it for years with pleasure, but I have that on Linux. If I had the luck of a Windows Hosting, I would give GraffitiCMS or subtext or dasblog a try.
I can sympathize with the WP folks. When I am writing a web application, I don't want to also be writing the platform it runs on at the same time. It's not exactly clear what sort of "caching" the author is talking about, but whether it's caching of DB results or rendered pages, it's not really something I would expect to care about as a web app developer. It should be handled by the platform.
But then, I don't use PHP. ;-)
A Google search for "wp cpu usage" was evidence for how bad wordpress is with CPU usage. All the top results for that search are links to this post which cites as evidence an install on a virtual machine on a windows server, database unknown.
Running PHP with Windows is not a very good idea. But I agree that Wordpress is really a heavy web application. I only use it if my client wants a real quick solution, otherwise I always implement custom solutions.
As a Movable Type user myself, i totally agree with you.
"You're right, there is a fundamental problem with WP making 20 db calls; however, caching would help because WP wouldn't have to go out to the db to display the article, it could grab it from the cache - therefore utilizing less resources."
I get that, I just don't get why anyone would blame PHP for that... thats the programmers fault for using the programming language to make multiple requests and not implementing a caching solution.
If it was written in something lower level this would be a no brainer... languages aren't supposed to provide optimal, or even good solutions to arbitrary problems, they are just supposed to provide the means to implement them. PHP provides a lot of unnecessary bonus material... which is good, but I don't think its fair to criticise it for not providing more... particularly something as non-trivial as caching, which can't be implemented blindly for a guaranteed performance increase. :P
I note that just two comments above me is another one about PHP running like a dog on Windows... completely irrelevant given that the problem stems from db requests and bad programming.
20 queries is considered much?
While that piqued attention and became an assumed source of ills, when I did the benchmark's listed above MySQL contributed very little to the workload -- overwhelmingly the work was in the PHP CGI handler.
Sometimes people forget that databases have the whole concept of caching pretty proven out -- a tiny option table, for instance, will almost certainly be cache-resident for read operations, with lookups basically being a hashmap lookup with proper indexing. Querying it a million times won't differ that much from just looking it up in a collection presuming there isn't expensive connection setup/teardown type activities.
Wow! I didn't even notice that the native cache has been removed in WP 2.5! Thanx for the hint. After reading here about WP-Super-Cache I gave it a try, and it seems to do a really good job.
@Dennis Forbes, thank you very much for bringing some empirical sense to this discussion.
I love the ignorant assclowns who come here and say that they've never had any problems with WordPress and that is must be Windows fault, when the bottom line is that their blogs get little to no traffic and therefore they wouldn't have any reason to be concerned with CPU usage.
WordPress blows, and it's not news.
Man, they sure are hacking our gibsons.
Insanely archaic caching aside, WP still holds a special place in my heart.
1. Stable, high performance
2. Exciting, fast development
Pick one. And as previously noted, everyone seems to pick number 2. Me included frankly.
Wordpress screams on my box, uses about 5% CPU at the worst of times, and does, oh, say 30k page views a day, which is nothing I know. But even when I get a hit of traffic that spikes up to 250k a day, same story. Funny thing is I'm running on linux, and all the bits and piece of the stack (mysql, php, apache, wordpress) are caching...
"I get that, I just don't get why anyone would blame PHP for that... thats the programmers fault for using the programming language to make multiple requests and not implementing a caching solution."
I don't think Jeff was saying that PHP should having caching built in (maybe he did, I read the article yesterday morning), I think he is saying that Wordpress should have it built in... although I'm not sure why he compared .NET to Word Press (unless Word Press is considered a platform and not an application... I don't know, I've never used it).
We had a couple of posts get on the front page of digg. Wordpress with wp-cache handled 24,000 unique visitors in three days just fine on a PIII Xenon running FreeBSD. You have to make sure apache is configured properly for your resources though, otherwise it's child processes can get pretty crazy. The MT approach is good too - very utilitarian - throttle once render many!
I get that too... I am just curious about all the smug comments about PHP running badly on Windows.
It really grinds my gears when comments like this are posted without any justification... its typical Linux vs Windows crap. No justification and strong opinions. As much as it maybe true it seems to me that it has nothing to do with the matter at hand...
I've got an install of WordPress running (without WP caching) on a site serving 10-20k pages per day on a small VPS with other sites, sometimes hitting hundreds of simultaneous connections. Our loads are nothing like yours listed.
1. bad-behavior is not a cache plugin at all
2. you can't compare movabletype to WP: Movabletype's slow-as-molasses perl can't render pages in realtime at ANY realistic realtime load - try "re-compling" 1000 movabletype pages just to affect a simple page change
3. You are running on a windows box with PHP as a cgi, possibly the slowest way possible to execute PHP
4. You're obviously running without a PHP opcode cache while I suspect comparing it to fast-cgi perl which is cached?
I will never, ever, go back to Movabletype after having the flexibility of WordPress. Neither is perfect but I'll take WordPress's attitude over MT's ways anyday.
thats the programmers fault for using the programming language to make multiple requests and not implementing a caching solution.
Remarkably, that is the one constant in software: there's no shortage of terrible programmers in all languages across all platforms!
thats the programmers fault for using the programming language to make multiple requests and not implementing a caching solution.
1. WordPress originally had a caching system which was abandoned for reasons I never followed, possibly when it was externalized as a plugin (wp-cache) which was superior to what they had.
2. Near-future versions of WordPress (and bbPress) are supposedly going to have object based caches where internally (and plugin developers) can save any results to it's cache in a structured manner (ie. userlists, front page topics, etc.)
I recommend doing some comparison testing using something like ab [http://en.wikipedia.org/wiki/ApacheBench]. Test your current setup. Test with Apache/mod_php on Windows. Test with Apache/mod_php on Linux. Test with and without WP-Cache. Get some real numbers and repeatable testing.
Of course, I also think that changing a database-backed blog to purely file-backed is kind of silly, and that fastcgi is a problematic way to run code (though I currently use it). So I'm sure my views are considered suspect.