Dreamhost Thinks I Use Too Much CPU

Ominous email:

PLEASE READ THIS ENTIRE MESSAGE VERY CAREFULLY!

If you have read everything and have any questions, please let us know.

Unfortunately it appears that your site is using more than your fair share of system resources on your shared hosting machine dawber. Our system has flagged your account because it is using a large number of CPU minutes per day on dawber. We need you to trim down your resource consumption considerably. Should you ignore this or subsequent warnings your account may be moved to an evaluation server which could cause downtime.

Specifically amf’s CPU minute usage for today is 56.70.

(What does this mean? Read this:
https://panel.dreamhost.com/kbase/index.cgi?area=3079)

* To understand exactly which of your scripts is using what resources, please examine /home/amf/logs/resources/ starting tomorrow morning. This file can also be downloaded via ftp or accessed via the stats page at http://domain.com/stats/resources .

* Every day at 8AM PDT, a new file named “amf.sa.analyzed.0” will be created in the resources directory, and contains a breakdown of your usage. Older files are kept as “amf.sa.analyzed.1”, “amf.sa.analyzed.2” and so on, for 7 days.

* If you only see php.cgi in your report, you can find out how to get more fine detail at:

https://panel.dreamhost.com/kbase/index.cgi?area=3080

* You may also want to evaluate our dedicated server offerings at http://www.dreamhost.com/dedicated/, which we recommend for busier sites, sites which experience extremely high volumes of traffic and / or use a lot of resources. Plans start at just $99 a month.

You will continue to receive this warning message until your resource usage goes down.

Here’s the key data

Process               CPU seconds      user   machine   count  average
mt-comments.cgi 1492.7900 43.880% 6.220% 2115 0.706
mt-tb.cgi 1180.9000 34.712% 4.920% 2007 0.588
mt.cgi 525.5200 15.447% 2.190% 574 0.916

The script that is causing the problem is mt-comments.cgi. Which is odd as there were no comments to speak of yesterday, and it was a light spam day too.

So far I have gotten away with a very cheap hosting plan for what I think of as my hobby. Moving to a dedicated server would push the cost waay up. Not high enough for me to take ads, but high enough for me to half-wish I did.

But unless this is a blip, that’s where I’m headed.

This entry was posted in Discourse.net. Bookmark the permalink.

13 Responses to Dreamhost Thinks I Use Too Much CPU

  1. Cathy says:

    I would ask them whether there is some misfire going on in their systems’ configuration which is causing the mt*.cgi scripts to choke/churn more than usual. Or to at least diagnose what might be causing the problem. And if the problem is unavoidable, then see if they can simply throttle the execution of your scripts to keep your site running, yet obviously not at the expense of others’ on the system.

    (Note: I’m not actually a sys admin myself, but I’ve hung out with a lot with people who are… and from what I gather I think these are reasonable questions to ask. A competent sys admin should at least be able to have some insight into the problem.)

  2. Alexander says:

    My hoster semi-gently pushed me from MT (2.something) to WordPress. But maybe upgrading to a newer MT version is an idea — at least before mt-comments.cgi makes you move to a dedicated server.

  3. i’d suggest migrating to wordpress on dreamhost…

  4. cafl says:

    You don’t seem to have very many real comments relative to likely amounts of spam. I notice you put “Did you happen to see..” section on your comment page, which obviously is doing some kind of searching/analysis of your article database. Try turning this off (seems pretty useless/off point to me, given the small number of actual commenters — but it gets computed for the spammers too).

    There is probably some lame code in the script that is doing an expensive operation n**2 (or some non-linear function) times for something on your site where n is large. Typically the solution will be to turn off that feature if you don’t know how to fix the code or reduce n. Look in the script for nested loops.

  5. Seth Gordon says:

    I agree with Cathy and cafl’s suggestions above. That CPU number is suspicious.

  6. Michael says:

    Here are yesterday’s stats (I took Google Analytics’s stats javascript off the page, as this was my only recent change):

    Process               CPU seconds      user   machine   count  average
    mt-comments.cgi 1111.7300 47.275% 4.632% 1393 0.798
    mt-tb.cgi 703.6500 29.922% 2.932% 1174 0.599
    mt.cgi 222.8200 9.475% 0.928% 133 1.675

  7. I had my account suspended for a couple of days some time ago because a similar mail about mt-comments.cgi did not reach me (overlooked because it was buried in mail spam).

    This might happen because you have code in place that defends against comment spammers. Every attack by a spammer will count on your CPU usage, even if the comment in question does not make it through your line of defense.

    That is one reason why I have moved all my comment and trackback functions to a WordPress shadow site with no Google rank. I don’t want scripts running defending successfully against the comment spammers. I want to hide my site from their robots in the first place instead.

  8. eszter says:

    This exact same issue happened to me last summer. It led to a frantic few days. There is absolutely NO WAY that my account (with my blog plus everything else) has enough traffic to justify anything like this. That is the time I decided NOT to recommend Dreamhost anymore. I came close to switching to another host, but there are, of course, additional costs (timewise) associated with that, and after all the time I had already put into fixing this problem, I had none left for a switch.

    Don’t let them bully you around (that’s what I consider this). They will be VERY LITTLE help in fixing this issue, but just keep emailing them info to let them know that you are working on it and you need more help from their end.

    By the way, I was already using WordPress then so that’s not the only issue. I did upgrade and install a few spam-fighting techniques. I seem to be okay now.

  9. eszter says:

    Oops, sorry, I forgot an important point in my last comment: The entire incident was due to spam. A huge chunk of all the CPU usage was due to spam (in the form of both messages and trackback links). That was it. Ever since I’ve fixed that problem all has been fine. I’m nowhere close to using up too many resources.

  10. mr.wobble says:

    There are a lot of very annoyed dreamhost customers here:

    https://panel.dreamhost.com/kbase/index.cgi?area=3079
    (its possible they censor the comments)

    Unfortunately, I too am in the same boat as the people in the above thread. They waited until my 3 month money back guarantee was up and 3 days later started sending me these nag emails. No chance of a refund. They screwed me. I wont be recommending them to anyone who buys hosting and plans to do anything with it. If you want hosting to run your own email and share a photo or two, then they are perfect.

    I am shocked and disgusted by their conduct and treatment of their customers.

  11. mr.wobble says:

    There are a lot of very annoyed dreamhost customers here:

    https://panel.dreamhost.com/kbase/index.cgi?area=3079
    (its possible they censor the comments)

    Unfortunately, I too am in the same boat as the people in the above thread. They waited until my 3 month money back guarantee was up and 3 days later started sending me these nag emails. No chance of a refund. They screwed me. I wont be recommending them to anyone who buys hosting and plans to do anything with it. If you want hosting to run your own email and share a photo or two, then they are perfect.

    I am shocked and disgusted by their conduct and treatment of their customers.

  12. Note that there is, in fact, a bug in older versions of mt-comments.cgi that causes high server load, so the CPU numbers probably are real, and you should fix it.

    Search for “mt-comments.cgi load” on Google, or see: http://www.sixapart.com/movabletype/news/2004/12/comment_spam_load_issue.html

  13. Eyal says:

    I dont remember seeing anything about CPU time when signing up!!!
    i am in the process of switching hosts from DreamHost to StartLogic just because of this. i called 4 Hosting companies and all of them said that they dont pull this BS like DreamHost does!!!

Comments are closed.