How To Reach Your Hosting Bandwidth Limit By Front Paging Hacker News

Last week I wrote a blog post that got way more exposure than I ever expected it to. It was a post that I had been brainstorming for a while and I finally sat down on Thursday night and actually banged it all out rather than just incrementally adding to my Evernote outline like I normally do.

Now before I even begin to go into the story of the post and the subsequent traffic it has received, I want to start by thanking everyone for reading, commenting, and sending over such kind feedback. It was really a great experience overall for me to see the response from you all.

The Traffic! Look At All The Traffic!

So I would imagine most of you are interested in what kind of traffic being in the top 5 of Hacker News can actually send to your site on a relatively normal day. Below are some screenshots of the traffic over those days, along with the referrer and social breakdowns:

Now here are the referrer stats for that same time frame along with bounce rate and all that jazz:

Pretty neat eh? Lot’s and lots of traffic, more than this blog has gotten all year combined I think. Sweet.

Also seems that I picked up another 60 or so RSS subscribers and roughly the same amount of twitter followers. Very cool.

A Journey Down Fail Blvd

I woke up Saturday morning to many reports of my blog hitting a Bandwidth Limit, effectively taking it down for most of the weekend. I reached out to my host’s 24/7 support line and didn’t get a response until late Sunday (I don’t think they know what 24/7 actually means). That was frustrating as I probably missed out on a whole extra day of traffic, but oh well.

On Sunday I was able to move this site over a fresh Linode running behind nginx and using W3 Total Cache. You can probably already tell it is much snappier. I still need to tweak my nginx settings to make sure the site won’t buckle under heavy load again, but since everything is cached, it’s pretty rock solid now. I might even hook things up to CloudFlare soon as well for that extra security blanket.

Lesson to be learned from my mistake:

So What’s Next?

Well, I have a few plans:

  • A few of you asked if I could expand on some of the rules in my last post, to which I responded “absolutely!”. Now I should probably actually do that. In speaking with some friends, the idea of expanding each rule into its own few thousand word post could be a really nice way to fully capture what I’m going for with each idea, as well as provide a means of giving both good and bad examples of each rule.
  • Even cooler is that if I can expand each rule into its own post, I could compile them all and clean them up into a single eBook that I would just give away here for free (I might sell a printed version I guess, mainly to cover the costs of printing and to say that I’ve officially printed a damn book). We’ll see, that will take some time to fully see through, but would be a fun project.
  • I have plans to write out a Manifesto of my own personal beliefs when it comes to life and business that will extend far beyond these 12 Rules I wrote out. I have that fully outlined out, I just need to start cranking out some content. I’m hoping to start writing and publishing a post every other day from now on throughout the summer, so we’ll see how that goes.
  • I have a few other random blog post ideas that go along with startup mentality and strategy that I think will be good to share here as well.
  • Finally, I’ve been thinking about trying to contact and interview fellow startup founders and posting the interviews in a similar fashion to how UsesThis.com does their posts (a site which I am obsessed with)

So those are my ideas for the time being. I’m always open for suggestions, so send them over. You’ll probably also notice the snazzy new WordPress theme, gotta love ThemeForest 🙂

Happy 4th of July to all my American friends.

  • I’ve had my blog on the front page of HN 3 times so i can relate. The first time i overloaded my DNS server, amusingly! Personally, i host a static Jekyll blog on S3, so traffic isn’t an issue. Plus s3 is insanely cheap, so it might be worth looking into?

    Congrats though, it was a brilliant post, keep em coming please!

    • Thanks Chris! Yeah, I’ve had a few post tickle the bottom of the front page but none got as high (or held the spot) as much as that last one. Right now, I have a wordpress behind nginx with php-fpm on a small linode, and everything is fully cached so it should be able to take a bit of a beating (hopefully).

      I plan on stress testing it a bit after the fourth to make sure I haven’t put my foot in my mouth here.

      How do you find the publishing experience is with Jekyll? I’m a huge Ruby fan and have looked at it a few times, but I’m always drawn back to WordPress’ simplicity + massive plugin selection…

      • I find jekyll to be simply awesome, however you seem to have a few ‘gadgets’ around your page here and there so you might not have as much fun dealing with the limitations of being purely static. Plus, deploying changes takes a while (because you generally have to reupload everything), but for my needs (a simple blog) it works great. You’ve just got to be willing to accept that it means you’ll possibly have to cut out some features.

  • This happened to me a few months ago and I experienced a similar surge.

    It completed blasted my server. The host accused me of running harmful scripts to drain the resources maliciously (not sure why I’d do that!) – and the account was suspended for about a day.

    Lesson learnt though. I’ve installed WP Super Cache since then… and upgraded my server just in case.

    Congrats on the traffic boom!

  • So obviously you will be paying more for your server now, but what if the traffic stay at the lower end again. Do you then return back to the old (cheaper) configuration that has served you well, before the spike anyway, or do you keep paying for the more expensive server with the hope that you may hit the front page and have the high traffic again?

    • Well I actually only “upgraded” to a $20/month Linode server so I won’t be doing any downgrading as I’m fine paying that much for my hosting. I’d say for 99% of blogs, the cheapest Linode combined with caching is more than enough to handle quite a bit of traffic.

      If I were in a situation where I needed a more expensive Linode to handle higher traffic, I could upgrade my server in their admin panel for the time being then downgrade the server when necessary, so that’s an option. Other options include hosting on Heroku (they’ll scale automatically for you) or EC2.

      Hope that answer helps!

Leave a Reply

Your email address will not be published. Required fields are marked *