August 16, 2010: Self-hosting?

Right now all of The Undermine Journal is hosted using VPSes. The web and database servers are hosted at Linode for their superb support and uptime, and the crawlers are on various no-name cheap VPS hosts that don't have the same uptime requirements. The DB servers encounter high I/O wait on occasion, which slows down returning results to users. This is caused mostly by the huge number of random data writes this project requires, but also from sharing those disks with other VPS customers. This is mitigated by having a large-ish database cache in memory, but memory ain't cheap, either.

I recently considered hosting the whole project at home instead of using a VPS or dropping huge money at a colocation center with dedicated servers. I could build a box specifically for this project, with high-speed drives and gobs of memory, and use a business-class cable modem with a dedicated IP to host the website. The bandwidth requirements for The Undermine Journal are modest. It doesn't serve up many bytes as far as images or objects; the pages are spartan. The bandwidth shouldn't be a problem. And if I have my own box, I can upgrade it or split it myself. I pay once for the box instead of monthly. And the box would only run stuff for The Undermine Journal instead of split up among other users.

To do it right would mean buying a server-class motherboard and ECC memory, and server-purposed hard drives in a RAID array. That's not cheap. ;) To do it 90% as good (which is fine, since this is just data about a game running on donations, after all), I'd get a performance desktop mobo, a bunch of fast memory, and one or two 10k RPM drives. Initial pricing out of all that was about $750 for a server that could handle at least half of the US realms, if not all of them. The Internet connection would be $75/month, and wouldn't increase if I add more realms, unlike my current monthly costs.

I don't have the money for that right now, and the project may never require it. But it was interesting to think about, and I have another avenue for growth if the VPSes just aren't doing it for me.

Category: General | Posted by: admin

Comments

August 16, 2010, 16:41:43 James wrote:

I'm not sure if self-hosting is a good idea, the bandwidth may not seem like much but there is some unreliability with internet connections when hosted locally, at least, that's what I've experienced. May not be the case where you're at, additionally, have you considered a different database type, say MongoDB, Cassandra, or some other NoSQL database, where memory is not as much of an issue, but instead more disk space is used. I am using MongoDB currently, and I've seen major improvements in speed, and handling much larger loads, this is a system similar to Google's bigtable database that they use for their projects.

BTW, I had actually started working on a project like this, then I found this website, unfortunately my realm (Fenris/Horde) is not available, you'll see my vote every day until it is. Also, if a donation will speed things along (wink wink), let me know.

August 16, 2010, 17:55:54 Shaun wrote:

Willing to donate to help things along and get frostmane listed =]

August 16, 2010, 18:17:29 admin wrote:

No bribes! ;)

On-topic: I thought about going column-based instead of row-based for the DB, but all the queries and mechanics behind The Undermine Journal are based on MySQL and not easily changed. If MySQL had a column-based engine, I'd definitely give it a shot. I even looked into Amazon's SimpleDB, but the inability to delete many rows at a time turned me off, never mind the amount of write traffic I'd generate..

August 17, 2010, 20:29:34 Shaun wrote:

It's not a bribe, it's financial incentive!

August 19, 2010, 00:16:36 Falls wrote:

Reliability shouldn't be a massive concern, there are many businesses that run their server off a business-class ISP plan. Of course, if you're only granted one IP you may have the same issues you'd have with your current solution, with the rate limiting by the Armory. My company's DSL provider gives ~5 static public IP addresses to play with, just adding a switch to the modem and having each server pull the external IP it needs.

As for hardware, it shouldn't cost too much to build a reliable server. I'm not sure how much space you think you'll need at any given time (especially if you're culling the database every week or so), but you could get a VelociRaptor (or a few for redundancy) for performance. Pair that with 8GB RAM and a semi-fast CPU and you're good to go.

NewEgg has 150GB VelociRaptors for ~$140 and 8GB DDR3 RAM for <$200. Add that with a cheap motherboard/CPU combo, get a simple case if you don't already have one, grab Ubuntu server and you're good to go.

Also, about your earlier post regarding authentication; I agree with what most people were saying about using some kind of open authentication system (FBConnect/OpenID/Google). I work with many people who insist on having their browser automatically delete any offline content at close. Needing to go to your e-mail every time would be quite a hassle. Of course, it's so easy to tie into those other authentication APIs, you could always use more than one mechanism.

PS I love reading this blog, I'm learning quite a bit.

August 19, 2010, 20:05:40 admin wrote:

Falls: I won't run into rate-limiting for the Armory since the web server doesn't do any crawls anyway. All my crawls are fed out to cheap VPSes with their own IPs.

I agree, the hardware doesn't have to be expensive. It's not what you call cheap (which means I won't do it right away) but there's potential for high bang-for-your-buck once you reach a certain size and have those large requirements.

Will definitely open with a couple existing auth APIs so the user can pick and choose. The trick is to convince them that they need to use that same API every time they login, otherwise it'll think it's someone else. dude@google.com might be dudeman@livejournal.com but I'll never know unless I ask them to auth as one while logged in as another.

Add Comment

This item is closed, it's not possible to add new comments to it or to vote on it