Welcome, Guest. Please login or register.

Username: Password:
Pages: [1]   Go Down

Author Topic: short question and testing 123  (Read 8398 times)

jaRf

  • Fan Calendar Participant
  • Tuftear
  • *
  • Posts: 269
  • just another Rayek fan
    • View Profile
short question and testing 123
« on: March 27, 2009, 05:06:15 PM »

Good to have an EQ artsite back!
Is there any implementation of SSL/TLS for login or entry of email adresses etc. planned?

*sigh*
I should really consider trying to do some EQ art...


That's all for now.
Logged

Obi-Rak Kaeliri

  • The Silent One
  • Treewee
  • *
  • Posts: 2
    • View Profile
    • Jungle Fire webcomic
Re: short question and testing 123
« Reply #1 on: March 27, 2009, 05:24:37 PM »

Hi there, and welcome :)

We would love to do SSL/TLS for logins and account management, but our current host has an architecture that does not lend itself to SSL. We have plans for moving to a more dedicated hosting situation, which will enable us to use SSL/TLS as we see fit.

Once we have the ability, it is high in our list of priorities.

jaRf

  • Fan Calendar Participant
  • Tuftear
  • *
  • Posts: 269
  • just another Rayek fan
    • View Profile
Re: short question and testing 123
« Reply #2 on: March 28, 2009, 02:08:41 AM »

Ah, good to hear (read). I'm glad someone just did it and set up the whole system to have an EQ fan art site. Thanks!
So you're the /root here?
May I ask which system/distribution the server is running on? And how much is hosting cost? Will you need donations to cover cost?
Logged

Obi-Rak Kaeliri

  • The Silent One
  • Treewee
  • *
  • Posts: 2
    • View Profile
    • Jungle Fire webcomic
Re: short question and testing 123
« Reply #3 on: March 28, 2009, 11:36:28 AM »

I'm glad we did -- and I'm glad you're all here :)

At the moment this site is on a shared host, so none of us is really /root. I just play one. We are planning on promoting our staging environment into a colo-box, fairly shortly actually, in which case I would indeed be the root. These servers are running FreeBSD 7.1, and I will continue to use the same in the colo-box down the road.

Right now the hosting is dirt cheap, because we use a pay-as-you-go type host; the projected monthly cost is about $4 for this month. That amount will rise, of course, as more people discover the site, but for the time being we are happy footing the bill. We have set sort of a soft-ceiling of $10/month real cost before we start exploring other options. At that point we would not solicit donations, but possibly add a couple of non-intrusive ads somewhere on the site. Either that, or see if we can just write the whole thing off in taxes...

Obi-Rak Kaeliri

  • The Silent One
  • Treewee
  • *
  • Posts: 2
    • View Profile
    • Jungle Fire webcomic
Re: short question and testing 123
« Reply #4 on: July 06, 2010, 11:47:28 PM »

<thread-necromancy>

SSL is now enabled :)

</thread-necromancy>

jaRf

  • Fan Calendar Participant
  • Tuftear
  • *
  • Posts: 269
  • just another Rayek fan
    • View Profile
Re: short question and testing 123
« Reply #5 on: July 17, 2010, 02:37:21 AM »

Aaaah! Nice. And I just noticed a new design. One that I seem to like.
First I was confused about the missing "gallery" on the left side but now... seems to render correctly everything til now... nice job!
Logged

Obi-Rak Kaeliri

  • The Silent One
  • Treewee
  • *
  • Posts: 2
    • View Profile
    • Jungle Fire webcomic
Re: short question and testing 123
« Reply #6 on: July 17, 2010, 12:08:56 PM »

Yar, we's trying to make things better this time around. Fixing a lot of old bugs and generally cleaning up things. It's so much nicer having a direct access to the server -- there were a bunch of memory allocation related bugs causing issues we never knew about because we didn't have access to certain logs.

A bit of trivia: one of the new design criterias we used when re-skinning the site was that the gallery is supposed to look and function ok on a smaller touch-based interface, like that of an iPad. We have been using tablet-PC type devices, with active digitizers, for better part of the past decade, and we figured we might as well embrace the future now that this whole touch fad doesn't seem to be going away. A nice side-effect of this is that the QA is essentially done on WebKit-based browser.

Obviously this site still has ways to go before it's truly cross-platform in a graceful way, but things should be better than they used to be. Plus the changes in the back-end means a lot easier time updating things in the future.

jaRf

  • Fan Calendar Participant
  • Tuftear
  • *
  • Posts: 269
  • just another Rayek fan
    • View Profile
Re: short question and testing 123
« Reply #7 on: July 27, 2010, 11:51:15 AM »

Analyzing logs and testing on various HW platforms and OS/browsers sounds like some bunch of work to me. Thanks for doing that job!
What does mem alloc problems mean exactly? Not enough RAM/Swap? Bad RAM? Or some program having a memory leak and wasting more and more RAM?
If I may ask: is it a real server where you have root access or a parted cake (like one of some vitual machines running on one large box)?
Logged

Obi-Rak Kaeliri

  • The Silent One
  • Treewee
  • *
  • Posts: 2
    • View Profile
    • Jungle Fire webcomic
Re: short question and testing 123
« Reply #8 on: July 31, 2010, 02:17:43 PM »

Just part of the service :)

In this context, the PHP script has been trying to allocate more memory than it really needs. This has caused some instances where the script runs into per-process memory limits and fails. The user basically tries to upload an image of certain type and proportions and encounters a nice, juicy, 500 server error. We nipped that right out once we got access to detailed logs.

It's not a real server. It's basically a Xen domU hosted on Gigenet's cloud service. So technically we have root access to our domU, but we're not masters of our own hardware.

Continuing a little bit into how we have things set up, we have two cloud instances serving these sites. One instance serves as the front-end web/app server, and the second serves as the dedicated database server. I separated these two roles out just to dedicate some resources to the database, and to gain some extra security over a single-server solution. I also do all of the builds on the database server, and just copy the resulting binary or package to the web server.

The web server talks to the database service over a persistent SSH tunnel over a private back-end network. All this on the assumption that the front-end web server will get compromised at some point or another; we're doing everything we can to minimize the potential damage. Including taking backups and storing them off-site.

I've included a diagram that shows how it's all put together. Ultimately it's a pretty simple layout, and I like simple because complexity is the number 1 enemy of security.

We have endeavored to beef up security as much as possible including migrating some of our dynamic sites to static HTML, since those sites didn't really gain much from being dynamic. These sites are still pseudo-dynamic in the sense that we use dynamic scripts to generate the static content, so maintaining these converted sites remains easy. We just don't front load the dynamic content.

To summarize...
  • "Servers" are "cloud" instances provided by Gigenet Cloud.
  • The front-end server software is Hiawatha as opposed to Apache.
  • The back-end database software is your run-of-the-mill MySQL.
  • A lot of the static content is served out of Cachefly CDN, who in many ways beat Akamai when it comes to serving small static content.
  • Except for the automated emails sent by the dynamic scripts, email service is handled by Tuffmail whose spam filters are outstanding. We do not hide any of our contact information, having it visible in unobfuscated form on our web sites and whois records, and yet our inboxes are not bombarded by spam.
  • DNS services are provided by DNSMadeEasy.
  • External uptime monitoring and alerting by Hyperspin.

All in all it's a pretty robust and surprisingly cheap stack that off-loads a bunch of the administrative overhead from Foxeye and myself. We have way more headroom than we need, but the extra cost for achieving this headroom is near trivial. We also have pretty high-intensity dayjobs, so low-intensity administration is a high priority. Fortunately at this point the stack pretty much runs itself, and has decent ability to self-correct any non-code related problems.




(Click on the picture to see the large version)
« Last Edit: July 31, 2010, 03:46:12 PM by Obi-Rak Kaeliri »
Logged

jaRf

  • Fan Calendar Participant
  • Tuftear
  • *
  • Posts: 269
  • just another Rayek fan
    • View Profile
Re: short question and testing 123
« Reply #9 on: August 08, 2010, 02:16:30 PM »

Thanks for the vast and detailed explanation. I can understand so well that you're looking for a less time consuming administration of web content with the RL day jobs. When I came/come home from work the recent years I rarely found the time to mess around with things on my boxes. (Ok, my fault that I chose Gentoo on 3 boxes, but then my gaming Windows NT5.1 can be a p.i.t.a. when something just doesn't want to work). So I really appreciate the work you're doing here.
Readings logs can be really helpful sometimes (my primary fails come in Xorg.0.log when I try a new GPU driver and the emerge output) but it can also be exhausting to search through a lot and then get a solution from an error message.
But do you trust cloud services? I still think that if something there goes wrong... huh. All that data spread around somewhere where you don't know but the system thinks it's a good place for the work balance.
The idea of havin two separate servers is nice, and then the virtual machines make it possible at low cost. Redundancy is helpful and though I'd normally expect to make a targeted crack of this site a generic takeover for building zombienets or whatever could be possible. Yes, might save a lot of work and nerves in case. And an offsite backup is the only way to go. I remember Gentoo-wiki going down because the hoster went bankrupt or something and nobody had any physical access and there was no backup (besides what people mirror, you know, Linus "real me don't need backups, they upload their important stuff to an ftp and let the world mirror it).
It's also interesting to see for me (with low web management experience) how you can split up services belonging to a website to multiple hosters.

Thanks!
Logged

Obi-Rak Kaeliri

  • The Silent One
  • Treewee
  • *
  • Posts: 2
    • View Profile
    • Jungle Fire webcomic
Re: short question and testing 123
« Reply #10 on: August 08, 2010, 10:18:03 PM »

Do I trust cloud services? That's a multi-faceted question with equally multi-faceted answers :)

I don't really trust anything I don't directly operate myself, and even then I recognize that there are inherent limitations present in my own capabilities.

That said, I trust the cloud services enough to use them for non-critical stuff like these sites. It's a simple cost-benefit tradeoff: I trade running my own show for access to much larger and more robust infrastructure than what I'm willing to pay for.

In order to provide a truly high quality and trustworthy service you need dedicated servers in a reputable datacenter with multiple transit carriers and a team of engineers who know their stuff.  All under your control. I do that IRL. That costs a lot of money, but on the flipside the fate of the service is thoroughly on me. The sort of money I blow on achieving that last nine in my day job is absolutely out of the question for low- or no-revenue sites like these.

Mind you, if you're willing to play fast and loose you can run a fairly reliable service with just a single server on a consumer-grade DSL connection. Most of the time it works. Sometimes it doesn't, and then it sucks. Been there; done that. You also still have to accommodate for power and network outages, or physical moves to a different town, which introduces cost and complexity into the whole setup.

On the other hand, there's this bunch of folks out there on the Internet who have built large infrastructures who are happy to lend you a slice of it. You get a fair bit of the benefits of a large infrastructure for reasonable opex and no capex whatsoever.

You just have to recognize the platform's inherent limitations. IOW, you don't trust it implicitly with your data. You avoid relying on the platform for all of your data. You store none of your critical data on it. You avoid storing sensitive data on it, and you assume it's going to get whacked at some point or another, and figure out whether you can live with it when it does happen.

This sort of circles back to what you mentioned about the targeted hacks. I'm pretty confident that if this site, or these servers, were subjected to a true targeted attack, we'd get whacked. Were that to happen, we'd just look at the logs -- which we store off-server in real time -- fix the vulnerability that got us whacked, and move on. There's no sensitive data on these servers -- even your passwords are stored as one-way hashes of the real password -- so the only thing we'd lose is being bad netizens, spewing bot-traffic at the Internet at large, until we whacked and re-built the servers from scratch.

In the final analysis, it's all an attempt at the lowest possible cost for maximum possible performance and minimum possible management overhead :)

You always trade off something.

I try to trade off the bit that's most irrelevant.

And users always come first. If they don't, they go elsewhere.


~K

BTW, I do remember the gentoo-wiki going belly-up. It was sad, too, because that wiki had some real useful info on making the HP-proprietary bits of ACPI on HP 2710p work on linux.
« Last Edit: August 08, 2010, 11:14:31 PM by Obi-Rak Kaeliri »
Logged
Pages: [1]   Go Up