Design Discussion (offtopic from Fav Petitions)
Printed From: Illyriad
Category: Miscellaneous
Forum Name: Technology & data
Forum Description: Discussions on data dumps, downloads, and third party applications.
URL: http://forum.illyriad.co.uk/forum_posts.asp?TID=204
Printed Date: 16 Apr 2022 at 21:10
Software Version: Web Wiz Forums 12.03 - http://www.webwizforums.com
Topic: Design Discussion (offtopic from Fav Petitions)
Posted By: HonoredMule
Subject: Design Discussion (offtopic from Fav Petitions)
Date Posted: 24 Mar 2010 at 15:50
Nah, I was just giving you all a hard time. I'd find the use of asp over php and the awesomeness of Smarty templating strange, but it's apparent you guys are a Microsoft shop (error '80020009' => MSSQL), and I do get the benefits of tight integration that works. I'm building a php RAD framework around such "Microsofty" design principles myself. "Classic" asp, I'm guessing, scales better than more robust, business-logic-centric .NET stuff. That or asp is more flexible for loose scripting. What I do find curious is the constant polling of json_currentresources.asp. That seems like a lot of unnecessary overhead for a server designed to handle 100,000 users, when you could just use javascript timers to maintain projected resource levels, and at least make the server polling less frequent. |
GM Stormcrow wrote:Replies:
Posted By: HonoredMule
Date Posted: 24 Mar 2010 at 16:07
|
Also, I'm stumped as to why the theft report's table showing resources
stolen has rows highlighted in somewhat random fashion. Example: ... Hmm...I guess the html posting doesn't work like before. At any rate, it didn't appear to indicate resource types cleaned out or any other special indication. |
Posted By: GM Stormcrow
Date Posted: 24 Mar 2010 at 16:36
Very much the latter. Plus the fact that I'm also quite incredibly incompetent at almost any sort of front end stuff - which is the domain that ThunderClap roams in, as the beastmaster of a herd of scripts, functions and "cascading style sheets" (whatever they may be). Javascripty-stuff makes me shiver, I still believe Ajax was the son of Telemon and I never bothered to learn anything about .NET. I'm led to believe that moving stuff into .NET would actually give us a staggering reduction in server-side page render clock time, and so things (as they settle) will indeed start moving across to .NET, prioritised by the harder hits parts of the front end.
Not my bit of the code I'm afraid, so can't comment - though I can say for sure that the DB I/O is quite beautifully petite. |
Posted By: HonoredMule
Date Posted: 24 Mar 2010 at 16:56
|
Isn't processing/latency generally more of an issue than data throughput when it comes to DB I/O anyway? I generally do complete platform stuff (i.e. back-end, db, front-end, and client-side design/scripting), but not targeted at high-volume usage, so my curiosity is piqued. With how you're designing things, whom would you consider to be in control of the number of db queries per page request--the back-end team or the web-front coders? Is there a middle tier between? Also, do you use direct SQL, traditional stored procedures, or full .NET in the db? I know design philosophies are radically different in MS shops, but you guys seem to float somewhere between what works for MS products and it/business-centric software design, and what works for the standard internet technologies and massively scaled operation. |
Posted By: GM Stormcrow
Date Posted: 24 Mar 2010 at 17:32
|
Hi HM, I was using I/O fairly loosely to mean an aggregate of all the important measurables regarding getting stuff into and out of the DB, including Stored Proc Execution time etc. No, no middle Tier. And we fundamentally believe that it is the back-end DB team who have final authority over anything and everything, and can change / countermand / force redesigns (or different designs) on anything the front-end people want to put together. However, it helps that most of us have strengths in the SQL back end, and both ThunderClap and I share the same fairly rigorous views about SQL... all those classic old-school things like "If you want to use a trigger, you've broken it already" and "If you need a cursor, you've already failed". Stored Procs for everything. In fact, pretty much the entire logic of everything that happens in the game comes out of Stored Procs, and the front end pretty much exists to show the output. Yes, I think you have the characterisation spot on. We have backgrounds in extremely high-volume transactional DBs, but we've also been in the commercial / business world enough to know that architecture considerations (and oftentimes the wishes of a tech team to use X or Y technology because it's "better") are often at the expense of rapid development, deployment, stability, flexibility, and more often than not, profit. The best teams I've ever worked with have been small, like-minded people who are flexible enough not to dismiss ideas and technologies because they don't (personally) understand or work in those technologies, or because those technologies are "old". After many years of putting the "con" and the "insult" into "consulting", I've lost count of the number of very large companies I've worked in with vast development teams who have ultimately hamstrung themselves through a lack of understanding of business objectives & business necessities and a zombie-like devotion to "we must use Y, Y is new, Y is better, Y is faster, Y is cool", when X worked perfectly well and probably didn't actually need replacing. Through years of painful experience, I became (and still am) massively cynical about anyone who describes themselves as a "Business Analyst", "Systems Architect" or "Data Consultant" - except in very specific industries and applications. Anyway, you'll start me ranting for hours, so I'll stop now  |
Posted By: GM ThunderCat
Date Posted: 24 Mar 2010 at 18:05
I may slip some .NET into the db when SC's not looking (Shhh...); but mainly just for things SQLServer is bad at: email, raw file i/o etc - naturally kicked off into an asynchronous thread to keep up the linear execution speed of the procs. But we don't talk about that  |
Posted By: HonoredMule
Date Posted: 24 Mar 2010 at 18:10
I didn't mean that in an authoritative/responsibility sense, but rather "whose code actually defines the amount of db usage." Without a middle tier, I would presume that means the front-end makes a lot of direct calls, but being limited to stored procedures puts the optimization/I/O control back in the db admins' hands.
Spot on. :)
So very true. After this term in university ends, I will be taking my time finding a job, not only to have time to develop my pet project, but also to take my time finding a group of "small, like-minded people" with which to work. I had such a position, and may return to it yet...but that crew was drifting further and further away from web development, where my passion lies...and their clientele was "a bit seedy" (mostly telemarketers). But one of the first things I learned while there was that perl in the right hands can still be pretty freaking awesome. |
Posted By: HonoredMule
Date Posted: 24 Mar 2010 at 18:17
I'm pretty keen on prepared statements myself. I'm of a mind that if the front-end team can't be trusted to systematically handle SQL safely, they're probably going to create even bigger security holes via arbitrary code execution, which can include SQL anyway--and the restrictions just increase the back-and-forth/red tape, slowing development. But if I were working with a larger team on larger deployments (i.e. the front and back end teams are actually separate entities as opposed to the same person or team of 3 people), I'd disable non-stored-procedure queries on the back end too. |
Posted By: GM ThunderCat
Date Posted: 24 Mar 2010 at 18:21
Thefts, completing resource buildings, trades, attacks, spells etc all alter the resource levels and the client side doesn't really know anything about. Its an area we are keeping an eye on but it isn't currently causing any issue. |
Posted By: HonoredMule
Date Posted: 24 Mar 2010 at 18:27
| You may also wish to note that it keeps accounts logged in indefinitely. I tend to leave tabs open, which means generating hundreds of queries when I'm not even there. That also means I'm reported as online all the time unless I intentionally log out (which I never do, preferring to keep tabs open as an indicator/reminder of future intentions). It's perhaps not a common use pattern, but one that exists nevertheless. |
Posted By: GM ThunderCat
Date Posted: 24 Mar 2010 at 18:30
 Obviously, it depends on the situation. If its an internal or closed user group business app then direct SQL isn't such a problem. |
Posted By: GM Stormcrow
Date Posted: 24 Mar 2010 at 18:33
Fixed it for you, HM |
Posted By: HonoredMule
Date Posted: 24 Mar 2010 at 18:44
|
lol. I admire your fatalistic optimism. |
Posted By: GM Stormcrow
Date Posted: 24 Mar 2010 at 18:47
A witch! Burn him! |
Posted By: fluffy
Date Posted: 24 Mar 2010 at 18:50
You don't look like a newt  |
Posted By: HonoredMule
Date Posted: 24 Mar 2010 at 18:52
|
Question: In more general situations (the game's event queue being a unique circumstance), if a webserver is compromised, doesn't the attacker pretty much have run of the palace just using any of the methods needed to accomplish actual functionality anyway? Normally, a web-based app is designed to manage an information system, making it the already critical point of failure, security wise. Coming from my background, possibility of a compromized db was a serious concern not because of elevated access to the app's database, but because of access to other databases of other applications (otherwise protected by the requirement of different access credentials), including such things as customer information and credit card numbers (even CVV numbers for some naughty apps...ouch). Such compromise was not in any way possible under any circumstance by through a hacked app, regardless of how the DB was accessed. |
Posted By: rescendent
Date Posted: 24 Mar 2010 at 19:03
|
Posted By: GM ThunderCat
Date Posted: 24 Mar 2010 at 19:03
If you were use direct sql the web site access to the database would give it the ability to do updates etc on the tables - in which case you do a whole host of things you shouldn't be able to do. |
Posted By: HonoredMule
Date Posted: 25 Mar 2010 at 00:24
|
Right. What I'm meaning to say is that you have far more capacity to control what can be done by an authorized entity (i.e. the webserver) than you would more typically. There's clearly a lot that is done to carry out game progress and comparatively very little the website needs to be allowed to do. CRM software, for example must be able to access and arbitrarily modify so much sensitive information, and do so much with it interactively (such as charging credit cards or reversing charges) that compromising the CRM itself is about as serious a security leak as one can have, regardless of how tightly DB access from the webserver is controlled. Of course that's where good software should ideally be splitting the front end into an actual front end and a middle tier for business logic and some more dynamic/heuristic security checks. At least then you can, for example, keep full credit card numbers away from the web server, though it must still be allowed to do things with the credit card numbers it can't see. In practice, I've never actually seen a proper 3-tier web-based project. To be honest, the stuff I've seen in the wild kind of scares me. But like I mentioned earlier, the clientele for which I was indirectly working was a bit seedy, and a hard sell for security improvements that require a rewrite for existing architecture that just plain sucks. One already takes his chances, I guess, when dealing with telemarketers or borderline-fraudulent web services (like sites selling access to casting calls that were just scraped from other sites like craigslist). Just don't be surprised if having a hard time canceling your account isn't your only grief. I'd love to find a similar shop but working on more respectable projects. My coworkers and boss were awesome, but their business associations less so. I absolutely love the hat, by the way. It's impossible to find a decent bowler hat around here that fits my narrow head. So I wear a soft fedora instead.  |
Posted By: HonoredMule
Date Posted: 25 Mar 2010 at 05:02
|
Stormcrow, are you familiar with thedailywtf.com? There's a thread in my alliance's forum labeled "Perhaps
the purpose of your life is to serve as a warning to others." Sadly,
everyone frequents that thread often, myself included. thedailywtf.com is the software engineer's version of that thread. Being a DB admin yourself, I figured you'd especially enjoy the site's latest article: http://thedailywtf.com/Articles/The-Certified-DBA.aspx - http://thedailywtf.com/Articles/The-Certified-DBA.aspx |
Posted By: GM Stormcrow
Date Posted: 25 Mar 2010 at 06:46
Never, never a truer word spoken, HM. My personal favourite is when the 3-tier project spirals out of control, creating a Frankenstein's monster that suddenly gets reclassified as n-tier simply because no one can actually define where the "tiers" start, end or how many there are.
Yes, I love thedailywtf. Hadn't seen that Article though, and it was very lol :) Thanks! |