Play Now Login Create Account
illyriad
  New Posts New Posts RSS Feed - Design Discussion (offtopic from Fav Petitions)
  FAQ FAQ  Forum Search   Register Register  Login Login

Topic ClosedDesign Discussion (offtopic from Fav Petitions)

 Post Reply Post Reply Page  123>
Author
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650
Direct Link To This Post Topic: Design Discussion (offtopic from Fav Petitions)
    Posted: 24 Mar 2010 at 15:50
Originally posted by GM Stormcrow GM Stormcrow wrote:

Originally posted by HonoredMule HonoredMule wrote:


I noticed this myself.  Is testing for functional errors or security holes the same as cheating?  I would want to know if someone could read my email, and given the rough edges still in the game, I take nothing for granted.

My next question was how long a database could support a single-column index for everyone's messages when the server is sending dozens of spam system messages to 100,000 players a day.  The answer for a bigint index and average 50 messages per player per day, yes--for about 5 billion years (and change).  And for a standard int, 117.5 thousand years.  I discontinued this line of inquisition.

Of course the real reason (G)UIDs are often used instead of ints isn't directly for the larger address space, but for easier generation of non-consecutive ids.  After all, with consecutive ids, you inevitably get some enterprising individual who realizes he can guess real ids finding messages like:
"error '80020009' /view_msg.asp, line 26"
I just hope there's no consecutive pattern in session ids AND that sessions are restricted to the IP that created them (since cookies on a non-secure connection can be compromized, after all).

To be honest, it's here more for the smile it produced on many people's faces. 

If we had truly considered this "cheating" we would have suspended the players' account - which we didn't. And yes, we like/love the reporting of non-standard error messages, however they have been generated.

So please take this post, and indeed this thread, in the spirit in which it is intended - a giggle rather than a serious comment on anything.

As to your other items, we're happy to discuss (some) elements of the database design philosophy, methodology, implementation and many other things (such as why on earth why we  chose to write the current UI in classic ASP) but this thread probably isn't the place for it. I would suggest the Technology & data forum?


Nah, I was just giving you all a hard time.  I'd find the use of asp over php and the awesomeness of Smarty templating strange, but it's apparent you guys are a Microsoft shop (error '80020009' => MSSQL), and I do get the benefits of tight integration that works.  I'm building a php RAD framework around such "Microsofty" design principles myself.  "Classic" asp, I'm guessing, scales better than more robust, business-logic-centric .NET stuff.  That or asp is more flexible for loose scripting.

What I do find curious is the constant polling of json_currentresources.asp.  That seems like a lot of unnecessary overhead for a server designed to handle 100,000 users, when you could just use javascript timers to maintain projected resource levels, and at least make the server polling less frequent.
Back to Top
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650
Direct Link To This Post Posted: 24 Mar 2010 at 16:07
Also, I'm stumped as to why the theft report's table showing resources stolen has rows highlighted in somewhat random fashion.  Example:

...

Hmm...I guess the html posting doesn't work like before.  At any rate, it didn't appear to indicate resource types cleaned out or any other special indication.


Edited by HonoredMule - 24 Mar 2010 at 16:10
Back to Top
GM Stormcrow View Drop Down
Moderator Group
Moderator Group
Avatar
GM

Joined: 23 Feb 2010
Location: Illyria
Status: Offline
Points: 3626
Direct Link To This Post Posted: 24 Mar 2010 at 16:36
Originally posted by HonoredMule HonoredMule wrote:

"Classic" asp, I'm guessing, scales better than more robust, business-logic-centric .NET stuff.  That or asp is more flexible for loose scripting.

Very much the latter.

Plus the fact that I'm also quite incredibly incompetent at almost any sort of front end stuff - which is the domain that ThunderClap roams in, as the beastmaster of a herd of scripts, functions and "cascading style sheets" (whatever they may be).  Javascripty-stuff makes me shiver, I still believe Ajax was the son of Telemon and I never bothered to learn anything about .NET.

I'm led to believe that moving stuff into .NET would actually give us a staggering reduction in server-side page render clock time, and so things (as they settle) will indeed start moving across to .NET, prioritised by the harder hits parts of the front end.

Originally posted by HonoredMule HonoredMule wrote:

What I do find curious is the constant polling of json_currentresources.asp.  That seems like a lot of unnecessary overhead for a server designed to handle 100,000 users, when you could just use javascript timers to maintain projected resource levels, and at least make the server polling less frequent.

Not my bit of the code I'm afraid, so can't comment - though I can say for sure that the DB I/O is quite beautifully petite.


Edited by GM Stormcrow - 24 Mar 2010 at 16:37
Back to Top
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650
Direct Link To This Post Posted: 24 Mar 2010 at 16:56
Isn't processing/latency generally more of an issue than data throughput when it comes to DB I/O anyway?  I generally do complete platform stuff (i.e. back-end, db, front-end, and client-side design/scripting), but not targeted at high-volume usage, so my curiosity is piqued.  With how you're designing things, whom would you consider to be in control of the number of db queries per page request--the back-end team or the web-front coders?  Is there a middle tier between?

Also, do you use direct SQL, traditional stored procedures, or full .NET in the db?

I know design philosophies are radically different in MS shops, but you guys seem to float somewhere between what works for MS products and it/business-centric software design, and what works for the standard internet technologies and massively scaled operation.


Edited by HonoredMule - 24 Mar 2010 at 16:57
Back to Top
GM Stormcrow View Drop Down
Moderator Group
Moderator Group
Avatar
GM

Joined: 23 Feb 2010
Location: Illyria
Status: Offline
Points: 3626
Direct Link To This Post Posted: 24 Mar 2010 at 17:32
Hi HM,

I was using I/O fairly loosely to mean an aggregate of all the important measurables regarding getting stuff into and out of the DB, including Stored Proc Execution time etc.

No, no middle Tier.  And we fundamentally believe that it is the back-end DB team who have final authority over anything and everything, and can change / countermand / force redesigns (or different designs) on anything the front-end people want to put together.

However, it helps that most of us have strengths in the SQL back end, and both ThunderClap and I share the same fairly rigorous views about SQL... all those classic old-school things like "If you want to use a trigger, you've broken it already" and "If you need a cursor, you've already failed".

Stored Procs for everything.  In fact, pretty much the entire logic of everything that happens in the game comes out of Stored Procs, and the front end pretty much exists to show the output.

Yes, I think you have the characterisation spot on. 

We have backgrounds in extremely high-volume transactional DBs, but we've also been in the commercial / business world enough to know that architecture considerations (and oftentimes the wishes of a tech team to use X or Y technology because it's "better") are often at the expense of rapid development, deployment, stability, flexibility, and more often than not, profit.

The best teams I've ever worked with have been small, like-minded people who are flexible enough not to dismiss ideas and technologies because they don't (personally) understand or work in those technologies, or because those technologies are "old". 

After many years of putting the "con" and the "insult" into "consulting", I've lost count of the number of very large companies I've worked in with vast development teams who have ultimately hamstrung themselves through a lack of understanding of business objectives & business necessities and a zombie-like devotion to "we must use Y, Y is new, Y is better, Y is faster, Y is cool", when X worked perfectly well and probably didn't actually need replacing. 

Through years of painful experience, I became (and still am) massively cynical about anyone who describes themselves as a "Business Analyst", "Systems Architect" or "Data Consultant" - except in very specific industries and applications.

Anyway, you'll start me ranting for hours, so I'll stop now Embarrassed
Back to Top
GM ThunderCat View Drop Down
Moderator Group
Moderator Group
Avatar
GM

Joined: 11 Dec 2009
Location: Everywhere
Status: Offline
Points: 2150
Direct Link To This Post Posted: 24 Mar 2010 at 18:05
Originally posted by HonoredMule HonoredMule wrote:

Also, do you use direct SQL, traditional stored procedures, or full .NET in the db?
For a open publiclly accessable website direct SQL is very much in the "tread carefully" area as you can't slam the doors of security tightly shut at multiple points as easily and quickly. [Though obviously you should be using parametrised queries whatever - hello mr injection]

I may slip some .NET into the db when SC's not looking (Shhh...); but mainly just for things SQLServer is bad at: email, raw file i/o etc - naturally kicked off into an asynchronous thread to keep up the linear execution speed of the procs. But we don't talk about that Wink


Edited by GM ThunderClap - 24 Mar 2010 at 18:12
Back to Top
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650
Direct Link To This Post Posted: 24 Mar 2010 at 18:10
Quote "Whom would you consider to be in control of the number of db queries per page request--the back-end team or the web-front coders?

I didn't mean that in an authoritative/responsibility sense, but rather "whose code actually defines the amount of db usage."  Without a middle tier, I would presume that means the front-end makes a lot of direct calls, but being limited to stored procedures puts the optimization/I/O control back in the db admins' hands.

Quote "If you want to use a trigger, you've broken it already" and "If you need a cursor, you've already failed".

Spot on. :)

Quote The best teams I've ever worked with have been small, like-minded people who are flexible enough not to dismiss ideas and technologies because they don't (personally) understand or work in those technologies, or because those technologies are "old".

So very true.  After this term in university ends, I will be taking my time finding a job, not only to have time to develop my pet project, but also to take my time finding a group of "small, like-minded people" with which to work.  I had such a position, and may return to it yet...but that crew was drifting further and further away from web development, where my passion lies...and their clientele was "a bit seedy" (mostly telemarketers).  But one of the first things I learned while there was that perl in the right hands can still be pretty freaking awesome.
Back to Top
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650
Direct Link To This Post Posted: 24 Mar 2010 at 18:17
Originally posted by GM ThunderClap GM ThunderClap wrote:

Originally posted by HonoredMule HonoredMule wrote:

Also, do you use direct SQL, traditional stored procedures, or full .NET in the db?
For a open publiclly accessable website direct SQL is very much in the "tread carefully" area as you can't slam the doors of security tightly shut at multiple points as easily and quickly. [Though obviously you should be using parametrised queries whatever - hello mr injection]

I may slip some .NET into the db when SC's not looking (Shhh...); but mainly just for things SQLServer is bad at: email, raw file i/o etc - naturally kicked off into an asynchronous thread to keep up the linear execution speed of the procs. But we don't talk about that Wink


I'm pretty keen on prepared statements myself.  I'm of a mind that if the front-end team can't be trusted to systematically handle SQL safely, they're probably going to create even bigger security holes via arbitrary code execution, which can include SQL anyway--and the restrictions just increase the back-and-forth/red tape, slowing development.  But if I were working with a larger team on larger deployments (i.e. the front and back end teams are actually separate entities as opposed to the same person or team of 3 people), I'd disable non-stored-procedure queries on the back end too.


Edited by HonoredMule - 24 Mar 2010 at 18:20
Back to Top
GM ThunderCat View Drop Down
Moderator Group
Moderator Group
Avatar
GM

Joined: 11 Dec 2009
Location: Everywhere
Status: Offline
Points: 2150
Direct Link To This Post Posted: 24 Mar 2010 at 18:21
Originally posted by HonoredMule HonoredMule wrote:

What I do find curious is the constant polling of json_currentresources.asp.  That seems like a lot of unnecessary overhead for a server designed to handle 100,000 users, when you could just use javascript timers to maintain projected resource levels, and at least make the server polling less frequent.

Thefts, completing resource buildings, trades, attacks, spells etc all alter the resource levels and the client side doesn't really know anything about. Its an area we are keeping an eye on but it isn't currently causing any issue.
Back to Top
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650
Direct Link To This Post Posted: 24 Mar 2010 at 18:27
You may also wish to note that it keeps accounts logged in indefinitely.  I tend to leave tabs open, which means generating hundreds of queries when I'm not even there.  That also means I'm reported as online all the time unless I intentionally log out (which I never do, preferring to keep tabs open as an indicator/reminder of future intentions).  It's perhaps not a common use pattern, but one that exists nevertheless.
Back to Top
 Post Reply Post Reply Page  123>
  Share Topic   

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 12.03
Copyright ©2001-2019 Web Wiz Ltd.