Play Now Login Create Account
illyriad
  New Posts New Posts RSS Feed - XML Parsing
  FAQ FAQ  Forum Search   Register Register  Login Login

Topic ClosedXML Parsing

 Post Reply Post Reply Page  12>
Author
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650
Direct Link To This Post Topic: XML Parsing
    Posted: 31 Jan 2011 at 16:16
I have some big plans for new services and tools for Illyriad players, the most prominent of which being HarmlessButler 2.0 for the next interface.  The new version won't use the heavy, clunky local database like the current one, but that means I need an online database HB can query.  That in turn means I might as well make full use of that database with all kinds of other public tools and services, and believe me, I've got lots of ideas.

However, before doing any server-side stuff, I'm first hell-bent on finishing a very dear personal project on which I wish to depend for many of these services.  Seeing any of these new goodies (including HB 2.0, unfortunately) will be a few weeks yet at least.  In the mean time, I thought I'd share an important puzzle piece with which other boffins no doubt struggle: how to usefully parse through those big XML files.  Hopefully this will help other developers build their own little tools quickly and easily.

The "nice" XML handling tools can't deal with such large files and run out of memory, and the stream-based parsers are pretty ugly and hard to use.  So I wrote an abstracted class-based system specially for Illyriad's data files that anyone can use.  Rather than being a finished but also inflexible solution, it's up to you to write the classes that read specific information and post it to whatever data store you like using whatever structure you like.  In this manner, it supports all past, present, and likely future formats at any file size while remaining extremely simple to use.  The classes can be as complete or as brief as desired.

Each class you write (like AllianceParser extends RowParser) handles a single table, and each instance of the class handles one row.  Each protected method you write (like parse_allianceticker($data, $attrs) ) receives the data specific to that node which you can compile as desired into $this->data using whatever structure and format you like.  When the information within the handled row is complete, the method processData() will be called--this is the only one you must implement.  From there, you can add information from parent classes and push $this->data wherever you like.  Because YOU defined the data layout, query builders can easily and automatically submit the data to database tables YOU defined, and other software you write can work with it easily as well, having just what it needs and where it wants it.  For added convenience, when you run your PHP through a web server, the classes will spit out lists of the nodes that were found and not parsed, serving as a guide while you're still writing your classes.

The parser classes can be nested as well--for example, RoleParser will parse the roles in an alliance, and has access to the information from its parent AllianceParser.  The classes can stack to any depth and still find a particular parent by node name, so all the structural/relational information you need will be preserved.  The system is also clever enough to know an alliance node within an alliance node isn't really an alliance within an alliance--the outer one is an alliance object and the inner one is some of the important information.  So, when quirks like that are encountered, only one AllianceParser is created, with the inner data-rich node getting handled by parse_alliance($data, $attrs) and the encapsulating one by parse_outer_alliance($data, $attrs).

Developers can fetch the code and a usage example at http://illyriad.honoredsoft.com/wiki/Tool:IllyriadParser
Back to Top
sityviper View Drop Down
Greenhorn
Greenhorn
Avatar

Joined: 05 Jul 2010
Location: USA
Status: Offline
Points: 81
Direct Link To This Post Posted: 13 Mar 2011 at 01:31
how do i use the script?
.i...V.... 3
Back to Top
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650
Direct Link To This Post Posted: 13 Mar 2011 at 04:13
The linked page explains in a bit more detail and provides a simple example.  Keep in mind that how you structure/organize/use the data is your domain.  This is just one piece of the puzzle...a typically annoying and obtuse one.
Back to Top
Llyorn Of Jaensch View Drop Down
Postmaster
Postmaster
Avatar

Joined: 31 Mar 2010
Location: Sydney
Status: Offline
Points: 924
Direct Link To This Post Posted: 13 Mar 2011 at 12:36
I'd like all Illyriad players to realise and appreciate what HM has done.

HM has developed a huge contribution to Illyriad. His contributions are available to all players not simply his alliance mates.  In creating Harmless Butler he has through no personal gain refined Illyriad to a professional and state of the art system.

Personally I think he is due much more than the minimal recognition he has so far received. This applies to both Devs and simply any player who, without recompense, uses HB.


Edited by Llyorn Of Jaensch - 13 Mar 2011 at 12:38
Back to Top
Mr. Ubiquitous Feral View Drop Down
Forum Warrior
Forum Warrior
Avatar

Joined: 01 Jan 2011
Location: U.S.A.
Status: Offline
Points: 419
Direct Link To This Post Posted: 13 Mar 2011 at 14:22
I do not understand code.  Therefore, it does not exist.  OPP's.  (Other People's Problems)
I am a Machine.
Back to Top
JohnnyBravo View Drop Down
New Poster
New Poster
Avatar

Joined: 13 Mar 2011
Status: Offline
Points: 15
Direct Link To This Post Posted: 13 Mar 2011 at 14:54
Every time I go on the wiki site HM set up I appreciate that he did it and have tried to update some things to put something back. When i use HB, i am thankful that he put it together. and when H? was fighting White i was rooting for you guys as i felt the game would be weaker without players like HM (even though i came in to the game through RL players in white).

So HM, we really do appreciate what you do for the game.




Edited by JohnnyBravo - 13 Mar 2011 at 15:45
Back to Top
Nokigon View Drop Down
Postmaster General
Postmaster General
Avatar
Player Council - Historian

Joined: 07 Nov 2010
Status: Offline
Points: 1452
Direct Link To This Post Posted: 13 Mar 2011 at 15:41

Whoa! Calm it people!

Back to Top
Lionz Heartz View Drop Down
Forum Warrior
Forum Warrior
Avatar

Joined: 27 Oct 2010
Location: Megan Fox
Status: Offline
Points: 292
Direct Link To This Post Posted: 15 Mar 2011 at 01:06
BE WARNED!!!

HM wants you all to use Harmless Butler 2 because...

In Lord of the Rings, Dark Lord Sauron made several rings to all of the leaders of dwarves, elves, and humans. Little did they know that Dark Lord Sauron made one ring, TO RULE THEM ALL!!!

HonoredMule, just like Dark Lord Sauron, made an add-on to control all of the accounts in the game of Illyriad. HonoredMule made Harmless Butler 2 to rule and control all accounts in Illyriad.

We must stop this evil from ruling the lands of Illyriad. Day by day, HonoredMule and H? get a new account to control.

If any player uses HarmlessButler 2, HonoredMule and H? will get one step closer to ruling Illyriad.

Back to Top
Mr. Ubiquitous Feral View Drop Down
Forum Warrior
Forum Warrior
Avatar

Joined: 01 Jan 2011
Location: U.S.A.
Status: Offline
Points: 419
Direct Link To This Post Posted: 15 Mar 2011 at 01:41
But do they have superpowers?  I have invisible pants.
I am a Machine.
Back to Top
G0DsDestroyer View Drop Down
Postmaster
Postmaster
Avatar

Joined: 16 Sep 2010
Location: Ásgarð/Vanaheim
Status: Offline
Points: 975
Direct Link To This Post Posted: 15 Mar 2011 at 03:34
Originally posted by Lionz Heartz Lionz Heartz wrote:

BE WARNED!!!

HM wants you all to use Harmless Butler 2 because...

In Lord of the Rings, Dark Lord Sauron made several rings to all of the leaders of dwarves, elves, and humans. Little did they know that Dark Lord Sauron made one ring, TO RULE THEM ALL!!!

HonoredMule, just like Dark Lord Sauron, made an add-on to control all of the accounts in the game of Illyriad. HonoredMule made Harmless Butler 2 to rule and control all accounts in Illyriad.

We must stop this evil from ruling the lands of Illyriad. Day by day, HonoredMule and H? get a new account to control.

If any player uses HarmlessButler 2, HonoredMule and H? will get one step closer to ruling Illyriad.



You've found my father Sauron?????Wink
AWESOME!!!!!!!!!Big smileBig smile and i thought he was dead!!Shocked
Big smile
Back to Top
 Post Reply Post Reply Page  12>
  Share Topic   

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 12.03
Copyright ©2001-2019 Web Wiz Ltd.