Play Now Login Create Account
illyriad
  New Posts New Posts RSS Feed - Daily XML dumps down
  FAQ FAQ  Forum Search   Register Register  Login Login

Daily XML dumps down
 Post Reply Post Reply Page  12>
Author
  Topic Search Topic Search  Topic Options Topic Options
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650 		Post Options Post Options   Thanks (0) Thanks(0)   Quote HonoredMule Quote  Post ReplyReply Direct Link To This Post Topic: Daily XML dumps down
    Posted: 30 Jan 2014 at 04:51
My webserver alerted me that today's XML dumps cannot be reached - server returns 503.  Retried a few times over the next 20 minutes but no luck.  I'm hoping this can be fixed soon enough to avoid ending up with a one (or more) day gap in my archives.

Geek
"Apparently, quoting me is a 'thing' now."
- HonoredMule
Back to Top
ubluntu View Drop Down
Forum Warrior
Forum Warrior


Joined: 24 Aug 2013
Status: Offline
Points: 324 		Post Options Post Options   Thanks (0) Thanks(0)   Quote ubluntu Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 05:11
Im getting 503 for all datafiles as well.

$ curl -I http://elgea.illyriad.co.uk/data_downloads/datafile_{towns.xml,players.xml,alliances.xml,factions.xml,worldmap.txt,terrain.xml,terrain_combat.xml}
HTTP/1.1 503 Service Unavailable
Content-Length: 326
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Thu, 30 Jan 2014 05:08:28 GMT
Connection: close

HTTP/1.1 503 Service Unavailable
Content-Length: 326
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Thu, 30 Jan 2014 05:08:28 GMT
Connection: close

HTTP/1.1 503 Service Unavailable
Content-Length: 326
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Thu, 30 Jan 2014 05:08:28 GMT
Connection: close

HTTP/1.1 503 Service Unavailable
Content-Length: 326
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Thu, 30 Jan 2014 05:08:28 GMT
Connection: close

HTTP/1.1 503 Service Unavailable
Content-Length: 326
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Thu, 30 Jan 2014 05:08:28 GMT
Connection: close

HTTP/1.1 503 Service Unavailable
Content-Length: 326
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Thu, 30 Jan 2014 05:08:28 GMT
Connection: close

HTTP/1.1 503 Service Unavailable
Content-Length: 326
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Thu, 30 Jan 2014 05:08:28 GMT
Connection: close
Back to Top
Malek View Drop Down
Forum Warrior
Forum Warrior
Avatar

Joined: 09 Jul 2012
Location: Australia
Status: Offline
Points: 252 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Malek Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 09:21
Possibly a merging of the two maps, I know that SC hinted at that end of last year.  No clue if it would affect the xml's though. 
Back to Top
GM ThunderCat View Drop Down
Moderator Group
Moderator Group
Avatar
GM

Joined: 11 Dec 2009
Location: Everywhere
Status: Offline
Points: 2183 		Post Options Post Options   Thanks (0) Thanks(0)   Quote GM ThunderCat Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 13:31
We have moved the data to a different server to reduce the bandwidth requirements for this data on the game server; and improve the compression level for download, if you use http://data-root.illyriad.co.uk/ plus file name you should be able to access the data.

e.g. datafile_towns_new.xml was 45MB, now is served compressed at 3.9MB, though is the same data.


Edited by GM ThunderCat - 30 Jan 2014 at 13:52
Back to Top
GM ThunderCat View Drop Down
Moderator Group
Moderator Group
Avatar
GM

Joined: 11 Dec 2009
Location: Everywhere
Status: Offline
Points: 2183 		Post Options Post Options   Thanks (0) Thanks(0)   Quote GM ThunderCat Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 14:13
Originally posted by GM ThunderCat GM ThunderCat wrote:

We have moved the data to a different server to reduce the bandwidth requirements for this data on the game server; and improve the compression level for download, if you use http://data-root.illyriad.co.uk/ plus file name you should be able to access the data.

e.g. datafile_towns_new.xml was 45MB, now is served compressed at 3.9MB, though is the same data.
Back to Top
Albatross View Drop Down
Postmaster General
Postmaster General


Joined: 11 May 2011
Status: Offline
Points: 1118 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Albatross Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 14:23
Thanks TC.

It should be noted (in my setup at least) that the file is downloaded as the requested name, e.g. datafile_towns.xml, but it's actually a compressed file (rather than just being transported using protocol compression), so needs to be uncompressed prior to parsing, if your downloader or parser don't already do that.
Back to Top
GM ThunderCat View Drop Down
Moderator Group
Moderator Group
Avatar
GM

Joined: 11 Dec 2009
Location: Everywhere
Status: Offline
Points: 2183 		Post Options Post Options   Thanks (0) Thanks(0)   Quote GM ThunderCat Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 14:32
Originally posted by Albatross Albatross wrote:

Thanks TC.

It should be noted (in my setup at least) that the file is downloaded as the requested name, e.g. datafile_towns.xml, but it's actually a compressed file (rather than just being transported using protocol compression), so needs to be uncompressed prior to parsing, if your downloader or parser don't already do that.
It should have the gzip content-encoding http header on it; but if your downloader doesn't understand gzip compressed http streams, yes you will need to decompress it.

Add a .gz extension if this is the case.
Back to Top
Albatross View Drop Down
Postmaster General
Postmaster General


Joined: 11 May 2011
Status: Offline
Points: 1118 		Post Options Post Options   Thanks (0) Thanks(0)   Quote Albatross Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 14:43
Originally posted by GM ThunderCat GM ThunderCat wrote:

Add a .gz extension if this is the case.
*nods*
I'm using PowerShell's $webclient.DownloadFile() function, which seems to ignore that header info.
Yes, I told 7-zip it was a .gz file, and that worked.
Back to Top
GM ThunderCat View Drop Down
Moderator Group
Moderator Group
Avatar
GM

Joined: 11 Dec 2009
Location: Everywhere
Status: Offline
Points: 2183 		Post Options Post Options   Thanks (0) Thanks(0)   Quote GM ThunderCat Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 14:53
Originally posted by Albatross Albatross wrote:

Originally posted by GM ThunderCat GM ThunderCat wrote:

Add a .gz extension if this is the case.
*nods*
I'm using PowerShell's $webclient.DownloadFile() function, which seems to ignore that header info.
Yes, I told 7-zip it was a .gz file, and that worked.
My PowerShell is a bit rusty; but can you set?

$webclient.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
Back to Top
HonoredMule View Drop Down
Postmaster General
Postmaster General
Avatar

Joined: 05 Mar 2010
Location: Canada
Status: Offline
Points: 1650 		Post Options Post Options   Thanks (0) Thanks(0)   Quote HonoredMule Quote  Post ReplyReply Direct Link To This Post Posted: 30 Jan 2014 at 17:20
Thanks TC, all good now.
"Apparently, quoting me is a 'thing' now."
- HonoredMule
Back to Top
 Post Reply Post Reply Page  12>
  Share Topic   

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 12.03
Copyright ©2001-2019 Web Wiz Ltd.