Print Page | Close Window

05MAR19 - Overall Score Changes

Printed From: Illyriad
Category: News & Announcements
Forum Name: Announcements
Forum Description: Changes, patch release dates, server launch dates, downtime notifications etc.
URL: http://forum.illyriad.co.uk/forum_posts.asp?TID=10605
Printed Date: 19 Nov 2019 at 12:59
Software Version: Web Wiz Forums 11.10 - http://www.webwizforums.com


Topic: 05MAR19 - Overall Score Changes
Posted By: GM ThunderCat
Subject: 05MAR19 - Overall Score Changes
Date Posted: 05 Mar 2019 at 17:02

Overall Score Changes

We will shortly be applying a change to the way category contributions to overall score is calculated.

Overall score is a dynamic calculation across 8 categories; with the importance of value of a score in an category determined by the player community.

If competition in a category and scores are low, it doesn't take much to get a high contribution from that category to your score. However, if competition and scores are high it requires more effort to get the highest contribution from that category.

However, the formula we have been using to calculate the individual category contributions is very problematic.

We have been using roughly the following formula to convert a category score to the total score contribution:

Category Contribution = % of Max(Cat Score) * Total Score Available

Using the % of the Maximum score in that category has problems; which can be shown using the following examples:

Looking at one category in isolation; to simplify, given starting scores of Cat Score it would produce the overall scores Overall

PosCat ScoreOverall
11000013500
220002700
310001350
4500675

Using the old score calculation, if the 1st ranked player gains 500 in category score it produces the following changes to overall score:

PosCat ChangeCat ScoreOverall ScoreOverall Change
1+5001050014000+4000
220002667+667
310001333+333
4500667+167

If the 2nd ranked player gains 500 in category score it produces the following changes to overall score:

PosCat ChangeCat ScoreOverall ScoreOverall Change
11000014000+4000
2+50025003500+1500
310001400+400
4500700+200

And if the 4th ranked player gains 500 in category score it produces the following changes to overall score:

PosCat ChangeCat ScoreOverall ScoreOverall Change
11000014000+4000
220002800+800
= 310001400+400
= 3+50010001400+900

We consider it a surprising outcome that if you gain score in a category it can cause a player ranked higher than you to gain more score than you for doing nothing.

Category Contribution = % of Total(Cat Score) * Total Score Available

Using the new formula the overall score changes are more expected:

PosCat ChangeCat ScoreOverall ScoreOverall Change
1+5001050010500+500
220002000
310001000
4500500
PosCat ChangeCat ScoreOverall ScoreOverall Change
11000010000
2+50025002500+500
310001000
4500500
PosCat ChangeCat ScoreOverall ScoreOverall Change
11000010000
220002000
= 310001000
= 3+50010001000+500

Each category remains 1/8 of total score.



-------------



Replies:
Posted By: Lagavulin
Date Posted: 05 Mar 2019 at 17:15
Thank you for the clarity.  It looks like you have made a good change.


Posted By: Eagle
Date Posted: 05 Mar 2019 at 17:54
Thanks for looking into this issue and making the changes.  Much appreciated.


Posted By: Cesena
Date Posted: 05 Mar 2019 at 19:02
Your change in the old way for the first one has  a calculation fault.
It should be: 10500  14175 change is 675. etc for the following


Posted By: Cesena
Date Posted: 05 Mar 2019 at 19:44
Not a very good change I'm sorry to say.
Can't check how points are assigned. Some components are more difficult to develope/grow then others. Using prestige for growing towns is easier then growing without and sure easier then developing research. And even in components there are differences. Attacking one NPC (f.i dogs) is easier then attacking another NPC (f.i. Underworld troops) and way easier then attacking troops of another account.

And thereby I also can't check the sum of the totals, which aren't 1/8 of the points given.

In the old way we couldn't check the results also. So nothing changed there.
But a good change would have made it better and more thorough.


Posted By: Thirion
Date Posted: 05 Mar 2019 at 21:16
As mentioned above your values seem wrong, at least for the first player - he gets an increase of 500 points for each case instead of 4000 (as he has a 1 multiplier and the total score inceases by 500 points). I am quite sure that other calculations are wrong too, as a player that does not increase his own score should not be able to get an increase of more than 500 points (he would need a category contribution of > 1, or not?)

Originally posted by GM ThunderCat GM ThunderCat wrote:

We consider it a surprising outcome that if you gain score in a category it can cause a player ranked higher than you to gain more score than you for doing nothing.


I am 99% sure this did not happen in the last years. I got way more points while climbing to rank 1 in magic then i got when i was first and others were climbing.

I think your example is not comparable with the reality, as in reality the addition to the total score is negligible compared to the category contribution, thus making increasing your contribution way more important than growing the total score. One is an addition to an already huge number, the other is increasing the multiplier of an already huge number. Or in numbers: Lets start with 1000000 * 0.5. If you add 1 to 1000000 you get an increase of 0.5 points, if you add 0.001 to 0.5 you get an increase of 1000.

I do agree that the current calculation has some issues.

I do not think though that the new formula makes the score calculation better, as categories were players have a lot of points are going to contribute way less to overall score than categories were people have overall less points.


Posted By: Thirion
Date Posted: 05 Mar 2019 at 21:29
If the overall score calculation is correct then the example should look like this:

Originally posted by GM ThunderCat GM ThunderCat wrote:

PosCat ScoreOverall
11000013500
220002700
310001350
4500675

Using the old score calculation, if the 1st ranked player gains 500 in category score it produces the following changes to overall score:

PosCat ChangeCat ScoreOverall ScoreOverall Change
1+5001050014000+500
2
20002667-33
3
10001333-17
4
500667-8

If the 2nd ranked player gains 500 in category score it produces the following changes to overall score:

PosCat ChangeCat ScoreOverall ScoreOverall Change
1
1000014000+500
2+50025003500+800
3
10001400+50
4
500700+25

And if the 4th ranked player gains 500 in category score it produces the following changes to overall score:

PosCat ChangeCat ScoreOverall ScoreOverall Change
1
1000014000+500
2
20002800+100
= 3
10001400+50
= 3+50010001400+725



Without doing all the math (too late, sorry) this looks a lot better - as all players lose points if the first one increases his points. Not 100% sure if it is correct though.


Posted By: Thirion
Date Posted: 05 Mar 2019 at 22:02
Sorry for tripple post, can't edit the previous ones (anymore).

Seems like the new score system is implemented and in my opinion it (sorry) sucks. Magic, Trade, Diplo and Quests are way more impactful then defense, attack, build and research.

https://elgea.illyriad.co.uk/#/Player/Rankings/68643" rel="nofollow - https://elgea.illyriad.co.uk/#/Player/Rankings/68643 (dittobite, Rank 16)

https://elgea.illyriad.co.uk/#/Player/Rankings/334315" rel="nofollow - https://elgea.illyriad.co.uk/#/Player/Rankings/334315 (Shagoury, Rank 18)

Dittobite has Tech rank 1, Defense rank 2, Build rank 3 and everything else at least Top100. Meanwhile Shagoury has Magic rank 3 and everything else not even Top1000. They have almost similar overall ranks and overall score - how is that a good score system?

In addition most of the Top20 players now have a category (or a few two) that contribute 50-80% of their overall score - how is that good?


Posted By: BayNights
Date Posted: 06 Mar 2019 at 01:08
Not a fan of changing scoring systems after many years. The change dropped me significantly in the standings if it's calculating correctly. Weighting a category such as quests (no skill involved and hasn't been updated in years) the same as att/def values is not keeping in line with actual game play.


Posted By: Kimmyeo
Date Posted: 06 Mar 2019 at 01:09
Thank you!


Posted By: GM ThunderCat
Date Posted: 06 Mar 2019 at 01:36
@Thirion looking into it; as it shouldn't be working like that.

-------------


Posted By: GM ThunderCat
Date Posted: 06 Mar 2019 at 02:17
We've rolled this back while we give it more consideration.

-------------


Posted By: Kimmyeo
Date Posted: 06 Mar 2019 at 03:35
I like the new way, it makes more sense to me.


Posted By: Thirion
Date Posted: 06 Mar 2019 at 09:11
Originally posted by GM ThunderCat GM ThunderCat wrote:

@Thirion looking into it; as it shouldn't be working like that.


Thanks, that's what i though.

I think a scoring update is a good idea, it shouldn't have the obvious flaws i mentioned above though.

In my opinion Illyriad is a game with a lot of different ways to play it (PVP,Tournaments,Hunting, Trading, Building,...) thus each category should have the same or at least similar impact to the total score. And yes, some categories require more skill and others more endurance - but its your decisions what you want to do.


Posted By: Hucbold
Date Posted: 06 Mar 2019 at 17:31
Here's a suggestion: The overall score should be the average of the 8 rankings. As you climb the rankings in any category, your overall score goes down with 1 being the ultimate setting. Lose the pie chart that shows how much each contributes and give us a graph that shows movement up or down over time.

Example - current no 1 is Quentin the miffed has ranks 1,7,7,119,30,4,27 neatly giving an average of 12
Example 2 - I (current overall rank 52) have current rankiings of 29,244,410,200,11,360,294,41 giving an overall score of 198.63

If you don't like the idea of an overall score going down then take one over it and multiply by 10,000! 


Posted By: Eagle
Date Posted: 06 Mar 2019 at 17:55
I appreciate everyone's input and comments and formula's.  I'm afraid I'm not smart enough to figure this out.

All I know is that after maintenance or something early December 27th my ranking started dropping like crazy within 24 hours.  I dropped from around 51st to 71 then later down to 79.  Someone mentioned magic issues.  I have been using magic for example on my own cities and adding mana after exo's.  I was ranked 29th in magic on Dec. 26th and still ranked 26th in magic.

My diplo, research, trade, quest and even building scores have gone up a lot since Dec. 26th.  My attack, magic, and defense have stayed about the same.   However my over-all rankings have not.   I'm pretty high now in all 8 categories so something appears to be wrong with them.

What the issue is I can't say.  I hope someone smarter than me can figure this out.  I play a lot to boost my score for fun.   This is still a sand-box game and having a high score suits my competitive personality.  I know several others who enjoy the competition of trying to get to the top 10, top 20, or even top 50 as in my case.

I realize you rolled back the new fix from yesterday but hope you figure out what is up and can fix it to be fair and where I don't have to keep passing people I've already passed again.

Thanks for your help on this!  :)


Posted By: smilee
Date Posted: 06 Mar 2019 at 18:29
Suggestion: Composite z scores.

Per category: z_cat = (player_points - average_points) / variance_category

Then add up the eight z_cats per player and sort them from largest (#1) to smallest (#lowest).

If you want to weigh the categories, use factors in the sum. What those weights should be I don't know, but don't base them on the total sum of scores or the same problem might happen again. *Maybe* weighting by the variance within the categories can work because the variance is mean-insensitive, but I'm not betting on that.




Posted By: Thirion
Date Posted: 07 Mar 2019 at 00:40
Originally posted by Eagle Eagle wrote:

...

I'll try to explain how the ranking system currently works and what happened in december. 

How the score is calculated can be found at  http://forum.illyriad.co.uk/25aug11-overall-ranking-and-score_topic2314.html" rel="nofollow - http://forum.illyriad.co.uk/25aug11-overall-ranking-and-score_topic2314.html - i am assuming that these formulas are still in use.

The score is calculated by multiplying the overall score with the sum of all category contributions which are calculated by "your points" / "points of rank 1 player". Lets assume we only have 2 scoring categories, Magic and Building. Your total score would be calculated by overall score * ( "your magic points" / "Thirions magic points" + "your build points" / "Quentins build points"). This means that the Rank you have in a category doesn't matter and just the points you and the rank 1 player has matters. In addition if the rank 1 player grows faster than you then you might lose total score even though you are gaining category points.

What happened december. Before december snot was rank 1 in magic with around 500m magic points. Before december mana limited the amount of magic points you could get (because at some point you ran out). With the december update and crystals this changed.

1) Lesser Crystal give A LOT more points per mana compared to old methods (Lesser Mana Fusion gives 110 points per Mana, Adding mana to spells give 0.5 points/mana)
2) With buying Greater Mana Crystals and using them you never run out of mana - thus you can gain almost indefinite amount of magic points.

Using this method Shagoury gained 1250m magic points in one week. Later on i got to around 1900m magic points. What does this mean for your total score? Lets assume you had 200m magic score in december and now. Before december magic contributed 200/500 = 2/5 = 4/10 of total score. After Shagoury magic contributed 200/1250 = 1/6 of total score and now it is 200/1900 = 1/10 of total score. This means you lost probably 20% of your score in a week and another 10% later on.

The score system is usually ok because players do not gain points too fast, thus players do not lose (many) points. The problem was that the Magic points of Rank 1 doubled in a week, and quadrupelt in a month - thus all players lost a lot of points and the rankings got shuffled around.

And by the way, earning magic ranking points is possible but takes a lot of time/work and is annoying/boring (similar to quests). I think i spent 3 days just creating Crystals for 10h (while watching Netflix). It is kind of an infinite amount of quests.


Posted By: BayNights
Date Posted: 07 Mar 2019 at 07:35
Thank you for the explanation Thirion. I think this is a perfect case of the type of behaviors the devs should not reward/incentivize in Illy - it doesn't do much to enhance the community and rankings become meaningless.


Posted By: Thirion
Date Posted: 07 Mar 2019 at 11:20
Originally posted by BayNights BayNights wrote:

Thank you for the explanation Thirion. I think this is a perfect case of the type of behaviors the devs should not reward/incentivize in Illy - it doesn't do much to enhance the community and rankings become meaningless.

I disagree. What people did was getting points - and for some players thats their way of playing the game. If you put up more (or a lot more) work than others you should get more points than the others. Rankings are still an indicator of how much work you put up - that was always the same and never changed. Everybody can get an almost infinite amount of Magic points if you put up the time, same with all other categories (if you trade/hunt a lot you get a lot of Trade/Attack points, ...).

In my opinion the only problem was/is that the points you get for Lesser Mana Crystals are to high compared to other ways of getting Magic points. This increased the speed people got points by a lot - otherwise the Score increase would have been a lot slower (comparable to the other categories).

But well, the Devs implemented the points you get for Lesser Mana Crystals and i see no reason not to use it. If a game developer adds a powerful weapon to a game that everybody can use why shouldn't you use it?



Posted By: Cesena
Date Posted: 07 Mar 2019 at 12:33
Yes, there are different playing styles. But the amount of work to earn your points must be comparible to any other category contributions. Making cristals is very easy done. The only restriction is the time that limits the quantity.

And because this is a new phenomenon some are playing it to the extreme in a category that did not grow very fast, because it wasn't fully developed. So big scores will have big influences in the beginning. But I think the effect was multiplied by the poorly balanced allocation of points.


Posted By: Cesena
Date Posted: 07 Mar 2019 at 12:47
The idea of Hucbold isn't that bad, but you could use an overall percentage to get the overall standings. That way you can give every category the same weighting and you can calculate the corresponding points.


Posted By: Thirion
Date Posted: 07 Mar 2019 at 13:43
Originally posted by Cesena Cesena wrote:

Yes, there are different playing styles. But the amount of work to earn your points must be comparible to any other category contributions.

I completely agree.

Originally posted by Cesena Cesena wrote:

Making cristals is very easy done.

Yes, creating crystals is easy. Getting a lot of magic points with it is quite hard. Before crystals people could use spells with added mana to get Magic points which is A LOT easier as you can get millions of magic points with 4 clicks - with crystals you have to create thousands of crystals to get a similar amount.

Looking at other categories there are a lot of ways to get "easy" points. Trading and Hunting is way easier. Quests, Diplo and Building are kind of comparable to Crystals in my opinion. And well, tech / defense is limited by other factors (although you could probably get defense points quite easy too - need to do some tests some day).

With quests you have to spend a long time doing quests every day for a few minutes. With Crystals you (probably) have to spend a similar amount of time but you can do it in a few days if you want to.

My Point: Making points with crystals is ok and not easy or broken, the amount of points you get is currently too high (half that amound would probably be fine).


Posted By: GM Stormcrow
Date Posted: 07 Mar 2019 at 19:59
Hi everyone,

Please see https://forum.illyriad.co.uk/topic10607_post105533.html" rel="nofollow - this post  regarding Overall Score, and feel free to comment or make alternate proposals.

Regards,

SC


Posted By: Hucbold
Date Posted: 07 Mar 2019 at 20:37
Very good - that's exactly equivalent to my suggestion!



Print Page | Close Window

Forum Software by Web Wiz Forums® version 11.10 - http://www.webwizforums.com
Copyright ©2001-2017 Web Wiz Ltd. - https://www.webwiz.net