Discussion: World Rankings

The World Ranking Scheme currently doesn't provide a good reflection of the best elite women. Despite winning 2 individual gold medals at WOC last week, Simone Niggli is ranked only 8th. Neither her Sprint Distance gold or Long Distance gold performances are good enough to make her top 4 runs of the year.
http://iof.6prog.org/WR_Athlete.aspx?how=F&AID=SUI...
These races have been given an "importance factor" IP=1. Prior to 2012, World Championships finals were given an IP=1.05. This change to the system seems to be a backward step. Surely top performances at the highest level should be rewarded with appropriate World Ranking points?
Can anyone make a valid argument that Simon should not be the #1 ranked woman in our sport?

Simon is a boy.

Because it really should be someone else's turn? ;-)

Simone ist the leader in the World Cup.

As far as I know, the World Ranking (different to World Cup) calculation is rather complicated, reacts slower and reflects the performance in the past twelve months. Since Simone has missed races in 2011 it will take some further weeks (or even months) until she is leading the World Ranking again.

Currently it also doesn't reflect the performance in the past 12 months. It reflects performances in the major 2011 events.

They have not recalculated the old events to IP=1. All the girls above Simone have earned their points in 2011 WOC & WC events, so it's virtually impossible to overtake that.
Also fe Lundanes is underrated because of this.

You'll have to be patient until 14th October to have a valid ranking again (as far as the usual situation is good, but I'm not going to burn my fingers on that one).

It's comparable to Tennis rankings - if you're out injured or pregnant your ranking goes down and it takes time to go up once you come back. We all know she's definitely returned as strong as ever and should be No.1 by the end of the year.

Maybe she's not top ranked because the weighting factor applied to last years WOC is still skewing the rankings? Shows that this weighting is a nonsense and why it is good that it will be removed ;)

According to WOC results, she's ranked only fifth best at middle distance, and Thierry is only fourth, Lundanes 14th and Hubmann nowhere. Anyone agree with that ranking, or must we conclude that WOC currently doesn't provide a good reflection of the best elites :)

Serious point - rankings aren't perfect - its daft to criticise unless you have a better scheme.

Rankings aren't perfect - agreed.

Ranking are even less perfect if a massive discontinuity is introduced into the way the rankings are calculated pre and post a certain date - as in this case.

The fact that the rankings have barely changed as a result of the WOC results is a good example that they are not currently even close to achieving what they are intended to. They are still pretty much a ranking list for 2011 WOC/WC performances rather than having much relevance to 2012 performances (and hence to reality of current world rankings).

The only remaining World Cup races this year are the NORT races. World Ranking points will be used to decide the start order for the first NORT race. This was also true for World Cup 4, the PostFinance Middle Distance race. Not sure why the NORT start list should be largely based on athlete performances in 2011.

I take the point about criticism but the ranking system is not an end in itself - it needs to at least try to achieve a sensible outcome given that problems with it have a knock on impact on the set up of some of the major races of the year.

If bigger events are weighted heavier (or has this changed?), why not also weigh recent events heavier?

That would mean that your score at a particular race would continue to diminish as time went on-- but at more gradually, instead of being abruptly cut-off at 12 months. (Is that how it currently works?)

..I don't actually know how Ranking works, except off of what I've gleaned from this thread. So pardon me if my suggestion doesn't make sense. ;)

If someone wants to briefly summarize A) how they DO work and B) what the recent changes are and C) what the goal of those changes were.. That'd be great!

I would say the point of the cut-off is that your recent results would then replace the ones from a year ago. This would only work if races were held at exact one year intervals.

...although, for WOC specifically, it would be possible (at least in principle) to tweak the system so that the most recent WOC, and only that WOC, counted, regardless of when it was actually held. Probably too complicated to do that for other races, though.

In 2010 I did an analysis of the races from which ranking scores of over 1300 for men and 1250 for women came from - this was roughly what you need to average to be in the top 35 (in the context of a now-abandoned option to base WOC qualification on individual world rankings). From memory, something like 85% of such scores came from either WOC and World Cup, and almost all the others came either from elite races in Scandinavia or a couple of early-season races in Portugal and Spain that attracted large international fields. Apart from that, only 9 out of ~500 such scores were achieved elsewhere, 3 of them by Simone Niggli in Swiss domestic races, and another 2 by other Swiss women carried along on Simone's coat-tails.

The removal of IP points was intended to address this imbalance. I intend to repeat the analysis at the end of 2012 (by which time 2011 points will have fallen out of the system), which may indicate whether it's gone too far the other way.

The removal of IP points was intended to address this imbalance
It is great to have someone in the IOF to monitor WRE system year by year but after WOC we can already see the 2012 trend which is very similar to 2011 trend (Highest Scoring Runs 2012).

Formula now calculates points based on much higher 18 months average for WOC runners. Removal of IP factors should mean also recalculation of all events with IP factor in the past (not just for 1 year).

It would be nice from IOF to present more about future development of WRE system. It looks more like live Beta version now and this is not good.

We've had this beta version for 12 years now! (something like 15 if you include the n3sport alpha).

What I see as a failure is that after 15 years we still don't know what to do with this beta version.

In other threads (in the last 2 or 3 years) we have talked about how WRE lost importance, what are the most evident weaknesses, problems, inaccuracies and "out of rules" solutions but "WE" still have respect and confidence into WRE system that one day this Rankings will give us back what is written in foreword of WRE handbook.

Sometimes I wonder whether we really compare and test all possible solutions but unfortunately we just didn't find any better solution than what we have now. It is very sad&tragic for the WRE system that after 15 yrs O-Ringen and many, many other big multiday events with media and public interest are out of this system.

We probably don't have the power to convince IOF to make a working group and make WRE 1.0 but who does?

The World of O Ranking tries to get a good ranking on the top of the results list - including separate rankings for each discipline - by taking into account WOC, EOC, JWOC and WC races. Not perfect (of course), but gives a good picture of the top 10-15 runners. Worse when you go further down,

http://runners.worldofo.com/rankingm.html
http://runners.worldofo.com/rankingw.html

Nice, Sam and Ali have exactly the same points in the WorldOfO rankings!

Wild!

Weird how they picked the wrong face from Ali's photo... If you click on the face, you see Ali in the middle of the shot.

Even with the wrong face, it's still better than the one they have for Ida Marie Naess Bjorgul...

Hmm, Sandra Zurcher is ranked considerably higher than Sandra Lauenstein...

Bjorg Finlandia is well ahead of Swisse Ultivites. Incredible!

Graeme said: "its daft to criticise unless you have a better scheme"

Are you serious? Anyone can see that the current WR system is flawed. This wouldn't be such a problem if it wasn't for the fact the new WOC programme is very likely to use WR as the determinant for start order in the straight-to-final races.

I've seen you be critical of things that you couldn't do better. If enough people voice their concern then perhaps the people who run the system can find the people with appropriate skills to improve it.

NZL - Middle Chasing Start http://iof.6prog.org/WR_Event.aspx?EID=2284 counts for WR.

Is there any official explanation because the WRE rules 2012 are a bit unclear ("All World Cup individual A/B Finals count for ranking", "World Ranking Events Criteria -should not normally be mass start events nor should they have chasing starts") and in the past these type of events have remark "This non-standard format of race was not used for World Ranking."?

But it looks O.K. to me, content over form. That is a good sign. In general IOF would need only to present new rules first if they have decided to change the logic which events count for WR. WRE rules are always a few steps behind and I can't imagine a situation where athletes would have unclear rules for WC points in a same way as for WRE points.

If it is possible I would suggest also to exclude rule "Rule deviations" out of WRE Rules. Each rule deviation - during the season - causes lack of credibility into the WRE system.

So after the change I highlighted as good, Simone is back on top.
I'm quite lost by Nixon's comment that "anyone can see the system is flawed". I'm looking at the list and I don't see any major oddities, unless you think people who don't race should be highly rated. What is it in the list that anyone (else) can see?

(1687th equal, since you ask)

@graeme
Last year IOF made new wording of WRE Rules http://iof.6prog.org/IOF_Documents/FootO/wrscheme.... and in this way covers the most evident holes in the system which causes incorrect WRE points allocation at some events.

If anyone who was interested to gain WRE points saw that IOF award WRE points also to some events which do not qualify than the "system is flawed".

Now the Points Formula and specifications are corrected but I still think IOF should do more if we all want to give credit and importance to WR. Technical aspects of the system are not flawed but the importance for athletes, federations, media,....is still flawed.

There's a review of the World Rankings getting under way within IOF (with yours truly as one of the members of the review team).

The rankings have to count for something. That'd be a start.

One another micro change which might be important to review.

1) To clarify definition of 18 months avg. score.
To give all athletes same basis and chances to score high WRE scores. Today's definition still gives advantage to WOC/WC runners because they have gained high 18 months avg. score based on previous IOF events or can gain it because are part of the closed events. It is less evident than IP >1 but technicaly all elite runners don't have same starting position.

It would be good to consider how to calculate this AVG. to get better and more solid system also for athletes who can't afford/allowed to run WOC/WC races. One possible alternative option could be to calculate 18 months avg. based only on WRE races which don't count for WOC/WC/WG or are not among high profile IOF competitions.

In this way IOF could give credit to all athletes and to promote Ranking Scheme as it is written in the WRE Rules.
a) "as motivation for runners, including those outside national squads"
b) "should be events which competitors are known to be taking seriously, i.e. not just for training purposes"

Explanation
Elite runners take very seriously WOC/WC events but normal WRE events most of them consider only as a good training and they don't have intention to run them so often. In case IOF make changes into definition of 18 months avg. score and they would still consider normal WRE events as a training "only" they would also have lower 18 months avg. score than today and consequently also lower WRE points at WOC. With this change we could raise motivation and importance of WR at all WRE events. Also small federations and organizers can benefit on the longrun because also athletes who will take part only at these small WRE events can get better place (with good performance) on WR than today.

You have to be careful of Goodhart's Law - if something you measure drives policy, the measure itself becomes distorted.
At the moment, elites dont take the WR so seriously, but if it affects something they care about (like selection for WOC) they'll take seriously getting a good ranking. One way to avoid low scores is to avoid racing in unfamiliar terrains... OK, so at the moment, the worst scores are discarded, but another would be for a national team to distort the race points by having some of their top runners jog round in smaller WRE races.

Yes, "jog round in smaller WRE races" is not what we want to see at any WRE races and not just at small WRE races. At the moment Elite runners can afford to do this because they know that they will gain significant better WRE points at WOC races because "Mean Points of Ranked Athlete" is higher. It is not just a result of a good performance within a stronger field but also because of a definition of 18 months avg. score.

It is not a problem if someone don't want to compete on "unfamiliar terrains" only because he wants to hold high 18 months avg. score. If all WOC runners decide to skip the small WRE races their avg. 18 months score would be = 0. Fair enough. In reality this change would cause additional reduce in the difference in "Mean Points of Ranked Athlete" at WOC and at small WRE races. This could be only good for the WR. The best WR athletes should be able to perform good at any kind of terrain.

If we speak about which 4 races should count for WR overall score than maybe we should decide that only 3 races from one country can be valid for overall score. If you are not racing abroad you can't be considered as important "elite athlete" and be placed high on a WR just because you master home terrains.

@graeme
When you wrote this
"if something you measure drives policy, the measure itself becomes distorted"
my first thought was; Hey, we have got today's rule of 18 months avg. score based exactly on this kind of approach. What is wrong with this? If we would knew a universal solution IOF would probably already used it. I see this approach as development approach based on previous experiences.

If you recall: Before last year review 18 months avg. was calculated based on all previous events, including performances with 0 points. Because this cause problem for some runners to become R runners and consequently some small WRE races have problem to have enough R runners to allocate points to runners FOC has made a decision to exclude these very bad performances out of runner's 18 months avg. score. In my opinion this decision was not fair(-play) because the old rules were still in force.

In same time they knew that this solution can be good decision for development of WR so they make it (and brake the rules). Today we have a situation where runners can do DNF or jog through the course (when they make a lot of mistakes) without to compromise their 18 months avg. score. This is not good, because at next event "Mean Points of Ranked Athlete" is higher than it should be.

Possible upgrade:
1) To decide max. number of events which count for 18 months AVG. Let's say max. 6-12 races. We have to equalize number of races which count on the long run because today R runners have from 1 to +20 races included into their AVG. R runner with only 1 race is less representative than runner with 12 races.

2) We should somehow penalize runner's avg. for their 0 points performances. We should also penalize runner with DNF status in case runner is not injured. Medical staff at the event should give report to race director for all DNF runners.

3) We could calculate runner's AVG as
a) Based on max. 12 races: 6x (3 best, 3 worst) races from IOF membership fees groups (1-4) and max. 6x races from IOF membership fees groups (5-8) or any kind of similar variation
b) ?

4) Mean Points of Ranked Athlete
Weighted overall mean Points (70% from R runners with more than 6 valid races and with at least one race from both groups)

Today's definition of 18 months avg. should be upgraded into better avg. measure for the purpose of WR development.

I fail to understand where this is going.

The World Rankings seem to work fine for the top 50 or so (European) runners, and besides that, a WRE score often but not always gives a reasonable indication of how well one performed in a race.

They are adequate for seeding athletes into start groups, for the most part.

They are inadequate for team selection, country-woc-start-slot selection and so on, and always will be.

It is impossible to get a standardised WR system (where 'equivalent', whatever that means, performances give similar scores) across the globe unless you start paying huge numbers of people to travel a lot [Graeme, can we prove this?]. And cutting WOC results from the rankings would only exacerbate the problem, and would be a terrible idea regardless.

@#487 There are certainly things they could do better to give the impression they understand stats e.g.
R runner with only 1 race is less representative than runner with 12 races. and cutting any data out of the system is counterproductive, but I can't imagine it making a lot of difference. you could probably do something more sensible with outliers too.

What you can do is quantify how accurate the average scores are, from the spread of people's scores, it's just standard deviation divided by square root of number of scores-1: so e.g Kyburz' mean score is 1318 +-10. With fewer races you're 840+- 25
The Ranking Score 5633 is based on outliers, so is much more uncertain.

Given that it's close to the best that can be done, the only reason not to use it for country-woc-start-slot selection would be if you want to favour slower athletes from small O-nations over faster ones from larger nations. That's a respectable viewpoint, but hardly a fair criticism of the ranking list.

I ask myself. What added value can small WREs bring to the WR?

It should not be just a pure statistics and how WR works only for top runners. We are too much concentrated only on this part of WR scheme. WR should not be just some kind of tail of WC/WOC races because we have in the system more than 2.000 runners, trainers and many organizers and advisors who are willing to promote elite orienteering. If we want to give them credit to continue work in the future we should think how their events should have bigger meaning for WR and what are other/best option to calculate statistical data and what other rules should be important to stimulate racing.

With suggestion 4 I tried to give credit to more representative R runners which is the basis of every statistics. It is not just about +-10 points and why we should bother with small differences and changing the formula; it is about how we perceive athletes and their performances in a pool of runners. I tried to see the links between theory-technical aspects and real situation and importance for the system as such.

This was the basis for my suggestions and not criticism of WR. It was a constructive criticism.

Given that it[']s close to the best that can be done, the only reason not to use it
for country-woc-start-slot selection would be if you want to favour slower athletes from small O-nations over faster ones from larger nations. That's a respectable viewpoint, but hardly a fair criticism of the ranking list.

Sure. I wasn't criticising the list. I would happily criticise the use some wish to make of it.

And some of the faster ones from larger O nations (eg Russia) would also have problems with its use for woc slots.

The ranking formula calculates average scores and times of all ranked runners, so is at it's best for calculating score for those average runners. And bad for giving score for those furthest away from the average, both the best and the worst. That's where and why it fails.

Score should be based on two factors: strength of the field and how well one performs compared to those who got to the podium.

So, what is the best way to determine how strong field is? To me it's something like average score of about 8 best ranked athletes who started. 20th best ranked athelete shouldn't have no effect at all. Why should it?

The other factor. To me runners time compared to the average time of 5 fastest times gives pretty good idea how well one performed. Average time of all ranked runners, meh, why should we care how badly some lesser runners got lost when calculating score for those who did well?

Without standard deviation fast and easy races would be good for some and difficult races for some atheltes. Very top one would get best scores from tough/difficult races with big differences. And lesser runners (4+ placed) would get best points from easy and fast races with smal time differences. Nothing wrong with that if you ask me.

Some tweakings would be needed of course, like limiting (roof, like 115% of field average) how much bigger score one can get compared to the field - one should get the very best points only if the best runners are there, not by winning a lesser race with huge margin. And doing something (ignoring?) with chase/mass start races, qual races where top ahletes goal is cruising and saving energy for the final. And factor for winning time would be nice (sprint vs long, % time diferences are usually smaller in sprints because athletes doesn't get tired like they do in long races) .

Additionally, athele's ranking shoud not be penalized too much for having just one race (travel costs) and not at all for having some bad races. One would get low scores just by running a race with lesser field (roof factor). Having a low score races listed wouldn't necessaruly indicate you are crappy, it would tell the race wasn't a big one.

Something like this would make more sense to me. It wouldn't be good enough for WOC selections or anything but points would start making some sense, and that would be a good start. To get good points one would need to get to the right races, big ones with top runners on start list and do there well. And event organizers could do their best to make their annual race a big one, invite top ahletes to attract lesser atheltes. To get to the start line would need to have good enough ranking, so points would actually matter. WOC with ~1.08 ranking factor and not penalizing too much for having just one good race and lots low score races would get someone from let' say Barbados to this WRE game just by running once well at WOC. Scandi runners outside their national team would need to run first lesser (weaker start field) WREs to get better and better points to finally get to the big races (plenty of those atheletes, most of them would be happy to have a good excuse to travel). Compared to those atheltes these national team atheltes from smaller O countries would have the advantage of being able to run WOC with 1.08 factor. It should bring balance to the force.

To me it's something like average score of about 8 best ranked athletes who started.
I like it. And correction might be; 20% of the best R runners or minimum 10 best ranked "representative" athletes. It is needed at least 10 R runners to use SP and ST today and with this approach it is no need to penalize bad performances or DNF. Yes, it was a bad idea but only reasonable one when you need to calculate points based on all R runners like today.

To me runners time compared to the average time of 5 fastest times gives pretty good idea how well one performed.
Maybe it is good to use same approach also for ST. I was a little bit more polite and try to operate with all R runners but in reality your suggestion is more accurate about what WR should measure and give more value to rankings.

And in regard to your observation I think that deleting the standard deviation out of formula would be a step back. We can implement Tennis Rankings any time with WCup points scheme. Easy to implement. You just need to rank races. A little bit hard if you don't have Prize money. Maybe one day with TV on board.

There is really no need for R runners and non R . All runner could be "ranked", if one doesn't have previous races ahtletes points are lowest possible "base" points. Like 30. Would not matter much because only best 7 would count for the average.

No, get rif of the standard deviation. Using averade deviaton of just handfull of runners doesn't make any sense. We already know this is endurance sport and differences in times and points should go hand in hand. If someone is 10% slower he deserves 10% less points, no matter what deviation we happen to get by accident and luck. Standard deviation would make (some) sense with thousands of runners in same race, but not here. We already know better. O races are standardized races, these aren't any treasure hunts.

But this is still the best way to compare two events where you have two different pool of runners. IOF have bad experience with prioritizing some races (IP factor).

Your approach is good for more competitive sports (athletics). Clear hierarchy of events,....and if you can climb through the system you are rewarded with Prize money, honor and glory.

Using averade deviaton of just handfull of runners doesn't make any sense
Like you said, all what is important is how good are top runners at the event. So it make sense. We have now many races with less or a few more than 10 R runners. Nothing dramatically would change except that the criteria who are outliers or representative runners would be a bit sharper and defined.

And I don't have a problem if one athlete make extremely good or normal performance while all others make extremely bad performances at small technically WRE race so the winner grab a lot of points, more than usually. This kind of situation happens very rarely.
http://iof.6prog.org/WR_Event.aspx?FED=SLO&YYMM=13... (Katarína Labašová)

If someone is 10% slower he deserves 10% less points, no matter what

In the same way as Oystein Kristensen's bronze in WOC 2003 was no better than Alexey Bortnik's 33rd place in WOC2010 (both being 10% slower than the winner).

Yes, Oystein Kristensen bronze compared to the winner is as good as Alexey Bortnik's 33rd place compared_to_the_winner. But the winner just happens to be far better in Oystein Kristensen's race. Thats why my formula (uses average of top 5 times) would give Oystein about 7% more points than to Bortnik (without any duration factors I wrote about). And middle winner Thierry 7% more points than sprint winner Matthias Müller, if field strength points are about the same. What is wrong with that? If winner is only 0.4 % faster than 6th, maybe winner doesn't deserve as much points as a winner who wins with clear margin.

@kofols. If Katarína Labašová had been three minutes faster, she'd have got no ranking points at all!

That would be something for IOF's bronze pin :)

Yes, it is true if you study rules by the book, genau 2:07 better time. But in each case she'd have got ranking points because when you go over the edge then you are in the twilight zone and you need to know what are THE "special rules" of WR.
http://iof.6prog.org/WR_Event.aspx?EID=2099&det=2
http://iof.6prog.org/WR_Event.aspx?EID=1976&det=2
http://iof.6prog.org/WR_Event.aspx?EID=1998&det=2
http://iof.6prog.org/WR_Event.aspx?EID=2020&det=2
http://iof.6prog.org/WR_Event.aspx?EID=2098&det=2

Blair,
Are you allowed to comment these special rules. Is this really the only possible way to help some countries to get more R runners and what is the point at the end if people don't believe in WR as it is?

Can you share WR vision or what can be done by your knowledge because it is evident that federations, trainers and national team members don't care much about WRE rules, points, position on WR and if calculation for each event is really by the book and correct. How much time FOC will needed to find a way out of this mess or to say stop. We have got mess a few times each year and then we use this mess at all other events. And we haven't seen even once any official complaint about points calculation in 15 years. So sad for the competitive sport.

Antalya O-days 2012
I have seen similar examples in the past but this one is really something special.

1) Insufficient Ranked Athletes to proceed
2) Used finisher # 2 to find enough ranked athletes within 150% (Havva Kocabaş).
3) Havva Kocabaş (she was granted with R status with manual elimination of one very bad previous performances).
4) but why they didn't use Tuğba Cın (Gata Doga) as she was in the same situation as Havva but she was better placed? Maybe they forgot to use her but with her result she would probably not get enough points to become R runner. By the rules Tuğba Cın would be without R status at Cappadocia Cup 2012 and her brilliant performance would be rewarded with 1067 points instead of 978.
5) and side effects: Ingjerd Myhre NOR was not granted R status based on Antalya race at Danish spring. OK. But at next race World Cup Event 10 her 18 months avg. score was calculated based on Danish spring + Antalya. huh..I will not dig anymore.
6) What a mess to allocate points at Antalya and to get 1 more R runner.

We're going into this with a reasonably open mind - and I'm certainly interested to see more examples of anomalies under the present system. Would also appreciate links to details of any systems that are in use at national level (preferably in a form which can be fed into Google Translate if necessary).

Average of inverse times! properly scaled.

Technically saying these examples are not anomalies because they were caused by FOC decisions and not by the Rules or the system itself.

The anomaly of the system is this example
http://iof.6prog.org/WR_Event.aspx?EID=1963&det=2

From this example we can see that update of the WR database is not working properly by the rules. It shows that Angelica Riley was a "R" runner but based on WRE rule 2.3 "...who has scored World Ranking points in the 18 months before the event..." she shouldn't be among R runners. She got R status because her 18 months avg. score was calculated from her last event (22.10.2011) and not from the day of the event (19.11.2011). Because of this her 18 months avg. score included also 2 events (8 and 9.5.2010) which is not correct by the rule 2.3. but was enough for her to become R runner at Sprint the Golden Gate event.

Well yes, people in North America need to show up in order for the system to work. When people don't show up, we ask IOF to allow these tricks. Unfortunately as you see even the tricks don't help, so there's not much (any) hope for the WRE system in North America—there is just isn't enough incentive for the runners to participate. If the points counted for something, there would most definitely be an incentive.

True. It is almost the same everywhere. People run Elite class because they want to challenge themselves and not because to get WRE points. What kind of incentives would you like to see from IOF?

One problem is that federations were and are too passive about WRE. FOC did something but not enough so people get use to it. It is good to hear that they will try to address the main problems.

I am aware that big changes are almost impossible. But small changes can be done to polish the system. Points formula is important but also other things like: Name of the event director, course setter and map attached to the results; rule about official complaint to allocated WRE points, registered runners database with more info about each runner, maybe also a rule who are "Representative athletes" in each country, club rankings).

Take a look a Brazilian situation. They have organized 9 WRE events in 6 years but W21E was without WRE points at all events. What a failure of the system.

The goal should be:
1) Formula and rules to allocate appropriate points at each WRE event in M21E and W21E.
2) Each runner should be registered by official club and name of the club should be part of the official information in results and Rankings.

We need to promote competitive sport and the competitive environment and not just a runner as a person. In Slovenia you could run Elite class as an individual. Same goes for WRE. Nobody check this, no need for a rule and nobody try to hold a line on this issue. I don't know how is elsewhere but every organizer is just seeking more competitors. This is a recreational way of thinking but in same time we want to be seen as a seriously sport. You can't run among best in any seriously sport on a national level if you are not registered by a club but... you can register at international event such as WRE as an individual!?!
7. Entries: Organisers should allow easy entry by individuals as well as by groups as it is recognised that entries are usually made by clubs/groups in some countries and by individuals in others.

If the country is not ready for an official IOF international event (WRE) then make a different type of events for them (IOF challenge or similar) as an IOF promotional event. WRE should be more than that. It gives recreational thoughts to many sport leaders, officials who run sport in each country. "Why you ask for a help and seek support for professional sport....you are family, recreational sport." I think we just can't sell both things in one package.

I see any other support such as IOF official awards (event, annual), media support, possible qualification rule for WOC,... attractive only then and when the basic things get in order.

What kind of incentives would you like to see from IOF?

The rankings used for WOC qualification. It seems it's an if-and-only-if.

It seems a realistic and inevitable goal but base on today's system it would be unfair. It will probably need to start at roots to find a way how to pave the road towards it.

I've mentioned athlete and club registration. We have more than 3200 registered elite runners in WRE database. Based on today's WRE rules (at least 3 WREs per country) and IOF members (74) we could have around 215 WREs. Annual average is around 80-100 if we exclude major events and IOF collects around 20-25K EUR each year from WREs. IOF position when it comes to budget is:
A considerable income reduction on sanction fees would require increasing other fees and Council does not see that as a realistic alternative. Such a model could be considered only when external funding is available.

With registration rule ~8 EUR per athlete/season we would be able to change one source for another, make it free for organizers and prepare a room for more interest (increase to 150 WREs per year which is probably a realistic goal) and hope for better promotion within a new system. We have got enough experiences that today's system can't do miracles and I see this as a necessary small step towards this goal.

If you are really going back to basics there are a couple of things you might look at to help events with small entries...

1/ Using time spreads from the mens course to determine how gnarly the forest is when there isn't enough data in the women. (Poor data gives better average times than spreads)
2/ Lowering the 600 point threshold when there aren't enough runners. Athletes with a lot of consistent scores around 500 should be good for inclusion, better in fact than someone with a couple of scores, one of which is above 600. If you want to include countries where most "elites" are slow, you need to look at that range of people.
3/ Most radically, when entries are so low that stats are meaningless, you could give points based on who you beat rather than on stats, a bit like the chess rankings.
Some of these weird looking things would only be temporary: once there's a sufficient cohort of ranked runners in a country, the existing stats system will work out fine.

It would be nice to think that the IOF review would actually look at their huge database of past results to optimise a system that works best, rather than introducing random number cutoffs (like "over 600") or genuinely nonsensical concepts like the IP multiplier.

Lowering the 600 point threshold when there aren't enough runners
It could be interesting to test more options, especially if we could reduce number of valid formulas from 3 to 2.

I also like your national (BOF) approach...Do you cut off outliers only on one side?
For these four calculations, the final 10% (rounded up) are ignored e.g. if there are 38 ranked runners on a course, only the first 34 are used for the calculation.

I suppose that in case of lowering the 600 point threshold all entered runners above would be outliers and the threshold could go down so long until you get 10 best avg. runners. As Jagge suggest how to use the same approach when you have a lot of R runners. Than you cut off at the bottom. Maybe it would be good to test also an option to cut off 10% at both sides and then find the representative sample to calculate all the variables.

Do you cut off outliers only on one side?
Yes, only long times are cut off. This causes a systematic downward drift of the points through the year, so everything is rescaled.

600 point threshold ... would be outliers
Some will, but a better test for outliers is to ask how far from their personal average score they are.

Whole WR Points Calculation is now around the average, and if you can beat that average. I think that competitive sport and Rankings should not be only about average and pure statistics.

Beat the average
http://iof.6prog.org/WR_Event.aspx?EID=2160&det=2
For me it is strange that in competitive sport final points are based only on this premise. Adamski won the race but his final score is below his 18. months average score. Should we find a way to reward his victory better than today? What this tells us. Nice win, but he should won with larger margin to gain more points or maybe other runners did really good race and beat their average so his race was nothing special compare to them. My position here is that Points calculation is not focusing on what is important in competitive sport and that is winning. We need to find a solution how to award rank better than today.

1) Strength of the field
We have two samples: R runners >600 before the race and R runners within + 50% at the end of the race. Is any deeper reason why we need to exclude R runners who make DNF or +50% to calculate Mean Points? In this case Lars Hjerrild DEN and Andreu Blanes Reig ESP should be included in MP as in this way we would really have the correct MP points based on true strength of the field.

The strength of the field is important and this variable should be fixed based on start list and not on final results. Adamski won and beat all of them no matter if someone made a huge mistake or couldn't finish the course because of injury or it was to technically to find a control. In Skiing athlete get 100 Points even if Lindsey Vonn, Tina Maze and Maria Höfl-Riesch make a mistake. This is elite, competitive sport and rank is important no matter who is able to finish the course.

2) Rank points
We had IP factor but this variable was used for everyone to get additional points. To award Rank is important and we could use additional rank points for let say first 15. These points would not influence on 18 months avg. score but can serve as a important indicator who beats who. Also Small WREs should be rewarded, maybe with different Rank points based on how strong is the field = MP.

Maybe these solutions could led to other side effects but I see that at least calculation of MP should be corrected. From perception how competitive sport should be presented I think it would be justified.

It's a time trial, not a mass start. Why would running two seconds faster than someone else give a big difference to one's points?

I think 1 place is worth something and also others. It gives respect and clear information that winning is important. Can we communicate in this way or not. Also 3rd place is a much more valuable than 4th place even if you beat someone only by 2 seconds. It is important how good you can be when it matters the most. Can you beat your competitors on a same race or you can't? All professional, competitive sports work on this premise and not only how good is your run compare to your competitors.

And say, WOC races would be more important than other races? Maybe worth some IP factor?

You could give 60pts bonus to the winner, 25 to second, 10 to 3rd, but that would skew the already dubious predictive capability of the rankings. It depends what your goal is.

This is not my goal nor it was my intention. I mentioned "These points would not influence on 18 months avg. score". It should be used as final race points and used only for overall WR (for 4 best scores) and maybe for club rankings and Federation League table.

World champs middle:
http://iof.6prog.org/WR_Event.aspx?EID=2046&det=2

Average points of 7 best starters was about 1318. Bertuks won, he was 1.3 % faster than the average time of the top 7. He got only 1318 points. So, by winning with clear margin he got just average of the seven best starters!

Then for comparison, PostFinance middle:
http://iof.6prog.org/WR_Event.aspx?EID=2055&det=2

Average points of 7 best starters is about the same, 1318. Kyburz won, he was 2.3% faster than the average time of the top 7. He got 1357, that is 3% more than the average 1318.

Explain me why Kyburz's 2.3% win over the best rivals was worth 3% but Bertuks 1.3% was worth nothing? Similar races, both middles, same country, lots of same runners, best in the world.

As I see it, from elite runner's perspective the current formula is more or less a random number generator. And that's why averages of all and standard deviations should not be used at all.

Because you arbitrarily chose 7?

Kyburz is 6 mins ahead of the average time, Bertuks only 4. 50% better! but he only got a poxy 3% more points.

Or, since you don't like average runners, Kyburz is 2 mins ahead of the highest rated runner in his race, Bertuks is only 30 sec. ahead That's FOUR times better, but only 3% more points!

If you mess around with small number statistics in a single race, you can come to pretty much any conclusion you like. That's precisely why the ranking list takes averages.

Average points of 7 best starters was about 1318
Jagge I checked first example and I can't understand how you get the number and why final points should be an example of anomaly. It is just how the system works today. The only way to see differences is to apply new models to same races and see differences in final points.

..the current formula is more or less a random number generator.
I agree but I still don't get it how the system could possible work better without to use stats. Without stats you loose comparison which is the most important goal of WR. According to my knowledge we must first try to optimize the general formula and to test different models for races 10~20 and +20 R runners to find out if we really need two different formulas and if final results are any better than old ones. I think we could get more representative points which is the whole point. Then we could look also what additional correction might be needed for smaller races. For small races I also like what graeme said:
Most radically, when entries are so low that stats are meaningless, you could give points based on who you beat rather than on stats

Thierry Gueorgiou 1378,5
Olav Lundanes 1328,46
Peter Öberg 1324,22
Valentin Novikov 1300,27
Pasi Ikonen 1298,79
Fabian Hertner 1297
Marc Lauenstein 1296,23

average = 1317,6385714286

For average time of 7 best finisher, use these:
1 Edgars Bertuks
2 Valentin Novikov
3 Fabian Hertner
4 Thierry Gueorgiou
5 Francois Gonon
6 Peter Öberg
7 Magne Daehli

... and you should ge my numbers.

Seven is just as arbitrary as your N (all). Seven just gives much better interpretation of both strength of the field and how fast the best with good run can run a course. "All" gives, well, something else. How well average runners with mediocre run can run a course.

Why average? It brings same starting position for all races and if these 7 runners cheat or just run the race as a training it is easier for any other runner to score a lot of points and vice versa at small races. Almost all things in life have normal distribution, so with average it is easier to find out who are "the special one" on the right side.

And ...how fast the best with good run can run a course...
is not what I am looking for. Orienteering is not athletics because there is no standardized courses. I am looking only how to award (good, average or bad) runs compare to strength of the field and rank. You never now if good run is really a perfect run. It could end up with many (all) bad runs but still one person would win.

Poor data gives better average times than spreads

We have only tens of runners per race at max, not thousands or millions. That is not much for a statistical analysis. That is why it is far better to not use spread, only averages. And averages are better(more reliable, if the important figures (=scores and times of the winner and the very top) are not far from the average we use. That's why average of the best (~10% aka about seven) gives far more reliable scores for the best than the scores we get by using averages of all competitors.

@kofols, note, the best starters as base score and best finishers are not the same athletes. If they take it as a training or someting, they will not end up top 7 and their time will not count for time averages making it less of a problem. Is't that "only training" a problem (and bigger?) also today with average of all?

Almost all things in life have normal distribution

Ask Poincaré about baguettes. I certainly wouldn't expect a WOC final to have a normal distribution.

example
You have two races, one with very strong field (mean points =1.100 points) and one with very weak field (MP = 700 points). When you use only 7 best runners their times would give you a very low standard deviation of times in both cases so the overall points would be almost depending on strength of the field. Mean points are more important (because you use only R runners >600 and not all runners) than Mean time anyway. Mean time and standard deviation of times give you a chance only when you have a normal distribution of times on whole interval winner+50%. But in most cases you have more R runners on the right side of the average on top races and more chances to have more R runners at left side of the average at small WREs where you could have one visitor (TG) and he destroys the normal distribution.

With using all R runners before the race to calculate Mean points you would normally (at small WREs) lowering the Mean points because R runners around 600 points avg. score has more chances to make a bad result (winner+40-50% and over 50% so their avg. points are lost = that is today's system and Mean points is higher than it should be) which in same time give you a chance to have a larger standard deviation of times. This is a balance - Low points based on Mean points but more points on time spread. You could earn a lot of points at small WREs only when you have totally left sided distribution. It works fine in most races. At top races good runners normally do DNF instead to finish with bad result so the Mean points is lower than it should be. That is way all R runners should be included.

Neil, WOC is a special closed race where you have a race among best. This is not a normal sample and it is normal that at WOC distribution of times is not normal. It is a right sided. But most of WREs tend to have a normal distribution. (I might lie, because I use only common sense and didn't check it, maybe you could or IOF would do).

You do lie. See how many you can find which have more people two sd fast than there are two sd slow. I'm not sure what Neil expects, but even WOC finals usually have more "bad" outliers (like LTCM, whereas Poincare had more good outliers...).

Who cares, we just need to put out two-three-four different models and ask IOF if they could check it and report which lie is more acceptable.

Jagge, I'm interested in the comments you are making, particularly about the current scheme being a poor ranking for the top competitors. It seems sensible that a scheme based on mean/standard deviation would do well for the "average" competitor, but would do a poor job of ranking the outliers (i.e., the best and the worst) if the standard deviation is poorly estimated or if the distribution isn't Normal. Is there a way of quantifying this effect based on existing ranking lists, i.e., to show that the top competitors are systematically getting less accurate ranking scores?

I don't agree with your comments above that scores should simply be based on % behind the top competitors. Terrain matters - tough terrain makes it harder to be close behind the top competitors. We don't want a system that rewards the "average" competitor who seeks out easy terrain in an effort to get good points (as compared to the "average" competitor who runs well in tougher terrain).

Given that it is far more important to get the top half of the ranking list accurate (rather than the bottom IMO), I wonder about using a one-sided standard deviation calculation (i.e., only use those athletes above the "average"). That might give a way of getting both the average and top athletes accurately placed. Or, could the scores of the top competitors be locked to their pre-race average, then some measure of spread be used to give appropriate rankings on down the results list?

True, tough terrain makes it harder to be close behind the top competitors, so it also makes it easier to make gap and make good points for top 3. Also "average" competitor who runs well in tougher terrain may get to top 3 if field is not the strongest, and get reward there. In practice this hasn't been as big problem as one would think, but sure this effect exists. For exampel in sprints athletes don't get tired the same way, so the easy terrain effect is significant. But bigger problem has been chase starts and mass starts (no wonder).

I'd try something like average score of 7 best ranked and average time of top 5 finishers (kind of expect all of them will not run well). And also average for all, scores and times, about as of today. For athletes faster than top 5 time use simple % ahead of the top 5. For last half just the current formula. Between top 5 time and average time I'd use weighted average of what top 7 calculation and "current formula" gives. Kind of calibrate the better half by using those top 5/7 averages.

wrt eliminating dud results from the calculations, I tried the BOF scheme (throw out bottom 10%) and it still have the occasional wacko result when applied across nearly all events in the Australian results database.
After talking to one of our club members who has to deal with some ugly datasets, I changed the throw- out bit so that any runner whose performance was n deviations away from their norm wasn't included in determining score for the race calculation: after throwing out, recalculate and check remaining runners in the calculation. This seems to have worked pretty well. (plug)

@kofols Who cares,
That's probably the crucial question. I don't think the top elites care, after all, they have WOC to determine who is best. More likely to care are the people from smaller countries who can't make WOC finals and want to know where they stand, and whether they're improving. So that's where the system needs to be accurate.

@undy It has always seemed obvious that an outlier is someone having a bad run, not someone who is a slow/bad orienteering. But there has always been resistance to anything recursive into the code.
(how did you capture my post?)

I changed the throw- out bit so that any runner whose performance was n deviations away from their norm wasn't included in determining score for the race calculation:
It might work. If you cut off only outliers +-2sd than maybe it would be better to lowering the Time spread winner+50% to winner+ ~40%.
To surpass this anomaly whose performance is more credible for the calculation I suggested to create a new definition who are more R representative runners among the R runners. Same goal, different path, but still all runners have influence on race points. It is important to use them, because runners know that their performance have some meaning no matter if they make very well, average or bad runs.
1) Mean points = All R runners before the race (small correction)
2) Weighted overall mean Points = ~70% from R runners with more than 6 valid races (at least 1 runner) and ~ 30% from R runners with less than 6 valid races (at least 1 runner). Representative athlete is a runner with at least one race from events organized in countries from both membership fees group (1-4 and 5-8) in 18 months or option: they have valid races from 3 different countries (different terrains).
3) Mean time = Winner+50% (as today)

I believe that with well defined rankings and clear rules (WOC qualification rule, WRE awards system) the interest among elite would be increased. Formula is just one part of the WR and marketing, promotion and visibility is also needed to reach this goal. It is a question how much energy and budget IOF can put to WR or how important is WR for them when they have "Orienteering development" on the table.

Jagge, here is a perfect race to compare my and your model.
http://iof.6prog.org/WR_Event.aspx?EID=2283&det=2 (World Cup Event 2 - Sprint W21)

I just need to add Rank points. To simplify I am adding Skiing WCup points for best 30 runners (Rule 10.1) to final results.
+++
MP = All R representative runners are included
MT = All R representative runners within winner+50% (very close to normal distribution)
Representative R runner = Runner must have valid at least 3 races and avg. >600 points. I simplify my definition and it is probably more realistic and still better than today (only 1 race). All runners in the race meet this criteria.

Only one premise of 18 months avg. score is still open here. It would give better interpretation. As I said ~10 best races in 18 months are enough to see how good is someone in average. We don't need to count every race as this give us just a meaningless "average of runner's points" and not "average of runner's quality". It would be also very helpful to have more static 18 months avg. score in cases such is this e.g. Helen Palmer GBR did a very bad race but she is not worse runner than before. She come to the race with 1122.45 avg. points and finish it with 1086.50 (at next race). She has more than 10 valid races so it would be more logical to use same 1122.45 points as true value of her "average" quality.

If I typed figures right my formula gives winner only 1333 points. The average of the best 7 starters was 1297. Winners' score/ time is 2.7 % better than the average of 7 best starters/finisher. The official formula gives a lot more points to Tove than this.

Compare to mens' WOC middle and Bertuks. He got only the average of 7 best starters. Official formula gives Sara Lüscher (4th) more than the average of 7 best. At NZ For sure everything is not right.

Helen Palmer has not effect in my formula for the winner's score. She isn't top 7 starter and she did not end up top 7 in result list either.

I have no idea what score your formula gives to Bertuks and Tove, but I suggest tweaking it until it gives Bertuks at least as good score as to Tove.

I have no idea what score your formula gives to Bertuks and Tove, but I suggest tweaking it until it gives Bertuks at least as good score as to Tove.
It is enough to add only Rank Points (+100, 80, 60, 50, 45, 40, 36,....) to get better final points spread for all runners. Correction with MP=ALL R runners would be additional small theoretical tweaking but Rank Points gives deeper meaning to his win. It is a correction because stats can't do the job - who deserves more respect. It gives a clear signal that even if you beat the stats you can't beat the Rank points. You need top places and not just good runs (beat the average on races with high MP) to be at the top of the WR.

So you would still give Tove far higher score than Bertuks? That would not do.

That is how stats works. W class is less competitive (larger time spreads on longer interval) than M class and Women score more points. Simple. Comparison of Women's and Men's Points has no meaning to me.

Mean Points correction (with Rollier & Bobach) would give a slightly higher total MP than it actually was = 1218.966 because from the past points I can say that most probably at that time both runners had larger 18 months avg. than 1218 points. So this would change a little bit upwards.

The system works same for every runner with remark that you could expect more anomalies in Women class.

Comparison of Women's and Men's Points has no meaning to m

I think you are missing my point:. How much points winner gets compared to average of the best rivals. Bertuks got only the average of 7 best ranked starters, Tove and Kyburz got a lot more. And that is nonsense. Nothing to do with M and W. You can't give random scores like that and use four best scores for the ranking (or any average). Or you can, but you can't expect anyone gives any value to those rankings. So, if you like to get somewhere, tweak your formula until winner's winning margin and score compared to best ranked rival's base points go hand in hand. In other works Betuks gets more points compared to the Kyburz and Tove.

This is impossible mission for me. Can you find the way out because I don't think that this (time spread and whole % comparison) is really an anomaly of the system in this case?

Bertuks would get less points with your formula because he and Gonon enter the race with lower 18 months avg. score as others in a pool of 7 runners so it is normal that the MP of only 7 best runners would be lower in Bartuks case and because of this spread (points, times) of the avg. for the 7 best runners it was harder for him to beat the average (larger time spread would be needed or lower avg. points difference or extreme distribution of avg. points and times).

Bertuks case: avg. points diff./1378.50-1200.22 = 178.28; time diff. = 1:23 (3,76% of winner's time 36:45)
Kyburz case: avg. points diff./1328.27-1231.86 = 96.41; time diff. = 1:36 (3,93% of winner's time 40:41)
Tove case: avg. points diff./1322.50-1136.00 = 186.50; time diff. = 1:04 (5,46% of winner's time 19:32)

Kyburz got more points more or less because of lower avg. points difference and Tove got more points more or less because of larger time difference. Rank points could give Bartuks some deserved satisfaction. I don't see any mistakes here or false interpretation of the model.

Comparison of his final points to MP of 7 runners give you some information but not give you the entire picture how the system works for others and if it is objective enough for all in most of the races. Too much variables to say which one is more important than other to have an objective conclusion. For the sake of the comparison you should test final points also for the other runners, especially for the last runners.

But in general his points are based on same rules as Kyburz and Tove case. The variables here are: Time spread, R runners and 18 months avg. score. I have tweaked the formula on 2 fields and I don't see (have an idea) to find even more objective corrections. O.K., I didn't test the 18 months avg. score for all runners (10 best races) which would change something. I think it is not possible to do much on time spreads (you did) and on R runners because you have limited options (IOF's, my option or to introduce even more restricted definition which can limit the fairness of calculation for small WREs). 18 months avg. score is probably the key variable where many options are open. Which is the best? I don't know. Someone would need to test all of them which seems to have some meat. And I even don't know if you agree with explanation of these two so far.

I think that someone else will need to look at these suggestions to say what are possible other options and if they can be introduced or put this idea in trash.

Oh yes it now counts for something, a lot of something.

For better escorts and bigger tracks, get NEW Ranking Points (TM)

Bertuks would get less points with your formula because he and Gonon enter the race with lower 18 months avg. score as others in a pool of 7 runners so it is normal that the MP of only 7 best runners would be lower in Bartuks case and

What you mean by that? Bertuks would get more points with my formula (about 1335), a lot more, not quite as much as Kyburz in his race (1348), but pretty close. And Bertuks and Gonon are not in the pool of 7 best ranked starters. No effect there, you must got that wrong. They are only in the pool of 7 best finishers.

Maybe Bertuks got so poor points because ahletes peaked for this race, there was qual races so all of them are in good shape, most of them in better shape than their average scores. So "average of all" is slightly off. And "spread of points" is similar way off - too big figure, athletes with poor scores are far better in reality. That all makes it hard to make the gap big enough to score good points.

So, there is no any mistakes or false interpretation of the model here. It is natural the model works that way. My conclusion is the model (using average of all and spread of all) we use here is simply wrong model for scoring top competitors. The average of all is so far away from their time and score, so spread inaccuracies throw their scores easily off. That makes top scores always less accurate than scores near the average.

you should test final points also for the other runners, especially for the last runners
If implemented as I described, it would effect only the top half. Scores below the mean time would be the same. No need to re-calculate them.

What you mean by that?
I tried to say that his final points are lower because of (smaller MP) than final Kyburz's points (higer MP) in a pool of these 7 runners.

Yes, I did a mistake for 7 best ranked starters but the explanation stays. This is how I understand the numbers.
Bertuks case: avg. points diff./1378.50-1296.23 = 82.27; time diff. = 1:23 (3,76% of winner's time 36:45)
Kyburz case: avg. points diff./ 1349.83-1301.18 = 48.65; time diff. = 1:36 (3,93% of winner's time 40:41)

That all makes it hard to make the gap big enough to score good points.
Because of this reason IOF introduced IP factor and I've exchanged it for Rank Points.

...is simply wrong model for scoring top competitors
We all agree about this statement but if we want to have one Ranking for so many different WREs +100 races you need average, spread, etc. WR is not just a Wcup or WOC. It is only a question how accurate it should be or what solutions can promote quality of the runners in the best way. 100% of fairness is not possible. Stats can't do that, but it is not far away. If this is acceptable than we can change or tweak the formula where we see a chance to get a slightly better outcome.

No need to re-calculate them.
Two formulas for same race. I can't buy it...sorry.

And the points are not so important at the end, more importantly is how we run all together around WR and promote it. In a way that runners, trainers, federations, media, sponsors will start to respect WR. And first among equals should be IOF.

Bertuks case: avg. points diff./1378.50-1296.23 = 82.27; time diff. = 1:23 (3,76% of winner's time 36:45)
Kyburz case: avg. points diff./ 1349.83-1301.18 = 48.65; time diff. = 1:36 (3,93% of winner's time 40:41)

None of this makes sense to me. Could you explain why you are calculating point and time differences between the first and the 7th? That is obsolete here, especially the avg. points diff. has no effect at all. For sure my formula doesn't give Kyburz more points because of lower avg. points difference.

The use of average of top 7 of athletes instead of average of all to address the problem with top ranking score accuracy has been used for decades in small and large O events, lots of experience and pretty happy athletes. You seem to be fond of the current IOF model, but unfortunately it is hard to believe elite athletes will ever buy it. You will need to calibrate the top end scoring one way or an other in your formula. You can call it second formula if you like. I am saying this because I believe you can find a good way to do it if you really wan't. And then your approach may end up being pretty good.

I know small events should be handled nicely too. As I see it we should make the formula reliable for large events first. And then do what is needed to get smaller events right if it doesn't work for those out of the box.

I put one versrion of the top 7 calibrated scoring formula to a server. Just replace EID in url with any event id. Examples:

WOC 2012 Middle and Bertuks:
http://iof.6prog.org/WR_Event.aspx?EID=2046
http://routegadget.net/misc/wr.cgi?EID=2046

highest score ever (men), Thierry at POM 2007
http://iof.6prog.org/WR_Event.aspx?EID=1119
http://routegadget.net/misc/wr.cgi?EID=1119

Sure makes those highest scoring races look more reasonable (to me). If you know races with suprisingly low or high scores, you can try what happens with this formula. Top 7 method used here for calibration is a variation of the formula used happily for Finnish national O rankings for three decades - not something I just made up.

Hi Jagge,
I am saying this because I believe you can find a good way to do it if you really wan't.

I was thinking about what is good in your formula and what is good in my approach but I have realized that I would still end up inside the formula which was designed with some important limitations. Some of our solutions can make it a little bit better but still doesn't resolve some fundamental problems. I think that the future solution of WRE should go in the direction of FIS points. I don't know if any other country uses this approach beside SWE. There are some similarities about regional problems within skiing and orienteering and
from IOF council minutes 164: "Council agreed on three main principles for the revision work, i.e. that every participant who has completed the course according to the rules should get world ranking points, the distribution of points should be related to the value of the event, and that the ranking list should serve as a promotion and selection tool.
it is evident that we will need totally different approach to base formula.

FIS points also combines our proposals in one. I like this rule.
4.4.2 The five competitors with the best FIS points who started the race, are selected (classified, disqualified, DNF and did not start 2nd run). Take a look

Why not look at the rules for XC Skiing instead of Alpine?

They have similar "problems" and rules regarding Interval start/Mass start and Sprint/Pursuit races...

Which type of the events should be included into WRE calculation is another debate. We need to figure out first what kind of Points calculation can serve as the best model to give points for sprint, middle, long. I agree that we can look into Alpine and XC model or some other points models.

@Bubo
Can you describe what kind of rules should be applied to Mass and Pursuit races to become a more standardized type of events for WREs? Appendix 6: Competition Formats describes only Sprint, Middle, Long and Relay.

I suppose that after a few years of testing and IOF's discretionary right (we need to avoid this practice in the future - not good) we need to put the best solutions on the paper. As I managed to look all previous WRE - Mass races counted for WR and at least one chasing race.

Long shortened, Ultra-Long, Mass Start, Sprint Knockout, Knockout, Middle chase start, Long Chasing Start, Relay Mixed Sprint.... it is a small jungle of different words, type and variations of events. I suppose we need to decide what type of events we really have and it should be organized at WREs. I would also like to see a better information which course setting models are suitable for different type of events and official definition of course setting models with examples of each one of them.

I know that organizers put these information in the bulletin (Danish Spring - Mass start and 2-3 loops with “Farsta”) but IMO this information should be also included in the WRE rules. Someone who run farsta for the first time needs to have a quick access to complete and detail information what farsta is. This is more or less the same problem as we have had with control descriptions in the past.

I´m not really into creating solutions to this complex problem - there is no such thing as a perfect model that fits all situations. I merely wanted to point out that similar situations exist also in other sports - and that XC skiing may be more similar to us than Alpine skiing - how they solve their problems is quite another story.

I´ve been following this lengthy discussion since it started, but I´m usually quite lost as soon as complicated mathematical models are mentioned. Whatever we do about rankings there will never be a solution that creates total justice and fairness - it all depends on "man-made"/artificial models anyway...

I think one moral of the story from skiing (both at FIS level and at domestic Canadian level) is that the points you get sometimes depends more on how you choose the event than how well you ski.

1) Points often depend on who shows up. The better the competition is, the better the points on offer, which is not necessarily a bad thing. What is a bad thing is people skipping races because they know that they will get bad points if they race (i.e., if noone else with good points has signed up).

2) The type of race determines how easy it is to get good points. Tactical mass start races where the pack stays together for a long time are the best opportunities for middle-of-the-pack racers to get really good points.

Tactical mass start races where the pack stays together for a long time are the best opportunities for middle-of-the-pack racers to get really good points.

There is still one big difference. In XC and Alpine they all know the course in advance. Classical mass start races are out of date but mass races as one-man relay and Farsta are pretty good examples how to solve this problem to satisfactory level and to minimize influence on following and final ranking points (Blodslitet, Danish spring). If this is already standard in Scandi countries then it is time to put these examples in rules and see more such races at other WREs. In skiing they have rules to define what is good course setting and we are short here at least for these type of events.

At these mass start races (around 80 elite runners) I an wondering how this works. What is the optimal number of runners for this type of events to have a good spread among them from the start to finish. Can we put out runners in smaller waves (15 runners, 3 minute interval) as we see at Marathons. The best runners start in first wave because in each case we can expect that in 99% the winner will be from the first wave.