Register | Login
Attackpoint - performance and training tools for orienteering athletes

Discussion: Rankings

in: Chas; Chas > 2019-05-02

May 2, 2019 10:08 AM # 
Statistically it's all sound given that everyone always tries there best but as soon as you add some weightings then statistically its broken and liable to have even more quirky anomalies.

It's the trying there best assumption that is broken. Everyone tries there best for the big events. At the smaller events not everyone is trying there best and people that do try there best get better ranking points. If you want a good ranking score don't do the British nights but treat the event the next day very seriously. Ask Phil...

The original old system when I first started was very simple. 100 points for a win and then position based. Lesser events I think gave 80 for the win or something like that. e.g. there was a weighting. In practice we still have this system now. It's just called the UK O league not the ranking system and only has the bigger events.
May 2, 2019 1:23 PM # 
I'm sure there's some truth in what you're saying Ifor but my theory is that the main issue is the size of the field. At the big events you get a large number running each course therefore the base time is likely to be statistically accurate. At small events with just over the minimum of 10 running a course you tend to to get anomalies. These can either give scores which are too big or too small. To get the best ranking position you need to do as many of these smaller events as possible to maximise your chances of finding those with inflated ranking points. This is not how it should be.

Surely it would be better to weight the scores from the bigger events (as most people do these races) so that the rankings are based on the more statistically accurate races?

Interestingly my best score (of the last 5 years or so) was at the UKEOL race in the Lakes where I ran Black against some of the best M21's in the country. Maybe I should just run M21 all the time? If I was bothered about the rankings that is.....
May 2, 2019 4:10 PM # 
Some of us are not capable of Black any more. I do one black standard course a year at Jukola.

Historically my best scores come from the big events where I try properly. Other events I don't get the motivations so I am in amongst those helping distort the scores by not putting in the same effort.
May 2, 2019 7:51 PM # 
Yes I targetted Pembrey last year as I liked the idea of the Level B event, and yes i was not tired so i did get a fair few points by beating some knackered runners frmo the previous night. (except Megan, but I was close to her). I also tried very hard & had a good run.

The JK points are a joke, and I shall be writing to them (BOF) about them , also asking if a proper enquiry into the whole JK results debacle is taking place.

And when do we get our relay medals & prizes? No one will want to sit through BOC prizes *and* the JK relay belated ones too?

Sigh. Roll on Italy, Slovenia & Swiss O week! #getaway
May 6, 2019 9:43 AM # 
You are right about the size of field. We looked at the scatter of points with the number of entries, that's where the minimum number comes in - it's inaccurate to about the level that most people's event-to-event scores fluctuate.
(there are also bugs in the implementation with runners have very few events, and issues from people running under bogus names or slowly with sticks...)
So, yes, you get a better ranking position if you go to lots of ranking events. But, that is exactly how BOF want it to be. The ranking list is for everyone, especially improvers to measure how they're improving, not so much for "outliers".

UKOL is meant to be to determine who the best in class is. It's shite. Compare the ranking list standings with the standings from UKOL and you'll see.
May 6, 2019 10:37 AM # 
If it was just about size of field then you'd see this effect for all classes - but it doesn't seem to apply to M21E where all the leading JK performances are amongst the respective competitors' best 6 scores. A few years back Vets were over-ranked cf the elite - Clive was in the top 30 overall for a long time - but since the algorithm was amended this seems to have been reversed (I haven't quite worked out why)
As you say, your "anomalous" scores come from events where you've run up against M21E - where you're benefitting from their high ranking points (and potentially from a helpful tow from a quicker runner).
May 6, 2019 12:46 PM # 
jayh: yep, since the 'change', I've noticed best scoring comes by running the longer courses against higher ranked runners, a good run on Black beats a much better run on Short Brown
May 6, 2019 1:54 PM # 
I had a useful chat with our new Director, Graham Patten about rankings on Sunday. He has a lot of experience with the French system and agrees with me regarding weightings. He's on the case.
He also likes the idea (as done in France) of the start list being the ranking list in reverse order. I'm not so sure about that...
May 6, 2019 3:58 PM # 
(I haven't quite worked out why)
The main change is identifying outliers. Previously, the last 3 finishers were excluded. On elite courses, they were often decent M21s having a below-average run. Taking them out of the averages dragged everyone's scores down.
Even so, the total number of points given out on a course is the average of the competitors' previous scores. It's not possible that everyone scores above average. So why do you (and I) notice the effect?
Since its us old folks noticing, it implies that good run which still puts you in the bottom half scores better than an equally good run which puts you near the top on an easier course. It makes sense from the skewness of the times - there's longer slow-time tail than fast-time tail, but the "spread" is assumed to be same for both slow and fast runners.
Also, we only run up when we're feeling fit!
May 7, 2019 7:21 PM # 
So how do you like them apples, Chas?

Scott Collier got almost as many points for 1:58 (37th) on M21E as James Logue did for winning M50 by 6.5mins. I know who I'd have my money on if they were running the same course.

Summat's wrong.
May 7, 2019 8:31 PM # 
Jon X:
I am not a fan of the weighting idea but something is awry....relative to what I have scored for some other events and lesser performances in the past year, I would definitely have expected more from JK Long and BOC Long than I am getting!

I do also agree with what Ifor said earlier that part of what is broken is that assumption that people try equally hard in all races, I know I don’t :)
(I will leave the actual rankings gurus such as graeme and greywolf to explain whether that would explain relatively worse scores for the top runners at big events when (probably) most people are indeed trying their best).
May 7, 2019 8:34 PM # 
Jon X:
... and perhaps also true that at Jk Long and BOC Long this year, both courses were fairly easy and also the running was good/fast. If that makes most people feel they are doing well (so more people try hard all the way to the end than is the norm) plus the prevalence of errors is below the norm, would that have an impact on points?
May 8, 2019 8:40 AM # 
The JK points were so rubbish, I actually feel slightly grateful to get 1289 points for a close 4th in BOC...... I guess we recalibrate our expectations and realise 1300+ is for legends only?
May 8, 2019 8:53 AM # 
Legends? Don't know that I'd call Scott Collier a legend.
May 8, 2019 9:16 AM # 
I would argue that the ranking, as published, averaged over six races, are pretty accurate. It's not possible that each score Is super-accurate: In any statistical sample you'll find anomalies. e.g. Its a bit odd that the M21E winner tied his run in split with a W16: but one anomaly on one leg is only part of the story, we don't then jump up and down complaining that the whole BOC is invalid.

Mathematically, it's not possible for a course to give everyone high scores. (Unless they miscoded it: they won't let me check)
But it is possible for the top half to score well, and the bottom half to score badly. Or vice versa. That depends on the spread of results and (because the slowest times are further from the average than the fastest) generally its harder to get a big score by winning against slower people then by finishing mid-pack against faster ones. So likes of us score better running against the big-boys than beating up on other old guys.

Jon has a point too, BOC was pretty easy, most people came back thinking they'd had a good run. Fewer errors tends to favour the people who usually make most errors (i.e. not you lot).
May 8, 2019 9:29 AM # 
Don't know why you're so down on Scott - I'll bet none of you lot will ever get under two hours for an M21E long race...
May 8, 2019 9:40 AM # 
I agree that the ranking list is probably pretty accurate and that it is a misperception that a good run or performance should garner a good score.
Having said that, I am pretty sure I am over-rated in the ranking list by virtue of racing Richard Robinson and Anthony Squire in the East Midland League.
May 23, 2019 7:39 AM # 
Another top 6 score obtained by running a Black course in the Lakes last weekend (Stickle Pike). It seems to be the way to go if you want to maximize your ranking. Certainly working for Graham Patten. His score on Saturday was amazing, admittedly for an excellent result.
May 23, 2019 7:40 AM # 
oh, and I did beat Scott Collier as well ;-)
May 25, 2019 10:01 AM # 
Ha ha. See.

This discussion thread is closed.