Discussion: WOC Performance vs. WRE Ranking (by Nation)
in: Orienteering; General
Sep 17, 2006 3:24 PM
#
bmay:
For the past few years, I have calculated a rough ranking of performance at WOC by country. My formula, designed to measure performances at the US/CAN level, doesn't count medals, but rather uses all performances by runners from a given country whether they are in the top half, bottom half, only in quals, or also in finals, etc.
The formula assigns points for all starters in each race (9 quals, 3 finals + relay for each sex). The point formula is simple - 100 for 1st place and ~0 for last place (e.g., in a race with 50 runners 1st = 100 pts, 26th = 50 pts, 50th = 2 pts). DNF and DSQ are assigned 0, but still count in the number of runners. The points are totalled for each country. Note, theoretical top score is approximately 2100 men + 2100 women = 4200 total.
Tabulations for WOC 2006 in Denmark are now complete and available here. There are three tables, one counting all men's and women's scores, one for men only and one for women only.
Because the IOF is now planning to use the WRE Federation League Table for selecting # of runners at World Cups, I was curious to know how their national ranking compared with my calculation of "WOC performance". If the two rankings have validity, then I would expect the two to be pretty similar.
A comparison of WRE national ranking and WOC ranking is available here.
The formula assigns points for all starters in each race (9 quals, 3 finals + relay for each sex). The point formula is simple - 100 for 1st place and ~0 for last place (e.g., in a race with 50 runners 1st = 100 pts, 26th = 50 pts, 50th = 2 pts). DNF and DSQ are assigned 0, but still count in the number of runners. The points are totalled for each country. Note, theoretical top score is approximately 2100 men + 2100 women = 4200 total.
Tabulations for WOC 2006 in Denmark are now complete and available here. There are three tables, one counting all men's and women's scores, one for men only and one for women only.
Because the IOF is now planning to use the WRE Federation League Table for selecting # of runners at World Cups, I was curious to know how their national ranking compared with my calculation of "WOC performance". If the two rankings have validity, then I would expect the two to be pretty similar.
A comparison of WRE national ranking and WOC ranking is available here.
Sep 17, 2006 3:30 PM
#
bmay:
Now for some thoughts on the US/CAN situation ...
For men/women combined, the WOC ranking gives USA=24th and CAN=28th. Compared with 2005, the US has held steady, while Canada has dropped a few places (from 22nd). Clearly, the lack of a healthy Sandy hurt Canada this year.
With the US outperforming Canada, one might wonder if this year might see the Bjorn Kjellstrom Cup head south of the border? To do so, the US needs to send a few more quality runners north of the border (e.g., a couple of Saegers would really help).
Considering men only ... CAN=30th and USA=32nd. These results are a fair bit worse than the overall (men/women combined). Comparing WRE countries for these two countries CAN=34th and USA=25th, it's clear that the US has done a better job playing the WRE game this year than Canada. But, expect upward motion for both countries in a few weeks at NAOC.
Considering women only ... USA=23rd and CAN=27. The US women did really well this year. Comparing WRE scores for these two countries USA=24th and CAN=28th suggests that the WRE score is a pretty good indicator of WOC performance.
Currently, the cut-off for getting more than one World Cup runner is top-16* on the Federation League table. US and CAN are nowhere close to this by either measure, so it matters little directly to us what measure is being used by IOF. * Oops, so it is actually top-22, which the women aren't too far away from.
For men/women combined, the WOC ranking gives USA=24th and CAN=28th. Compared with 2005, the US has held steady, while Canada has dropped a few places (from 22nd). Clearly, the lack of a healthy Sandy hurt Canada this year.
With the US outperforming Canada, one might wonder if this year might see the Bjorn Kjellstrom Cup head south of the border? To do so, the US needs to send a few more quality runners north of the border (e.g., a couple of Saegers would really help).
Considering men only ... CAN=30th and USA=32nd. These results are a fair bit worse than the overall (men/women combined). Comparing WRE countries for these two countries CAN=34th and USA=25th, it's clear that the US has done a better job playing the WRE game this year than Canada. But, expect upward motion for both countries in a few weeks at NAOC.
Considering women only ... USA=23rd and CAN=27. The US women did really well this year. Comparing WRE scores for these two countries USA=24th and CAN=28th suggests that the WRE score is a pretty good indicator of WOC performance.
Currently, the cut-off for getting more than one World Cup runner is top-16* on the Federation League table. US and CAN are nowhere close to this by either measure, so it matters little directly to us what measure is being used by IOF. * Oops, so it is actually top-22, which the women aren't too far away from.
Sep 17, 2006 4:05 PM
#
bmay:
Thoughts on the WRE Federation League Table ...
While the Federation League Table doesn't seem so bad for US/CAN, the overall match between WRE ranking and WOC performance by country is pretty scattered.
Looking at the Men's rankings in particular illustrates that the WRE rank is not a great predictor of WOC performance. There is lots of scatter in the graph of WRE ranking vs. WOC performance (r^2 = 0.67). There are a number of countries that are ranked too high by the WRE, e.g., Austria (7th in WRE, 18th in WOC), Germany (9th in WRE, 19th in WOC), Hungary (10th in WRE, 24th in WOC). Similarly, there are a number of countries ranked too low ... Russia (16th in WRE, 6th at WOC), Ukraine (26th in WRE, 10th at WOC).
Looking at the Women's rankings, the agreement between WRE and WOC rankings is better. But, there are still some anomalies. Germany (6th in WRE, 12th in WOC) and Hungary (11th in WRE, 18th in WOC) are over-ranked by WRE. Czech (10th in WRE, 5th at WOC), Denmark (13th in WRE, 8th in WOC) and Russia (15th in WRE, 10th in WOC) are all under-ranked by WRE.
Overall, it seems to me the WRE ranking is one of "how many ranked runners do you have" vs. "how good are they". As time goes by (and more countries take the WRE system more seriously), I think this will improve. But, I still think using the top 20 scores in the Federation League Table is way too many, if one is trying to decide whether a country can send 1 or 2 runners to World Cup. If the League table was dropped to top 10 scores, I think we'd see a much more reasonable ranking.
While the Federation League Table doesn't seem so bad for US/CAN, the overall match between WRE ranking and WOC performance by country is pretty scattered.
Looking at the Men's rankings in particular illustrates that the WRE rank is not a great predictor of WOC performance. There is lots of scatter in the graph of WRE ranking vs. WOC performance (r^2 = 0.67). There are a number of countries that are ranked too high by the WRE, e.g., Austria (7th in WRE, 18th in WOC), Germany (9th in WRE, 19th in WOC), Hungary (10th in WRE, 24th in WOC). Similarly, there are a number of countries ranked too low ... Russia (16th in WRE, 6th at WOC), Ukraine (26th in WRE, 10th at WOC).
Looking at the Women's rankings, the agreement between WRE and WOC rankings is better. But, there are still some anomalies. Germany (6th in WRE, 12th in WOC) and Hungary (11th in WRE, 18th in WOC) are over-ranked by WRE. Czech (10th in WRE, 5th at WOC), Denmark (13th in WRE, 8th in WOC) and Russia (15th in WRE, 10th in WOC) are all under-ranked by WRE.
Overall, it seems to me the WRE ranking is one of "how many ranked runners do you have" vs. "how good are they". As time goes by (and more countries take the WRE system more seriously), I think this will improve. But, I still think using the top 20 scores in the Federation League Table is way too many, if one is trying to decide whether a country can send 1 or 2 runners to World Cup. If the League table was dropped to top 10 scores, I think we'd see a much more reasonable ranking.
Sep 17, 2006 7:45 PM
#
rm:
Currently, the cut-off for getting more than one World Cup runner is top-16 on the Federation League table. US and CAN are nowhere close to this by either measure, so it matters little directly to us what measure is being used by IOF
No, 22nd or higher get more than one runner in each World Cup. (So, after NAOC, the US and Canadian women and probably men will be close, or maybe there.) From the Special Rules:
"The allocations will be:
• Top 4 nations get 10 places
• Next 3 nations get 9 places (nation 5-7)
• Next 3 nations get 8 places (nation 8-10)
• Next 2 nations get 6 places (nation 11-12)
• Next 3 nations get 4 places (nation 13-15)
• Next 3 nations get 3 places (nation 16-18)
• Next 4 nations get 2 places (nation 19-22)
• All others get 1 place"
To which the extra special rule was added giving extra allocations based on runners in the top 150 (but only for countries that would otherwise get one or two runners).
It is possible for the US and Canada to get more World Cup allocations due to NAOCs, via one of the rules.
(An aside on something I noticed: Kind of an odd distribution of allocations versus country placing, eh, now that I look at it more closely, with both the allotment increment (1 or 2 runners) and the number of countries at each allotment (2 to 4) varying up and down? Kind of shaped like one of those huge kid's slides that has two bumps in it. They must have had specific countries and their strengths in mind. A big flat spot for the top ten countries (8-10 runners), then a rapid drop to half as many runners, then a slower decline again. Makes for an enormous drop between 10th and 13th place. I wonder why they didn't pick a simpler distribution (1 fewer runner for every 3 places in the country ranking, or something)? This distribution might match the distribution of country strengths now, but will it stay constant as the sport develops? Anyway, just an aside; I hadn't noticed all the peculiarities before.)
No, 22nd or higher get more than one runner in each World Cup. (So, after NAOC, the US and Canadian women and probably men will be close, or maybe there.) From the Special Rules:
"The allocations will be:
• Top 4 nations get 10 places
• Next 3 nations get 9 places (nation 5-7)
• Next 3 nations get 8 places (nation 8-10)
• Next 2 nations get 6 places (nation 11-12)
• Next 3 nations get 4 places (nation 13-15)
• Next 3 nations get 3 places (nation 16-18)
• Next 4 nations get 2 places (nation 19-22)
• All others get 1 place"
To which the extra special rule was added giving extra allocations based on runners in the top 150 (but only for countries that would otherwise get one or two runners).
It is possible for the US and Canada to get more World Cup allocations due to NAOCs, via one of the rules.
(An aside on something I noticed: Kind of an odd distribution of allocations versus country placing, eh, now that I look at it more closely, with both the allotment increment (1 or 2 runners) and the number of countries at each allotment (2 to 4) varying up and down? Kind of shaped like one of those huge kid's slides that has two bumps in it. They must have had specific countries and their strengths in mind. A big flat spot for the top ten countries (8-10 runners), then a rapid drop to half as many runners, then a slower decline again. Makes for an enormous drop between 10th and 13th place. I wonder why they didn't pick a simpler distribution (1 fewer runner for every 3 places in the country ranking, or something)? This distribution might match the distribution of country strengths now, but will it stay constant as the sport develops? Anyway, just an aside; I hadn't noticed all the peculiarities before.)
Sep 17, 2006 11:20 PM
#
I believe Brian is saying that the WRE rankings for the top 20 runners count towards a country's World Cup allocation. I agree that 10 would work better.
Sep 17, 2006 11:42 PM
#
rm:
Yes, I agree, top ten would be fairer for geographically large countries, rather than top twenty.
I was responding to Brians statement Currently, the cut-off for getting more than one World Cup runner is top-16 on the Federation League table. US and CAN are nowhere close to this by either measure, so it matters little directly to us what measure is being used by IOF....it's actually top 22, which the US and Canada are indeed within striking range of. (It may not look like it now, but at NAOC, the Canadian women could nearly double their rankings with typical runs by those attending.)
I've updated my posting above to make it clearer what I was referring to...sorry for the confusion.
I was responding to Brians statement Currently, the cut-off for getting more than one World Cup runner is top-16 on the Federation League table. US and CAN are nowhere close to this by either measure, so it matters little directly to us what measure is being used by IOF....it's actually top 22, which the US and Canada are indeed within striking range of. (It may not look like it now, but at NAOC, the Canadian women could nearly double their rankings with typical runs by those attending.)
I've updated my posting above to make it clearer what I was referring to...sorry for the confusion.
This discussion thread is closed.