Discussion: Agreed.

My thoughts exactly. Keep thinking about posting but always decide not to.

Agreed as well. However, if you haven't read it yet, you should check this, where Bob-F implemented the suggestion I made here. That is better evidence for noise in the ranking system than anything I've ever seen. I don't think it's crazy to suggest based on this example that rankings for a single race could have a +/- 5% error. I say this while completely understanding how almost everything in both Barb's document and the main discussion thread (USA Juniors II) is not convincing at all as evidence of any kind of problem.

More generally, Bob-F's point in the thread on barb's log that the calculation of CGVs could be noisy where a large fraction of the competitors competed in only a small number of ranked races is definitely worth some thought.

Yes, the main issue is that there is not much overlap between runners on the ISVM course and any other course at any other ranked event. This - the overlap - is what makes the rankings work across events. I've never thought carefully about how it affects the CGVs but I would expect it to be very sensitive to small changes when the overlap is not very high.

Yes, thanks, I did wander over there, and the bulk of the conversation went over my head a long time ago, so my take is more simplistic - the results on the course were very compressed, and the handful of ranked runners are not that highly ranked, so the scores are low.

When it comes down to it though, I am really just the data person, and I probably don't know what I am talking about, so I won't try. :)

I don't understand the numbers or the process either, I think they really are just playing with the deck chairs. JJ caught the main problem with using Rankings in their current form. The populations of the groups being measured is quite different. The boys compete on Red and pretty successfully as most competitive men continue to run Blue instead of their age grouping. OTH the girls compete against M50s who are taking it easy for a day by running the shorter course and blowing it up for rankings. Also Blue runners recovering from injury occasionally run M Open adding to the distortion. The problem occurs when a ranking of 75 is required to make the team. Would make a great team if they could do that, but they are going up against former US Champs still running very well. So some get left home missing the experience.

I wonder if I have understood this correctly - are rankings actually calculated per course only, regardless of who happens to be running at that particular event?

Then there´s a definite disadvantage to those competing against different age classes with different motivation/background (and women against men?) than their own.

In the Swedish ranking system - Sverigelistan - every age group is treated separately and results/rankings can only be trusted for comparisons within that group. Isn´t that really what everybody wants? If all juniors run the same course it makes things so much easier - and who cares (at least in the context of JWOC selections) if they beat any H50´s or not?

Rankings are calculated by course, not by class, in order to have large enough data sets for the math to work out. If we had enough people participating, we could do it by class (and also have separate courses for each class, for that matter).

I understand the reasons for this, but also see the danger of doing it affect rankings in an unwanted (?) manner...

I don't think there's any indication that the rankings are adversely affected. It's just that there's a possibility of their being misinterpreted.