Discussion: SportIndent data sharing policy?

Is there any?

To make adequate ratings for runners in a club one need at least data from all the events visited by club members.
Recently, trying to accomplish such ratings for BAOC, I was asking people holding the data to share it.

I must admit I felt awkward while doing this, and I'm afraid I was putting people in awkward position also...
Some shared, some had problems with it. I'm getting more and more uncomfortable looking like a needy person asking for favor all the time.

Usually all needed data is published on the web, but it is practically impossible to extract it as a batch.
All web pages look different and there is no way to get data in standard format in reasonable time frame.
While most of the data originally already exists in standard SportIndent format and sharing would be very easy.

I understand that sometime personal data, like phones and addresses are stored in SportIndent files - those could be stripped - couple of keystrokes....


Maybe some read only database (ftp server) could be be created for files upload by "Local Deciders" and for subsequent download by anybody interested?


Shura

Add consent specific to this to the waiver or have a form signed and you're good. Someone could argue that their results are their PI and depending on the jurisdiction they could be right. Express consent to do exactly and only what you describe in the form is the safest way to go.

Actual issue is not runner not wishing to be published, but the SportIndent data holder being extra cautious and not releasing the data

NOTE: I must repeat here I'm not judging or saying somebody is wrong!!!

Arguments to not release data can be from :
- It is pain to send file to everybody who asks -too many of you here...
or
- I was gathering data for some time and do not want somebody get it for free, I spent so much time tweaking it.
to
- I want information to be published on <..> web page and nowhere else because it makes more sense to me
or
- I would like to keep runner's trust in what I'm doing, while somebody else can misuse the data by plotting graphs with wrong colors or doing some other weird calculations.

Maybe web archive, accompanied with specially crafted waiver/ consent to use data only in predefined way and with reference to originator would be good solution?

I think Winsplits Pro will let you export data. You still have to rely on the organizers of a race to put the data on Winsplits. If you feel uncomfortable asking people to give you the data, you might try asking them to uploaded it to Winsplits.

Thank you, Spike,
Actually I have no problems with asking people.
I'm more concerned about putting data holders in awkward situation when they have to refuse, because they feel they must because of above mentioned reasons.
And they do refuse, about 50% of the time, from my own experience.

In this case having more or less generalized sharing policy ( I assume it doesn't exists, otherwise Tundra of JJ would already have pointed out by now) would make everyone easy and also would allow to ban somebody from getting data if he violates the policy.


My general idea is that if data from all clubs would be easily available, different people varying from statistics majors to "have-some-free-time-psychologists" could use it and produce various analysis based on it.

Winsplits Pro can not be considered as easy way...

Opening easy data access to many people would encourage analysis attempts.
Opening the data sources is the way google became Google after all.

At the end of the day everybody would benefit from somebody investing his time and making some cute applet plotting nice graphs and tables in addition to WinSplits.

I'm interested to know what you mean by 'SportIdent data'. There isn't really any results data produced by any SportIdent software. (well, SportIdentUK does, but it's not SportIdent)

In the UK there is more than 1 choice for what software to use to time an O event, there are 3 or 4. Each generally has a different way of exporting data, which makes what you're trying to do a bit of a nightmare.

What British Orienteering have done / do (they seem to be working on a replacement at the moment but we're in limbo a bit) is supplied an upload facility to their website where each organising club uploads a CSV file in a specific format which is used to calculate rankings. You can then get a list of rankings for each class and lists by club etc

This is it here: http://ccgi.orienteering.plus.com/rankings/html/ra...

Sadly it isn't as solid a service as it could/should be, but if every event was added (some are not, as the onus is on the volunteer running the timing systems) it would be exaclty the kind of thing you're looking for.

I believe the way forward is for the national body (BOF/USOF) to collect / enforce collection of such data for every event over a specified level. If only it were easy!

Is the issue technical or social?

I'm not sure I personally undertstand objections to race data being put in a database and analyized. All USOF members are presently subject to this now with finish times in the ranking database -- I see split times as being no different. I'm not aware of any objections to use of someone's personal race data, but WDIK?

As for technical issues, asking meet volunteers to use a specific format, specific software, etc., to get a consistent format is a bit more of a request. They may already be invested in whatever they are using, and may not want to do more work to convert it for this purpose. What is the purpose anyway? I know I wouldn't unless the value of the purpose clearly exceeded the cost of the work. That said, it certainly would not hurt to suggest a consistent format, and see if people follow along.

My suggestion, FWIW -- most of this stuff is already on the web. I think that makes it fair game to write scripts to scrape it and analyize it as you wish.

To andrewd:
> I'm interested to know what you mean by 'SportIdent data'.
There is a way to export csv file in US version also. you are talking about it on the ranking page. this is it.

> In the UK there is more than 1 choice for what software to use to time an O event, there are 3 or 4. Each generally has a different way of exporting data, which makes what you're trying to do a bit of a nightmare.

Well, nightmares are different. Compare N=4 formats with N->infinity actually existing on the web...

> I believe the way forward is for the national body (BOF/USOF) to collect / enforce collection of such data for every event over a specified level. If only it were easy!

I agree completely. And benefits are at the end.

Thanks to Spike's advice I found a relatively easy way to get information from publicly available place - WinSplits.
Even though splits data are restricted in free version, all final results can be downloaded from there.
Of course this is far from having freely accessible archive, but this is something big - well formatted place where people put their results by themselves.

and now to the randy's questions:

> is the issue technical or social?

the issue is sociotechnical :)

> I'm not sure I personally undertstand objections to race data being put in a database and analyized.

this is social part.

I mentioned that people never have problems with data being analyzed. It is database holders who are reluctant to release the data for open use. Notice the British site only concentrates on gathering the data and I found no words about " Get our data and make use of it!" ( But there is "request for information page" though...)

If major data gatherers would get together, we have only 5-10 big chunks of data to combine and support

> As for technical issues, asking meet volunteers to use a specific format, specific software, etc., to get a consistent format is a bit more of a request.

I agree to the point where I even do not bother asking for information which is not SportIdent or other digital format already. too much hassle.

As for any already-in-excel-like format - it would be piece of cake to convert any file into any other using scripts, as you mention...

and technical part of the problem comes below

> My suggestion, FWIW -- most of this stuff is already on the web. I think that makes it fair game to write scripts to scrape it and analyize it as you wish.

In a straight way it is not possible. Look on the BAOC web page http://baoc.org/wiki/Results/Current

it is a complete mess of text and data which is nonconvertible into anything.

Special attention need to be paid to the data before submission - then it will be useful fro everybody.

and for those who get through the post do far - reiteration of my open source idea benefits:

Open source database would(might) encourage many young computer savvy people to experiment with it and maybe produce some great product, which would never exist otherwise.

I personally would never get so far in my O-scores project if Valerie would not share the initial set of data -well organized and useful. Thanks again

FWIW, this is what we have done in Ireland for the past few years.

1. All organisers use Ór (http://orienteering.ie/or), OE2000 or a manual pin-punch system for results.
2. Ór and OE2000 can both produce results in the same csv format. I have provided a spreadsheet that organisers must use as a template for manual punching events.
3. After each event, the organiser can upload their results to the IOA website, orienteering.ie.
4. As a carrot, the process is very simple and is easier & quicker than sending results to a club webmaster. Also, we provide an RSS feed so each club can have their own results page with links to all their events for no effort eg. http://corko.orienteering.ie.
5. When results are uploaded they are inserted into a database. Only basic information is stored - name, club, class, course, times, splits.
6. Once in the dB, there are some basic search facilities. The IOA can also extract some statistics such as the number of competitors per year, number of competitors who have only been to one events etc.
7. We also have an automatic league results system which uses the data in the dB (http://orienteering.ie/wiki/doku.php?do=show&id=io... for examples). The stick for organisers is that their event will not count for the league unless they upload their results.

Of course, as a National Governing Body, we do have some control but we have found that clubs are very happy with the system. I think the key is that the system is easy to use and actually saves them work. It also means that results can be found in a single place and format, plus they do not suddenly disappear when a club does a spring-clean.

we don't really have such problem: there's one software provider (no need to pay) who requests to upload all the data to his website. so we can see all the results on one website (also they are collecting old results (including international competitions) which are not online anymore and uploading them as well)
website

Thanks for sharing the way it is done in other places. But I would like to revert discussion to the Subject:

Anybody has any policies on data sharing?
If I, or somebody else, ask Ireland or Lithuania database admin to share all his data in some (simple and convenient for both) format, what answer I would get?

I think that what we learned is that you cannot rely on people to perform an optional task, such as uploading data to a central database, when they have already put results on their club web site. Some will, some will not bother and many will not even know to do so.

If, however, you can make it worth their while, they will. It's a matter of convenience. In our case, we made it easier than what they were doing beforehand. For many events it is mandatory. This means that people are used to uploading their results.

Now that we have all the data stored centrally and in a consistent format, anything is possible. If a club wanted to create a system like yours, it would be relatively straightforward.

I'm not sure if this is very useful to you though, as it would require your National or Regional body to orgainse the central dB. Of course, if you were to volunteer to create the system, they might take you up on the offer:-) The benefits to everyone are significant.