Can we do anything for judging transparency?

[JoseM]

NOT releasing individual judges scores protects them from being pressured to score a certain way by their federation or friendly federations. Olympic assignments are determined by how you do on a number of designated competitions during the Olympic cycle. If you publish scores, another judge willing to judge more according to the federation's wishes will be sent.

I can't imagine any of you calling for public scores have ever experienced the harassment from coaches disagreeing with you - starting at the lowest levels where it is usually quite obvious what scores you gave.

Do reléase the individual scores, but not the judges names. Reléase the scores, but keep the judges anonimous. What is needed is the score, not the identity of the judge. Its that simple.

José M.

 

[cats]

I don’t think releasing individual scores tied to judges names, or even anonymized J1/J2, would really do much in terms of making the sport more fair.

Releasing itemized deduction lists would be far more interesting and much more likely to actually move the performances in line with what the FIG wants. I’d also much rather talk about whether a mistake was a .1 or a .3 than who the judge was or which total score was closer to right.

 

[RichardL]

I don’t think releasing individual scores tied to judges names, or even anonymized J1/J2, would really do much in terms of making the sport more fair.

Releasing itemized deduction lists would be far more interesting and much more likely to actually move the performances in line with what the FIG wants. I’d also much rather talk about whether a mistake was a .1 or a .3 than who the judge was or which total score was closer to right.

Agreed but I also don't think doing both would hurt anyone. We don't have to choose!

 

[Denn]

I do think that scores should be released, but I don't think it will change the results too much. It's more of a principle. There are more important issues with gymnastics judging than this.

@cats what you are asking for is such an enormous undertaking that it would never happen. Think about what would need to be done for that to happen! A single judge could score possibly hundreds of exercises in a single competition. Who's going to type all that up? (or even clean up scribbles done while watching routines so they are discernible to others)

NOT releasing individual judges scores protects them from being pressured to score a certain way by their federation or friendly federations. Olympic assignments are determined by how you do on a number of designated competitions during the Olympic cycle. If you publish scores, another judge willing to judge more according to the federation's wishes will be sent.

Judges talk with each other all the time about scores, though. And hiding the score doesn't stop friends from wanting generous scoring. The only benefit here is that a judge can try to lie to sound like they were more generous than they were, and I don't think a rule should stay in place to support that. Moreover, the scores DO become public sometimes — either a big mouth A tells person B about gymnast C's score. Or, as we saw, leaks.

I can't imagine any of you calling for public scores have ever experienced the harassment from coaches disagreeing with you - starting at the lowest levels where it is usually quite obvious what scores you gave.

Honestly, I never had issues with this because (a) I tried to do the right thing 100% of the time, (b) I accepted that talking with coaches is part of the job, and (c) most people know that there's going to be differences of opinion. Coaches would often come up to me to discuss scores during and after meets, but 9 times out of 10 it was respectful — they were just trying to understand things better. Occasionally, someone would say they disagree but that was it. It wasn't harassment.

Granted I know there can be pushy coaches — I never was the target of that, but I remember at one competition, a certain very well-known coach and TV commentator had an issue with my judging partner's refusal to give a stick bonus. LOL.

 

[Aeris]

@cats what you are asking for is such an enormous undertaking that it would never happen. Think about what would need to be done for that to happen! A single judge could score possibly hundreds of exercises in a single competition. Who's going to type all that up?

It should be done the same as figure skating. Every move of the routine is input into the computer by a technical controller and judges use a computer to click on each move and assign a deduction amount, and be able to have video replay as well.

 

[makam]

what you are asking for is such an enormous undertaking that it would never happen. Think about what would need to be done for that to happen! A single judge could score possibly hundreds of exercises in a single competition. Who's going to type all that up? (or even clean up scribbles done while watching routines so they are discernible to others)

This might be where AI can come to play. Or a combo of scribble the move and hit the button for .1, .3, .5 off. If the scribbles are done on a tablet, they can be registered immediately and cleared until the end and then listed for confirmation and validation. AI could be trained to recognize the scribble (or the judge gets retraining on how to scribble) and names the skill, attatches the deduction, and the coach gets copies at the end of the comp.

 

[FrolovasDoubleLayout]

If anyone has used the hand-held judging device it works in a fashion to above. Judges hold a Nintendo Switch-like device that has buttons for 0,1, 0,3, 0,5 and 1,0. The gymnast gets the green light and the judges just click deductions. The exercise is videoed and the camera and input devices work together. There is a time-line that corresponds to the exercise so you can actually see what the average deduction(s) is(are) for the elements as the exercise progresses. As a judge you need to be sure of your deduction(s) as the grace period is a mere couple of seconds. Pro is that it is incredibly fast to get the E-score, you just press send at the end (you are given a small period of time to enter the artistry/composition deductions to add on for BB/FX. Challenging to give feedback since you are not scripting, though I did see some judges trying to script and click using both hands - I'm not that trusting of my physical dexterity for that one. I think that the goal is to provide a print-out of the routine with the element values and the corresponding deductions. We trialed it along side the E-panel at an FIG Challenge Cup and it was interesting that the test group E-score was usually within 0,10-0,20 of the actual E-panal score. It is an Elevian initiative and I do hope that it takes off in the future.

 

[Denn]

@Aeris This has been tried and it doesn't work. It's not figure skating where there are SUBSTANTIAL gaps of time between elements. In gymnastics, particularly UB, SR, PH, PB, and HB, the elements keep coming and trying to sync this live — or even after the fact — is too hard.

@makam Maybe you have an idea there. Hmm... Something video oriented might work, too.

ETA: @FrolovasDoubleLayout beat me to it!

 

[JoseM]

As I said before, release the scores, but not the judge’s names.

Also, increase the amount of judges, to keep the judges anonymous and produce a more fair score.

José M.

 

[Aeris]

@Aeris This has been tried and it doesn't work. It's not figure skating where there are SUBSTANTIAL gaps of time between elements. In gymnastics, particularly UB, SR, PH, PB, and HB, the elements keep coming and trying to sync this live — or even after the fact — is too hard.

It definitely hasn't been tried like I am talking about. As a judge you would notate the routine as it happens and then at the end start entering the official scores into the computer, giving the tech controller enough time to enter the routine into the computer and provide video replay.

 

[cats]

The E score itemization is definitely a solvable problem. Judges already have to do the hard part, which is identifying all the deductions. We’re really talking about a data entry issue here, yeah?

 

[Denn]

The E score itemization is definitely a solvable problem. Judges already have to do the hard part, which is identifying all the deductions. We’re really talking about a data entry issue here, yeah?

Yes, but more specifically, the alignment of D and E scoring to each other.

I was part of a small group that tried building an app to do this, and I know others tried as well. (And did it differently.)

Entering specific elements in a timely way is super difficult because there are so many elements in gymnastics. Skating has a way easier list of elements, and there's way more time to enter elements and align deductions. Doing this with a pommel horse routine or even a bar routine (PB/HB/UB) is haaaarrrrrddddd. We tried making a searchable list of elements, and it helped, but it was just way too hard to do fast enough for a competition with tons of gymnasts where you have to get scores in.

So... we ended up just having the D-score judge list the difficulty values of the elements (A, B, C, D, E). Meanwhile the E-score judge(s) clicked deductions small, medium, large, and 1.0. Like @FrolovasDoubleLayout 's experience, we had a place for element groups, neutral deductions, and composition deductions. We also made it so that the judges could edit after.

The problem was the alignment between the D and E panels. As soon as a D-panel judge fumbled with the app, it messed up the E-panel judges. Or the E judge would have to shift all the deductions over if the D-judge missed an element, usually an A of some kind. So there were little arrows to shift everything left or right. Or sometimes, a skill is not an element and the E-judge would want to deduct and had to do it to the previous element. (We did this with juniors, where they get an A for almost everything, even if it's not in the Code, so that wasn't a big problem actually, but it would be for FIG scoring on the men's side, especially FX).

Also, if the D-panel judge put the wrong value, it threw off the E-panel. They could talk, of course, but it was bumpy. There are lots of skills on FX that have no letter, and pommel horse deductions sometimes bleed from one element to the next — a judge might put it one place or the other sometimes. Also, the E-panel judges found themselves "rolling up" deductions instead of listing them all separately, e.g., writing 3 instead of 1,1,1 because it was faster to click and didn't require looking down to check. ...Same score in the end, but less useful to the gymnast later.

We didn't even get into what the deductions were for, which would have been useful.

One place it DID work better was compulsories — the routine could be listed on screen and the judge could just tap deductions for each element. But even that required looking down a lot more than they would if they were just judging on paper — where you don't need to look at the paper at all (and honestly should not).

The idea of E-scoring in real time on a video instead while a D-panel judge indepdently does difficulty, like @FrolovasDoubleLayout mentioned, sounds better. It avoids the alignment issue completely, and it also makes a nice, tidy way to package the information for the coach/gymnast later. Nowdays, with video being so easy to do, it seems like a good method. That said, I would like to see how they deal with deductions that are clicked on the transition between two elements. How does the human and / or how does the computer decide which element to put it with? For example, let's say a gymnast does RO / BHS / double full, and the back handspring is sloppy and the judge taps the deduction a little too late in real time and it lands on the double full? They go back at the end and check everything, essentially rewatching the whole exercise? That could be time-consuming for a big meet....

 

[Aeris]

Entering specific elements in a timely way is super difficult because there are so many elements in gymnastics.

It really shouldn't be that difficult. Every element should have an abbreviation that the tech controller types as it is being performed. Every element also has a number, if the controller prefers to enter it that way. For example:

TK - tkatchev
TKP - tkatchev piked
TTKL - Toe tkatchev layout
STKH - stalder tkatchev with half-turn

It's also possible to implement voice input software. Speak the routine into the computer: "kip mount, cast, giant full, tkatchev, pak, cast, maloney, gienger" etc