FIG and Fujitsu discuss "Judging Support System"

[KristyJ]

I disagree that human judges should only need to judge artistry, because I do think that sometimes technically correct can be incorrect in some way unless we somehow write the perfect code of points. I do, however, very much agree that we should have half-tenth deduction increments and that a gymnast shouldn’t lose credit for a skill that is 1% shy of a turn plus gain the execution deduction on the lower value skill.

Technique and artistry are seen as separate things for the purposes of so much in the code, but they should not be separate. Technique should support aristry; artistry should flow at least partly from great technique. A skill done with exceptional extension and amplitude is artistic. And a skill generally cannot be as aesthetically pleasing if it is done with failing technique.

 

[Doug1233]

I don’t disagree with that. I guess my point is that we can write the perfect code (which calculates difficulty and execution) and humans can judge artistry (which could include overall impression of technique of elements plus content of dance and transitions).

 

[cats]

I hope the computer system abandons the whole idea of bucketed deductions for many things. For example I think it would be fine to award some small fraction of a point per centimeter of height and distance achieved on vault instead of bucketing - that guarantees some differentiation in an event that is sorely lacking it. It also would be relatively easy to define that fraction by vault family, so you don’t have judges marking down a Tsuk because it didn’t get as high as a Yurchenko.

If that concept works it could be extended to many areas. It’s always bugged me that the Code is supposedly objective but written in a way that both puts big limits on differentiation via bucketing and is also impossible for a human judge to actually achieve.

 

[makam]

Could you imagine if HB (or UB) gave points for stuff like “air time”? Like, this routine was 47 seconds long and you were off the bar(s) for 12 of them. +.4 bonus!

 

[onodian]

I’m sure we’d see dumb releases in all shapes and forms plus constant hop 1/2 and hop fulls

 

[KristyJ]

To be fair, I miss hop fulls (though seeing them in everything would not be great either)

 

[navyblue]

At the same time, at some point we take it to the edge of absurdity. Is it so bad to give an athlete the benefit of the doubt if they hit 179.9 degrees on an otherwise absolutely stunning leap that displays artistry, technique, toe point, and fluidity? There’s a point where taking that .1 is almost unfair, and I think that is what Denn means. (Of course, correct me if I am wrong, Denn!)

I don’t think this is an issue introduced by the robots, though. I’ve seen judges on this website arguing that a gymnast didn’t hit 180 degrees in “real time” (meaning, according to their failed perceptive faculties, because certain gymnasts like Leanne Wong often hit an over-split for a very brief second, which should technically fulfill the requirement, but doesn’t look the same as someone like Anna Pavlova who practically levitates in her split position). I think some judges are also being harsher on pirouette angles in bar routines than they would be with an actual digital protractor in use. So the cyborg world isn’t necessarily more punitive, and I think they would do a better job differentiating routines, because one element of human bias that creeps into all judging is a sort of conservatism that causes a lot of score clumping, where it becomes practically inconceivable that someone like Rebeca Andrade on bars or Konnor McClain on beam could earn a 9.0 e-score simply because that just doesn’t happen anymore.

 

[navyblue]

I feel judging is always adapting to whatever effects the code happens to unleash upon us, like the judging tends to change during the course of a quad (think “body posture” all of a sudden in Tokyo, whereas I think you can argue some of those deductions weren’t being taken in Stuttgart)

This only seems to me like another striking example against human judges in this debate. If the body posture deduction is written into the rules for a quad, it should be deducted identically in 2017 and 2020, and it shouldn’t be incumbent on gymnasts to respond in real time based on the intra-quad vicissitudes of technical committee directives (like “surprise, our pet peeve for this Worlds is heel position on floor turns!”). They should be able to work within a consistent set of codified rules, at least for 4 year chunks, which is already pretty damn short. If a quick change is required, it ought to be formally announced with a bit of headway both to allow for changes to the software and for gymnasts to adapt.

 

[KristyJ]

This is a fair point, as well, that sometimes judges are being unnecessarily strict. I did like the suggestion above that perhaps an actual mathematically graded scale could be taken in accordance with the degree away from perfection. If 45 degrees off is .3, then one degree off would be .007. It would result in weird decimal places, sure, but it would be far less boxed in and more precise.

I do think the example of body posture may be one of those things that is hard to put into AI though. All of the changing angles would make that much harder to measure without cameras everywhere, and it’s not a deduction that is a snapshot in time. Also posture can look different on different bodies.

 

[onodian]

Partially, yes. What happens too though, is that the judges react to things the gymnasts bring in through the quad (for example, I have no receipts but I would say the wolf turn proliferation had a say in the body posture thing). So then you would end up probably updating whatever it is the system is looking at.

I don’t think that’s necessarily a bad thing, other than, the gymnasts and coaches should know what they’re up against.

 

[KristyJ]

Coaches should also be either watching the judging errata or working with a judge who can, as well. If they are not I would argue that is a degree of negligence on their part.

 

[onodian]

I do think the example of body posture may be one of those things that is hard to put into AI though. All of the changing angles would make that much harder to measure without cameras everywhere, and it’s not a deduction that is a snapshot in time. Also posture can look different on different bodies.

Do we know how are they building the system? Is it through machine learning (so, feeding it tons of judging results and letting it “sort it out” -great description I just gave there of machine learning) or are they codifying a set of rules and ideals that the system looks for?

 

[KristyJ]

Not entirely sure.

And yes, machine learning would help, but I think we’d still need that set of human eyes to check things such as that posture deduction. That would especially be the case earlier on in the process but may never totally be perfect. Also, we’d need cameras from a lot of different angles for aspects of judging such as that one.

 

[Doug1233]

I’ve seen judges on this website arguing that a gymnast didn’t hit 180 degrees in “real time” (meaning, according to their failed perceptive faculties)

I think the majority of the board saw short shrift to that argument. Although, of course, you can give your “real time” score and note that, after the fact, it was wrong.

 

[makam]

I think they would do a better job differentiating routines, because one element of human bias that creeps into all judging is a sort of conservatism that causes a lot of score clumping, where it becomes practically inconceivable that someone like Rebeca Andrade on bars or Konnor McClain on beam could earn a 9.0 e-score simply because that just doesn’t happen anymore.

It is like when people want the perfect 10 back. Someone would have to confirm, but a quick internet search says that the last 10 given in international elite was in 1992. I don’t recall ever seeing one live. So for a decade+, gymnastst were aiming for a thing that simply didn’t exist anymore. No matter how well you did your routine, you simply never were going to get a 10. It feels cruel to hold up an impossible to achieve number and tell the hardworking atheletes that is what they should strive for.

I could get behind deductions based on a percentage deviation from ideal. Crossed toes (if the computer can see it), .05, crossed knees, .1-.4 depending on how akimbo the lower legs are, legs all over the place, .5-1. Foot shuffle: .05, foot little step: .1 or .2, all other steps based on distance from initial touchdown point.

I am imagining future gymnasts in unitards with dots all over them like those mo-cap suits.

 

[YurchenkoLoop]

Kui Yuanyuan got a 9.875 for this routine in the '96 TF. I honestly think without the step on the dismount and two minor adjustments after her tumbling series, there’s a chance the judges would’ve given her a 10–and the hardest apparatus to get a 10 on!

 

[navyblue]

Sure, my main point is just that these hypothetical robot judges wouldn’t necessarily be harsher than the human judges of today. And if it worked as expected, we wouldn’t have the scenario that had like a 5 tenth scoring differential on beam between the Olympics and last major pre-Olympic meet (2019 worlds).

 

[RAFIKI]

Someone would have to confirm, but a quick internet search says that the last 10 given in international elite was in 1992.

If we’re not limited to WAG then Urzica got a 10 on PH at the 2001 Glasgow Grand Prix.

 

[Denn]

I think a lot of this would be solved by allowing judges to take a 0.05 and a 0.2 deduction. In reality, though, I think most do. I certainly do. I’ll often ignore a very minor error that technically I should give a 0.1 for, and later combine it with another very minor error, and just give a 0.1 for the second one (and feel less bad about it, because I forgave the first one). Of course, against the rules. But I’m sure most judges do it. They just don’t say they do it.

Yeah, they do and they even say it. I heard numerous judge colleagues say at competitions while we were talking, say, between sessions.

 

[Doug1233]

Fujitsu Judging Support System expanding to 10 apparatuses

The FIG has approved the official use of the Fujitsu Judging Support System (JSS) for all apparatuses at the 2023 Artistic Gymnastics World Championships in Antwerp (BEL).

www.gymnastics.sport