Even with new rules in place, the scoring of Olympic figure skating competitions could declare undeserving winners.

skate.jpg Credit: Elaine Lanmon

Michelle Kwan’s much-lamented exit from Torino may not be the only tragedy that befalls this year’s Olympic figure skating competitions. Jay Emerson, an assistant professor of statistics at Yale University believes the new rules for scoring could create results that may be unfair and potentially contestable. Emerson published his critique of the new system—inaugurated at the 2004 World Championships— on his website earlier this week.

In past Olympics, scores were more subjective and prone to bias, often based on the order when a skater skates or a judge’s idea of what medal a particular skater deserves rather than what points they scored for certain elements in their program.

The reformed system, introduced in response to a controversy at the 2002 Salt Lake City Olympics that resulted in the awarding of two sets of gold medals during the ice skating pairs competition, is harder to manipulate. It awards points for technical elements, based on a prior analysis of each element’s level of difficulty— a triple axel gets more base points than a double axel—as well as five other criteria: skating skills, transition or linking footwork, performance/execution, choreography/composition and interpretation.

In the new system, a computer algorithm randomly disregards three of the 12 judges’ scores. Then, from the nine-score subset, the highest and lowest scores are dropped, a relic from the old system, which was installed to correct for favoritism.

Emerson claims that this judging process could lead to undeserved rewards or punishment if, for example, the computer happened to pick the nine judges most in favor of one skater, or most opposed to another. He also pointed out that he is not the first one to find fault with the changes to the scoring system. Katherine Godfrey, a professional statistician with a PhD from Harvard, lodged a complaint in 2003 about an interim system that fell on deaf ears.

“Different combinations of nine judges would yield slightly different scores,” Emerson said via e-mail. “If a competition is close, the medal standings could change. So far, I’ve shown this has been a possibility in past competitions. Fortunately, the actual medal results for competitions I’ve studied have successfully reflected the consensus of the full panel of twelve judges. But this is, to some extent, no more than good luck.”

To illustrate his objection, Emerson examined the scores for the Ladies’ 2006 European Figure Skating Championships. Russia’s Irina Slutskaya walked away with first place, but the scores for the next four competitors—Yelena Sokolova, Sarah Meier, Elena Gedevanishvili and Carolina Kostner— were very close to one another. The scores were calculated according to the new judging rules, and out of the 220 possible groups of nine judges, Emerson says, only 50 would have awarded the medals in the same order as the actual group did.

“Only Slutskaya’s standing was secure,” Emerson writes in his critique. “Each of the other skaters could have been placed as high as 2nd or as low as 5th in the short program.”

As it turned out, the group of nine judges picked the same order that the entire group of 12 would have picked, lending some credence to the official results. Still, as Emerson notes, such a close race could potentially have painful results at the Olympics this year.

“I think that’s something people would find pretty dissatisfying,” said Eric Zitzewitz, assistant professor of economics at Stanford and author of an earlier paper on nationalist bias in figure skating scoring.

Zitzewitz agrees with Emerson that the new rules could result in skewed standings.

“Basically what you’re doing when you take 12 and throw out three is you’re taking the average of the 12 and adding noise to it,” he said. “That doesn’t seem to make sense.”

Zitzewitz said an equally, if not more, problematic aspect of the new rules is the anonymity for judges, which, instead of eliminating bias in any kind of systematic way, takes away transparency. When a skater finishes a program, there’s no more listening to the scores from each country’s judge. All the scores are lumped together into a single figure.

“They’ve reduced accountability for the judges, because now they can be as biased as they like without any public accounting for it,” he said.

After all, it was the accountability of the old system that fingered the French judge for fixing the 2002 Olympic pairs competition and necessitating a new method in the first place.

Download podcast

Originally published February 16, 2006

Tags

Share this Stumbleupon Reddit Email + More

Now on SEEDMAGAZINE.COM

  • Ideas

    I Tried Almost Everything Else

    John Rinn, snowboarder, skateboarder, and “genomic origamist,” on why we should dumpster-dive in our genomes and the inspiration of a middle-distance runner.

  • Ideas

    Going, Going, Gone

    The second most common element in the universe is increasingly rare on Earth—except, for now, in America.

  • Ideas

    Earth-like Planets Aren’t Rare

    Renowned planetary scientist James Kasting on the odds of finding another Earth-like planet and the power of science fiction.

The Seed Salon

Video: conversations with leading scientists and thinkers on fundamental issues and ideas at the edge of science and culture.

Are We Beyond the Two Cultures?

Video: Seed revisits the questions C.P. Snow raised about science and the humanities 50 years by asking six great thinkers, Where are we now?

Saved by Science

Audio slideshow: Justine Cooper's large-format photographs of the collections behind the walls of the American Museum of Natural History.

The Universe in 2009

In 2009, we are celebrating curiosity and creativity with a dynamic look at the very best ideas that give us reason for optimism.

Revolutionary Minds
The Interpreters

In this installment of Revolutionary Minds, five people who use the new tools of science to educate, illuminate, and engage.

The Seed Design Series

Leading scientists, designers, and architects on ideas like the personal genome, brain visualization, generative architecture, and collective design.

The Seed State of Science

Seed examines the radical changes within science itself by assessing the evolving role of scientists and the shifting dimensions of scientific practice.

A Place for Science

On the trail of the haunts, homes, and posts of knowledge, from the laboratory to the field.

Portfolio

Witness the science. Stunning photographic portfolios from the pages of Seed magazine.

SEEDMAGAZINE.COM by Seed Media Group. ©2005-2012 Seed Media Group LLC. All Rights Reserved.

Sites by Seed Media Group: Seed Media Group | ScienceBlogs | Research Blogging | SEEDMAGAZINE.COM