Did Flawed Data Lead Track Astray on Testosterone in Women?

http://bit.ly/2zxFXnN
July 12, 2018 at 09:42PM

Citing Data Flaws, Researchers Question the Basis of Track’s New Testosterone Rules

The I.A.A.F. has argued that rules governing testosterone levels are needed to level the playing field and to reduce an unfair advantage.Ben Hoskins/Getty Images

Researchers have found flaws in some of the data that track and field officials used to formulate regulations for the complicated cases of Caster Semenya of South Africa, the two-time Olympic champion at 800 meters, and other female athletes with naturally elevated testosterone levels.

Three independent researchers said they believed the mistakes called into question the validity of a 2017 study commissioned by track and field’s world governing body, the International Federation of Athletics Associations, or I.A.A.F., according to interviews and a paper written by the researchers and provided to The New York Times.

The 2017 study was used to help devise regulations that could require some runners to undergo medical treatment to lower their hormone levels to remain eligible for the sport’s most prominent international competitions, like the Summer Games.

The researchers have called for a retraction of the study, published last year in the British Journal of Sports Medicine. The study served as an underpinning for rules, scheduled to be enacted in November, which would establish permitted testosterone levels for athletes participating in women’s events from 400 meters to the mile.

“They cannot use this study as an excuse or a reason for setting a testosterone level because the data they have presented is not solid,” one of the independent researchers, Erik Boye of Norway, said Thursday.

The I.A.A.F. has updated its research, which was published last week, again in the British Journal of Sports Medicine. “The I.A.A.F. will not be seeking a retraction of the 2017 study,” the governing body said in a statement on Thursday. “The conclusions remain the same.”

But the statement did little to dampen criticism by the independent researchers. The I.A.A.F. seems “bound to lose” an intended challenge by Semenya to the Court of Arbitration for Sport, a kind of Supreme Court for international athletics, said Boye, a cancer researcher and an antidoping expert.

The I.A.A.F. has argued for years that rules governing testosterone levels are needed to level the playing field and to reduce an unfair advantage gained in some women’s events by athletes with so-called differences of sexual development. The 2017 study was only one facet of 15 years’ worth of field study, the I.A.A.F. said.

Dr. Stephane Bermon, the I.A.A.F.’s senior medical and scientific consultant and a co-author of the 2017 study, last week acknowledged some errors in the data in an email sent to one of the independent researchers. But Dr. Bermon added in the email, obtained by the Times, that the mistakes “do not have significant impact on the final outcomes and conclusions of our study.”

Karim Khan, editor in chief of the British Journal of Sports Medicine, did not respond to emails seeking comment.

The disputed 2017 study in the journal examined results from the 2011 and 2013 world track and field championships. It found that women with the highest testosterone levels significantly outperformed women with the lowest testosterone levels in events such as the 400 meters, the 400-meter hurdles and the 800 meters, which distill speed and endurance.

But in examining the study’s results from those three races, plus the 1,500 meters, the three independent researchers said they found that the performance data used in the study’s analysis was anomalous or inaccurate 17 to 33 percent of the time.

The errors included more than one time recorded for the same athlete; repeated use of the same time for individual athletes; and phantom times when no athlete could be found to have run a reported time. Also included were times for athletes who were disqualified for doping.

“I think everyone can understand that if your data set is contaminated by as much as one-third bad data, it’s kind of a garbage-in, garbage-out situation,” said one of the independent researchers, Roger Pielke Jr., the director of the Sports Governance Center at the University of Colorado.

Referring to the I.A.A.F., Pielke said: “I really see no option for them other than to retract the paper. If they retract the paper, then the regulations don’t have a scientific basis.”

After reading the revised study on Thursday, Pielke noted that the I.A.A.F. was now acknowledging there were 220 errors in performance data found across every women’s track and field event at the 2011 and 2013 world championships.

“This is an effort at what I would call a do-over, and it’s embarrassing and it’s not how science is expected to be done,” Pielke said. “I think this adds considerably more weight to our call for the original paper to be retracted. This is everything but putting up a billboard saying, ‘We really screwed up the data in the original study.’”

Another of the independent researchers, Ross Tucker, an exercise physiologist who specializes in sports performance at the University of Cape Town in South Africa, agreed that the first study should still be retracted. The re-analysis, he said, included “too much uncertainty to trust.”

A re-analysis of the original study might, in fact, make even a stronger case for the I.A.A.F.’s position on the need to regulate testosterone levels, the independent researchers said in interviews. But any new analysis should be conducted only with a full independent audit and with publicly available performance data that could be replicated by independent scholars, the researchers said.

The I.A.A.F. has lacked transparency in providing and presenting its data, said Boye, the Norwegian researcher, who described the governing body as “doing everything with their hands over the data.”

Given the data errors, the original study is “entirely untrustworthy” and “an impossible position” for the I.A.A.F. to defend, said Tucker, the South African researcher.

He added, “If I was on Semenya’s team, this would be among the best news I could receive.”

If the challenge to the study succeeds, this would be the second major setback for the I.A.A.F. in trying to set testosterone limits.

In 2015, the Court of Arbitration for Sport suspended a previous I.A.A.F. rule, saying the governing body had not sufficiently quantified the performance advantage gained in women’s events by elevated testosterone levels. That case involved an Indian sprinter named Dutee Chand.

The challenge to the 2017 study is the second time this week that the I.A.A.F. has come under criticism. More than 60 current and former elite athletes, including the tennis legend Billie Jean King and the soccer champions Abby Wambach and Megan Rapinoe, signed an open letter in support of Semenya, calling for the pending testosterone rules to be rescinded.

The athletes described the regulations as discriminatory and invasive, arguing that “no woman should be required to change her body to compete in women’s sport.”

But the I.A.A.F. has said that, in some events, athletes with differences of sexual development could have a performance advantage of 5 to 6 percent over athletes with testosterone in the typical female range, an enormous difference in a sport where events can be decided by hundredths of a second.

The pending rules would affect women with testosterone levels of five nanomoles per liter and above. Most women, including elite athletes, have natural testosterone levels of .12 to 1.79 nanomoles per liter, the I.A.A.F. said, while the typical male range after puberty is much higher, at 7.7 to 29.4 nanomoles per liter.

No female athlete would have natural testosterone levels at five nanomoles per liter or higher without so-called differences in sex development or tumors, the I.A.A.F. said. In effect, it has said that athletes with such elevated levels are biologically male.

But the scientific discussion has now been somewhat sidetracked by questions about the independence and validity of the I.A.A.F.’s research.

“You don’t have drug companies doing their own studies that no one else can see,” Pielke, the Colorado researcher, said.

Advertisement

via NYT > Science