»Case Studies«
Case Study 1: Truth or Consequences
Case Study 2: In the Field, No One Will Know
»Case Study 1: Truth or Consequences«
Julio Cruz and Samantha Bergen are both graduate students working with Dr. Mark Chan, an eminent environmental chemist. Although both are in their fourth year of study, neither has published a manuscript. Both are beginning to worry that if they do not publish soon they will not be able to obtain good postdoctoral positions.
Finally, Julio's project starts to look promising. After many months of effort, he believes he has been able to synthesize RG198, a compound that is to serve as an intermediary in the formation of his thesis molecule, WX5, which he believes will degrade plastic in an environmentally sound way. Julio now has to repeat his experiment to make more RG198 and perform a series of analyses on the compound to verify some of its properties.
Dr. Chan is very excited about Julio's progress, and tells him to repeat his experiment and to begin to write up the results, because even the synthesis and some properties of the intermediate molecule are unique enough to be published in an important journal, such as Nature.
Although only small amounts of RG198 are available, Julio and Dr. Chan agree that they must push ahead and work quickly. In order to help Julio as he works on manufacturing more RG198 for the next set of experiments, Dr. Chan recruits Samantha to assist Julio in some analyses. Samantha has not been very successful with her project, which involves transforming asbestos into a non-toxic compound, and Dr. Chan feels that performing the analyses will teach her some skills that she could apply to her own project. Dr. Chan promises her a second authorship on the paper if the results of her analytic studies pan out. Although Julio does not think highly of Samantha, believing her to be sloppy, he wants to move ahead with his research. He gives her the RG198 in two batches for the analytic studies.
Samantha completes the first set of analyses on the first batch and is excited by the results, which verify three of the four chemical groups that RG198 is supposed to have. On the next batch, Samantha performs another set of experiments, using another analytical tool that will identify the fourth chemical group. On the day she is doing the first experiment on the second batch of RG198, she phones Julio from the analytical facility across the street from the lab and asks him if a contaminant might have gotten mixed up in the compound, since the spectral pattern is not what is expected for the molecule.
Julio asks Samantha to save the remaining material from the second batch, telling her that he will perform the second round of analyses. But when Samantha comes back to the lab a few hours later, she does not give him the leftover RG198. She tells Julio that she obtained positive results and that her mistake in the original interpretation was due to tiredness, and to the fact that she had focused inadvertently on a reference sample, not on RG198. There is no way for Julio to validate her findings, since there is not enough RG198 left to do another run. Samantha tried to reassure Julio by showing him the graphical readout from the instruments from the experiments on the second batch, pointing to the results for the fourth chemical group.
Dr. Chan is ecstatic about the findings, and tells Julio to quickly write up a manuscript. Julio doesn't want to accuse Samantha of manipulating research results, but later in the day he looks through her research notebook and sees a written procedure and data for the first batch of experiments. For the second batch, he sees that she has put only the readout in the notebook, which looks too clean to him. It also has no accompanying text. He wonders what might have happened. Perhaps she used a reference sample and some mechanical manipulation to make the fourth chemical group peak appear so pure.
Julio is unsure about whether he can trust Samantha's findings, but he proceeds to write up the manuscript about his synthesis of RG198 and its analysis by Samantha. The article is published in Nature, but in the next several months other scientists who repeat his synthesis are finding different spectra than what he reported in his second batch of experiments. During that time, Julio has been able to synthesize more of the compound, and even succeeds in making WX5. When he repeats the analysis on the fourth chemical group in RG198, he finds a different spectral pattern from what Samantha found and what was published. He believes that she must have done something to the data.
»Case Study 2: In the Field, No One Will Know«
LaToya Johnson and Sandra Rajeev are first-year graduate students in social work at a major research institution in New York City. They are working with Dr. Francine Lockheart, who specializes in studying the effects of homelessness on children. Dr. Lockheart has received a major federal grant to study the education of homeless children in New York City compared with children who are poor but have more steady housing. The study will follow children over a five-year period, from the fifth to the tenth grade.
A significant part of the grant involves fieldwork, in which investigators must go to homeless shelters and obtain informed consent from parents so that they and their children can participate in the research project for the duration of the study. Researchers must get permission to review the educational records of the children, and also must get the same informed consent and approvals from the parents of the children who have homes. Part of the study involves parents filling out a survey of information about themselves and the children with an interviewer. Another part involves an educational test of the children, to validate the school records. Both the survey and the test are specifically designed to be performed in the field.
Besides LaToya and Sandra, Dr. Lockheart has a staff of five paid surveyors and five paid testers, to make it easier to obtain a cross section of children from the five boroughs of New York City. Dr. Lockheart holds a training session for the 12 people involved, explaining to them the research tools and the importance of accuracy and honesty in reporting the results of the surveys and tests. Each of the surveyors and testers is given a wireless laptop computer, which allows them to input the data and transmit it to a main computer as soon as the surveys and tests are performed.
Each interviewer or tester is expected to handle 25 children in one year, for a total of 300 children. LaToya is an interviewer and Sandra is a tester. Dr. Lockheart had hoped to remove the potential for tester bias by separating the surveying aspect of the study from the educational testing part. The testers, who do not know whether a child is homeless or not, meet the children at a school after the surveyors get the demographic information. Dr. Lockheart also tells the surveyors and testers that their work will be monitored or verified, to serve as a deterrent to falsification.
LaToya is assigned a section in Queens and Sandra is assigned Staten Island. The two young women are excited about participating in such an important study. After the first month on the job, they talk to each other about the experience, and are saddened by the circumstances of the children in both groups. Sandra also says that she feels guilty about exploiting the children, since the results of the study will come out only after it would be possible to help them. Sandra discusses her feelings with Dr. Lockheart, who is sympathetic, but she explains that the only way society can know whether there needs to be more resources for these children is to do these kinds of studies.
Sandra agrees, but she still feels that she needs to do more to help. So although she knows that her work might be validated by someone else and that she is not supposed to ask the child she is testing whether he or she is homeless, she does so anyway. She decides to change a few of the right answers from the homeless group to wrong answers and a few of the wrong answers from the children from homes to right answers. She figures that her minor changes really won't make a difference in the overall study results, because she thinks that the homeless children probably will fare more poorly, educationally, than the children who come from homes. After another tester repeats the test with the same children, the results of the two are averaged.
When Dr. Lockheart analyzes the data from the first year of the study, she finds that homeless children in Manhattan and Queens actually do a little better on their educational tests than their counterparts from homes, and that homeless children in the Bronx, Staten Island, and Brooklyn don't do as well. Dr. Lockheart said she was surprised by the findings in Staten Island, because the amount of money spent per capita on providing educational services for homeless children is the same as in Manhattan and Queens, and more than the amount spent in the Bronx and Brooklyn. Other demographic factors and results from other studies also seem to suggest that the homeless children in Staten Island should have done better than the study found.
Although the results are preliminary, politicians have begun to discuss the results in the press and are asking that more money be spent in Staten Island, Brooklyn, and the Bronx. Sandra realizes that her actions probably contributed to the brouhaha and tells Dr. Lockheart what she did. They redo the testing of the Staten Island homeless children and find, in fact, that the Staten Island homeless children fare as well as the Manhattan and Queens homeless children. When the truth comes out, a furor erupts within the university, in the press, and at the federal funding agency.
Continue to the next section: → Questions & Answers