Point of View: Teaching troubleshooting skills to graduate students

Researchers are routinely confronted with experiments and equipment that do not work as expected. A negative control, for example, might yield an appreciable signal, or a piece of equipment might fail to produce any signal at all. Diagnosing and fixing such problems – a process known as troubleshooting – is, therefore, an essential skill for any researcher (Jonassen, 2000; Verderame et al., 2018). However, PhD students rarely receive formal training in troubleshooting, and are expected to acquire this skill ‘on the fly’ as they progress through graduate school (Barnard and Shultz, 2020). In this article, we describe an approach called Pipettes and Problem Solving that has been used to teach troubleshooting skills to graduate students at the University of Texas at Austin.

Our approach is designed to be similar in nature to a journal club, with more experienced researchers passing on their knowledge to those with less experience at a meeting that lasts between 30minutes and an hour. Each meeting is led by a different member of the research group who, prior to the meeting, draws on their own experiences to create a scenario involving an unexpected experimental outcome. The overall goal of the meeting is for the group to work together to identify the source of the problem.

In this article, we describe how we have implemented Pipettes and Problem Solving at our own institution (Figure 1) and provide resources that will help other researchers experiment with this approach in their own labs and departments (see Table 1). These resources span a range of topics in the biological and chemical sciences. Moreover, we are confident that this approach could be extended to other fields of research.

Figure 1

Download asset Open asset

Two types of scenarios can be proposed by the meeting leader: the first involves an experiment with a known outcome that is returning atypical results (such as a negative control giving a positive result, and vice versa). This type of scenario is useful for teaching the use of appropriate controls, troubleshooting fundamentals, and instrument/laboratory/analytical techniques. Such scenarios are also helpful in illustrating how researcher-driven shortcuts, material aging, and general randomness can all contribute to an unexpected result.

The second type of scenario is more complicated in that the ‘right’ outcome is not known in advance. Most of the problems that require troubleshooting in research labs are of this second type. A typical example might be developing a new assay and demonstrating that it works. The assay is unlikely to work as intended the first time around, so it will be necessary to perform additional experiments with different controls, and/or characterize various compounds, strains or samples, before attempting the original experiment again. This type of scenario is useful for teaching hypothesis development, proper controls, new analytical techniques, and more advanced troubleshooting.

In preparation for either type of scenario, the leader must create 1–2 slides describing the hypothetical experimental setup and mock results from this experiment. The leader must also prepare hypothetical background information that could be useful for troubleshooting. For example, when were the relevant instruments last calibrated or serviced?; what are the temperature and humidity levels in the lab?; and what are other researchers in the hypothetical lab working on?

The leader presents the slides containing the experimental setup and the mock results to start the meeting, and then answers any questions the students have about the setup (such as timings, concentrations, temperatures, solvents and so so). Students can also ask questions throughout the meeting – ideally these will be fairly specific questions that the leader (as the person who ‘performed’ the hypothetical experiment) will be able to answer, possibly using the hypothetical background information they have prepared. The leader is encouraged to not answer questions that require an amount of subjectivity (Figure 2).

Figure 2

Download asset Open asset

A major challenge for the students is to understand the science behind the hypothetical experiment: with only 1–2 slides at their disposal, the leader is not able to go into detail about the scientific background underlying the experiment, so the students are encouraged to research and discuss the topic during the meeting.

The students must then propose a limited number of experiments to help them troubleshoot the hypothetical problem. To foster conversation and collaboration, the students in the group must reach a full consensus on the proposed experiments. The students are also encouraged to discuss all possibilities and to settle on a single experiment to do first. The leader, who knows the source of the problem, then provides mock results for the proposed experiment. Based on this information the group can then either guess what is causing the problem, or propose another experiment. After a set number of experiments (typically three), the group must reach a consensus on the source of the problem, after which the leader will reveal the results.

The number of experiments can be increased or decreased at the discretion of the leader. The leader can also reject experiments that they deem to be too expensive, dangerous, or time-consuming, as well as experiments that require equipment that is not readily available at the host institution.

The students are encouraged to ask questions and work together; however, they should avoid asking questions like “is it the xxxx?” or “did the yyyyy go ‘bad’?”, as the only way to answer such questions is to run additional experiments! Conversely, they could ask the color of a solution or the time something was purchased. When proposing experiments the students are not allowed to iterate on the experiment in a way to circumvent the failure. Although proposing a new experiment to circumvent the problem could be an interesting exercise, it would not help the group to identify the source of problem, which is the purpose of Pipettes and Problem Solving.

There is often an element of chance or luck in research, and bad luck is sometimes the reason why an experiment does not work. For instance, mechanical noise might create a strange signal in one experiment, but not another, so we encourage leaders to include such seemingly mundane sources of error (such as contamination, miscalibration, instrument malfunctions and software bugs). In such cases we suggest that the leader gives the group a hint about the problem being mundane. The leader should keep in mind that the goal of the exercise is to foster troubleshooting instincts, not to ‘stump’ the group.

Another twist that can be added is to use dice to reflect various aspects of the research process. For example, the leader could roll dice to determine how long the proposed experiment could take: a low number could mean that the group has to propose an experiment that can be done in a day, whereas a high number could mean that the experiment is allowed to take a week or longer. A similar approach could be used to determine the budget for the proposed experiment. Adding game-related elements like dice can improve participation, particularly among less experienced group members.

An effective round of Pipettes and Problem Solving should take between 30 and 60minutes. The students should be challenged, but not so much as to decrease their confidence. If the meeting appears to be taking longer than the allotted time, the leader can intervene to provide additional time, information, or support in the form of hints – however, it should remain a balanced and student-led discussion.

To give readers a better idea of how Pipettes and Problem Solving works in practice, we describe an actual meeting at the University of Texas Austin in which the leader describes a hypothetical cell viability assay (MTT Assay; see example 1 in Supplementary file 1). In the hypothetical experiment, a researcher is interested in the cytotoxic effect of a protein aggregate on human neuroblastoma cells; however, unexpectedly, there are very high error bars, and much higher than expected values. The leader described the workflow and presented simulated data with a high amount of sample variance. The rest of the group started by returning to the plotted data and discussing two major points: Were the appropriate controls run? What are the culturing conditions of this particular mammalian cell line? The group began researching both questions, and discussing what elements should be included in the first experiment.

The first discussion was around finding an appropriate negative control in the form of a cytotoxic compound. They discussed that an ideal compound would exhibit a range of behavior from low- to high- cytotoxicity. Next, the discussion turned to the culturing of the particular cell line, which was revealed to be dual adherent and non-adherent. The group discussed considerations of culturing dual adherent cell lines and whether this property was contributing to the high degree of variability. It was at this point that someone who had experience with mammalian cell viability assays mentioned the source of the error: aspiration of cells during the washes. This was taken up by the group dialog which led to a conversation surrounding the wash steps.

The group proposed only two experiments, despite discussing the scenario for nearly an hour: the first was to remove wash steps and to look only at a negative control instead of the protein aggregate. However, this introduced a new source of error (background noise from fetal bovine serum (FBS) and phenol red). The group quickly realized their error after looking at the detailed protocol and decided that the washes were strictly required. They examined the possibility that insufficient washing was the initial source of noise the entire time.

The second experiment they proposed was to add an additional wash step and to specifically use a pipette to aspirate the supernatant. They emphasized taking care to examine the cell density after each step, and several people mimed appropriately slow aspiration mentioning things like “placing the pipette on the well wall” and “slightly tilting the plate”. In addition to the wash step, the group proposed this experiment be done with both a negative control and the protein aggregate in order to directly compare the two.

Ultimately, the source of error was user generated, and by focusing on the aspiration technique the group was able to uncover the source of error and achieve appropriate results. The inclusion of a negative control in the final experiment allowed them to see that the assay was functioning correctly and that the protein aggregate cytotoxicity study yielded useful data. Importantly, we note that there was also a discussion about particular lab techniques including specific motion and speeds, which is a conversation that might not have been facilitated otherwise.

Pipettes and Problem Solving was developed in October 2022 largely because of challenges with training new graduate students during and after the COVID-19 pandemic. This student-led meeting takes the form of sharing interesting methods, journal articles, and discussing particular instrumentation. Since the advent of Pipettes and Problem Solving, we have added it as a regular component to group meetings and have performed a session roughly once a month for almost two years. Between 3 and 10 graduate students, undergraduate students, and postdoctoral researchers attend per session. Participation is optional, but most of the group chooses to attend. It has also been piloted at another two research groups at our university.

We have found that the challenge of a troubleshooting scenario, with the near instant gratification of hypothetical experimental results, is a very engaging environment for graduate students, postdocs, and undergraduate students alike. A senior graduate student in a lab that had recently tried Pipettes and Problem Solving for the first time said: “It stimulated a lot of great discussion both from newer lab members thinking about problems they haven’t experienced yet and from experienced lab members who were able to easily share their thought processes behind troubleshooting”. In our lab, it has provided the opportunity to talk about specific instrument quirks, common lab errors, and potential experimental pitfalls, preserving this information as students graduate and transition into new roles (Jonassen and Hung, 2006).

1. 1. Barnard RA
   2. Shultz GV

   (2020) “Most important is that they figure out how to solve the problem”: How do advisors conceptualize and develop research autonomy in chemistry doctoral students?

   Higher Education 79:981–999.

   https://doi.org/10.1007/s10734-019-00451-y

   • Google Scholar
2. 1. Jonassen DH

   (2000) Toward a design theory of problem solving

   Educational Technology Research and Development 48:63–85.

   https://doi.org/10.1007/BF02300500

   • Google Scholar
3. 1. Jonassen DH
   2. Hung W

   (2006) Learning to troubleshoot: A new theory-based design architecture

   Educational Psychology Review 18:77–114.

   https://doi.org/10.1007/s10648-006-9001-8

   • Google Scholar
4. 1. Verderame MF
   2. Freedman VH
   3. Kozlowski LM
   4. McCormack WT

   (2018) Competency-based assessment for the training of PhD students and early-career scientists

   eLife 7:e34801.

   https://doi.org/10.7554/eLife.34801

   • PubMed
   • Google Scholar

Author details

1. Gina Partipilo

   Gina Partipilo is in the McKetta Department of Chemical Engineering, University of Texas at Austin, Austin, United States

   Contribution

   Conceptualization, Writing – original draft, Writing – review and editing, Refined the protocol, wrote example scenarios and led example sessions

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-6353-3419

2. Yang Gao

   Yang Gao is in the McKetta Department of Chemical Engineering, University of Texas at Austin, Austin, United States

   Contribution

   Refined the protocol, wrote example scenarios and led example sessions

   Contributed equally with

   Alexis J Holwerda, Yassir Lekbach, Ismar E Miniel Mahfoud, Trevor R Simmons, Bailey M Tibbett and Rebecca E Wilen

   Competing interests

   No competing interests declared

3. Alexis J Holwerda

   Alexis J Holwerda is in the Interdisciplinary Life Sciences Graduate Program, University of Texas at Austin, Austin, United States

   Contribution

   Refined the protocol, wrote example scenarios and led example sessions

   Contributed equally with

   Yang Gao, Yassir Lekbach, Ismar E Miniel Mahfoud, Trevor R Simmons, Bailey M Tibbett and Rebecca E Wilen

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-0911-3533

4. Yassir Lekbach

   Yassir Lekbach is in the McKetta Department of Chemical Engineering, University of Texas at Austin, Austin, United States

   Contribution

   Refined the protocol, wrote example scenarios and led example sessions

   Contributed equally with

   Yang Gao, Alexis J Holwerda, Ismar E Miniel Mahfoud, Trevor R Simmons, Bailey M Tibbett and Rebecca E Wilen

   Competing interests

   No competing interests declared

5. Ismar E Miniel Mahfoud

   Ismar E Miniel Mahfoud is in the Interdisciplinary Life Sciences Graduate Program, University of Texas at Austin, Austin, United States

   Contribution

   Refined the protocol, wrote example scenarios and led example sessions

   Contributed equally with

   Yang Gao, Alexis J Holwerda, Yassir Lekbach, Trevor R Simmons, Bailey M Tibbett and Rebecca E Wilen

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0003-0373-8337

6. Trevor R Simmons

   Trevor R Simmons is in the McKetta Department of Chemical Engineering, University of Texas at Austin, Austin, United States

   Contribution

   Refined the protocol, wrote example scenarios and led example sessions

   Contributed equally with

   Yang Gao, Alexis J Holwerda, Yassir Lekbach, Ismar E Miniel Mahfoud, Bailey M Tibbett and Rebecca E Wilen

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-2198-6863

7. Bailey M Tibbett

   Bailey M Tibbett is in the McKetta Department of Chemical Engineering, University of Texas at Austin, Austin, United States

   Contribution

   Refined the protocol, wrote example scenarios and led example sessions

   Contributed equally with

   Yang Gao, Alexis J Holwerda, Yassir Lekbach, Ismar E Miniel Mahfoud, Trevor R Simmons and Rebecca E Wilen

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0009-0000-3174-5134

8. Rebecca E Wilen

   Rebecca E Wilen is in the McKetta Department of Chemical Engineering, University of Texas at Austin, Austin, United States

   Contribution

   Refined the protocol, wrote example scenarios and led example sessions

   Contributed equally with

   Yang Gao, Alexis J Holwerda, Yassir Lekbach, Ismar E Miniel Mahfoud, Trevor R Simmons and Bailey M Tibbett

   Competing interests

   No competing interests declared

9. Marcus S Benyamin

   Marcus S Benyamin is in the McKetta Department of Chemical Engineering, University of Texas at Austin, Austin, United States

   Contribution

   Wrote example scenarios

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-7081-9914

10. Benjamin K Keitz

    Benjamin K Keitz is in the McKetta Department of Chemical Engineering, University of Texas at Austin, Austin, United States

    Contribution

    Conceptualization, Resources, Supervision, Writing – original draft, Project administration, Writing – review and editing

    For correspondence

    keitz@utexas.edu

    Competing interests

    No competing interests declared

    "This ORCID iD identifies the author of this article:" 0000-0003-3314-0053