At many stages during your medical training or career you may want to (or have to) perform a research study. For some of you, this is the opportunity to sink your teeth into solving the unknown – a step towards innovation and impact. For others, this is just another hoop to jump through on your ascent to other professional aspirations. Regardless of the ‘why’, performing research is hard but starts with a crucial step. Formulating the research question.
Creating a focused and answerable research question
Expert problem solvers often discuss that their first step is not focusing on solutions, but rather thinking about the problem itself. Research is similar in that the solution (the study design, methodology, and statistics) are secondary to what question you want to answer. In fact, creating an excellent research question will often dictate the methodology needed to solve it. There are two main things to focus on when creating a research question: 1) creating a focused question 2) creating an answerable question. Lets dive into each of these.
Creating a focused research question
There are many ways to develop focused research questions, but all rely on clearly defining the population, intervention/exposure, comparison, and outcome you want to study. This has been summarized into the PICO framework, which can be modified as needed for different study designs. The key to this framework is it forces researchers to focus on the individual components of the question.
PEARL #1: Most research questions are not focused enough (initially).
Lets dive into PICO and see how it can be used to focus research questions:
Population: The population refers to all people in a defined setting or with certain characteristics. This differs from the sample which is a subset of people from a population that are studied to make inferences about the broader population.
When defining a research population here are a few things to consider: 1) age 2) disease including indicators of severity 3) comorbidities 4) hospitalization status
Intervention/Exposure: Whether you use intervention or exposure as your I/E in PICO (PECO) depends on whether you are performing a controlled trial or an observational study.
An intervention in a controlled trial is something you are giving to one of the trial arms. It could be a new drug, technology, or really any type of therapy. When specifying the intervention, it is important to define the exact dose, root of administration, and duration of therapy.
An exposure is the term used in an observational study. The exposure is any factor one part of your population is exposed to, that you want to see if it is associated with developing an outcome of interest. For example, the exposure of smoking and its association with the outcome of lung cancer. Alternatively, an exposure could be a defining characteristics of part of a population (e.g. biological sex). In that situation, you may be interested in knowing whether the male sex is associated with increased rates of cardiovascular disease.
Control: In an interventional study, the control is the group that does not receive the intervention. This could be a placebo in a placebo controlled trial, or could be routine standard therapy. During observational research, the control group is referred to as the ‘unexposed’ group.
Outcome: The outcome is what you are trying to measure. This could be an outcome like mortality, quality of life, morbidity, or anything else you are interested in seeing whether the intervention or exposure results in. Some outcomes are distal (e.g. mortality) because there are many steps between an exposure/intervention and the patient developing that outcome. Other outcomes are proximal (e.g. change in blood pressure) because an intervention may directly impact the outcome.
Although the PICO framework is a good starting point, certain research questions are better suited to other approaches. For diagnostic accuracy research, you can use the Population, Index Test (the test you are studying), Reference Standard (the gold standard test you are comparing against), and Outcome (e.g. sensitivity, specificity, accuracy) framework.
Regardless of the exact framework you use, focusing your question is the first step in creating the perfect research question. Unfortunately it doesn’t end there – there are many questions that are focused but not be answerable in your setting within the real-world constraints of doing research. Lets dive into the often overlooked question of how to create an answerable research question.
Creating an answerable research question
When I first started doing research I had grand ideas of identifying the unsolved problems in my specialty, and designing flawless, unimpeachable, trials that would definitively answer the question and change practice. Unfortunately, I quickly realized the complexity of performing quality research, and also the limitations of thinking of research studies in such a isolated fashion.
PEARL #2: Individual research studies are unlikely to definitively answer a research question.
Think about where your study sits on a research ark
Every large randomized controlled (RCT) trial is built upon observational studies, pilot studies, and other smaller RCTs. The collection of studies needed to definitively answer a research question can be conceptualized as a research ark, where early studies focus on biological mechanism, feasibility, identifying differences in proximal outcomes, and then as evidence in the field grows, later trials focus patient important outcomes.
This differs from how many clinicians view research, where at a trial level we often think of studies in a very binary fashion: positive or negative. This reductionist approach to research does clinicians and researchers a disservice, as it disregards the evidentiary baseline leading up to RCTs.
What does this mean for you designing a research study? Simply, you need to ask yourself, do you want to do one study and be finished, or do you want to explore this field or topic in a systematic fashion? Both are acceptable, but have important implications for what preliminary questions you may choose to answer.
If you want to create a program of research in a given field, consider starting with an evidence synthesis study (systematic or scoping review), an observational study of local data, or even consider a prospective study focusing on feasibility of recruiting patients. These will give you crucial information on the disease prevalence, ease of administering study procedures, recruitment rate, and baseline demographics. This will help you determine the feasibility, and potential impact of additional prospective research. It also may help inform you whether you are actually like the research question enough to spend years of your life to answering it!
If you are looking to do a one-off study, that’s okay too! The key is though – you won’t be able to definitively answer questions about the impact on distal outcomes (e.g. mortality, hospitalization) as many of these will have small effect sizes and require large sample sizes to recruit. Instead, consider focusing on questions related to mechanism, feasibility, biological plausibility, as even with a small sample size, these can contribute to the field and set the stage for future prospective studies.
While these are just general guides, they serve as a starting point when thinking about creating answerable research questions. It is better to robustly answer an important preliminary study, than to stall out part way not answering anything at all. The next thing to consider when creating an answerable question is the feasibility of your research question.
Determining the feasibility of your research question
The next question to ask yourself about any research question, is it feasible to answer in my setting. If this is tough to gauge, it means you might benefit from collaborating with a colleague or mentor.
To determine feasibility, the first question I ask myself is: “has something similar been done before in my setting or institution?”. If a similar study has been done, I reach out to the research team to learn more about their experience running the study. The unseen challenges and limitations are important, and often not present in traditional publications or presentations.
Next, I ask myself several focused questions: 1) what is my anticipated recruitment rate? 2) what is the prevalence of the outcome? 3) what is the effected effect size (of treatment or exposure) 4) what is the time frame I am willing to commit? 5) what is my budget (and need for funding). If the answer to any of 1-3 is “I don’t know”, a small feasibility study focusing on answering those questions might be a good first step. That way, instead of starting to recruit 200 patients (to find out you can only recruit 20/year), you can recruit 20 patients as a feasibility study and change your study population or research questions as needed.
Strategies to creating focused and answerable research questions
You have made it this far in the post, congrats. Here are a few concrete strategies that will help ensure you create focused and answerable research questions.
- After you create a research question, create a research protocol that describes exactly (in detail) how you will go about answering your question. If you are having trouble with this, it may be a sign that your research question is not focused and/or answerable. This is often a great reality check.
- Start small and focused. Don’t underestimate the potential impact of a well done, focused, and highly relevant research study. It might not change practice overnight, but will set the stage for definitive research.
- Avoid the temptation of only focusing on a very distal outcome like mortality, as the event rate (and potential impact of an intervention) is likely to be small, requiring large sample sizes to detect differences.
- Collaborate widely. Having experienced colleagues and mentors on your team will enable you to learn, focus your research question, and ultimately increase the likelihood your study has the impact you desire.
Good luck on your research endeavors, and if you have questions or comments feel free to reach out to me at @ross_prager on twitter, or at ross.prager@simplesage.io. Remember, it all starts with the perfect question.
Ross Prager, MD