>>Please note, today’s session is being recorded. If you have any objections, you may disconnect
at this time. Thank you very much and on behalf of the National
Cancer Institute, I wish to welcome everyone to the June Advanced Topics in Implementation
Science Webinar. Today we are delighted to welcome Drs. Stamatakis,
Shelton, and Craddock Lee, who will be joined by our own Dr. David Chambers to this campfire
session. A brief word about logistics and we’ll be
off. We ask that if you are not already on mute
to please keep your phone on mute for the duration of today’s presentation. As mentioned, the session is being recorded
and muting all lines will help us to avoid any background noise. We encourage questions, they can be submitted
using the Q&A features on the right hand side of your screen. Type your question in the provided Q&A field
and hit submit. Feel free to submit your questions at any
time. We’ll be moving through your questions and
some we have for our guests throughout today’s webinar and with that, I will turn it over
to David.>>Thank you Sarah, and here you see the smiling
faces that we are able to learn from today. For those of you who haven’t attended these
series of webinars that we’ve had, this has been our attempt to move a little bit away
from some of the more didactic sessions that we did in previous years, to have more in
depth conversations with various leaders and in this case at different stages of their
career. We found that the ability to learn from the
wisdom of those who entered the field at a particular point and what they found there,
how they’ve seen the field of implementation science evolve, and what advice they have
for others who are following in their footsteps has been really well received by our community,
and we think that today’s session is a wonderful complement to those that came before. We started out in the dead of winter I guess,
with the fireside and now as we are finally in summer, although those who have been tracking
temperature might feel like we got there a little bit ago, we move more to the campfire
and so again, the idea of this is to be more of an informal conversational opportunity
both to learn from those folks, and I have a set of initial questions to get us going,
but then also to feature your thinking and what you would like to hear from our esteemed
guests. And as Sarah had indicated, we have three
wonderful earlier career investigators I would say with us, Katie, and Simon and Rachel. Just a few words, so Katie was actually part
of our first Training Institute for Dissemination and Implementation Research in Health back
in 2011, and at the time I believe was at Wash. U. and is now Assistant Professor at St. Louis
University. Katie’s work has really focused on trying
to understand evidence based practice implementation within public health and learning from the
experiences of those working in local Health Departments to give broader insights that
help us as we think about how do we move our evidence based to the benefit of public health
practitioners. Simon Craddock Lee, who joined us in 2012,
the second edition of TIDR, which was over on the west coast in San Jose, I believe,
was a Cancer Prevention Research Fellow here at the NIC, is as you see at University of
Texas Southwestern Medical Center, and his work has focused on screening, and survivorship
in underserved populations. He’s also played some leading roles within
the CTSA in thinking about community engagement and we’re grateful for his participation as
well. And then you’ll see on the right, Rachel Shelton,
who was part of our fifth cohort in 2015, which was again in California in Pasadena
and Rachel, who is at Columbia University at the Mailman School of Public Health, has
worked a lot on the roles of lay health advisors in reach underserved populations, has also
focused on communication and decision making to try and motivate better care for colon
cancer patients, and has had a particular recent focus in thinking about how do we advance
our understanding of sustainability. So welcome to the three of you and thanks
so much for joining. Want to get it kicked off with a sense of,
for each of you, entering the field what was it that inspired each of you to pursue work? Why did implementation science call out as
an area that you wanted to receive training and dive into and as I see Katie, and then
Simon, and then Rachel, why don’t we go in that order so again, in chronology of TIDR
and here, you see the first of our many campfire scenes to come so Katie, do you want to tell
us a little bit about what led you to work in implementation science?>>Yeah thank you David and thanks for the
explanation of that picture on the screen. I was afraid I’d have to try and explain that. I think what initially inspired me was that
I was seeing from the public health side of things, that so much of what seemed to be
successful in public health interventions seemed to be very setting specific and it
made me start to think about context in a different way and to wonder if there were
certain features of implementation setting that could themselves be modified to improve
successful programs and then ultimately community outcomes. And then also, I had mentorship in DNI early
on from senior people that were already working on dissemination and implementation science
and so I had inspiration from some senior researchers and that really helped spur me
on.>>Great, Simon do you want to tell us a little
bit about your entry into the field.>>I stumbled into implementation science
and dissemination or at least DNI from that perspective. I mean, as an anthropologist I went through
a bit of a crisis of conscience leaving the NCI and coming out to an academic medical
center where you know, the days of purely observational RO1s are probably long gone
or at least few and far between and anthropologists have this sort of prime directive thing that
we joke about which is we watch and we don’t intervene, which means that that really limits
what you think you’re doing in proposing an RO1 and I realized that people called dissemination
and implementation different things in different traditions so we’ve talked about in Canada
the language that they use for knowledge transfer but I stumbled on it by thinking about evaluation
in anthropology where applied anthropologists do a lot more in evaluation so the idea for
me was without giving up the methods that I feel like I have the strongest grasp of,
that I could evaluate before, during, and after an intervention and assuage my sort
of initial gut reluctance to engage and intervening, but then I had to think about who could partner
with me to help think through that.>>And Rachel, same question for you.>>Sure, so I think two things came together
to really inspire my work in implementation science. One, during my doctoral studies at Harvard
I did a pre-doctoral fellowship in cancer prevention and control at Dana Farber Cancer
Institute and I was really fortunate that a lot of the early leaders in implementation
science were based there so I think as early as 2007 or 2008, I had spent a day long training
on the topic and that really got me inspired. There were national leaders from the field
and I think that was eye-opening for me that there was this whole other layer that I wasn’t
considering when I was thinking about social and behavioral interventions so there was
that piece, kind of the mentorship and inspiration. And then, when I did my post-doctoral work
at Mount Sinai School of Medicine, I started doing a lot of work on lay health advisor
and patient navigation programs and I developed this great partnership with The National Witness
Project which is this community based, evidence based lay health advisor program that was
one of NCIs RTIPs, the Research Tested Intervention Programs and my research was really driven
by their need. They were finding that they had huge issues
in implementing and sustaining the program you know, despite their success and it was
really important to them to understand this better so I pulled together my resources and
I wrote an RO3 that as funded and so I kind of rolled into it partly because of their
interest and need and really developed my research program from that. And at the time, I didn’t really have the
language or training in implementation science but I think in effect that’s what I was working
on and then kind of developed that as I went.>>Cool yeah so it seems like again, each
of you had slightly different starting points and also it seems like when you entered the
course, the specific TIDR training, each of you were at a little different point in terms
of your familiarity with some of the terms, some of the concepts, some of the tools and
resources and so along those lines, I just wanted to get a sense from each of you of
how have you perceived since entering the field, how do you perceive that the field
has changed over those years of working in it? Maybe we’ll go back to Katie, do you want
to kick us off, we’ll go sort of same order Katie, and then Simon, and then Rachel?>>Sure, so I guess thinking back a few years,
it seemed like a smaller field early on and because of that, I think there was a lot of
focus on it being an emerging field and it seemed like a lot of kind of self-reflective
discussions happening at DNI meetings about the fact that it was an emerging field. And then, all of the things that go along
with that in terms of maybe fewer established experts from whom early stage investigators
might seek mentorship, a perceived and perhaps real lack of institutional supports particularly
in academic settings for doing DNI research, inconsistency in terminology and then you
know, as far as methods an expressed need for research designs other than RCTs and ways
to approach the inherent complexity in delivery systems that DNI work targets. And so, I think those were some of my reflections
of kind of where the field was early on and I think some of those have changed. I mean you know, the terminology I think still
is something that the field struggles with to some extent but has become more consistent. I think there are ways and a lot more examples
now of people that have been successful doing really rigorous work and recognized for doing
rigorous work that doesn’t necessarily fit in that RCT gold standard framework anymore
so I guess those are some of the changes that I’ve seen.>>Gotcha great, Simon your thought?>>I’d agree with that. I think for me, what impressed me was sort
of the explosion of interest in measures and within that, or I guess above that, a much
deeper approach to thinking about how you frame where your research question is within
the continuum of say healthcare delivery so the interest I think in trying to advance
that rigor was balanced by I think a lot more nuance in slicing up the continuum and recognizing
that there were a lot more unanswered problems than I think I certainly had initially realized. And the challenge I think, for me, has been
balancing the rigor of advancing an implementation science or dissemination science without losing
track of the reason we want to implement in the first place which is to change practice,
but to change practice in the right way that’s evidence based that’s therefore replicable
and generalizable. But I think the nuance in efforts at measuring
and thinking about the larger frameworks within which you approach your research question
has really exploded.>>Cool, Rachel?>>Yeah, so I think I originally got involved
in the field around 2008 to 2010 and I agree like, at that time there was a lot of initial
excitement about the field and I think the field is really just growing and defining
itself and I started a course in 2013, and I remember I was so excited that there was
a textbook that had come out, Ross Johnson’s [assumed spelling] textbook around that time,
because it really helped me coalesce a little bit more in terms of definitions, and methods,
and I really feel like that’s around the time where it really started to coalesce a bit
more and I’ve really noticed in the last five years it’s really grown from being this smaller
niche area to one where really everyone across disciplines and topical areas, I feel like
there’s an interest in really expanding work in this area and there’s definitely greater
institutional commitment from schools, and deans, to department chairs, and faculty and
I feel like on a regular basis now I’m really approached about different opportunities to
get involved. And I’ve even seen in my course you know,
it was originally mostly health promotion and [inaudible] students and now I have people
docs, post-docs, [inaudible] students, health policy students, people from the medical school
so it really signifies that there’s interest across disciplines and across topical areas. And I agree, I think that while there’s still
room for improvement in terms of kind of coalescing around the terminology and the methodologies
and the theoretical frameworks and models, I do think there is an evidence base that’s
really developed a bit more around what works.>>Great and so, given that we have a fair
number of representatives across the field listening to us, each of you as you said have
seen the field grow and this is your chance certainly to give your sense of what you see
as particular priority areas for research that you’d like to see the field undertake
so we’d love to get each of your takes on what would you see particularly for folks
listening or others, what would you encourage people to be thinking about for that next
set of research studies? Why don’t we start with Simon and then we’ll
go Rachel and then Katie.>>I knew there’d be a switch there.>>Yeah [inaudible].>>I’m going to cheat and go back to something
that Katie said though, because I think you know, the issue of trying to define the field,
I don’t know if it fits right here but the lead in to where I’m thinking is I think that
if we can be sort of practical for early stage investigators, there are sort of two different
phenomenon of trying to get funded by NIH and one is recognizing that the program announcement
that’s calling for dissemination and implementation research is different from the study section
called dissemination implementation research and health. And I think in terms of establishing the field,
there was a really big push early on to create that evidence base and demonstrate that DNI
was a real thing with a rigorous science behind it but I think, and the fastest way to do
that for example, would be to push [inaudible] randomized trials but I think I’m really eager
to see more work, as Katie said, tying back theory to quasi-experimental designs that
I think really help us vary in how we think about applications in particular settings
because not everything is ready. We don’t necessarily know enough about how
an evidence based intervention is going to work in a particular setting to then start
running towards looking for data for generalized ability so I think trying to parse your research
question and decide what is it you’re in the next three to four years trying to solve or
at least shed light on because we can’t do everything in each grant application but I
think trying to be very clear about what your goals are would really push the field forward.>>Okay gotcha, and so Rachel, thoughts on
the same theme.>>Yeah, and I actually agree with Simon there. I was just doing a review on factors that
influence sustainability across settings and I think initially you know, previously we
just saw kind of broadly what factors influence but I think it’s really important, especially
for early stage investigators like Simon said, to start to develop what’s their setting focus
going to be. Is it going to be more community, is it going
to be more clinical, because I think those are very different in terms of what implementation
science looks like and is carried out so I do think that’s an important point. I think one really important issue that’s
been evolving over the course of this field developing is this tension between adaptation
and fidelity so you know, we’ve traditionally seen this idea that you have to keep to the
core components with strict fidelity but there’s really been growing consensus that some adaptations
of evidence based interventions might be beneficial particularly for sustaining interventions. So I think a really important question that’s
going to involve a lot of, I think, partnerships across faculty members and across systems
is to start to understand and tackle this question of what kinds of adaptations are
useful and which ones are detrimental. I think that’s one really critical area and
the other really critical area I think that’s been kind of wrapped up in dissemination and
implementation science but not always pulled out separately is sustainability. You know, I think it’s often thought of we’ll
follow up in one year and see what’s happened but I really think that the field needs to
invest and to understand much more in terms of what the long-term impact is of evidence
based programs, what can sustain them, and as the scientific evidence evolves over time,
how do we think about again de-implementation or de-adoption of what’s not working so I
think those are important questions that the field needs to tackle that you know, we don’t
have a lot of answers for so those are areas that are kind of ripe for early stage investigators
to start focusing on.>>Great, Katie your thoughts?>>Yeah, I guess I have maybe two different
ways of answering the question and the first I think also relates back to what Simon was
saying so as far as priority areas for research, you know the major emphasis of the field is
really testing DNI strategies and then thinking about early stage investigators, this may
not be something that’s immediately feasible for an early stage investigator to jump right
into. There’s a lot of formative research that has
to be done perhaps before you’re ready to test a particular DNI strategy and so, as
far as research and funding priorities, I think making sure that some of that formative
research, that it’s prioritized, that there’s [inaudible]. And then, I think in my field one of the things
that I’ve been really excited to see as far as change is that I see in population health
and public health that DNI is really starting to become more integrated with intervention
work and integrated with other theories of behavioral change that are now expanding frameworks
beyond interventions that act solely on individual characteristics and individuals to lead the
behavior change but also looking at the context of implementation settings and where those
individuals might be aggregated or the prevention setting that they may come in contact with
through certain interventions. So, I’m excited to see how that continues
to develop.>>Katie, thank you for that answer. I’m just going to jump in and remind our attendees
that you can submit your questions using the Q&A feature on the right hand side of your
screen. We do have more speakers today and so in order
to sort of give you as many perspectives as we can on each question, we want to make sure
that we are receiving those early and often if you have them so feel free to submit. We have more questions for them but we’d love
to hear from you all as well.>>Yeah, absolutely and while you’re thinking
of those and while you’re typing them into the Q&A feature of the website, one of the
reasons why we thought it would be particularly helpful to have the, and thank you for our
first person who has already sent it in and we’ll get to that you know, for the three
of you is that you each applied for and completed training in dissemination and implementation
research, implementation science. Wanted to have a sense from you as we’re into
summer, we’re in the midst of certain summer training institutes, we’re also looking ahead
to the next version of our TIDR training, to learn from you in terms of those experiences
of completing that training, what sticks out from your training experiences as helping
you to move your ideas forward and are there specific lessons that you’ve learned that
you feel like through that training improved your success in getting research support? Why don’t we start with Rachel and then we’ll
go to Katie and then Simon?>>Sure, so I did the 2015 training and honestly,
I had doubts about applying originally because I had developed a course in this area years
before that and I was always doing work in the area, but a mentor of mine had convinced
me that it would be really critical because of kind of the in person relationship building
and I’m so glad I did. It was honestly one of the best mentoring
experiences that I’ve had. I think the in person and not only developing
relationships with peers but the mentors and the leaders in the field and the program officers
was really, really critical in terms of me just feeling more comfortable asking questions,
feeling like I knew the content, getting the in person feedback on my aims, being able
to go to leaders in the field and ask them to become investigators or consultants on
my studies or even just bounce off ideas or strategize so I think that really gave me
the confidence that I knew the content but that I could also approach them and that I
had the relationships in place. And I think again, just having that in person
commitment and getting feedback on the aims, even though the projects really developed
after the TIDR training, I was still able to kind of reach out to them and work with
them over the course of the year before I submitted my RO1 which was so, so helpful. And then, I think it also gave me credibility
as an expert within my institution so I think part of the mandate of the TIDR training was
that you really come back to your institution and you know, kind of present yourself to
the school as I’m here as this resource for you, I’m doing presentations on it, and I
think that this really allowed me to kind of have the credibility again to be an expert
within my institution and it gave me a platform for doing so, so I really appreciated those
aspects of it.>>Katie?>>Yeah, I think for me the TIDR training
really, so at that point going back to 2011, I had started doing some work in public health
systems, research, and so what TIDR gave to me was a starting point to really then start
understanding and applying theories in implementation science to some of my research interests and
so really helped me formalize research questions in that respect and that really helped move
some things forward as far as funding and research projects. And then, I think the other part of training
that I had through an NIH Clinical and Translational Science Award was some additional methods
areas that I wanted to fill out that I thought were important so my doctoral training was
primarily in epidemiology but to address some of the research questions that I was interested
in, in DNI work, I recognized that I needed to apply some mixed methods, I needed to fill
in some training and qualitative research, and also in some system science methods and
that gave me an opportunity to do that as well, which has been really helpful.>>Great, Simon?>>I would say you know, I think one of the
challenges we have in field is that you know, the whole premise of becoming an expert is
that you become more and more knowledgeable about less and less and what that means is
in a day-to-day world we don’t all read each other’s journals. What the TIDR program did for me and what
I would urge other people to look not just for TIDR but several of the different offerings
both at the institutional level or the regional level, is that it got me out of the disease
site, organ system specific biases that I think we inherent from how NIH is structured
to realize that for dissemination and implementation, the most interesting piece of science you
could be reading about may not actually be in your disease field, that it’s funded by
another institute happening somewhere else in a different population and potentially
a different setting but they’re asking the same kind of question that you’re grappling
with and I really hadn’t quite wrapped my head around that until I went to TIDR and
I met folks from very different places doing very different work, who were arguing around
the same ideas when we got together as a group. And that was really illuminating because they
could help you go find the literature that you were looking for. Now, that’s even more evolved now as I think
the DNI literature itself and implementation science the journal has evolved so it’s not
as difficult but for me, it was really eye opening and I think particularly conversations
with folks from behavioral health and psychology really helped me think about a lot of the
issues that I was struggling with because they were already thinking about how do they
adapt and the challenges of practitioners delivering manualized training in different
settings that really helped me think analogically and I don’t think that would really have occurred
to me if I hadn’t sort of been doused with listening to people from very different fields
coming together within DNI.>>Speaking of dousing, we just wanted to
update the campfire pictures for folks. The previous one, just so you know is not
a fire sort of uncontrolled, but it was actually you’ll see four different fires representing
the four voices that you were able to see so there we are, okay cool. We are dynamic [laughter]. Thanks to those of you who have already started
to submit some questions so I’m just going to pull in a few of these. One of them that I see right here is asking
given, as you’ve moved on and sort of think about submitting different grant applications,
the question for any one of you is how much success you’ve had with grant applications
that have incorporated implementation research that are reviewed outside of DIRH so do you
have any advice or any experiences where you have a DNI related application but it might
be pegged for a different student section where there may not be that specific expertise
in the membership? Any of you have any experiences to share or
advice to give?>>I’d be happy to jump in. My RO1 with my colleague at the School of
Public Health here at UT [inaudible], we struggled mightily because we were definitely on board
with the DNI language, we were excited to see the establishment of the study section,
we were thrilled to see the PA. And our RO1 did not fare well in that venue,
particularly over the key challenge of cluster randomized control trials which we weren’t
ready for working in a safety net setting in an integrated system. But, it did very well in nursing and related
clinical sciences. It had a very meaningful review I think, thinking
very carefully about how you can use NIH RePORTER to look at funded grants that match your keywords
or match your abstract and look at the funding provenance is really useful because I think
there are different goals. David may have other thoughts about that from
sort of a program perspective but I absolutely would encourage people to think distinctly
between the methods you’re trying to use versus whether you’re trying to advance the field
etc. and some of those challenges.>>Yeah, no I think->>Definitely and, oh->>Please, go ahead.>>I was going to say I don’t have the experience
specifically with other study sections within NIH but I have had luck with American Cancer
Society with two grants and I think one thing that was slightly different that I did compared
to the RO1 application to the DNI study section was instead of writing your background section
where you start with DNI and then skip to the health issue, I still framed the issue
for the ACS application. I started it with the cancer issue and then
I embedded the implementation science piece within that and took a bit more time to define
things and use really simple terms that was maybe not as complex as I would have for the
implementation science study section so, and I have mentors and colleagues who have success
with the query and AHRQ so there are other options outside that study section.>>And just an overall point that at least
that we’ve learned is again you as an applicant, you have the ability to take a look at the
different rosters and the different themes or the specific objectives of different review
committees and once you submit the focus, the sort of locus of control at NIH really
is with the scientific review officer and so it’s absolutely your right as an applicant
to connect with them early on and just ask the questions about are the sufficient numbers
of experts in the key areas of my application on the review committee? Because, even up towards the meeting they
do have the opportunity to bring on ad hoc reviewers and so you might find where if the
initial match doesn’t quite seem right and that could be for DIRH where you feel like
the clinical or the disease, or even the context area might vary from what you see on the review
committee or it’s DNI expertise on those other committees it often can be helpful to at least
ask the question because I know the scientific review officers work incredibly hard to try
and make sure that the reviews have adequate expertise and they’re often quite enthusiastic
to get some guidance as to what are the key dimensions of all of these different applications
that might need particular expertise. So just another thought for folks to consider. I want to go to another question, which I
think came from the recognition that each of you have experience in working with a range
of different settings. The specific question just asked are there
specific resources dissemination implementation and perhaps research outside of the public
health space and more within the acute care setting? Do any of you have any ideas of resources
that might be useful if people are thinking about acute care hospital etc?>>I’m not sure what the questioner was getting
at per resources. My work is in an integrated health system
that’s a safety net. It’s a public county hospital so it’s not
public health broadly lit in the field sense but it is definitely a tertiary hospital with
primary care clinics out in the community so my work is specifically looking at care
coordination between complex cancer survivors and their return to primary care post diagnosis
and we’re struggling with the challenge of patients with a new incident diagnosis when
we’re trying to track them have slightly different issues of care coordination when they’re in
active therapy as opposed to after sort of ambulatory discharge as it were. But I think DNI maps quite broadly from both
sort of community settings if that’s what she’s thinking about in terms of public health
and certainly the broad range of work that Katie’s leading but I think it’s actively
involved in acute care settings, if that’s what the question was.>>Yeah, I think that’s how I interpreted
it and of course you know, questioners feel free to if there’s helpful clarification to
provide. I would note that a number of the different
dissemination implementation frameworks did arise out of looking at efforts to change
clinical practice in healthcare settings and so it may be that as you start to think about
resources related to particular frameworks, that you’ll find there’s more connection to
particular care settings but good question. I think it also underscores what I heard from
each of you just talking about as the field has matured there’s been value in being able
to learn lessons from varying settings and apply different models as well as measures
as they’ve been advanced to think about. Are they getting at the provider level, at
the healthcare system level, at the hospital level etc.>>And for our clinical colleagues, I think
there’s a burgeoning literature in quality improvement which has become increasingly
more systematic and rigorous about their evaluation. I know we all, I think at least many of my
ilk, draw distinction between research and quality improvement but the quality improvement
science is changing and has grown a lot and there’s some really interesting working going
on there too. It’s aimed at a different audience with a
slightly different language yet again.>>Right, let me move on to another question
which and maybe we’ll start with Katie and then Rachel and then finish up with Simon
with anything to add, is there anything that you would have done differently in your doctoral
trainings if you knew you were going to focus on DNI at that point and as a PhD student,
what’s available to help groom students in dissemination and implementation?>>Yes so the first part of that question
for me is an easy one because as an epidemiology doctoral student, much of my training was
really focused on quantitative methods and analyzing large data sets, secondary data
sets, and things that are not really a huge part of my life anymore as a DNI scientist
so I think, as I mentioned, I really sought other ways to fill out some additional methodologies
that I felt like I needed to build up to do that work in DNI as far as qualitative work,
systems, complex systems, methods and systems thinking have been really important to me
as far as filling in some of those gaps. You know and again, really understanding the
utility and the application of theory in developing a research question and framing an approach
to research which for any of you fellow epidemiologists out there, you probably agree is not a huge
part of our core training necessarily. I’m sorry, what was the second part of the
question?>>As a PhD student, what is available to
help groom students in DNI so those I gather who are interested, while a student, is there
anything that you would point to, to help them get going?>>Hmm, well I’m not sure. I would open that question up more broadly
and ask you as well David but I think you know, just as a starting point the journal
implementation science is a really good place just to start browsing, put in some key words,
see some areas of interest that you have and see what work if any has been done in the
field and how it’s discussed, how it’s framed, what methods are used, what are the areas
of expertise of the co-authors that are on those papers as a starting point is one thing
that comes to mind.>>Yeah no, good idea and let me also just
say certainly on NCI’s implementation science website we’ve been trying to curate any number
of resources, Implementation Science the journal is another good one. Also if there is any ability for folks to
join us in December at what will now the be the 10th annual conference on the science
of dissemination and implementation you’ll have like-minded individuals numbering 11
or 1,200 at this point and it’s a good way to network because you do have people coming
from all different work environments, all different settings that they’re interested
in, all different populations and so it’s a great way just to get to see the breadth
of the field. Actually Rachel, if you don’t mind, there’s
a question here that I think relates to a comment that you had made before when you
were talking about areas that might benefit from further research and that’s in your view,
how does cost or coverage for example, business models fit into sustainability? Ultimately, we have to have a way to pay for
interventions to disseminate and sustain them. How does this fit into research careers or
grants?>>I agree and I think this has been a huge
missing issue. I think when we look at implementation science
grants people are starting to increasingly add cost as an outcome but we honestly don’t
know much and the cost not only of the intervention but the cost of the implementation strategies
to implement the intervention across different settings. I think we have the most information about
cost probably in the clinical setting, we probably have the least in the community setting. But in terms of the value of sustainability,
and the cost that’s inherent in sustaining them and what the value of that is, we know
very little. There’s been very little work done on that
and I think that’s a huge priority for sustainability research. I think it’s been a huge missing piece. The work that I have done is mostly on volunteer
run lay health advisor programs and you know, this has been a huge issue in terms of what’s
been able to keep them going is this issue of who’s covering it, what’s the cost? Even if it is a volunteer program, there still
is a cost of the organizational infrastructure, etc. so I think this has been a huge missing
piece particularly in the sustainability literature so we need more work in this area and I would
encourage people to build that into their studies.>>Great, anything else to add Katie or Simon?>>I just wanted to throw in a piece because
what I was thinking about when Rachel first brought up the sort of key question about
sustainability is if you are a graduate student or a trainee, if you are looking for opportunities
to be involved in studies that are at the implementation stage, that’s in your early
sort of in the training phase but when you’re trying to think about a K award or an RO3,
thinking about how to become involved in a team with an eye to the idea that the piece
that is going to be yours will be about exploring a question of sustainability, I think it opens
up, I was excited when Rachel talked about it because it opens up a framework to say
I can build a relationship while I’m a trainee, with an existing site or with a team that’s
doing work there but the spin off for me as I think about my independent research is going
to be focusing on sustainability after that implementation piece is over and I think that’s
a different way to think about sort of where you’re trying to position yourself within
a research question that works but is also feasible in terms of how you develop your
career.>>Great.>>And just as a quick flag for folks, if
you are not already following along in the chat, some of the resources or suggestions
that are being made I am definitely moving over into that space. There’s a website with our resources, some
suggestions about what graduate students will do, the textbook that was mentioned, all of
that can be found within the chat and I’ll be continuing to update that.>>Great, so I wanted to turn attention to
another question that I think would be great to get each of your take on and that’s you
know, certainly we recognize that with any of these programs and we’ve tried to take
a look across different training opportunities in dissemination implementation research,
we have many, many more applicants than the slots that we’re able to account for and so
wanted to get a sense so it’s not just that, it’s also the fact that a lot of these training
opportunities are time limited so for example the TIDR course has really only been a single
span of time and it doesn’t recur, some of the other programs might be up to two years
but they still may not overlap an entire career. So, I wanted to ask each of you and maybe
we’ll start with Katie and then Simon and then Rachel, following the training experiences
that you’ve had, what types of ongoing mentoring or training did you seek out or receive and
also are there suggestions for those of us to think about as far as providing additional
support beyond that initial training that you were enrolled in and participated in? Any thoughts about mentoring and sort of next
steps for training?>>Yeah so again, I guess I have two kind
of separate answers to that so on one side of the mentoring, at some point I realized
that I had wonderful academic mentors but a piece that I really needed to fill out was
having somebody in a practice setting that could also mentor me in other ways and so
when I had some ideas for a project, somebody that I could just informally toss those ideas
around and get some immediate feedback from a practice perspective but from someone that
also appreciated the utility of the research as well, but just to get that immediate feedback
on is this feasible, is this something that people in the practice community would get
excited about so that I would have some buy in for the research. So that kind of practice mentorship I guess
is one way of answering the question and then another, so I was really fortunate to have
here in St. Louis there’s a group at Washington University that meets frequently and just
sets aside a chunk of time every month or so to get people together that are interested
in DNI science and to discuss what they’re doing and then have opportunities to review
each other’s grant proposals and specific aims pages before submitting them and that
has been a really useful resource as well and it’s something that is pretty open to
just outside of the immediate Wash. U. DNI community, I believe as well if anybody
is interested.>>Great, Simon?>>So, as an anthropologist, I was trained
in the idea that I was going to be a solo researcher and would do everything by myself
and that sort of died quickly when I shifted into the work that I was doing and so I’ve
become a really big proponent of team science. My expertise is in qualitative methods and
the first thing that I want to do is find someone who has the skills that I am never
going to be strong enough in and fast enough in so my caution to asking about sort of more
training is shifting the mentorship side of the question to thinking about peer mentors
and the kinds of coinvestigators that you want to surround yourself with. I am not going to become a quantitative methodologist,
I just need to know enough to ask the right questions and have the relationships with
people who do have that skill set so by analogy, I think it behooves early stage investigators
to be thinking not just about the kinds of training that you want but the kinds of training
that you need just enough to have intelligent, meaningful conversations with people to develop
projects where their skills complement your own.>>Great, Rachel?>>Great so as I mentioned, in the follow
up period, the training follow up period, I really did stay in touch with a lot of the
people that I met through the TIDR training. I invited people to be coauthors on papers,
perusing my aim and get feedback so I actually did a lot working with that initial group
of people that I met and kind of continued those relationships and I actually would send
kind of regular emails to mentors that I met through the program where I would say you
know, I’m really developing my work in this, I submitted this grant, do you have feedback
on other stuff, other things I could be doing? And I actually found that that opened a lot
of doors for me and a lot of invitations for other kinds of development of expertise on
my end you know, being a coauthor on the textbook and being involved with the review papers
so I think really that training really opened up relationships by just staying in touch
with a lot of the initial people that I met. And then another piece of that is the peer
mentoring, I completely agree with that. I think within the training we almost have
a couple of levels within TIDR. We had people who were kind of deans and more
senior and people who were more junior and I think what would be useful is almost to
kind of have an ongoing trajectory of people who are more junior and who are early in their
career and people who are more senior because it seems like we all have different need and
I’ve kind of done that informally through kind of the DNI conference in December, hosting
social events, inviting everyone from TIDR to come and meet but I think there could be
a more formal process that would be useful whether it be a monthly phone call or something
where it’s kind of an ongoing established chance to kind of connect with people who
are at a similar level and just kind of bounce off ideas from each other. So I think that piece would be really helpful
in terms of the follow up of the training and next step.>>Sure no thanks, that’s great ideas. Wanted to ask each of you and I think this
sort of relates to what you were just talking about, people who in some cases are entering
the field and advice you might have for them. So it’s sort of a general question I see and
I’m going to try and pull in another question that we received here as well and so one is
just broadly are there additional pieces of advice that you have for early career investigators
who are thinking about entering the field and then the related question for that subset
who are on the job market and are thinking about DNI research but not sure what institutions
are supportive of it, would you advocate thinking about joining an institution that’s already
saturated with DNI researches? Would you encourage people to join an institution
where they might actually be the lone researcher, which may mean they don’t have as much collaboration
but they might have a niche area? Really broadly any advice you have for early
career investigators and anything specifically related to those who are looking at the job
market. Maybe we’ll start with Simon, and then we’ll
go to Katie and Rachel.>>Sorry, I hit mute [laughter]. So, I think that one of the biggest challenges
is trying to think through where what’s the bite sized morsel that you are biting off
for your first grant application because I think that we, speaking for myself, I think
it was very easy to get stars in my eyes and start thinking about really, really big projects
and realizing that that’s not how for example, most federal or large agency funding streams
work, that the idea is that you are biting off a small piece so you have the long-term
trajectory of your research or the research program you hope to develop but the smaller
piece that you’re really thinking about what you are going to accomplish in this grant
in the next year and a half is also an approach you can take to thinking about the different
kinds of elements you need to build a larger portfolio and that it isn’t going to happen
overnight. Once it does happen, it tends to happen a
little too quickly but I think trying to think what the small piece that you’re trying to
accomplish that you can then build on because so much of the funding ladder has a structure
that we really should be taking advantage of. So thinking about, at your local institution,
what the local pilot funds could be used to apply to develop preliminary data to move
to the next piece.>>Great, Katie?>>Yeah I think some of my thoughts around
that question relate to the fact that it really depends on the institutional context and part
of what I mean by that is what is already available maybe through the institution or
through ready collaborators at the institution that will give you that leg up in terms of
for DNI science, partnerships with practice settings? So are there some established partnerships
or linkages with certain practice settings via contacts at the institution and if not,
then that’s something that you’re going to have to build and that takes some time and
energy and resources to do that and so you know, making sure that work in terms of developing
partnerships with practitioners in practice settings, some of that might involve you know,
providing service through maybe serving on an advisory board or volunteering some of
your time so that you develop those relationships and start giving back to practitioners in
a way that they then are willing to give some of themselves to help out with the research
and so you know, making sure that is time and energy that’s recognized by your particular
institution and then I think now I guess the question, it may be easier to ask in terms
of is there some recognition of and appreciation for DNI science and so you know, I think maybe
it’s easier to open up that conversation now when you’re trying to understand some of that
context and whether this kind of work would be recognized and prioritized at your institution.>>Right Simon, additional comments?>>I think it’s me [laughter], Rachel.>>Sorry Rachel.>>Whoops.>>So, and I agree with that piece. I think if I were a doctoral student now,
I would be trying to learn as much as I could from my mentors about the relationship building
process because I do think whether you’re in a clinical or a community setting, that’s
one of the most critical pieces and it’s often not clear how those partnerships developed,
how did they create this network so I think learning from mentors on that piece will really
set people for success in implementation science and that’s something that I think you have
to observe and be a part of to learn from. I think if I were a doctoral student now too,
I would be thinking again really specifically about what my niche area is going to be and
I think one lesson I didn’t learn is that the titles of your grants and your articles
are really important, that you want them to be cohesive and you want to have this area
of expertise you’re developing so I have a lot of implementation science embedded in
my articles but I didn’t necessarily use that language so think really strategically too
about developing titles that really reflect the program of research that you want to develop
in implementation science. And, I would also really recommend using conferences. I think I used to kind of hide at them, we
don’t think of using them for networking and relationship building and you know, this field
has an incredible set of mentors. I think people are really, really open with
approaching them and setting up meetings. I’ve been completely appreciative of this
so I would really take advantage of trying to meet with leaders in the field and learn
from them and I would have you know, at TIDR we did kind of the elevator speech where you
did your two or three minute speech if you’re caught with the dean or your department chair
and you want to tell them really quickly what your research is, I think having that down
in your doctoral program or at least starting to practice that will also really help you
clarify for yourself and for others, they’ll know you’re kind of the go-to person in that
area.>>Great yeah, and I think there was an additional
question that was just asking about different prioritization among different institutes
of DNI research and I would say that in the same way that often mentors are surprisingly
at times open and certainly happy to talk to anyone and everyone at these conferences,
workshops, etc. also don’t be shy about getting in touch with the different representatives
from our NIH institutes who are listed on the announcements to field questions to give
you a sense of what some of those local priorities are so I want to get to one final question
and you’ll see our final campfire set of pictures so enjoy those, and this has to do with I
think there was a recognition from one of the questioners that funding climates are
becoming increasingly challenging. Since you’re talking also to those of us at
NCI and those of us at the NIH and beyond, any advice you have for those of us funding
organizations, what can we do to advance implementation science further? Why don’t we get Katie, and then Simon, and
have Rachel finish us off so your thoughts on that and then we’ll have to close.>>Hmm, I wish I had a really great answer
right now. I don’t know if I do but I guess one thing
that comes to mind is as far as when grants are reviewed, innovation is obviously a really
important piece and I think the way that gets often interpreted is innovation in research
strategy but I think what DNI science also offers is a lot of innovation in terms of
the settings, and target populations that a study is trying to address and the types
of partnerships, as Simon was saying team science approaches so that all of that innovation
may be considered more broadly considering some of those other domains.>>Okay, Simon?>>I wanted to give a shout out before I answer
the question. One is that as people are, a lot of the questions
that have been raised today, David you and other folks have done a really good job of
trying to build up the literature about training and developing a career pathway. Katie has a really great paper from 2013 that
addresses some of these core issues so a lot of these questions are out there and are in
Implementation Science or the American Journal of Preventive Medicine just to pursue if people
haven’t made those connections in PubMed. When I think about funding, I think there
are key challenges that aren’t unique I think, to DNI but I think trying to think about where
implementation ends and dissemination starts, I’ve had arguments with colleagues about whether
the cart comes before the horse or the horse before the cart and I think particularly trying
to think about funding in the obviously investigator initiated work drives the field but I think
from a convenor perspective in terms of producing program announcements and invited issue kinds
of things, trying to think about how we advance the science to make those connections because
there are elements of implementation science I think that are not ready for dissemination
yet and vice versa, where some of the dissemination you know it’s all iterative so these spirals
and whirlpools, sort of eddy’s and floes move depending on the application of your question
because if you take an evidenced based intervention in one setting that you’re going to learn
from and then apply to others, that is both the dissemination and implementation when
you move to the next setting because you have to think about how you implement in multiple
settings while disseminating more broadly and I think that comes back to this adaptation
fidelity challenge so trying to think about where the gaps are is where I am sort of theoretically
that I am still trying to figure out how the funding can drive some of that.>>And in terms of thinking about mechanisms,
I think it can be intimidating to think about moving from the RO3 to an RO1, particularly
if you need a large number of sites so I think if funding agencies could build more intermediary
either exploratory or partnership building or mid-level career awards to help address
that period where investigators are trying to sustain those relationships and build for
that larger RO1, I think that would be really helpful.>>Awesome so thanks to the three of you and
thanks to everyone who has continued with us even a few minutes past the hour. I’m going to turn it over to Sarah to close
things out.>>Wonderful so I’m going to echo my thanks
to Katie, Rachel, Simon and David for their time today. Your feedback is important to us and we encourage
you to complete our online evaluation. A link to the survey will open a new window
once this session is concluded. An archive of today’s session will be made
available shortly on our website. We hope to see you at our next session, held
in July. Registration details will be shared in the
coming weeks. Thank you again for joining us today. You may disconnect at this time.>>And thanks everybody, that was great.>>Thank you all.

Developing Programs of Research in Implementation Science: Lessons from Early-Career Investigators
Tagged on:                                 

Leave a Reply

Your email address will not be published. Required fields are marked *