Measuring Impact in patient-centered drug development conference

Humanising key performance indicators to make patient engagement in medicines R&D a greater reality: a report of the DIA-organised conference on ‘Measuring the impact in patient-centered drug development’, by Mathieu Boudes, PARADIGM coordinator.

DIA organised its second conference on measuring impact in patient-centered drug development on 2-3 October 2018 in Bethesda (USA). This conference was designed to help pharmaceutical and biotech companies, as well as patient organisations, enhance their capacity to measure the effectiveness of patient engagement efforts. The focus included the development and implementation of qualitative and quantitative metrics.

The specific objectives of the conference were:

• to describe how to develop and use metrics to assess the implementation, quality, and impact of patient engagement
• to identify objectives of patient representatives, industry and regulators for patient engagement and their influence on the selection of measures of engagement effectiveness
• to describe a framework for patient engagement metrics in the medical product lifecycle and how to assess the usefulness/fit for purpose of current and emerging approaches and measures
• to assimilate lessons learned by the early adopters to anticipate potential challenges and optimise measurement of patient engagement impact.

The message I conveyed in my session is that patient engagement needs a profound cultural change that can be supported by the development of an assessment framework to measure the engagement of patient representatives in medicines R&D.

However, metrics are just metrics. They can support a shift in operations within an organisation but it is very easy to fall into the ‘Tyranny of metrics’, as described in the book of Jerry Z. Muller, a historian who traces back the origin of the measure and why it became so popular. He explains that the development of key performance indicators boomed at the turn of the 20th century as Taylorism was introduced. The goal of Taylorism was to replace the implicit knowledge of workers by mass production methodologies, developed, planned and monitored by a manager. This theory was developed by engineers and accountants. Muller also explains that metrics became so popular because they allow going beyond individual judgement. However, in our ecosystem, personal judgement is so crucial in triggering a cultural change towards the ethical dimension of patient engagement that is intrinsically linked with business aspects. Humanising the numbers behind the KPIs is key to making patient engagement a greater reality and ensuring that the needle moves from ‘engaging with patients is the right thing to do’ to ‘it is the right thing to do for my business’.

The conference started with a keynote address abouta roadmap for patient engagement in healthcare: change strategies and measurements that drive them, from Lee Thompson, senior research at the American Institutes for Research.She introduced the patient and family engagement roadmap, aiming to improve population health to provide better experiences of care at lower healthcare costs.

This was followed by a presentation from Linda Sullivan, the president of the Metrics Champion Consortium. She presented the process of maturity model that was developed by the Software Engineering Institute in 2002 at the Carnegie Mellon University, which shows that metrics can be at different stages (initial, repeatable, defined, managed and optimizing). This framework was used throughout the conference to keep track of metrics described in the different sessions, determining whether they are examples of basic, advanced or exploratory metrics and to which stage of the framework they align.

Interestingly, it was highlighted that some metrics are used for planning, others assess performance, and therefore the use of each metrics must serve the purpose for which it was developed otherwise, its use can often be counterproductive.

The next session featured Roslyn Schneider from Pfizer, Nicholas Brooke from PFMD and Dawn Richards from Clinical Trials Ontario and was about applying the metrics to patient engagement in the product lifecycle to potentially improve both health and business outcomes.

This was followed by a session on where and when we should be measuring the impact of patient engagement. The session was chaired by Cynthia Grossman, from FasterCures, and featured Bray Patrick-Lake and James Valentine, a former FDA staff member involved in patient engagement. My message was that it is never too early to manage and therefore to measure, when relevant. This statement got a wide buy-in from the audience.

Stella Stergiopoulos, from the Tufts Center for the study of drug development chaired the last session of the day about what measures/metrics are in use, and featured Ellen Coleman from MK&A, Carrie Corboy from Janssen, Karlin Schroder from the Parkinson’s Foundation and Heidi Ross from the Clinical Trial Nursing services. One of the messages is that, when involving people living with Parkinson diseases in fundamental science grant review led to changes in funding decisions.

The second and last day of the conference was about developing, implementing, demonstrating value and disseminating the metrics across organisations with a key focus on engaging in multidisciplinary exercises. Two sessions were chaired by Cynthia Grossman from FasterCures and Patricia Jones from the US National Institutes of Health (NIH).

Finally, the last session of the day, chaired by Suzanne Schrandt from the Arthritis Foundation, and featuring Jennifer Miller from the Yale University and Theresa Mullia from the US Food and Drug Administration (FDA) focused on ‘Where are we going in the implementation of metrics?’. This session was much in line with my statement that metrics should be developed and implemented to support the much-needed cultural change, and this sheds some light on this need in both sides of the Atlantic Ocean.

Overall, with EPF as co-leader, it appears clear that the IMI-PARADIGM project, with its work stream on the development of a monitoring and evaluation framework is going in the right direction. It is encouraging, and it is exciting for EPF to be part of this line of work.

Mathieu Boudes, PARADIGM Coordinator