ClavystBio Conversation: Future of Clinical Trials

TRANSCRIPT:

Khoo Shih

00:00:03:18 - 00:00:05:13

Today it’s our great pleasure to have Dr Amy Abernethy. Amy, first of all, is a trailblazer in healthcare change.

She's really looking at the convergence of delivering the advancement of healthcare, leveraging on innovation data and technology. I think all of us here, because of our passion in health, we know clinical development is a very critical path as we advance innovative medicines and health solutions.

So Amy, let me start out with the first question. Can you comment on how you're looking at clinical trial changes, and how you see these changes progressing in the next 5 to 10 years?

Amy Abernethy

00:00:56

So first of all, thank you very much for having me. And it's fun to be back. You know, I counted on my way here that this is my 10th trip to Singapore, and I love coming here, so thank you.

I've been part of and watching the shifts in clinical trials and clinical development, for the last 30 years or so, and it's interesting to see the change. Historically, clinical development is very formulaic. Phase 1 through 4 clinical trials. We understand that Phase 1 helps us document safety. We then are doing appropriate dose finding and understanding efficacy endpoints in Phase 2, and move to Phase 3 to demonstrate the effectiveness of a product before a regulatory decision, like an approval. But it's shifted to something that looks much more blended.

Oftentimes, regulatory approvals are happening earlier in the lifecycle of the medical product, for example, after Phase 2B. We're moving more of the understanding of the performance of the medical product into the post-market setting so that actually there's systematic data collection.

Now in the part of the process that we've historically thought of as post-market or Phase 4. So that's one big shift is moving towards earlier approvals and more post-market analysis. Another activity that's changing in the clinical trial space is that we're using Phase 1 not just as assessment of safety, but now trying to understand efficacy in Phase 1.

In fact, we're seeing lots of changes to the Phase 1 clinical trial itself, including, for example, expansion cohorts and understanding different indications in the Phase 1 setting, so big change in what Phase 1 itself looks like. And overall, one of the things that we're seeing is much more of an expectation of continuous assessment of a medical product across the lifecycle of the medical product.

So it's not just getting to a regulatory decision as of approval and then being done. We're thinking about safety and for example, long term assessment of safety in the post-market setting. We’re thinking about new indications and indication adjustments in the post-market setting, or thinking about systematic data collection, including understanding drug combinations, treatment sequences, biomarker development.

All of these now are routine parts of the clinical development process. All of this in total means several things. First of all, the amount of evidence being generated, the demand for evidence generation is going up, not down. We may have smaller clinical trials and a smaller corpus of data in the beginning getting to a regulatory approval, but there's much more expectation that we have more data about a drug all the way across the lifecycle, which means we've got more work to do in clinical development. We're seeing much more use of data and technology to help make the process more efficient. I think we'll come back to that, so I'm not going to spend a lot of time there. We're seeing much more focus on biomarker development and being able to stratify populations and identify patients at greatest opportunity to derive benefit. And we're also, and I think we'll talk more about this, seeing more of an expectation of combining multiple study types to get a more complete picture of the product traditional clinical trials, plus pragmatic studies. Traditional clinical trials, plus real world data and starting to see an expectation of a corpus of knowledge, not just a single study or a pair of studies.

Khoo Shih

00:04:53

Amy, you have been also a personal champion, for many years, including when you're at FDA, of real world data. Can you talk a bit about where are we on the real world data, and what is it and what is not yet contributing to clinical development?

Amy Abernethy

00:05:11

So first let's start off definitions. When we think about real world data, typically we're talking about data that is collected outside of the traditional clinical trial data. So it may be, for example, data that exists in the electronic health record or administrative data. It may be registry data or information in a genomics data set. Real world data these days also means sensor data, such as a sensor that it's in a ring or a watch. It might be social determinants of health. It might be environmental data. All of these types of data sets are becoming a part of an overall type of data, traditionally called real world data.

Now, as we think about real world data, I would say until about the last 2 or 3 years, a lot of the conversation around real world data and real world evidence meant taking the data that exists off an electronic health record and claims data and analyzing that which has already happened in the past. So basically asking is drug A better than drug B, because we've been using both of those drugs for the management of hypertension for a long time, and we want to know which ones are best in a population of 55-year-old gentlemen. That's kind of the historical approaches to real-world data and real-world evidence.

But that story has been shifting. So the most obvious way that it's been shifting is new data types have been added. So it's not just the electronic health record, but it might be a combination of the electronic health record plus sensor data or electronic health record plus genomics data, or electronic health record plus imaging plus sensor, plus genomics plus social determinants of health. So the number of data types has been rapidly increasing, meaning that we have now more variables to actually give more details about the population and information about safety and effectiveness. So that's one way real world data, and real world evidence has been changing.

The second way it's been changing is we've seen a lot of evolution and the analysis of capabilities and techniques. I remember when propensity score matching was all the rage, like it was very cool. We could do a propensity score matched analysis. Yes, we still leverage propensity score matching. But in fact, these days we're leveraging causal inference or leveraging hierarchical models and repeated measure analysis to really now start to have a number of ways of looking at real world data and more sophisticated approaches, which also incorporate machine learning and artificial intelligence. So the analysis approaches have started to get better, and Regulators around the world are starting to recognize that more sophisticated analyzes are possible that lead to more confident findings.

Third way there's been remarkable change and real world evidence space is we've stopped thinking about it as only a retrospective set of activities. We've stopped talking about real world data, as only the data about what used to happen or happened yesterday. But we're combining data that tells the story that came before, together with new data collected going forward.

The most classic example is I'm Amy. For Amy. We've got my information that exists in my electronic health record. And now perhaps I've consented to participate in a registry to understand hormones and women who have gone through menopause. I say yes, and I answer questions across time. There's information about me. My story goes forward. Maybe I got new medicines, maybe I didn't. But now that information can become part of a larger part of real world data.

The fourth way that the real world data story is changing, and I think the most important is we're starting to see a blurring of the lines between traditional clinical trials and real world data studies. Go back to my story. Perhaps my information sits in electronic health record at Duke University in Durham, North Carolina. Duke already knows I'm female. Duke already knows what medicines I'm on. Maybe I now go on a study for a new medicine. And that's a traditional clinical trial. Duke can flow my data, my gender, my medicines, my electronic health record information of what vaccines I've received into the clinical trial, data set, reducing the need to document information in that study itself, and reducing the tasks required for that study. You're starting to see now more and more blurring across the boundaries of how clinical trials get done that use data that came before. And pair that with new data that gets collected in the future.

I'm going to jump to another big topic, what else you are seeing that can be further harnessed to really, you know, change how we look into how we do clinical development. So when I think about artificial intelligence, first, let me kind of describe some big buckets.

So first, I think about artificial intelligence based health care solutions. So this might be an Algorithm that acts as a biomarker to distinguish specific patient groups and distinguish whether or not that patient group is at greater risk, for example at greater risk of stroke. So those algorithms, depending on the risk of the algorithm and the artificial intelligence based product, goes through a regulatory process to understand safety and effectiveness and whether or not it should be used at population scale.

The second category of algorithms all the way on the other side, low risk do dozens of sort of background administrative tasks. Whether the algorithm is helping us check our spelling or the algorithm is helping us process some billing sort of administrative tasks.

And then the third category of algorithms I'm thinking about are essentially algorithms that are helping us make the process of understanding safety and effectiveness more efficient. So algorithms for clinical development, reason why I'm dividing them into those three categories is because, in fact, as I think about clinical development and artificial intelligence, I have to always figure out which category we're talking about before I think about the algorithm and the development itself and how we can use it.

Right now, in the context of clinical development, we're seeing algorithms show up to make clinical development more efficient in many different ways. So first of all, you know, on the tip of everybody's tongue would be algorithms that are supporting AI based drug discovery. So developing novel targets, improving the design characteristics of the drug itself, does it fit in the pocket, all those good questions. There's also leveraging algorithms to conduct studies themselves to conduct clinical trials themselves. So there's some operational tasks where the algorithms don't need to be reviewed too carefully. So for example, algorithms that help with patient finding for participant recruitment in clinical trials.

And then there's tasks where algorithms really need to be vetted very carefully. So for example, algorithms that are curating data, taking for example, unstructured documents and now pulling out clinical endpoints as details. And those algorithms need to go through a process of not only validation but continued performance monitoring across time. And we're seeing how algorithms show up there. In fact, FDA put out guidance about two months ago now, really specifying how we should think about the development validation performance of those algorithms.

The other two places to highlight algorithms, and I probably have a very long list of others that I could name. One is algorithms that are helping us more and more in adaptive design clinical trials. In the very first question, I sort of alluded to the fact that our clinical trial designs are getting progressively more complex, especially in phase one, and what we're now seeing is not only algorithms to help with Bayesian decision making and allocation of patients to study arms or algorithms, or helping with decision making, for example, a model of informed drug development. So simulating Phase 2 to enable essentially this jump from Phase 1 directly to Phase 3.

So we're seeing algorithms show up there. And then interestingly going back to my first category of artificial intelligence based medical products, we're seeing the importance of the clinical development of those products. So we need to oftentimes run clinical studies to understand if an algorithm based product is safe and effective. So algorithms can help all across that process. But also algorithms are helping with the development of the underlying data sets that then the AI algorithm based products are being developed from, that then ultimately are being studied in algorithm based studies. So we're seeing kind of the layering of algorithms all the way across.

It's a complex story as I've just told. You can imagine sitting and looking at that as a regulator and needing to parse through and ask the question, where do we need to essentially trust but verify?

What do we need to stop and ask algorithm developers to basically document how we got here and it's basically in those first and third categories.

AUDIENCE Q& A

00:15: 10

What indications are best suited for accelerated approval, given the reliance on post-market monitoring?

So historically this has played out first in HIV, then oncology, and oncology has been the poster child for a long time. In rare diseases, so diseases where we have biologic confidence, and biologic plausibility together with clinical outcomes, that makes sense.

You can imagine that also in those settings there is the ability to leverage the accelerated approval pathway, which relies on a clinical surrogate so that you don't have to wait all the way until survival outcomes.

Or you can actually have a regulatory decision based on, for example, progression free survival. Importantly, leveraging accelerated approval pathways comes with the expectation that post-market, there'll be essentially the confirmatory studies to be done to confirm those findings.

One of the problems that has been happening, especially in the oncology space, is that those confirmatory studies have not been happening. And so now there's been a lot of pressure to essentially pressure companies to do the trials. And in January, there was guidance put forward about, essentially how those confirmatory trials should be designed.

I'm going to come to other indications in just a second, but this is a set up for other indications. What's interesting is in that guidance, part of what the guidance described was being able to start to think about how your confirmatory trial is designed and set up that agreement with FDA at the time of, as you start to go into your Phase two. So that that's already getting planned, and maybe your confirmatory study is your continuation of your Phase two for long term, or is a new set of studies that have been predesigned. It may even be real world studies, but having that now be a clear part of the conversation for accelerated approvals. And that's a lot of been lessons learned from the oncology space.

The poster child, really for the accelerated process has actually been the Covid 19 vaccines. So mRNA vaccines were authorized and then approved on smaller clinical trials. These were shorter studies with a small number of patients. They were randomized studies. And then we looked towards real world data in particular, but also other randomized trials to understand the mRNA vaccines. Unfortunately, the mRNA vaccines have had a complicated story over the last six months or so, and the US, some of which may have been a byproduct of this, more efficient development system. And so I think it remains to be seen right now how that story will fully play out.

But thus far, I at least, my instinct is the pressure towards a better understanding of biology and having biologic plausibility will still continue us in this direction of having more accelerated approvals, and earlier approvals - including, I've heard two stories now, of better use of model informed drug development pathways. Now, what are the indications we're starting to see, that this story play out in those other indications and what are the features. When there's clear genetic and sort of essentially biologic plausibility that is really one of the key defining factors, clear ability to identify the population, clear ability to understand the biology.

But as you know, we don't always fully understand the biology. Right now we're seeing, moves towards accelerated approvals type approaches happening in neurology. We're seeing it in liver disease, we’re seeing it in metabolic disease, psychiatry. So I think there's a number of different places. But this very clear patient population and patient definition and understanding of biology theme seem to be key features.

Will current technologies and FDA guidance on in silico modeling truly transform drug development and transform the need for large clinical trials?

00:19:38

So your first question was really are we just nibbling at the edges? Right. Like we're making cute little improvements to the process, but it really isn't transforming clinical trials. I 100% agree. So the reason we started Highlander Health was for exactly that reason. The entire mission of Highlander Health is literally to make the catalytic change that reduces clinical trial timelines by at least 50% in the next five years, and we should be able to do more than that. Like, you know, when we look at making major changes to clinical development timelines, it really is across many different parts of the cycle, though now all starting to come together. Sort of stepping back, we set up Highlander Health as an organization in two parts because the thesis is attacking this problem requires having multiple ways of solving it.

So the first part of Highlander Health is actually a nonprofit. So we're giving grants to health systems and others to test questions that we've been talking about as potential solutions to accelerate clinical trials for a long time, but haven't made progress against them. And the question is, why? And so there we are studying, for example, how can you do what I described a few minutes ago and take Amy's data and Electronic Health record and flow it into a clinical trial data set, and do that reliably for every clinical trial and help regulators trust that and that be just a part of how we do clinical trials.

So that's just one example of what we're spending time and money trying to address. And the reason that we have that on as a nonprofit is because we think there are a lot of problems that are higher water floats all boats. Everybody needs to be able to see and comment and have access to the solutions as they come through in order to make catalytic change.

But the other part of the story is we have to build new, basically new infrastructure and new technologies and new scientific methods to really change how this looks in the future.

And so the other part of Highlander Health is an investment fund, where we invest in companies that we think are a part of those novel solutions of the future, with a key thesis that that includes data related solutions and includes building what I call footprint, which is basically building out ways of having all of the right interactions between doctors and patients so that clinical research activities can happen. And then also what I call the third piece credibility. So it's novel scientific methods. It's dealing with privacy and security. It's thinking about trustworthiness and trust and built and building all of those capabilities that builds the credible science, because the science has to look different. If we truly want to make catalytic change and make all this different.

Can countries outside US, like Singapore, be sandboxes for catalytic change in clinical development?

00:22:49

Absolutely. So it's interesting, I have this little figure that I drew for myself, and there's a fourth piece. I gave you my three pieces, but the fourth piece is actually having whole of system coordination. And there's really only a few places in the world that that can really happen, where all of the pieces come together. It's not something that happens easily in the United States.

In the US, we're not even really an archipelago. Right? So it's, you know, part of the US's value is that it's very, very diverse and many different, capabilities, but it's actually really hard to harness all of that in a coordinated fashion. But Singapore is one of the places you’ve got the advantage of. Singapore isn't too huge. You've got the advantage of coordinated government. You've got advantage of a educated, reasonably healthy, long living, smart population. These pieces all coming together, provides a really unique opportunity to leverage what you know and data and science, as well as how you can coordinate across the whole population to do something that is completely transformative. And so that is my fourth box on my story, which is that there really is the opportunity in only certain places in the world, Singapore being one of them, where you actually could now start to pull all of the pieces together in a much more sort of catalytic demonstration zone of what it takes to make it all work.

Can you share your vision for the health pilot beyond vaccine testing and what it could look like in the coming years?

00:24:26

So, you know, my expectation in the artificial intelligence space is that we're going to be moving towards a landscape where the regulation of

AI based health care products includes a clear focus on post clearance or post-market performance monitoring, and then that may also allow then the entry of learning algorithms in the post-market space. Like that's kind of like one of the directions I anticipate this story might go. I could be completely wrong. It may need to still be locked predetermined change control plans. But the reason that that's an important story is that in reality where that takes us, which is just essentially a shortened timeline version of what then would be the drug story, is that essentially now you have data sets that likely combine traditional clinical trial with real world data to get to market entry, and then an expectation of continuous assessment and analysis, post-market that leverages the same kind of data elements and repeat measures analysis across time. And I think that's how you start to see this blended landscape.

It's going to require the methods. It's going to require the data sets to do that. And also it's going to require some kind of regulatory apparatus that allows that repeat assessment across time and has essentially the scale and structure for that to happen. But I think that AI is going to demand it because AI is going to demand, and then we're going to be able to transfer those capabilities to our drug development landscape. And so, funny enough, I spend a lot of time thinking about this for artificial intelligence, because I feel like it is the use case to solve that has drug applicability as well.

How can pharma companies collaborate with providers to create sustainable commercial models for new therapies?

00: 26:28

I think really what you're getting at is how might pharma companies work better with health tech companies to both help their products. But actually sort of help make sure that the entire landscape is frankly, ready for moving the needle on health, which is better for their products in the end. Yeah. You know, it's interesting, having sat inside of health tech companies, I've been in health tech, big tech, I've been on boards of pharma companies, biotech companies. So sort of like been looking at this from multiple sides.

You know, historically, if the products that a health tech company is building, don't make sense to a pharma company or sort of aren't required right at this moment in time, then it's sort of outside the vision of the pharma company, and they don't really quite know how to think about the health tech company. And frankly, pharma companies do what many other companies do. Or historically did. I think this is getting better is they have vendor think Right. They sort of think of anybody as a tech company, as a vendor, as opposed to a co-developer or a contributor to the overarching ecosystem. That's been shifting and I think will continue to shift. It's helped as pharma companies have started to see handshakes and partnerships with health tech companies to build together, because there's capabilities that aren't inside the pharma company that they see health tech bringing to the table. And a classic example of this is artificial intelligence based drug design and a lot of that, you know, capabilities that sit inside of the pharma company. And how do they do that together. But if you want to know my thesis on how do we make this even better and move faster, it's two things. The first is what I call lingua franca, Esperanto, a common vocabulary. In reality, pharma companies and health tech companies and health systems and doctors and scientists and engineers and product managers don't speak the same language. And until health tech and pharma can speak the same language and are talking about the same things, it becomes really hard to have mutual understanding and trust. And so the challenge is as health tech, that often you're the little guy and they're the big ones, right. And how do you sort of inspire common language and playing in equal, you know, playing ground if you're the little guys. And actually I think that is a really, really important thing to sort of start to try and support. But it is really important because just talking two different languages.

The second thing is that ultimately there is a group that's often left out of this conversation, and that's what I'm going to call mostly the health systems. It may be the doctors, it may be the systems of care, it may be the governments, but they have the most to win or lose by health, tech and pharma learning how to work together. And it's interesting what happens when you have health systems and government and others advocate for it.

One little caveat, which is there's always worry inside of pharma companies around some FDA regulations, for example, around digital products that enhance their drugs, so kind of staying away from those points of concern. But it says two things lingua franca and the overall ecosystem supporting it that I think make the biggest difference.

With payment models shifting to value-based care and FDA approval not ensuring reimbursement, how do health tech firms prove both clinical value and financial ROI?

00:30:21

So there are very few instances where something costs more to the health system or to payers that payers go for it. Right. Or the health system goes for it. Sort of.

The two big ones that kind of come to mind. One is dramatic effect size. So when you clearly Change the outcomes of patients in ways that nobody can ignore, you often can name your price, right? Although that's becoming a bit harder even. But so that's one. And then the other is when you're clearly changing the lives of the most constrained resource in the system, which is clinician, you can start to demand pricing. And there the perfect example is ambient listening companies are getting paid for, although it's interesting to see now they're starting to creep into the conversation. Are they really making a difference? after that, you end up in this landscape of demonstrating value, which often is demonstrating value with respect to value for the payer by reducing costs. Now, as you do that, your question may be well, but I can improve clinical outcomes. Again, lots of people can nibble at the edges in improving clinical outcomes. But unless you have a product that truly changes effect size, figuring out how you price a product, especially software based products, so that they see that what actually happens is an improvement in cost in the overall system, sort of the overall direction.

The last thing I would say there is a word of caution. I've heard multiple times today with the language of, with the move towards value based care in the United States and the move away from fee for service, beware of the assumption that that is entirely going to happen, because there's been kind of this interesting set of questions about value based care models, how much they really are working in the United States, where that story is really going.

And I think that you're going to see, probably a middle zone that's not one or the other.

And, you know, the example that kind of comes to mind, right this second was that there was a Statement that all Medicare patients were going to be on value based care models by 2030, and that's now being walked back. Similarly, you're still seeing lots of fee for service in the ortho space and other areas. And if anything, you're seeing the concierge model start to, to really come up. So it'll be interesting to see what happens. It'll probably be somewhere in the middle is my bet. I think that we still have metamorphosis to go in that in that space.

How should early-stage biotechs and investors adapt to the changing regulatory landscape?

00:33:28

You know, practically speaking, for new companies that are identifying targets, drug candidates trying to figure out, like their AI and enabling studies like kind of living in that landscape, the work still looks the same, and I would not assume the work is going to look a lot different, at least for right now, because there's a lot of steps between here and there of the work looking different.

And if I was advising one of the companies, I'm not a regulatory strategist I don't, like, you know, I sit on the boards of companies that are relevant, but that that's the only thing. I would tell my companies, to follow the normal pattern, watch what others are doing. But one of the challenges can be it gives you, you know, just whiplash to watch so much of what's going on around you and not be able to figure out what you should be. So you know, being able to look down on the path is pretty important.

Khoo Shih

Okay. I think with that very serious topic, I want to end with one last question.

How will clinical trials look in 10 years time?

34:41

So my prediction is I'm going to have two pieces to this. So the first is that, the way you sort of described it, I think is directionally correct, which was, you know, we'll erase this very set of very clear bright lines between Phase one, two, three, four. And we'll essentially have a continuous set of evolving studies that evolve and adapt in Bayesian ways as we get new knowledge and continue to segment to the right patient population, and indication, as we get to the place of saying we have enough safety confidence, enough efficacy confidence that this medication should be ready to be used on larger scale. And then we'll very smoothly flow to the the section of the story where medicine is being used on larger scale. And we're going to much more consistently monitor that product in a very fluid way as it gets used. It goes back to the artificial intelligence story. That's the reason why I use that as my kind of like core use case.

In artificial intelligence products we need to continuously monitor them as they get implemented. And I think what you're going to see is there's going to be essentially a seamless set of data and analyzes that first document, safety and effectiveness.

So we have confidence, we can use something and then continuously watch a medicine goes on market. And we're going to append new data elements to the data that already exists. In order to collect the new data that you need along the way to answer the different questions in that sort of continuous fashion. So that's how I think it will look.

My second prediction is that I think longitudinal data sets that longitudinally collate clinical stories across time is the foundational model. Like essentially these are all longitudinal models and so what we are going to do is we're going to first form a foundation of longitudinal data sets.

And we're going to do the math on top of them. We're going to add new data elements as we need. We're going to make regulatory and policy decisions on top of that longitudinal story. And we will continuously say yes, no, yes, yes, no. As we see more information come out. But it'll be a continuously evolving story. And so it will be a longitudinal data story. So the reason why I say that's the second piece is I think that's where the maths going to need to be.