In science, criticism is perhaps the highest compliment, showing that we take work so seriously that we are inspired to spend time thinking about and formulating how work can be improved. It is in this spirit that I lay out some of the many issues that I have with the model of
Zhang et al (2017) developed by my friends and colleagues at the Moffitt Cancer Center. This model has been widely cited as providing support for the appealing idea of adaptive therapy in prostate and other cancers, the approach of adjusting therapy continuously in response to tumor status in order to maintain long-term control by delaying or avoiding resistance. However, I have been arguing for some time this model is severely flawed in its assumptions, the way it implements those assumptions, and in the presentation and interpretation of results. I take this opportunity to present a brief outline of these flaws. How much they weaken the justification for adaptive therapy is a broader question, as is the question of whether I have wasted my career trying to write models that avoid flaws of this sort.
The work is built on the twin mantras of resistance management: the centrality of competition between sensitive and resistant cells, and the cost of resistance. I question both of these. Cancer cells have many challenges to deal with besides each other, particularly during invasion and metastasis. These models use carrying capacity to model competition, which is best justified as capturing limitations of space for growth, a strange idea for a cancer, particularly in a model that includes no compartment of normal cells that might be taking up some of that space. I have gone to some effort to attempt to derive something like the carrying capacities in this model from a more mechanistic model of competition for androgen, but even a naive version that neglects the realities of the mechanisms, further simplified with a string a steady-state assumptions, does not come close.
The assumption is here implemented in a rather confusing way, by making the growth rate of the resistant type HIGHER than that of the other types, although with a lower carrying capacity in the absence of treatment. I am unclear what sort of cost this models. Some case is made that the competition coefficients, all rather close to 1 in this implementation, capture some of the costs, but our experiments with the model show them to have only a limited effect on the results.
Difficult as it is to justify the use of carrying capacity to model competition and the cost of resistance, its use to capture the effects of treatment with abiraterone is almost dizzying. This treatment inhibits production of androgens by both testosterone producing (TP) cells in the prostate and residual production elsewhere, which is neglected in this model. Rather than halting growth or increasing a death rate, the model, which includes no explicit death rate, reduces the carrying capacity of the TP cells by the huge factor of 100. Because the model uses the Lotka-Volterra or logistic framework, this reduction creates a catastrophic negative growth rate (not a death rate because there is no term for death) of these cells, which propagates even more strongly to the androgen-dependent (here T+) cells which are completely dependent on TP cells, and thus catastrophically decline the instant after therapy begins. At this point, the androgen-independent (here T-) cells are released to exhibit their higher growth rate and unchanged carrying capacity to rapidly grow. Is it any surprise that any modeled reduction of treatment delays escape of AI cells?
This model makes an interesting choice of fixed cell strategies, but neglects plasticity. Switching among states, such as through upregulation of androgen receptors, provides a much faster time scale response to treatment and might qualitatively change the selection regime. We have spent much time discussing this in our group, and definitely think it warrants careful investigation. The paper also focuses on the single outcome of maximizing time to emergence of resistance. It is easy to prevent emergence of resistance; never treat. With the assumed carrying capacity, the total population of cells would remain constant. The cost of maintaining that high burden, the only thing that delays emergence of resistant cells in the model, is not included.
I like to tell my students that behind every scientist is a little voice chanting in their ear "so what?". Responses I hear to critiques of these and other models are that these are minimal models to capture the fundamental processes, that they do a good job of fitting the data, and that new and better models are on the way. All reasonable retorts. But if models are "accurate representations of our pathetic thinking," as so delightfully quoted by Jeremy Gunawardena in his
paper of that title, they should at least teach us something about our thinking that wasn't obvious from a verbal argument, nor deliberately embrace the pathetic when stronger alternatives are available. By making the effects of treatment so extreme, there is no way that reducing therapy could fail to delay emergence of resistance. Secondly, I think that the ability of even these models to fit data and distinguish trajectories of individual patients says less about the power of the models than about the low information content of PSA dynamic data. Effectively, each round of growth and treatment provides two numbers, and almost anything, including linear models, can fit and distinguish patients. In this case, I argue, somewhat sadly, that a statistical model is more honest than a pseudo-mechanistic model.
I have long thought that it is worse to be right for the wrong reasons than simply wrong in the first place. The success of the adaptive therapy clinical trial does not support the key assumptions of the primacy of competition and costs. No trial that I know of provides a genuinely critical test of the mechanisms underlying the outcomes by including measurements of quantities specific to mechanisms. If we choose to believe in competition and costs as the central mechanisms justifying this approach, we are sure to be led astray in attempts to apply to other cancers. The world is full of great ideas for cancer therapies, and few of them work because the world is a bit more complicated than we knew. When the stakes are this high, both for patients and for an important and potentially transformative approach to treatment, we must hold ourselves to a much higher standard.
At some point, everybody started quoting "all models are wrong but some are useful." True as it is, its appeal has started to wear thin. Criticisms like the ones I've given here are answered precisely that way. Given the influence of this model, its usefulness seems hard to deny. But I'm arguing here that this model fails to accurately portray our pathetic thinking and provides a false sense of understanding. My friends and colleagues behind this work, all among the people I most admire in this field, are committed, like me, to create better models, the right models, to optimize treatment of individual patients and shake up the conventional wisdom with new thinking. If modelers are going to genuinely contribute, we need to hold ourselves to a more rigorous and self-critical standard.
Minor addendum:
I became concerned about this paper immediately for two small but telling details. When I sat down with a student to read the model, we noted that it states "50% of the PSA decays out of the serum each time step," but then uses a decay rate of 0.5 for PSA rather than ln(2). Of course this makes no difference because PSA dynamics are sufficiently rapid to be proportional to cell numbers (a point not mentioned in the paper). Secondly, the paper refers to what is really intermittent therapy as "metronomic therapy", a term reserved for "continuous and dose-dense administration of chemotherapeutic drugs with lowered doses" (from a
2019 review by Simsek, Esin & Yalcin in the Journal of Oncology).