A stochastic model of tumor response to fractionated radiation: limit theorems and rate of convergence

The iterated birth and death Markov process is defined as an n-fold iteration of a birth and death Markov process describing kinetics of certain population combined with random killing of individuals in the population at moments τ1,…,τn with given survival probabilities s1,…,sn. A long-standing problem of computing the distribution of the number of clonogenic tumor cells surviving an arbitrary fractionated radiation schedule is solved within the framework of iterated birth and death Markov process. It is shown that, for any initial population size i, the distribution of the size N of the population at moment t⩾τn is generalized negative binomial, and an explicit computationally feasible formula for the latter is found. It is shown that if i→∞ and sn→0 so that the product is1⋯sn tends to a finite positive limit, the distribution of random variable N converges to a probability distribution, which for t=τn turns out to be Poisson. In the latter case, an estimate of the rate of convergence in the total variation metric similar to the classical Law of Rare Events is obtained.