Kalman filter for inhomogeneous population Markov chains with application to stochastic recruitment control of muscle actuators

A population of stochastic agents, as seen in swarm robots and some biological systems, can be modeled as a population Markov chain where the transition probability matrix is time-varying, or inhomogeneous. This paper presents a Kalman filter approach to estimating the population state, i.e., the headcount of the number of agents in each possible agent-state. The probabilistic state transition formalism originated in Markov chain modeling is recast as a standard state transition equation perturbed by an additive random process with a multinomial distribution. An optimal linear filter is derived for the recast state equation; the resultant optimal filter is a type of Kalman filter with a modified covariance propagation law. Convergence properties are examined, and the state estimation error covariance is guaranteed to converge. The state estimation method is applied to stochastic control of muscle actuators, where individual artificial muscle fibers are stochastically recruited with probabilities broadcasted from a central controller. The system output is the resultant force generated by the population of muscle fibers, each of which takes a discrete level of output force. The linear optimal filter estimates the population state (the headcount of agents producing each level of force) from the aggregate output alone. Experimental results demonstrate that stochastic recruitment control works effectively with the linear optimal filter.