Co-design of hardware and algorithms for real-time optimization

It is difficult or impossible to separate the performance of an optimization solver from the architecture of the computing system on which the algorithm is implemented. This is particularly true if measurements from a physical system are used to update and solve a sequence of mathematical optimization problems in real-time, such as in control, automation, signal processing and machine learning. In these real-time optimization applications the designer has to trade off computing time, space and energy against each other, while satisfying constraints on the performance and robustness of the resulting cyber-physical system. This paper is an informal introduction to the issues involved when designing the computing hardware and a real-time optimization algorithm at the same time, which can result in systems with efficiencies and performances that are unachievable when designing the sub-systems independently. The co-design process can, in principle, be formulated as a sequence of uncertain and non-smooth optimization problems. In other words, optimizers might be used to design optimizers. Before this can become a reality, new systems theory and numerical methods will have to be developed to solve these co-design problems effectively and reliably.