Utility Optimization for Event-Driven Distributed Infrastructures

Event-driven distributed infrastructures are becoming increasingly important for information dissemination and application integration. We examine the problem of optimal resource allocation for such an infrastructure composed of an overlay of nodes. Resources, like CPU and network bandwidth, are consumed by both message flows and message consumers; therefore, we consider both rate control for flows and admission control for consumers. This makes the optimization problem difficult because the objective function is nonconcave and the constraint set is nonconvex. We present LRGP (Lagrangian Rates, Greedy Populations), a scalable and efficient distributed algorithm to maximize the total system utility. The key insight of our solution involves partitioning the optimization problem into two types of subproblems: a greedy allocation for consumer admission control and a Lagrangian allocation to compute the flow rates, and linking the subproblems in a manner that allows tradeoffs between consumer admission and flow rates while satisfying the nonconvex constraints. LRGP allows an autonomic approach to system management where nodes collaboratively optimize aggregate system performance. We evaluate the quality of results and convergence characteristics under various workloads.