On scheduling redundant requests with cancellation overheads

Reducing latency in distributed computing and data storage systems is gaining increasing importance. Several empirical works have reported on the efficacy of scheduling redundant requests in such systems. That is, one may reduce job latency by 1) scheduling the same job at more than one server and 2) waiting only until the fastest of them responds. Inspired by the empirically observed gains of such schemes, several theoretical models have been proposed to explain the power of using redundant requests. Although the proposed models in the literature provide useful insights such as when scheduling redundant requests can be beneficial, all these results rely heavily on a common assumption: all redundant requests of a job can be immediately cancelled as soon as one of them is completed. In this paper, we study how one should schedule redundant requests when such assumption does not hold. This is of great importance in practice since cancellation of running jobs typically incurs non-negligible delays. In order to bridge the gap between the existing models and practice, we propose a new queueing model that captures such cancellation delays. We then find how one can schedule redundant requests to achieve the optimal average job latency when cancellation delay is considered and accounted for. Our results show that even with a small cancellation overhead, the actual optimal scheduling policy differs significantly from the optimal scheduling policy when the overhead is zero. Further, we study optimal dynamic scheduling policies, which appropriately schedule redundant requests based on the number of jobs in the system. Our analysis reveals that for the twoserver case, the optimal dynamic scheduler can achieve 7% to 16% lower average job latency, compared with the optimal static scheduler. This observation is in stark contrast to the known fact that the optimal static scheduler performs as well as the optimal dynamic scheduler when cancellation overhead is ignored, affirming that misleading conclusions result if the cancellation overhead is ignored completely.