Non convexity of the stability domain of discrete time linear filters

If an autoregressive digital filter of nth order is represented by a point a_n of the n-dimensional space, the set of all stable filters defines a region S in this space called the stability domain. For n ⩾3, the geometry of S is complicated compared to that of its representation in the n-dimensional space of the reflection coefficients introduced by the lattice representation of stable filters. If nth-order filters are considered, it is shown that for n⩾3, S is not convex and the expression of both the radius of the greatest sphere included in S and that of the smallest sphere containing S are established