Information theoretic validity of penalized likelihood

Building upon past work, which developed information theoretic notions of when a penalized likelihood procedure can be interpreted as codelengths arising from a two stage code and when the statistical risk of the procedure has a redundancy risk bound, we present new results and risk bounds showing that the l₁ penalty in Gaussian Graphical Models fits the above story. We also show how the traditional l₀ penalty times plus lower order terms which stay bounded on the whole parameter space has a conditional two stage description length interpretation.