论文标题
通过相对熵正则化的经验风险最小化：最佳和灵敏度分析
Empirical Risk Minimization with Relative Entropy Regularization: Optimality and Sensitivity Analysis
论文作者
论文摘要
研究参考是Sigma-Finite度量而不是概率度量的情况，研究了经验风险最小化问题（ERM-RER）的经验风险最小化问题（ERM-RER）的最佳和敏感性。这种概括允许在模型集中纳入先验知识时具有更大程度的灵活性。在这种情况下，表征了正则化参数的相互作用，参考度量，风险函数以及由ERM-RER问题解决方案引起的经验风险。这种表征为存在的正规化参数带来了必要和充分的条件，该参数具有任意的经验风险，并任意高。研究了预期的经验风险对偏离ERM-RER问题溶液的敏感性。然后，灵敏度用于在预期的经验风险上提供上限和下限。此外，结果表明，敏感性的期望是在模型和数据集之间的Lautum信息的平方根上的上限，最高为恒定因素。
The optimality and sensitivity of the empirical risk minimization problem with relative entropy regularization (ERM-RER) are investigated for the case in which the reference is a sigma-finite measure instead of a probability measure. This generalization allows for a larger degree of flexibility in the incorporation of prior knowledge over the set of models. In this setting, the interplay of the regularization parameter, the reference measure, the risk function, and the empirical risk induced by the solution of the ERM-RER problem is characterized. This characterization yields necessary and sufficient conditions for the existence of a regularization parameter that achieves an arbitrarily small empirical risk with arbitrarily high probability. The sensitivity of the expected empirical risk to deviations from the solution of the ERM-RER problem is studied. The sensitivity is then used to provide upper and lower bounds on the expected empirical risk. Moreover, it is shown that the expectation of the sensitivity is upper bounded, up to a constant factor, by the square root of the lautum information between the models and the datasets.
