$k$ Nearest Neighbor (kNN) method is a simple and popular statistical method for classification and regression. For both classification and regression problems, existing works have shown that, if the distribution of the feature vector has bounded support and the probability density function is bounded away from zero in its support, the convergence rate of the standard kNN method, in which $k$ is the same for all test samples, is minimax optimal. On the contrary, if the distribution has unbounded support, we show that there is a gap between the convergence rate achieved by the standard kNN method and the minimax bound. To close this gap, we propose an adaptive kNN method, in which different $k$ is selected for different samples. Our selection rule does not require precise knowledge of the underlying distribution of features. The proposed adaptive method significantly outperforms the standard one. We characterize the convergence rate of the proposed adaptive method, and show that it matches the minimax lower bound.

中文翻译：

---

最小最大速率最优自适应最近邻分类和回归

$ k $ 最近邻（kNN）方法是一种用于分类和回归的简单且流行的统计方法。对于分类和回归问题，现有工作表明，如果特征向量的分布具有有限支持，并且概率密度函数的支持在零范围内，则标准kNN方法的收敛速度如下： $ k $ 对于所有测试样品都是相同的，是minimax最优的。相反，如果分布具有无穷大的支持，则表明在通过标准kNN方法获得的收敛速度与minimax界限之间存在差距。为了弥补这一差距，我们提出了一种自适应kNN方法，其中 $ k $ 选择不同的样本。我们的选择规则不需要精确了解要素的基础分布。所提出的自适应方法明显优于标准方法。我们表征了所提出的自适应方法的收敛速度，并表明它与minimax下界匹配。