Delineation of agricultural fields is desirable for operational monitoring of agricultural production and is essential to support food security. Due to large within-class variance of pixel values and small inter-class difference, automated field delineation remains to be a challenging task. In this study, a strategy is proposed to effectively address this issue. Firstly, a framework was developed using the Canny operator connected with the Watershed segmentation algorithm (CW) to quickly label the training dataset, which minimizes the workload of dataset generation in comparison with the commonly used manual vectorization. Secondly, a CW-trained deep semantic segmentation network, recurrent residual U-Net, was selected to mine the low level and deep semantic features. Finally, a boundary connecting method (to integrate fragmented boundaries) was used to generate the agricultural field boundary. The proposed methods are tested over smallholder agricultural landscape in Heilongjiang province, China, using Sentinel-2 imagery. Compared with the U-Net (overall accuracy (OA) 82.18%), the residual U-Net (ResU-Net, with OA 85.78%), traditional object-based image analysis (OBIA, with OA about 82%), and the existing 10-m resolution global land cover map (FROM-GLC10), the proposed method shows an improved performance (OA 89.28%, and Kappa 0.85). The successful application of the proposed method suggests that the recurrent residual U-Net model has great universality in agricultural field boundary extraction, and the automated technique has the potential of being applied to other regions.