Okra is widely used as vegetable, food ingredient, and traditional medicine in China for various applications. The main healthy properties are accepted to originate from the high polysaccharide content of okra. In this study, a homogenous pectic polysaccharide WOP-2 was extracted, isolated, and purified from okra by DEAE-cellulose and Sepharose CL-6B column, and its molecular weight was estimated to be 580 kDa. Monosaccharide composition was analyzed to be Rha (21.4%), GalA (34.9%), Gal (29.6%), GlcA (4.5%), Glc (5.9%), and Ara (3.7%) by HPLC with PMP derivatization. Combined with NMR analysis, WOP-2 was elucidated to be a rhamnogalacturonan I backbone with type-II arabinogalactan side-chains substituted partly at O-4 of Rhap. WOP-2 exhibits significant anti-hyperglycemic activity on STZ-induced diabetic mice by inhibiting the lipid peroxidation chain reaction probably. Overall, these results provide new insight into structure–activity relationships of polysaccharide from okra and provide impetus towards the development of polysaccharide-based therapeutics on diabetes.