Abstract: The main color extraction algorithm is proposed to extract the color feature of clothing image in this paper. The idea of this algorithm is to firstly judge the clothing is whether flowery or non flowery according to the main color of the garment, and then arrange the eigenvalues of color histogram based on descending order ,remove the smaller part of the eigenvalues and keep the main color eigenvalues. In clothing image retrieval, firstly, we should make the screening of the image in the database according to the clothing image is whether flowery or non flowery ; and then the second round selection was conducted according to the main color characteristic value, and the final retrieval results were obtained. The experimental results show that the recall ratio and precision ratio of the the improved algorithm compared with other existing color feature extraction methods, whether the clothing of simple background or complex background, flowery or non flowery, has obvious advantages.