Abstract: Aiming at the problem of the interactive and complex background extraction effect of Grab Cut algorithm applied in the foreground retrieval of garment image in garment retrieval system, a fully automated Grab Cut algorithm is proposed to adopt different methods to automatically generate the initial rectangle and then extract the foreground for single backgrounds and complex background images, depending on the background. As the algorithm still has under-segmentation and over-segmentation problems，a new fully automated Grab Cut algorithm that combined with region growing algorithm is proposed. The results show that, the improved Grab Cut algorithm is superior to classical algorithms for both single background and complex background，and can not only accurately extract the foreground of clothing images, but also effectively reduce the over-segmentation within the images.