In order to realize single target tracking and target statistics in video monitoring, this paper proposes a method of combining background difference and inter-frame difference. The method can detect moving objects effectively and improve precision of target detection. Firstly, median filter is adopted to remove salt and pepper noise in the image and morphology processing is introduced to reduce noise of extracted target binary image. Secondly, target tracking is realized well by using multi-point location algorithm. Lastly, human is detected according to moving target characteristic parameters, such as body ratio between height and width, target area, center of mass. When pedestrians go into a specific area in video monitoring, system can statistics the pedestrian. Experimental results show that proposed method owns a stability and high accuracy. The proposed system could satisfy the real-time requirement.