Abstract: The theory of ontology is applied to the video feature description together with its interface in MPEG-7 and video information based on semantic hierarchy, to build up a video semantic description model based on MPEG-7 and ontology .This model makes use of the RDFS and Owls good capability to describe the semantic information and self-extend capability to describe itself. It overcomes the weakness of the XML schema used in MPEG-7 that is hard to express the semantic information of video. By using this model, we can build an effective engine for semantic reasoning.