Hrishikesh Bhaumik¹, Siddhartha Bhattacharyya¹ and Susanta Chakraborty², ¹RCC Institute of Information Technology, India and ²Bengal Engineering and Science University, India

Segmentation of a video into its constituent shots is a fundamental task for indexing and analysis in content based video retrieval systems. In this paper, a novel approach is presented for accurately detecting the shot boundaries in real time video streams, without any a priori knowledge about the content or type of the video. The edges of objects in a video frame are detected using a spatio-temporal fuzzy hostility index. These edges are treated as features of the frame. The correlation between the features is computed for successive incoming frames of the video. The mean and standard deviation of the correlation values obtained are updated as new video frames are streamed in. This is done to dynamically set the threshold value using the three-sigma rule for detecting the shot boundary (abrupt transition). A look back mechanism forms an important part of the proposed algorithm to detect any missed hard cuts, especially during the start of the video. The proposed method is shown to be applicable for online video analysis and summarization systems. In an experimental evaluation on a heterogeneous test set, consisting of videos from sports, movie songs and music albums, the proposed method achieves 99.24% recall and 99.35% precision on the average.

Real time video segmentation, spatio-temporal fuzzy hostility index, image correlation, three-sigma rule.