Video Summarization Techniques: A Review

With the advancement of technology, the enormous amount of video data is generated on the daily basis. The processing of such huge chunky videos requires high storage, high computational processing power, and consumes a lot of time. Extraction of features from the video is a time-consuming task because the user has to watch the entire video. A large number of editing tools exist that require expertise that is highly expensive. The video summarization is used to overcome these issues that deal with lengthy videos and condense those, based on the various features. This paper presents a review and comparative analysis of video summarization techniques. Initially, these methods have been classified into different categories based on their characteristics. After that, the discussion is made related to the domain directions, applications, pros/cons, and challenges for existing video summarization approaches.

REFERENCES

[1] A. S. Murugan, K. S. Devi., A. Sivaranjani, P. Srinivasan, “A study on various methods used for video summarization and moving object detection for video surveillance applications”, Multimedia Tools and Applications, 77(18), pp. 23273-23290,2018.
[2] A. Bora, S. Sharma, “A Review on Video Summarization Approcahes: Recent Advances and Directions”. In 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN), pp. 601-606,2018.
[3] R. Kansagara, D. Thakore, M. Joshi, “A study on video summarization techniques”. International journal of innovative research in computer and communication engi-neering, 2(2014).
[4] K. Darabi, G. Ghinea, “Personalized video summarization by highest quality frames”. In 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW) (pp. 1-6), 2014.
[5] Z. Zhao, S. Jiang, Q. Huang, G. Zhu, “Highlight summarization in sports video based on replay detection”. In 2006 IEEE international conference on multimedia and expo, pp. 1613-1616, 2006.
[6] G. Irie, T. Satou, A. Kojima, T. Yamasaki, K. Aizawa, “Automatic trailer generation”. In Proceedings of the 18th ACM international conference on Multimedia, pp. 839-842, 2010.
[7] P. Kalaivani , S. M. M. Roomi, Towards Comprehensive Understanding of Event Detection and Video Summarization Approaches”. In Recent Trends and Challenges in ComputationalModels (ICRTCCM), 2017 Second International Conference, IEEE (2017, February): 61-66
[8] K. Kumar, D. D Shrimankar. & N. Singh. “Event BAGGING: A novelevent summarization approach in multiview surveillance videos”.In Innovations in Electronics, Signal Processing and Communication(IESC), 2017 International Conference. pp: 106-111, 2017.
[9] U. Damnjanovic, V. Fernandez, E. Izquierdo, “Event Detection and Clustering for Surveillance Video Summarization”. In: Proceedings of the Ninth International Workshop on Image Analysis for Multimedia Interactive Services. IEEE Computer Society, Washington, USA (2008).
[10] S. S. Thomas, S. Gupta, V. K. Subramanian, Perceptual video summarization—“A new framework for video summarization”. IEEE Transactions on Circuits and Systems for Video Technology, 27(8), pp. 1790-1802, 2016.
[11] F. Cricri, S. Mate, I. D. Curcio & M. Gabbouj. “Salient event detection in basketball mobile videos”. In Multimedia (ISM), 2014 IEEE International Symposium. IEEE pp. 63-70, 2014.
[12] S.J. Andaloussi, A. Mohamed, N. Madrane & A. Sekkaki. “Soccer video summarization using video content analysis and social media streams”. In Proceedings of the 2014 IEEE/ACM International Symposium on Big Data Computing, IEEE Computer Society,pp.1-7, 2014.
[13] M. Cote, F. Jean, A. B. Albu & D. Capson. “Video summarization for remote invigilation of online exams”. In Applications of Computer Vision (WACV), 2016 IEEE Winter Conference, pp. 1-9, 2016.
[14] K. Kumar & D. D. Shrimankar. “F-DES: Fast and Deep Event Summarization”. IEEE Transactions on Multimedia, 20(2), pp. 323- 334, 2018.
[15] W. Wolf, Key frame selection by motion analysis. In: ICASSP, vol. 2, (1996) pp. 1228– 1231.
[16] E. Bulut, T. Capin, “Key Frame Extraction from Motion Capture Data by Curve Saliency”. In: Proceedings of 20th Annual Conference on Computer Animation and Social Agents, Belgium (2007).
[17] C. Li, Y.T. Wu, S.S. Yu, T. Chen, “Motion-focusing key frame extraction and video summarization for lane surveillance system”. In: 16th IEEE International Conference on Image Processing (ICIP), pp. 7–10, 2009.
[18] M. Ajmal, M. Naseer, F. Ahmad, A. Saleem. “Human Motion Trajectory Analysis Based Video Summarization”. 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA) (2017):550-55510.1109/ICMLA.pp. 0-203, 2017.
[19] J. Almeida, R. D. S. Torres, N. J. Leite, “Rapid video summarization on compressed video”. In 2010 IEEE International Symposium on Multimedia , pp. 113-120,2010.
[20] H.J. Zhang, J. Wu, D. Zhong, S.W. Smoliar, “An integrated system for content based video retrieval and browsing”. Pattern Recognition 30, pp.643–658, 1997.
[21] O. Elharrouss, N Almaadeed, S. Al-Maadeed, A. Bouridane, and A. Beghdadi,. “A combined multiple action recognition and summarization for surveillance video sequences”. Applied Intelligence, pp. 1-23, 2020
[22] F. Wang, C.W. Ngo, “Summarizing rushes videos by motion, object and event understanding”. IEEE Transactions on Multimedia 14 (2012).
[23] Y.J. Lee, J. Ghosh, K. Grauman, “Discovering Important People and Objects for Egocentric Video Summarization”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2012).
[24] M. Miniakhmetova & M. Zymbler, “An approach to personalized video summarization based on user preferences analysis”. In Application of Information and Communication Technologies (AICT), 2015 9th International Conference, pp.153-155, 2015.
[25] A. A. Rav, Y. Pritch, S. Peleg, Making a long video short: Dynamic video synopsis. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), Vol. 1, pp. 435-441, 2006.
[26] K. Davila , R. Zanibbi. “Whiteboard Video Summarization via SpatioTemporal Conflict Minimization”. In Document Analysis and Recognition (ICDAR), 2017 14th IAPR International Conference, vol 1, pp. 355-362, 2017.
[27] N. Baghel, S. C. Raikwar and C. Bhatnagar. ‘Image Conditioned Keyframe-Based Video Summarization Using Object Detection”. 2020 arXiv preprint arXiv:2009.05269.
[28] C. W. Ngo, Y. F. Ma, H. J. Zhang, “Video summarization and scene detection by graph modeling”. IEEE Transactions on circuits and systems for video technology, 15(2) pp. 296-305, 2005.
[29] M. A. Nahian, A. S. M. Iftekhar , M. T. Islam, S. M. Rahman, D. Hatzinakos. “CNN-Based Prediction of Frame-Level Shot Importance for Video Summarization”, arXiv preprint arXiv:1708.07023, 2017
[30] J. Xu, L. Mukherjee, Y. Li, J. Warner, J. M. Rehg ,V. Singh. “Gazeenabled egocentric video summarization via constrained submodular maximization”. In Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference, pp. 2235-2244, 2015.
[31] M. Srinivas, M. M. Pai, R. M. Pai, “An Improved Algorithm for Video Summarization–A Rank Based Approach”. Procedia Computer Science, 89, pp.812-819, 2016.
[32] Y. Pritch, S. Ratovitch, A. Hendel, S. Peleg, “Clustered synopsis of surveillance video”. In: 6th IEEE Int Conf. on Advance Video and Signal Base Selection (AVSS 2009), Genoa, Italy, pp. 2–4, 2019.
[33] S. Jadon, and M. Jasim, “Video summarization using keyframe extraction and video skimming”. 2019. arXiv preprint arXiv:1910.04792.
[34] M. Otani, Y. Nakashima, E. Rahtu, J. Heikkilä, and N. Yokoya. “Video summarization using deep semantic features. In Asian Conference on Computer Vision, pp. 361-377, 2016.
[35] V. Raut, and R. Gunjan, “Video summarization approaches in wireless capsule endoscopy: A review”. In E3S Web of Conferences, Vol. 170, pp. 03005, 2020.
[36] F. Chen, M. Cooper, J. Adcock, “Video Summarization Preserving Dynamic Content”. In: Proceedings of the International Workshop on TRECVID Video Summarization, 2007.
[37] J. Mohan, and M. S. Nair, “Domain independent static video summarization using sparse autoencoders and K-means clustering”. Journal of Intelligent & Fuzzy Systems, 36(3), pp. 1945-1955, 2019.
[38] K.A. Peker, F.I. Bashir, “Content-Based Video Summarization using SpectralClustering”. Mitsubishi Electric Research Laboratories Cambridge, MA. Universityof Illinois at Chicago, Chicago, IL (2009).
[39] V.T Chasanis, C.L Likas, N.P Galatsanos,”Scene detection in videos using shot clustering and sequence alignment”. IEEE Trans. Multimed. 11(1), pp. 89–100, 2009.
[40] W. Sabbar, A. Chergui, A. Bekkhoucha, ‘Video summarization using shot segmentation and local motion estimation”, Innovative Computing Technology (INTECH), 2012 Second International Conference. pp.190- 193, 2012.
[41] G. Evangelopoulos, K. Rapantzikos, A. Potamianos, P. Maragos, A. Zlatintsi, Y. Avrithis, “Movie Summarization Based on Audio-Visual Valiency Detection”. In: IEEE Intl Conf. Image Processing (ICIP), San Diego, CA,2008.
[42] S. Uchihachi, J. Foote, L. Wilcox, “Automatic Video Summarization Using a Measure of Shot Importance and a Frame Packing Method”. United States Patent 6, pp.535-639, Mar.2003.
[43] Z. Lu, K. Grauman, “Story-Driven Summarization for Egocentric Video. 2013 IEEE Conference on Computer Vision and Pattern” Recognition, pp. 2714–2721, 2013. http://doi.org/10.1109/CVPR.2013.350.
[44] S. Lu, M. R. Lyu, I. King, H. K. Sar, “Semantic Video Summarization Using Mutual Reinforcement Principle and Shot Arrangement Patterns”, 2005.
[45] J. Varghese, K. R. Nair, “An Algorithmic Approach for General Video Summarization”. In 2015 Fifth International Conference on Advances in Computing and Communications (ICACC), pp. 7-11, 2015.
[46] X. Wang, X. Nie, X. Liu, B. Wang, and Y. Yin, “Modality correlation-based video summarization. Multimedia Tools and Applications”, pp. 1-16, 2020.
[47] K. Muhammad, T. Hussain, and S.W. Baik, “Efficient CNN based summarization of surveillance videos for resource-constrained devices”. Pattern Recognition Letters, 130, pp. 370-375, 2020.
[48] M. Ma, S. Mei, S. Wan, Z. Wang, D. D. Feng. “Robust video summarization using collaborative representation of adjacent frames”. Multimedia Tools and Applications, 78(20), pp.28985-29005, 2019.
[49] P. K. Lai, M. Décombas, K. Moutet, R. Laganière, “Video summarization of surveillance cameras”. In 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 286-294, 2016.
[50] K. Li, S. Li, S. Oh & Y. Fu. “Videography-Based Unconstrained Video Analysis”. IEEE Transactions on Image Processing, 26(5), pp. 2261-2273, 2017.
[51] S. Mei, M. Ma, S. Wan, J. Hou, Z. Wang, D.D. Feng, “Patch based Video Summarization with Block Sparse Representation”. IEEE Transactions on Multimedia, 2020.
[52] M. Ma, S. Mei, S. Wan, J. Hou, Z. Wang, and D.D. Feng, “Video summarization via block sparse dictionary selection”. Neurocomputing, pp. 378, 197-209, 2020.
[53] S. Tinumol, J.P. Jiby, “A survey on video summarization techniques”. Int J Comput Appl 132(13) pp.31–33, 2015.
[54] S. Khan, S. Pawar. “Video summarization: survey on event detection and summarization in soccer videos”. International Journal of Advanced Computer Science and Applications, 6(11), 2015.
[55] H. B. U. Haq, H. U. R. Kiyani, S. K. Toor, S. Zafar, I. Khalid. The Popular Tools Of Data Sciences: Benefits, Challenges and Applications. IJCSNS International Journal of Computer Science and Network Security, VOL.20 No.5, 2020. http://search.ijcsns.org/07_book/html/202005/202005008.html
[56] S. Patil, D. Phalke, International Journal for Research in Applied Science & Engineering Technology (IJRASET). Retrieved from https://www.ijraset.com/fileserve.php?FID=19635, 2019.
[57] A. Workie, R. Sharma, Y. K. Chung. “Digital Video Summarization Techniques: A Survey”, 2020.