Video fragment processing by Ky Fan norm

Abstract

In this study, we focused on the formalization of video frame descriptions in the context of solving video segmentation problem. Since native video data can have various sizes, dividing each frame into blocks allows present image frame as a square matrix for a formal description. The frame block is a matrix of arbitrary dimensions. The ability to skip the step of matrix transformation to a square dimension or vectorization using some descriptor allows to reduce computational costs, freeing up computational resources required for this transformation. In our study, we used Ky Fan norm value as image frame block descriptor. The Ky Fan norm is built on top of matrix singular values. A singular decomposition does not impose restrictions on either the dimension or the character of the elements of the original matrix. We conducted a comparative analysis of the effectiveness of the obtained descriptor for different video data sizes and with different aspect ratios, showing that the change in the descriptor for each block is independent of the video size and aspect ratios. Changes in the descriptors for each block from frame to frame are identical for video data of varying sizes. This means that as a result of such fragment transform, a square matrix of a fixed size is created, regardless of the output video size. This makes it possible to unify further processing of the video, which can be useful for the task of information search in large video databases under the conditions of providing a query "ad exemplum". In this case, we can analyze the existing database in offline mode and match each video with a fixed square matrix of descriptors, which will significantly reduce the time and amount of resources when matching with the query. Also, this approach can be effectively used to analyze video data for the motion detection and scene change tracking.