Automatic Segmentation of Moving Objects in Video Sequences

The general approach that motivates the need for segmentation is that the problem of analyzing a complete scene can be subdivided into a set of simpler problems, of analyzing its primitives. Then, each primitive object in the scene can be processed independently. This approach greatly simplifies many high-level vision problems, such as recognition, classification, scene analysis,etc. It is important to make a distinction between low-level segmentation and high-level segmentation. low-level segmentation seeks to partition images into regions of similar attribute. Some of the most fundamental attributes are luminance , color, and optical flow. The obtained partition consists of primitive regions that usually do not have a one-to-one correspondence with physical objects. However, the ultimate goal of the segmentation process is to divide an image into the physical objects composing it, so that each region constitutes a semantically meaningful entity. In order to make a clear distinction between these two problems, the later process is often referred to as high-level segmentation. The main problem with this definition of high-level segmentation, is that it is a semantical definition, rather than a formal one. In other words, without some sort of a-priori contextual knowledge, this problem cannot be solved unambiguously. The only known system that solves this problem is the human visual system. Yet, the human brain does not specifically address the segmentation problem, rather, it considers the problems of segmentation and recognition simultaneously. Clearly, the solutions to these two problems are highly dependent. Once an object is recognized, it is much easier to segment it correctly, since we now have information about the properties of the object, such as shape, texture, etc. On the other hand, it is much easier to identify an object once it has been separated from its surrounding [login to view URL], theoretically, considering segmentation and recognition jointly, provides the contextual information needed to solve the problem. However, recognition in itself is probably the most difficult problem in high-level vision, hence it would be impractical to consider both problems jointly if we are only looking for the solution of the segmentation problem. A different approach would be to gain the additional information needed by analyzing a sequence of images, instead of just one image. If we assume that all objects of interest in the scene are moving throughout the sequence, then we might have enough information in order to segment the object. In this case, we use motion as cue for segmentation. Note that this extension does not provide us with enough information to fully segment the scene, since any non-moving objects will be considered as static background. However, a partial solution can now be obtained in the form of segmentation of moving objects in the scene. This problem is commonly referred to as spatio-temporal segmentation, since it uses both the spatial information (in the image domain), and the temporal information (in the time domain). Spatio-temporal segmentation is a very important problem, that can be used as a basis for many high-level vision tasks. However, in the last two decades, most of the interest in this problem has come from the field of video coding.