Socials

1. What Is a Feature in Computer Vision?

In computer vision and image processing, a feature is a piece of information about the content of an image — typically about whether a certain region of the image has certain properties. Features may be specific structures in the image such as points, edges or objects, or they may be the result of a general neighborhood operation or feature detection applied to the image.

More broadly, a feature is any piece of information that is relevant for solving the computational task related to a certain application — the same sense as "feature" in machine learning and pattern recognition generally, though image processing has a very sophisticated collection of features.

There is no universal or exact definition of what constitutes a feature, and the exact definition often depends on the problem or the type of application. Nevertheless, a feature is typically defined as an "interesting" part of an image, and features are used as a starting point for many computer vision algorithms.

2. Why Does Feature Detection Matter?

Since features are used as the starting point and main primitives for subsequent algorithms, the overall algorithm will often only be as good as its feature detector. Consequently, the most desirable property for a feature detector is repeatability — whether or not the same feature will be detected in two or more different images of the same scene.

Feature detection is a low-level image processing operation. It is usually performed as the first operation on an image and examines every pixel to see if there is a feature present at that pixel. As a built-in prerequisite to feature detection, the input image is usually smoothed by a Gaussian kernel in a scale-space representation, and one or several feature images are computed, often expressed in terms of local image derivative operations.

Occasionally, when feature detection is computationally expensive and there are time constraints, a higher-level algorithm may be used to guide the feature detection stage so that only certain parts of the image are searched for features.

3. Types of Features

3.1 Edges

Edges are points where there is a boundary between two image regions. In practice, edges are usually defined as sets of point in the image that have a strong gradient magnitude.

Furthermore, some common algorithms will chain high gradient points together to form a a more complete description of an edge, usually placing constraints on properties such as, smoothness, and gradient value.

Common detectors: Canny, Sobel, Prewitt, Roberts Cross.

3.2 Corners / Interest Points

The terms corners and interest points are used somewhat interchangeably and refer to point-like features in an image which have a local two-dimensional structure. The name “corner“ arose since early algorithms first performed edge detection.

These algorithms were then developed so that explicit edge detection was no longer required — for instance by looking for high levels of curvature in the image gradient. It was then noticed that the so-called corners were also being detected on parts of the image that were not corners in the traditional sense; for instance, a small bright spot on a dark background may be detected. These points are frequently known as interest points, but the term "corner" is used by tradition

3.3 Blobs / Regions of Interest

Blobs provide a complementary description of image structures in terms of regions, as opposed to corners that are more point-like. Nevertheless, blob descriptors may often contain a preferred point — a local maximum of an operator response or a center of gravity — which means that many blob detectors may also be regarded as interest point operators. Blob detectors can detect areas in an image that are too smooth to be detected by a corner detector.

Consider shrinking an image and then performing corner detection. The detector will respond to points that are sharp in the shrunk image, but may be smooth in the original image. It is at this point that the difference between a corner detector and a blob detector becomes somewhat vague — a distinction that can be remedied by including an appropriate notion of scale.

Common detectors: Laplacian of Gaussian (LoG), Difference of Gaussians (DoG), Determinant of Hessian (DoH), MSER.

3.4 Ridges

For elongated objects, the notion of ridges is a natural tool. A ridge descriptor computed from a grey-level image can be seen as a generalization of a medial axis. From a practical viewpoint, a ridge can be thought of as a one-dimensional curve that represents an axis of symmetry, and in addition has an attribute of local ridge width associated with each ridge point.

Unfortunately, it is algorithmically harder to extract ridge features from general classes of grey-level images than edge, corner, or blob features. Nevertheless, ridge descriptors are frequently used for road extraction in aerial images and for extracting blood vessels in medical images.

Common detectors: Principal curvature ridges.

4. Feature Vectors and Feature Spaces

In some applications, it is not sufficient to extract only one type of feature. Instead, two or more different features are extracted, resulting in two or more feature descriptors at each image point. A common practice is to organize the information provided by all these descriptors as the elements of one single vector, commonly referred to as a feature vector. The set of all possible feature vectors constitutes a feature space.

A common example of feature vectors appears when each image point is to be classified as belonging to a specific class. Assuming that each image point has a corresponding feature vector based on a suitable set of features — meaning that each class is well separated in the corresponding feature space — the classification of each image point can be done using standard classification methods.

4. Feature Vectors and Feature Spaces

In some applications, it is not sufficient to extract only one type of feature. Instead, two or more different features are extracted, resulting in two or more feature descriptors at each image point. A common practice is to organize the information provided by all these descriptors as the elements of one single vector, commonly referred to as a feature vector. The set of all possible feature vectors constitutes a feature space.

A common example of feature vectors appears when each image point is to be classified as belonging to a specific class. Assuming that each image point has a corresponding feature vector based on a suitable set of features — meaning that each class is well separated in the corresponding feature space — the classification of each image point can be done using standard classification methods.

5. Feature Extraction and Descriptors

Once features have been detected, a local image patch around the feature can be extracted. This extraction may involve considerable amounts of image processing. The result is known as a feature descriptor or feature vector. Among the approaches used for feature description, one can mention N-jets and local histograms. In addition, the feature detection step itself may also provide complementary attributes, such as edge orientation and gradient magnitude in edge detection, and the polarity and strength of the blob in blob detection.