Digital image interpolation. What is camera interpolation in a phone and why is it needed?

All tips

The mobile phone market is filled with models with cameras with huge resolutions. There are even relatively inexpensive smartphones with sensors with a resolution of 16-20 megapixels. An unknowing buyer is chasing a “cool” camera and prefers the phone with a higher camera resolution. He doesn’t even realize that he is falling for the bait of marketers and sellers.

What is permission?

Camera resolution is a parameter that indicates the final size of the image. It only determines how large the resulting image will be, that is, its width and height in pixels. Important: the picture quality does not change. The photo may turn out to be of low quality, but large due to the resolution.

Resolution does not affect quality. It was impossible not to mention this in the context of smartphone camera interpolation. Now you can get straight to the point.

What is camera interpolation in a phone?

Camera interpolation is an artificial increase in image resolution. It is images, and not That is, this is special software, thanks to which an image with a resolution of 8 megapixels is interpolated to 13 megapixels or more (or less).

If we draw an analogy, camera interpolation is similar to binoculars. These devices enlarge the image, but do not make it look better or more detailed. So if interpolation is indicated in the phone's specifications, then the actual camera resolution may be lower than stated. It's not good or bad, it just is.

What is it for?

Interpolation was invented to increase the size of the image, nothing more. Now this is a ploy by marketers and manufacturers who are trying to sell a product. They indicate in large numbers on the advertising poster the resolution of the phone's camera and position it as an advantage or something good. Not only does resolution itself not affect the quality of photographs, but it can also be interpolated.

Literally 3-4 years ago, many manufacturers were chasing the number of megapixels and in various ways tried to cram sensors with as many megapixels as possible into their smartphones. This is how smartphones with cameras with a resolution of 5, 8, 12, 15, 21 megapixels appeared. At the same time, they could take photographs like the cheapest point-and-shoot cameras, but when buyers saw the “18 MP camera” sticker, they immediately wanted to buy such a phone. With the advent of interpolation, it has become easier to sell such smartphones due to the ability to artificially add megapixels to the camera. Of course, photo quality began to improve over time, but certainly not because of resolution or interpolation, but because of natural progress in terms of sensor development and software.

Technical side

What is camera interpolation in a phone technically, since all the text above described only the basic idea?

Using special software, new pixels are “drawn” on the image. For example, to enlarge an image by 2 times, a new line is added after each line of pixels in the image. Each pixel in this new line is filled with a color. The fill color is calculated by a special algorithm. The very first way is to pour new line colors that the nearest pixels have. The result of such processing will be terrible, but similar method requires a minimum of computational operations.

Most often, another method is used. That is, new rows of pixels are added to the original image. Each pixel is filled with a color, which in turn is calculated as the average of neighboring pixels. This method gives top scores, but requires more computational operations.

Fortunately, modern mobile processors are fast, and in practice the user does not notice how the program edits the image, trying to artificially increase its size.

There are many advanced interpolation methods and algorithms that are constantly being improved: the boundaries of the transition between colors are improved, the lines become more accurate and clear. It doesn't matter how all these algorithms are built. The very idea of camera interpolation is banal and is unlikely to catch on in the near future. Interpolation cannot make an image more detailed, add new details, or improve it in any other way. Only in films does a small blurry picture become clear after applying a couple of filters. In practice this cannot happen.

Do you need interpolation?

Many users, out of ignorance, ask questions on various forums about how to do camera interpolation, believing that this will improve the quality of images. In fact, interpolation not only will not improve the quality of the picture, but may even make it worse, because new pixels will be added to the photos, and due to the not always accurate calculation of colors for filling, the photo may have undetailed areas and graininess. As a result, quality drops.

So interpolation in the phone is a marketing ploy that is completely unnecessary. It can increase not only the resolution of the photo, but also the cost of the smartphone itself. Don't fall for the tricks of sellers and manufacturers.

Why does an image scaled with bicubic interpolation look different from Photoshop? Why does one program resize quickly, while another does not, although the result is the same. Which resizing method is better for increasing and which for decreasing. What do filters do and how are they different?

In general, this was an introduction to another article, but it dragged on and resulted in a separate article.

This man is sitting among the daisies to draw your attention to the article.

For a visual comparison, I will use images of the same resolution 1920x1280 (one, the other), which I will reduce to sizes 330x220, 1067x667 and 4800x3200. Under the illustrations it will be written how many milliseconds it took to resize to a particular resolution. The numbers are given only to understand the complexity of the algorithm, so the specific hardware or software on which they are obtained is not so important.

Nearest neighbor

This is the most primitive and quick method. For each pixel of the final image, one pixel of the original image is selected that is closest to its position, taking into account scaling. This method produces a pixelated image when zoomed in and a highly grainy image when zoomed out.

In general, the quality and performance of any reduction method can be assessed by the ratio of the number of pixels involved in the formation of the final image to the number of pixels in the original image. The larger this ratio, the more likely the algorithm is better and slower. A ratio of one means that at least every pixel in the original image contributed to the final one. But for advanced methods it can be more than one. So, if, for example, we reduce the image using the nearest neighbor method by 3 times on each side, then this ratio is equal to 1/9. Those. most of the original pixels are not taken into account in any way.

1920×1280 → 330×220 = 0.12 ms
1920×1280 → 1067×667 = 1.86 ms

The theoretical speed of operation depends only on the size of the final image. In practice, when decreasing, processor cache misses contribute: smaller scale, the less data is used from each line loaded into the cache.

The method is deliberately used for reduction extremely rarely, because gives very poor quality, although it can be useful when enlarged. Due to its speed and ease of implementation, it is included in all libraries and applications that work with graphics.

Affine transformations

Affine transformations are a common method for distorting images. They allow you to rotate, stretch and flip an image in one operation. Therefore, in many applications and libraries that implement the affine transformation method, the image modification function is simply a wrapper that calculates the coefficients for the transformation.

The principle of operation is that for each point in the final image, a fixed set of points in the source image is taken and interpolated in accordance with their relative position and the selected filter. The number of points also depends on the filter. For bilinear interpolation, 2x2 source pixels are taken, for bicubic interpolation, 4x4. This method produces a smooth image when zoomed in, but when zoomed out the result is very similar to its nearest neighbor. See for yourself: theoretically, with a bicubic filter and 3 times reduction, the ratio of processed pixels to original ones is 4² / 3² = 1.78. In practice, the result is much worse because In existing implementations, the filter window and interpolation function are not scaled according to the image scale, and pixels closer to the edge of the window are taken with negative coefficients (according to the function), i.e. do not make a useful contribution to the final image. As a result, an image reduced with a bicubic filter differs from an image reduced with a bilinear filter only in that it is even sharper. Well, for a bilinear filter and three-fold reduction, the ratio of processed pixels to original ones is 2² / 3² = 0.44, which is not fundamentally different from the nearest neighbor. In fact, affine transformations cannot be used to reduce by more than a factor of 2. And even when scaled down to two times, they give noticeable ladder effects for the lines.

Theoretically, there should be implementations of affine transformations that scale the filter window and the filter itself in accordance with the specified distortions, but in popular open-source libraries source code I've never met anyone like that.

1920×1280 → 330×220 = 6.13 ms
1920×1280 → 1067×667 = 17.7 ms
1920×1280 → 4800×3200 = 869 ms

The running time is noticeably longer than its nearest neighbor and depends on the size of the final image and the window size of the selected filter. It practically does not depend on cache misses, because the original pixels are used at least two at a time.

My humble opinion, that using this method to arbitrarily reduce images is simply is a bug, because the result is very bad and similar to its nearest neighbor, and this method requires much more resources. However, this method has found wide application in programs and libraries. The most surprising thing is that this method is used in all browsers for the canvas drawImage() method (illustrative example), although more accurate methods are used to simply display images in an element (except for IE, it uses affine transformations for both cases). In addition, this method is used in OpenCV, the current version of the Python library Pillow (I hope to write about this separately), and in Paint.NET.

In addition, this is the method used by video cards to render 3D scenes. But the difference is that video cards prepare a set of reduced versions (mip levels) in advance for each texture, and for final rendering a level with such a resolution is selected so that the texture is reduced no more than twice. In addition, to eliminate a sharp jump when changing the mip level (when a textured object approaches or moves away), linear interpolation is used between adjacent mip levels (this is already trilinear filtering). Thus, to draw each pixel of a 3D object, you need to interpolate between 2³ pixels. This gives a result acceptable for a fast-moving picture in a time linear with respect to the final resolution.

Supersampling

Using this method, the same mip levels are created, with the help of which (to greatly simplify) full-screen anti-aliasing works in games. Its essence is to split the original image into a grid of final pixels and add up all the original pixels per each pixel of the final image in accordance with the area covered by the final pixel. When using this method for enlargement, for every pixel in the final image there is exactly one pixel in the original. Therefore the result for augmentation is equal to the nearest neighbor.

There are two subtypes of this method: with rounding of pixel boundaries to the nearest whole number of pixels and without. In the first case, the algorithm becomes unsuitable for scaling less than 3 times, because any one final pixel may have one original pixel, and the neighboring one may have four (2x2), which leads to disproportions at the local level. At the same time, the rounding algorithm can obviously be used in cases where the size of the original image is a multiple of the size of the final image, or the reduction scale is quite small (versions with a resolution of 330x220 are almost the same). The ratio of processed pixels to the original ones when rounding boundaries is always equal to one.

1920×1280 → 330×220 = 7 ms
1920×1280 → 1067×667 = 15 ms
1920×1280 → 4800×3200 = 22.5 ms

The subview without rounding gives excellent quality when zoomed out at any scale, but when zoomed in it gives a strange effect where most of the original pixel in the final image looks uniform, but there is a transition visible at the edges. The ratio of processed pixels to original pixels without edge rounding can be from one to four, because each original pixel contributes either to one final pixel, or to two neighboring pixels, or to four neighboring pixels.

1920×1280 → 330×220 = 19 ms
1920×1280 → 1067×667 = 45 ms
1920×1280 → 4800×3200 = 112 ms

The performance of this method for reduction is lower than that of affine transformations because all pixels of the original image are involved in the calculation of the final image. The version with rounding to the nearest bounds is usually several times faster. It is also possible to create separate versions to scale by a fixed number of times (for example, reducing by 2 times), which will be even faster.

This method is used in the gdImageCopyResampled() function of the GD library, which is part of PHP, found in OpenCV (INTER_AREA flag), Intel IPP, AMD Framewave. Libjpeg works on approximately the same principle, when it opens images reduced several times. The latter allows many applications to open JPEG images pre-resized by several times without much overhead (in practice, libjpeg opens smaller images even slightly faster than full-size ones), and then use other methods to resize them to the exact size. For example, if you need to resize a 1920x1280 JPEG to 330x220, you can open the original image at 480x320 and then downsize it to the desired 330x220.

Convolution

This method is similar to affine transformations in that it uses filters, but instead of having a fixed window, it has a window proportional to the scale. For example, if the filter window size is 6, and the image size is reduced by 2.5 times, then (2.5 * 6)² = 225 pixels take part in the formation of each pixel of the final image, which is much more than in the case of supersampling (from 9 to 16). Fortunately, convolutions can be counted in 2 passes, first in one direction, then in the other, so the algorithmic complexity of calculating each pixel is not 225, but only (2.5 * 6) * 2 = 30. The contribution of each original pixel to the final one is as follows: times determined by the filter. The ratio of processed pixels to the original ones is entirely determined by the size of the filter window and is equal to its square. Those. for a bilinear filter this ratio will be 4, for a bicubic filter 16, for Lanczos 36. The algorithm works great for both reduction and augmentation.

1920×1280 → 330×220 = 76 ms
1920×1280 → 1067×667 = 160 ms
1920×1280 → 4800×3200 = 1540 ms

The speed of this method depends on all parameters: the size of the source image, the size of the final image, and the size of the filter window.

This is the method implemented in ImageMagick, GIMP, and in the current version of Pillow with the ANTIALIAS flag.

One of the advantages of this method is that filters can be specified as a separate function that is not tied in any way to the implementation of the method. However, the function of the filter itself can be quite complex without much loss of performance, because the coefficients for all pixels in one column and for all pixels in one row are calculated only once. Those. The filter function itself is called only (m + n) * w times, where m and n are the dimensions of the final image, and w is the size of the filter window. And you can rivet a lot of these functions, if only there was a mathematical justification. In ImageMagick, for example, there are 15 of them. Here is what the most popular ones look like:

Bilinear filter (bilinear or triangle in ImageMagick)

Bicubic filter (bicubic, catrom in ImageMagick)

Lanczos filter

It is noteworthy that some filters have zones of negative coefficients (such as the bicubic filter or the Lanczos filter). This is necessary to give the transitions in the final image the sharpness that was in the original one.

An image digitized on a scanner is presented on a computer monitor during the editing process and processed in an editor so that a printed version is obtained High Quality. This chain of operations is so familiar to most users that few people think about the difficult transformations that the original undergoes along this path.

The screen version of the image is simply a matrix of dots, which is described by its dimensions in height and width. An image with dimensions of 600 by 400 will occupy a fixed proportion of screen space on any monitor, regardless of its operating principle. It will cover almost the entire screen if the resolution is 640*480, on a screen with a resolution of 1024*768 it will take up about a quarter of the space, and finally, with a resolution of 1600*1200 it will take up a little more than one-ninth of the screen area. At the same time, the physical dimensions, i.e. dimensions, which are calculated in inches and centimeters, will depend on the diagonal of the monitor.

What will be the size of the image when it is printed? For an experienced Photoshop user, the answer is obvious. The dimensions of the printed version coincide with the dimensions of the scanned original (to be extremely precise, with the dimensions of the scanning area). This natural convention is the default setting for all graphics programs; but most raster editors have by special means changing print sizes.

To set the size of the screen version of the image, which coincides with its printed version, you need to execute the main menu command View - Print Size Photoshop editor or use the panel button with the same name.

Let's say you want to print an image with a size of 600*600 pixels. These dimensions are a given; now the method of obtaining them, scanning resolution and printing installation does not matter. If you set the dimensions of the printed version to 10 inches, the resolution will be 600 dot / 10 inch = 60 dpi. Here are a number of resolution values for different print sizes:

600dot/5inch = 120dpi;
600dot/3inch = 200dpi;
600dot / 2inch = 300dpi.

All these changes do not affect the screen version at all; all its advantages and disadvantages are introduced at the scanning stage and changes in the printing area do not affect the quality of the digitized original. But this has a significant impact on the quality of the printed version.

For any printing equipment there is some optimal resolution value digital image, when the printing device will be able to convey the maximum number of details of the original. The quality of the result also depends on the type of paper chosen. This influence is especially pronounced for the most popular printing devices of our time - color inkjet printers.

Let the optimal resolution value for the selected printer and type of paper be 200 dpi. What consequences will result from printing the selected original with a resolution of 120 dpi? This solution will lead to a loss of quality, since some of the details will be lost during printing. What if you compete for results by choosing a higher print resolution? If, for example, you set it to 300 dpi or more, then redundant information will be transmitted to the printer, which it simply cannot use.

Let's say that the scanned version of the image shows mediocre quality when displayed on the monitor. Is it possible to improve the situation by printing it on high-quality paper with high resolution? The trick will not work, since printing does not add new information to the original; the printer uses only the data that was included in the image at the digitization stage. These thought experiments, of course, simplify real situation affairs, but the action of the principle of reasonable sufficiency for choosing the optimal print resolution can hardly be disputed.

So, if you fix the point dimensions of the image, then any changes in resolution entail a modification of the print area. The opposite statement is also true. IN raster graphics this transformation is usually called scaling.

Why scale the image? The reasons for this are varied and often very compelling. Many modern digital cameras middle level produce images small size, which, when printed, occupy the area of a postage stamp. Desktop Publishing require images of fixed dimensions, which may not coincide with the original dimensions, etc.

Scaling does not change physical dimensions graphic file, since it does not affect any of the parameters (number of dots, color depth) on which its value depends.

Sampling

Changing the number of pixels in an image is called sampling. This operation obviously affects the size of the screen version of the image, which on a monitor with unchanged characteristics becomes larger or smaller, depending on the specified values.

Rice. 2.4.

Let us explain this operation using an example of an image from the editor’s standard collection (Fig. 2.4). The original version of the picture, which occupies the middle position, has a resolution of 72 dpi. Doubling the resolution to 144 dpi entails an increase in the number of pixels and an increase in the linear dimensions of the screen version of the image (right sample). Reducing the resolution to 36 dpi produces exactly the opposite effects (left sample).

Unlike upscaling, sampling is not a computationally straightforward operation because it strongly interferes with the structure of the image.

Let there be an image of size 400*400 pixels. If you reduce its screen size to 300*300, then, at first glance, this means a slight intervention in the original - a reduction of only three quarters. A different picture emerges if you count the number of points before and after surgery. The original image consisted of 400*400 = 160,000 pixels, and after transformation it has 300*300 = 90,000 pixels - almost half as much. It is clear that such a large-scale operation cannot but affect the quality of the picture.

Even more complex problems have to be solved when the number of points increases. If, when reducing them, the program simply discards extra pixels, then when increasing the matrix, additional points must be “invented”. Adding new pixels is performed using special interpolation algorithms.

Reducing the number of image pixels is a relatively safe procedure that does not directly affect the quality of the original. Increasing the pixels is more complex in its algorithms and consequences. A small increment in the raster does not entail noticeable negative consequences. This kind of large-scale conversion almost always degrades the sharpness of the image, partially blurring the image.

In raster graphics, three main sampling methods have become widespread (all of them are supported by the Photoshop editor), which differ in speed and accuracy of results:

Nearest Neighbor(By adjacent pixels). The simplest interpolation method with high speed work and results are not of the highest quality. The characteristics of its nearest actual neighbor are taken as a sample for a new pixel. The method gives good results for areas with regular geometry, such as straight lines, rectangles, etc.;
Bilinear(Bilinear). This method is somewhat more difficult to implement, but gives better results compared to the method Nearest Neighbor. Options new point are calculated by averaging the color or tone characteristics of adjacent real image pixels. The method shows its advantages when the number of image pixels is reduced. A rational area of its application is the processing of images of average quality;
Bicubic(Bicubic). This is the best interpolation method and for this reason it is the default in Photoshop editor. New points are calculated from existing neighbors using slightly more complex algorithms than in the previous method;
Bicubic Smoother(Bicubic with smoothing). A variant of the bicubic interpolation method. It is designed to sample high quality images while increasing their size;
Bicubic Sharper(Bicubic with sharpness adjustment). A variant of the bicubic interpolation method. It is designed to process high-quality images while reducing their size.

What happens to the resolution and print area when the sampling procedure is performed? The answer gives the definition of resolution: Length ( inch ) * Resolution (dpi)= Number of dots.

This relationship shows that any image sampling must change its actual length or resolution. From a mathematical point of view, both possibilities are equal; it is only important to maintain the equality of the right and left sides of the equation. When you resample an image in Photoshop, the print dimensions change, but the resolution remains the same. By slightly more complex manipulation of the numeric fields of the same dialog box, you can compensate for the change in the number of points using new resolution values.

The sampling operation can also be performed by digitizing devices. When processing an original at a resolution that is not an integer part of the maximum optical scanner resolution A procedure has been implemented that is in many ways reminiscent of bilinear interpolation performed by raster editors when changing the number of image pixels. Let's look at this situation in more detail. Suppose you want to digitize an original three inches wide on a scanner with a maximum optical resolution of 600 dpi. By simple multiplication you can find the number of photosensitive ones that will be involved in this procedure. It is equal to 600 * 3 = 1800. If the resolution is set to half the maximum (300 dpi), then 900 sensors will participate in the digitization process, i.e. every second. Operation in this mode can be organized using elementary means, without making deep changes to the device control algorithms. A completely different situation arises if you choose a digitization density that is not an integral part of the maximum optical resolution. This will lead to a violation of the regularity of the location of active sensors, so the true appearance of the scanned original can only be formed with the participation of special corrective algorithms working on the principle of software interpolation.

The choice of scanning resolution is often justified by rational arguments, but despite strong physical arguments and logical reasoning, the user almost always remains with significant freedom of choice. Even in a thought experiment, it is difficult to imagine a situation where it is impossible to deviate from the calculated scanning resolution. In most cases, image quality does not undergo critical changes even with significant deviations in resolution from the calculated ones optimal values. Therefore, you should choose a digitization density that approximates the calculated value from above and at the same time is an integral part of the maximum optical resolution of the selected scanning device. In other words, if the scanner is capable of operating at a resolution of 300 dpi, then multiples of 75, 100, 150 dpi are preferable to scan settings that are not an integer part of 300, such as 120 or 175 dpi. If for some original, by calculation or other means, an optimal resolution of 140 dpi is obtained, then in a real scanning session it is advisable to set 150 dpi.

Let us note once again the fundamental differences between scaling and sampling. The first operation only affects the printed version of the image, it does not affect the actual pixels in any way, so screen version the image does not undergo any changes even with significant scale transformations. Its results can only be seen when the document is printed. The second operation is more complex in technique and more responsible in its results. It performs a deep restructuring of the image, under certain conditions, affecting each of its pixels.

In the English-language literature, a subtle terminological distinction is often made between increasing and decreasing the number of points. The first operation is called upsampling, and the second is called downsampling, and the generic term is resampling. In Russian literature you can find a literal, tracing translation of these operations into Russian - upsampling, downsampling and resampling! If you can still come to terms with the last term, then the first two words clearly do not agree with the structure of Russian speech and their existence is not dictated by technical necessity.

Scaling and sampling in Photoshop

Photoshop is a professional raster editor, so it fully supports scaling and sampling functions. All possible operations of this type are performed using one Image Size dialog box. To display it on the screen, just run the command Image - Image Size (Fig. 2.5).

Rice. 2.5.

Let's look at the main features of this window:

Pixel Dimensions. This section displays the image dimensions, specified in pixels or percentages, and the total size of the current document in kilobytes or megabytes. The fields in this section are available when the Resample Image option is active. Otherwise, the program prohibits direct changes to these values;
Document Size. This section displays information about the actual dimensions of the printed print and the resolution at which the image receives its current dimensions. These fields can be directly modified. An increase in the size of the printed print entails an increase in the number of pixels in the digital version of the image and vice versa;
Constrain Proportion(Keep proportions). This option controls the preservation of document proportions. If it is enabled, then any operations with the image will preserve the original aspect ratio. If the option is not selected, then the dimensions of the sides can be changed independently of each other.
Resample Image (Interpolation). This option controls the sampling process. If it is enabled, the program allows you to change the point dimensions of the original, and therefore the total number of pixels, at the user's discretion. Otherwise, all fields in the Pixel Dimension section become unavailable, and dimensions are controlled only by setting the printed size or resolution.
Raster resizing is performed using interpolation algorithms. To select the interpolation method, use the nameless list located next to the switch. In it you can choose one of five available methods of image resampling Nearest Neighbor(By adjacent pixels), Bilinear (Bilinear) and three variants of the Bicubuc method (Bicubic). The features of these sampling algorithms were discussed in the previous section.

If manipulations with the settings of the Image Size dialog box were unsuccessful, you can reset them and return to the starting parameter values. To do this, press and hold Alt key, as a result, the Cancel button will turn into Reset, which serves to discard the changes made to the window. This standard editor technique is used in many program dialogs.

What reasons force you to resort to scaling? It is impossible to mention all the situations in which this technique is necessary. Let's give just one example. Many digital cameras produce low-resolution and large-sized originals. Let the image be obtained with a resolution of 72 dpi and dimensions of 30 by 20 centimeters. If you send it for printing in this condition, the quality of the print will be low. You can confidently predict the appearance of irregularities at the edges of lines and clearly visible steps in areas with smooth color transitions. The number of dots in the original is large enough (almost half a million) to produce a high-quality printed version. You just need to reduce the size of the printed version. To do this, you need to disable the Resample Image option and enter reasonable print sizes in the Document Size section fields, for example 10 centimeters in width. The program will recalculate all the others available options. The height will be 6.5 cm, and the resolution will be 215. These values guarantee printing of sufficiently high quality.

Key terms

Bicubic interpolation- interpolation method in which new points bitmap are created from the color and brightness values of several neighboring pixels. This interpolation method is based on a more complex calculation algorithm than the nearest neighbor method, so it requires more computational resources but produces better results.

Color depth- the total number of binary bits per pixel of a raster image. Let some image be recorded in the RGB system, and 8 binary bits (1 byte) are allocated for each chromatic coordinate R, G, and B. One could argue that this image has a color depth of 8 bits per channel or 3 * 8 = 24 bits per pixel.

Sampling- changing the number of pixels of the raster image.

Interpolated Resolution- characteristics of digitizing devices in which digital image points are obtained by a combination of physical reading and software interpolation.

Bitmap Interpolation- procedure for calculating and adding new points to a raster image.

Scaling a bitmap- changing the resolution of the image when displaying it on various media, provided that the point dimensions of the original are preserved.

Nearest neighbor method- an interpolation method that generates new points in a raster image, borrowing color and brightness characteristics from the nearest neighboring point. The fastest and crudest interpolation method.

Mechanical resolution- characteristic of flatbed scanners equal to the vertical sampling density. This parameter is usually equal to the minimum carriage offset of the flatbed scanner, and is therefore called vertical resolution.

Optical resolution- characteristic of flatbed scanners equal to the density of photoreceptors. This parameter is sometimes called horizontal resolution.

Digitization- procedure for converting an image into raster electronic form.

Image Resolution- characteristic of a raster image that specifies the density of dots (pixels) per unit length. Usually measured in dots per inch inch, dpi) or pixels per inch inch, ppi).

Digital camera resolution- the total number of receptors of the photosensitive matrix, for example 8 megapixels = 8 million pixels.

Screen resolution- two numbers equal maximum number independent addressable points in width and height, for example 800*600, 1024*768.

True Color color space- RGB color space with a color depth of 24 bits per pixel. The set of colors in this space is 16,777,216. This palette is sufficient to represent and process most digital images in electronic publishing and printing.

Image interpolation occurs in all digital photographs at some stage, be it dematrixing or scaling. It occurs whenever you change the size or scan of an image from one grid of pixels to another. Resizing an image is necessary when you need to increase or decrease the number of pixels, while changing the position can occur in the most various cases: Correct lens distortion, change perspective, or rotate the image.

Even if the same image is resized or scanned, the results can vary significantly depending on the interpolation algorithm. Since any interpolation is just an approximation, the image will lose some quality each time it is interpolated. This chapter is intended to provide better understanding what influences the result - and thus help you minimize any loss of image quality caused by interpolation.

Concept

The essence of interpolation is to use available data to obtain expected values at unknown points. For example, if you wanted to know what the temperature was at noon, but measured it at 11 o'clock and at one o'clock, you can guess its value by applying linear interpolation:

If you had an extra measurement at half past twelve, you could notice that the temperature rose faster before noon and use that extra measurement to perform a quadratic interpolation:

The more temperature measurements you have around midday, the more complex (and expectedly more accurate) your interpolation algorithm can be.

Example of resizing an image

Image interpolation works in two dimensions and tries to achieve the best approximation in pixel color and brightness based on the values of surrounding pixels. Next example illustrates how scaling works:

	planar interpolation

Original		before	after	without interpolation

Unlike fluctuations in air temperature and the ideal gradient above, pixel values can change much more dramatically from point to point. As with the temperature example, the more you know about the surrounding pixels, the better the interpolation will work. This is why the results quickly deteriorate as the image is stretched, and also why interpolation can never add detail to an image that is not there.

Image rotation example

Interpolation also occurs every time you rotate or change the perspective of an image. The previous example was misleading because it is a special case in which interpolators usually work quite well. The following example shows how quickly detail can be lost in an image:

		Image degradation

Original	turn 45°	turn 90° (no loss)	2 45° turns	6 turns at 15°

A 90° rotation introduces no loss, since no pixel needs to be placed on the border between two (and therefore divided). Notice how much of the detail is lost on the first turn, and how the quality continues to drop on subsequent turns. This means that you should avoid rotation as much as possible; If an unevenly exposed frame requires rotation, you should not rotate it more than once.

The above results use the so-called "bicubic" algorithm and show a significant degradation in quality. Notice how the overall contrast decreases due to the decrease in color intensity, how dark halos appear around the light blue. The results can be significantly better depending on the interpolation algorithm and the imaged subject.

Types of interpolation algorithms

Common interpolation algorithms can be divided into two categories: adaptive and non-adaptive. Adaptive methods vary depending on the subject of interpolation (hard edges, smooth texture), while non-adaptive methods treat all pixels equally.

Non-adaptive algorithms include: nearest neighbor method, bilinear, bicubic, splines, cardinal sine function (sinc), Lanczos method and others. Depending on complexity, they use from 0 to 256 (or more) contiguous pixels for interpolation. The more adjacent pixels they include, the more accurate they can be, but this comes at the cost of a significant increase in processing time. These algorithms can be used for both scanning and scaling of images.

Adaptive Algorithms include many commercial algorithms in licensed programs such as Qimage, PhotoZoom Pro, Genuine Fractals and others. Many of them use different versions its algorithms (based on pixel-by-pixel analysis) when the presence of a border is detected - in order to minimize unsightly interpolation defects in places where they are most visible. These algorithms are primarily designed to maximize the defect-free detail of enlarged images, so some of them are not suitable for rotating or changing the perspective of an image.

Nearest neighbor method

This is the most basic of all interpolation algorithms and requires the least processing time because it only takes into account one pixel - the one closest to the interpolation point. As a result, each pixel simply becomes larger.

Bilinear interpolation

Bilinear interpolation considers a 2x2 square of known pixels surrounding an unknown one. The weighted average of these four pixels is used as the interpolated value. The result is images that look significantly smoother than the result of the nearest neighbor method.

The diagram on the left is for the case where all known pixels are equal, so the interpolated value is simply their sum divided by 4.

Bicubic interpolation

Bicubic interpolation goes one step further than bilinear interpolation, looking at an array of 4x4 surrounding pixels—16 in total. Because they are at different distances from the unknown pixel, nearby pixels are given more weight in the calculation. Bicubic interpolation produces significantly sharper images than the previous two methods and is arguably the best in terms of processing time and output quality. For this reason, it has become standard in many image editing programs (including Adobe Photoshop), printer drivers and built-in camera interpolation.

Higher order interpolation: splines and sinc

There are many other interpolators that take more surrounding pixels into account and thus are more computationally intensive. These algorithms include splines and cardinal sine (sinc), and they retain most of the image information after interpolation. As a result, they are extremely useful when an image requires multiple rotations or perspective changes in separate steps. However, for single increments or rotations, such higher-order algorithms provide negligible visual improvement with a significant increase in processing time. Moreover, in some cases, the cardinal sine algorithm performs worse on a smooth section than bicubic interpolation.

Observable interpolation defects

All non-adaptive interpolators try to find the optimal balance between three undesirable defects: boundary halos, blur, and aliasing.

Even the most developed non-adaptive interpolators are always forced to increase or decrease one of the above defects at the expense of the other two - as a result, at least one of them will be noticeable. Notice how similar the edge halo is to the defect caused by sharpening with an unsharp mask, and how it increases the apparent sharpness through sharpening.

Adaptive interpolators may or may not create the defects described above, but they can also produce textures or single pixels at large scales that are unusual for the original image:

On the other hand, some “defects” of adaptive interpolators can also be considered as advantages. Because the eye expects to see detail down to the smallest detail in finely textured areas such as foliage, such patterns can deceive the eye at a distance (for certain types of material).

Smoothing

Anti-aliasing or anti-aliasing is a process that attempts to minimize the appearance of jagged or jagged diagonal borders that give text or images a rough digital appearance:

	300%

Anti-aliasing removes these jaggies and gives the appearance of softer edges and higher resolution. It takes into account how much the ideal border overlaps adjacent pixels. The stepped border is simply rounded up or down without intermediate value, while a smooth border produces a value proportional to how much of the border is included in each pixel:

An important consideration when enlarging images is to avoid excessive aliasing resulting from interpolation. Many adaptive interpolators detect the presence of edges and adjust to minimize aliasing while maintaining edge sharpness. Since the smoothed boundary contains information about its position at more high resolution, it is quite possible that a powerful adaptive (boundary-detecting) interpolator could at least partially reconstruct the boundary upon magnification.

Optical and digital zoom

Many compact digital cameras can provide both optical and digital zoom. Optical zoom is achieved by moving the vari-lens so that the light is amplified before it hits the digital sensor. In contrast, digital zoom reduces quality because it simply interpolates the image after it has been received by the sensor.


optical zoom (10x)		digital zoom (10x)

Even though a photo using digital zoom contains the same number of pixels, its detail is clearly less than when using optical zoom. Digital zoom should be almost completely eliminated, minus the times when it helps display a distant object on your camera's LCD screen. On the other hand, if you typically shoot in JPEG and want to crop and enlarge your image later, digital zoom has the advantage of interpolating before introducing compression artifacts. If you find that you need the digital zoom too often, invest in a teleconverter or, better yet, a longer focal length lens.

To increase or decrease the size of an image, Photoshop uses the Interpolation method. So, for example, when you enlarge an image, Photoshop creates additional pixels based on the values of neighboring ones. Roughly speaking, if one pixel is black and another is white, then Photoshop will calculate the average and create a new gray pixel. Some types of interpolation are fast and poor quality, others are more complex, but they achieve good results.

First, let's go to the main menu Image - Image Size or Alt+Ctrl+I.

If you click on the arrow next to the parameter Resample Image, then several interpolation options will appear in the pop-up window:

Automatic. Photoshop chooses a resampling method based on the document type and how it is scaled up or down.
Preserve details (enlargement). When this method is selected, the Noise Reduction slider becomes available to smooth out noise when the image is scaled.
Preserve Details 2.0. This algorithm gives a very interesting result of image enlargement. Of course, the detail does not become more detailed, but what is there increases quite significantly without losing clarity.
. Good method for image enlargement based on bicubic interpolation, designed specifically for smoother results.
Bicubic Sharper (reduction). A good method for reducing image size based on bicubic interpolation with increased sharpness. This method allows you to preserve the details of the resampled image. If Bicubic Down interpolation makes some areas of the image too sharp, try using Bicubic interpolation.
Bicubic (smooth gradients). A slower but more accurate method based on analyzing the color values of surrounding pixels. By using more complex calculations Bicubic interpolation produces smoother color transitions than adjacent pixel interpolation or bilinear interpolation.
Nearest Neighbor (hard edges). A quick but less accurate method that follows the pixels of an image. This method preserves sharp edges and produces a reduced file size in artwork that contains unsmoothed edges. However, this method can create jagged edges that become noticeable when you distort or scale the image, or perform many selection operations.
Bilinear. This method adds new pixels by calculating the average color value of surrounding pixels. It produces results of average quality.

Usage example Bicubic Smoother (enlargement):

There is a photo, dimensions 600 x 450 pixels, resolution 72 dpi

We need to increase it. Opens a window Image Size and choose Bicubic Smoother (enlargement), units of measurement are percentages.

The document dimensions will immediately be set to 100%. Next, we will gradually enlarge the image. Change the value from 100% to 110%. When you change the width, the height will automatically adjust itself.

Now its dimensions are already 660 x 495 pixels. By repeating these steps you can achieve good results. Of course, it will be quite difficult for us to achieve ideal clarity, since the photo was small and low resolution. But look at the changes that have taken place in the pixels.

How big can we make photos with interpolation? It all depends on the quality of the photo, how it was taken and for what purpose you are enlarging it. Best answer: take it and check it yourself.

See you in the next lesson!