8/18/08

How 1080i is converted into 720p

(Converting 1080i into 720p. Cross-converting to 720p.)

All comments about this page are welcome. Send them to kq6qv@aol.com.

Some terms

Artifact – A flaw in an image caused by the way the data is processed. Interference and noise are not usually called artifacts. So artifacts are in a sense predictable.

Aliasing – Jaggedness that appears at diagonal edges of objects in a pixel or raster image. Often this is barely noticeable. But if the object moves, the jaggedness moves seemingly independently and can be a major distraction. (Aliasing is a mathematical term with applications that have no obvious connection to this one.)

De-aliasing – (Anti-aliasing) A smoothing of the jagged edges by blending adjacent pixels slightly.

FIR Filter – A mathematical process that considers samples taken from several points in time, without feedback. If the rescaler considers several 1080 pixels surrounding the 720 pixel then the rescaler is an FIR filter. (FIR means Finite Impulse Response. Don’t ask.)

Methodology

This page will not use the SMPTE definition of “lines of resolution”. Instead it will use a scheme consistent with 480p, 720p, and 1080p, as if there is a smooth continuous scale linking these three numbers. The method measures the amount of blurring at the edges of objects.

The effective resolution achieved by each of the following examples will be given. A supporting document is provided that shows how all of these resolutions were determined. The document is an HTML printout of a Mathcad spreadsheet. It is entirely mathematical. It is not light reading.

Example 1 – 1080p

First we must examine rescaling 1080p, 24p and 30p.

Converting the image to 720p is possible by simply moving some of the 1080 pixels slightly and discarding the rest. The result is quite good. A software evaluation of the image reveals that the resolution is about 760.

A 1080 test pattern and the 720 result:

The following map shows how the 1080 pixels were used. The black pixels were discarded.

Question: How can a 720p image have 760-line resolution?

Answer: Some of the discarded pixels were de-aliasing pixels. De-aliasing always lowers the image resolution some. So stripping off the de-aliasing pixels causes the resolution to rise, possibly above that of a properly de-aliased 720 image. Although this resolution improvement is real, the image will not look better and might look worse. The definition of “resolution” used by the software is appropriate for many examples but probably not this one.

Example 2 – 1080p

The problem with example 1 is that the deletion of lines causes some jaggedness on diagonal edges. Also thin bright horizontal lines tend to shimmer if they move vertically. One way to reduce these is to add some fraction of each deleted pixel to the image. In this example, each line is 67% of the closest 1080 line plus 33% of the adjacent deleted line:

The jaggedness and shimmer are gone, but the resolution is only 625, rather poor. Can we do better? Changing the percentages can raise the resolution, but it makes the artifacts worse.

Example 3 – 1080p

This is the same as example 2 except some edge enhancement is added. Edge enhancement has a bad reputation because it has been overused, especially in trying to turn 480-line images into something better. The high-frequency components of a signal can be restored to full strength if they were not completely filtered out. Many attempts at this have fared poorly because the TV usually can’t determine how much over-filtering occurred. But the losses in a rescaling are predictable.

Rather than use standard edge enhancement, this example will use an enhancement that applies only to those lines used more than once. This technique looks for de-aliasing pixels and adjusts them so that they appear in only one line. The algorithm appears in the supporting document.

The resulting vertical resolution is 720. This sounds perfect, but image enhancement usually causes some artifacts. There are two preferred offsets. Let’s try the other one.

Example 4 – 1080p

Here, the odd-numbered lines are taken from the 1080 frame unaltered, but each even-numbered line is an average of two 1080 lines:

The resolution is 700, vertical and horizontal. A small amount of jaggedness and shimmer are present, but nothing like example 1. This is a preferred method, used in many rescalers. (If the 1080 frame is over-filtered then example 1 is possibly the best method.)

Example 5 – 1080p

An FIR filter is considered here. The “taps” are the inputs, in this case 1080 pixels. Most rescalers use 9-tap (3 by 3) up to 64-tap (8 by 8) filters. For simplicity I did the horizontal and vertical rescaling in separate steps. Using two 8-tap (8 by 1) filters gives about the same result as one 64-tap filter. I launched a computer search for the optimum filter coefficients. When the 720 pixel was directly over a 1080 pixel the optimum coefficients are [0 0 0 0 1 0 0 0]. For the remaining pixels the best coefficients are [0 0 0 .5 .5 0 0 0].

In other words the filter does exactly what example 4 does and gets exactly the same result: 700.

But this is a special case: a rescaling by exactly 2/3. When the image must be rescaled to an odd size (such as 768) an FIR filter usually gives the best result. But the result will always be around 600 unless additional tricks are employed.

Example 6 – 1080i

Now, on to interlace.

For 1080i, the present field can be combined with the previous field to make a complete frame, which can be rescaled by the methods above. This works fairly well, but does tend to blur motion slightly, sometimes giving moving objects a double image.

Example 7 – 1080i

One way to avoid the double image is to distribute the data as in example 4 but never actually combine fields. Since no 720p frame has information from the wrong point in time, motion blurring is eliminated.

Note that each 720p frame is composed from one 540-line image. Many people have concluded from this that the resulting 720p has the same apparent resolution as a 540-line image. This would be a 50% reduction in vertical resolution, not the 33% reduction that one would hope for. But that is wrong. The resulting image can look about as good as example 4. However some flicker is reintroduced.

Many people don’t understand how images with 540 lines can have 720-line resolution. But by their reasoning, a 1080i monitor has only 540 lines of resolution since only 540 lines are lit at one time.

The truth is that on a CRT fewer than 100 lines are lit at any instant, and over 90% of the screen is dark. If you have a digital camera that you can set to a high speed, you can prove this to yourself. But the human eye is just not that fast. Interlacing works because the eye retains information from the previous field. This happens whether you wish it or not.

If the rescaling is performed as described above, the human eye will perceive near-720 resolution. The rise in resolution above 540 is caused by information the eye retains from the previous frame (which would be the previous field of the 1080i image).

Put another way, the image has not been de-interlaced. You are seeing an interlaced image (somewhat compacted) even though your monitor is supposedly progressive.

An inescapable flaw in this process is the reintroduction of some flicker. But due to the way the image is compacted, there is somewhat less flicker than the same images would have on a 1080i monitor.

This method works great most of the time. The vertical resolution will be 700. But if your eye locks on to an object that is moving upward or downward at about 120 lines per second, the image suddenly looks like this:

This picture is 426x240, which is one ninth of a frame. The picture also shows what a single 720p frame would look like with this method

Many LCD monitors are too slow to show this artifact. Ordinary interlaced images show this phenomenon, but to see it you have to be seated way too close.

All interlacing schemes based on examples 1-4 have this problem. Avoiding this artifact requires that the 720p frames not have brighter and darker lines. All lines should have even brightness.

Example 8 – 1080i

This is like example 2 except all lines are used full strength, and fields are not mixed.

The vertical resolution is 580, not very high. If the horizontal rescaling is done as in example 4 then the visual impact is between 580 and 700, or about 635. Motion results in a lot of shimmer, which is caused by some 1080 lines having 200% representation while most lines have 100%.

Example 9 – 1080i

This is the same as example 8 but with some edge enhancement and a shimmer filter.

Standard edge enhancement will not work here since we cannot look at the other field. A single 540-line field has all frequencies above 540 eliminated, so there is nothing to enhance. But the same technique used in example 3 will work here.

The shimmer filter, like the edge enhancer, is applied only to pixels that appear in two lines, and is applied only to image objects that are thin lines or points. It just reduces their intensity slightly.

The resulting vertical resolution is 700, lower than example 3 because edge decisions are based on pixels two lines away, not one. The shimmer is mostly gone. What remains resembles example 4. (Also there is some minor contouring (terracing) in shaded regions with uniform gradients. But this contouring is never more than two pixels wide and is hard to see at the correct viewing distance, and harder still in a moving object. A smarter algorithm might eliminate this.)

Question: So directly combining lines beats an FIR filter?

Answer: Since many 1080 pixels are examined for each 720 pixel, it could be argued that this is an FIR filter.

Conclusions

Example 8 is a safe and simple method. Many rescalers use it. Manufacturers can get away with such poor performance because it is usually not noticeable. As the following graph shows, when the 1080i signal is over-filtered, the drop in resolution caused by rescaling is less than one might expect.

Presently, not much 1080i material is as good as it could be. (Even if it is as low as 540, it’s not terribly bad. Although this sounds close to 480, most 480i material is well below 400.)

When the receiver is the rescaler, it can detect 24p and 30p, and would be smart to switch to example 3 or 4.

When an LCD monitor must rescale 1080i it can use example 6 or 7 since LCDs are too slow to avoid motion smear.

So generally there are two techniques that will result in a vertical resolution of 700:

1. The method of example 9.

2. A unit with motion adaptive de-interlacing can use the method of example 3 or 4. (This technique is fairly new. Older units do not have it.)

The ultimate rescaler would combine these two techniques. Probably it would use the example 3 method for areas where the fields are merged (weaving) and the method of example 9 for areas with motion (bobbing). (Methods 4 and 9 cannot be combined since they have different offsets.) Horizontal rescaling should be done by method 3 or 4. All parts of the screen will be high resolution, and flicker will be maximally suppressed.

So, which of the above methods have manufacturers used? Probably all of them.

How can you tell which method a rescaler uses? Good luck figuring that out.

The Gallery

The following are the all the test patterns from above. They are shown at 300% of actual size so that the individual pixels are more discernable.

Example 1 - test pattern 300%

Example 2 - test pattern 300%

Example 3 - test pattern 300% (Horizontal rescaling as in example 4.)

Example 4 - test pattern 300%

Example 8 - test pattern 300% (Horizontal rescaling as in example 4.)

Example 9 - test pattern 300% (Horizontal rescaling as in example 4.)

The following are full 1280-by-720 frames created as described above. Make sure your imager is set to 100% scale, minimum. These are BMP files with 24-bit pixels (8 bits per color). For some people these files will load very slowly.

Example 1 - 1280 x 720

Example 2 - 1280 x 720

Example 3 - 1280 x 720 (Horizontal rescaling as in example 4.)

Example 4 - 1280 x 720

Example 8 - 1280 x 720 (Horizontal rescaling as in example 4.)

Example 9 - 1280 x 720 (Horizontal rescaling as in example 4.)

The original 1920-by-1080 image is also available. The original image came from a 7.1 megapixel digital camera. The 720-line versions of it were generated by a Pascal program written by the author.

Misinformation

You have probably heard the following statements. Each of these statements is partially correct, but also partially wrong.

· 1080i is the same as 540p.

· 1080i is no better than 540p.

· 1080i is converted to 720p by first converting it to 540p.

(The term “540p” is used mostly by 720p proponents who want to belittle 1080i. Like “right-to-life” or “pro-choice” it lets you identify the speaker’s bias instantly.)

This page is part of “An HDTV Primer”, which starts at www.hdtvprimer.com