20 Color

7/28/2019 20 Color

1/6

Image processing operations in color space using finite-state machines

Frederick M. Waltz

2095 Delaware Avenue, Mendota Heights, MN 55118-4801 USA

ABSTRACT

The SKIPSM (SSeparated- KKernel IImage PProcessing using finite-SState MMachines) has recently been extended with excellent

results to various grey-scale operations (morphology, Gaussian Blur, ranked filters, etc.) as well as 3-dimensional binary

operations. Another interesting direction, initiated in this paper, is the extension to color spaces.

After a discussion of some possible goals of such image processing, this paper presents implementation techniques and

examples. Because of the extra costs involved with the printing of colored images in journals, the example images are

presented here in grey-scale only. Color images will be provided via e-mail on request.

Keywords: image color spaces, binary morphology, grey-scale morphology, separability, finite-state machines

1. INTRODUCTION

The motivation for this paper is the opposite of the usual one: Instead of looking for techniques to solve a specified problem,

we here investigate possible problems to which a certain technique will apply. To paraphrase a common expression, we have

at our disposal an excellent and powerful hammer, and we are looking for things to nail with it. The excellent hammer

is the class of image processing implementation techniques called SKIPSM (SSeparated-KKernel IImage PProcessing using

finite-SState MMachines)1-20, which provides significant speed increases (in comparison to conventional implementations) in a

wide range of neighborhood image processing operations. For binary morphology with very large structuring elements and

such operations as the Grassfire transform, the speed increases are truly phenomenal, with speedup ratios of 100 or more

being common. For other operations, the speedup ratios are typically not as large, but are still worthwhile.

Recently, the technique has been extended to a number of grey-scale operations15, 18, 19, 20 and even to 3-dimensional and n-

dimensional morphology13, with excellent results. This paper explores a few of the possibilities of applying SKIPSM to

color images.

As a starting point, let us discuss Adobe Photoshop, a widely-used multi-platform software program for manipulating

images, and especially color images. A fairly wide range of standard image processing operations is built into this program,including Gaussian and other blurring operations, edge enhancement, nonsharp masking, something they call a median filter,

etc. Their standard user documentation does not give theirdefinitions of their operations, but it appears (on the basis of

operations investigated so far) that all operations applied to complete color images operate in RGB (Red-Blue-Green) space,

with the same grey-scale operation being performed on each color plane. The user can also separate the color planes into

three separate images, operate on each plane separately, and then recombine the results to form a new color image. Outputs

can be saved in CMYK (Cyan-Magenta-Yellow-Black) format and some other formats, but apparently not in HSI (Hue-

Saturation-Intensity) format, which is the one that generally makes the most sense for industrial inspection. In no case

examined to date is there any interaction between the RGB color planes, as there would be if the RGB information was first

converted to HSI, these signals were processed separately, and then the results were converted back to RGB.

An aside: For experimental purposes (although obviously not for on-line applications), it has proven to be very useful to be

able to move images back and forth between Photoshop and a spreadsheet, to allow virtually any kind of image processing

operation to be performed without the need to get inside Photoshop or some other image processing engine, or to spend

time writing and debugging C code which will the be discarded in a few minutes after your experiment is completed.

Techniques for moving images between Photoshop and Excel can be provided on request.

2. SOME POSSIBLE OPERATIONS

As indicated above, conventional color-space operations, on all planes together or on individual planes, include edge

enhancement, neighborhood averaging, etc. Figure 1 shows another possibility: Grey-scale erosion or dilation on one of the

color planes (RGB or HSI), an operation for which SKIPSM is particularly appropriate15, 19, especially if the desired

Image processing operations in color space using finite-state machines SPIE Paper 3521-33 Frederick M. Waltz SK20-1

SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII Originally published Boston, Nov. 1998

Copyright July 1998 by F. M. Waltz. Duplication by permission only.

7/28/2019 20 Color

2/6

structuring element is not a simple rectangle or circle. The wavy line in

this figure is an attempt to represent the three-dimensional surface of the

color variable.

One possible place where operating on one of the RGB planes might

provide useful results is that in which there are image spikes or noise of

predominantly one color, say red. Eroding only the red plane wouldreduce this noise without affecting the other color planes, producing a

better signal. Figure 2 presents a synthetic and highly simplified 32x32

example of this type, in which 3x3 spikes in the red plane are completely

removed by eroding with a 5x5 circular structuring element (shown

below), with only minor changes a 2-pixel enlargement of the central

dark square in the main structure in this plane. (Because color plates are

not practical in this Proceedings volume, grey-scale versions of the

before and after images are shown here. These contain only the

intensity information (the I plane of the HSI representation). The smallest discernible squares in these images

represent one pixel. Color versions of these and the other images from this article will be provided on request via

e-mail. Contact the author at .

Similarly, dropouts or dark spots can be reducedby grey-scale dilation on one or more of the

color planes. Figure 3 shows grey-scale

representations of a color image before and

after the application of a diamond-shaped 7x7

dilation operator (shown at right) to the green plane only.

There has, of course, been some distortion of the signal

aspect of the image, in that the central light area of the green

plane of the image has been enlarged by three pixels all

around. Except in very unusual situations, a filtering

operation distorts the signal while removing the noise. The

point being made here is that if additional kinds of imageprocessing operations are available, there is a better chance

that one can be found to reduce the noise effectively

without distorting the signal too much.

These are clearly highly artificial examples. Should larger

erosion or dilation operators be desired, SKIPSM can provide

efficient implementations, including automatically-generated

compileable main-loop computer code19.




xy

huesaturation

or intensity

image plane

erosion

Figure 1. Grey-scale erosion on individual

color-space variables.

Figure 2. A simple 32x32 synthetic image before and after

erosion of the red color plane with a 5x5 circle.

Figure 3. A simple 32x32 synthetic image before and after

dilation of the green color plane with a 7x7 diamond.

Red Before

Green

Grey Before

Red After

Blue

Grey After

7/28/2019 20 Color

3/6

3. USING DIFFERENT OPERATIONS SIMULTANEOUSLY ON THE RGB COLOR SIGNALS

Figure 4 shows another possibility: Applying the same operation (erosion, in this example) to all the color signals or planes,

but using different-sized structuring elements. This would be appropriate for RGB images if the noise to be filtered out had

different size and shapes in the color planes note likely, but certainly possible, depending on the source of the noise. For

example, varying amounts of light in the red band might be affecting the

acquired image adversely. Varying light intensity and color has been a realproblem in outdoor automated vehicle recognition and control systems on

high-speed highways, because the ambient light is much more red near

dawn and sunset. (Human eyes readily adjust to this, so that humans are

generally unaware of slow ambient light changes, but electronic cameras

usually dont handle this problem very well.)

Rather than continue with more morphology examples at this point, a

linear processing technique will now be considered. A standard method of

finding particular features in an image, such as holes or corners, is to use

linear filters designed to trap the features in question. These are called

n-tuples in the PIP image processing environment16, but are not

available in typical image processing designed to produce images for

human viewing. The n-tuple operation involves computing the correlation

of pixels in the image with the selected (usually non-contiguous) pixels of

the n-tuple. SKIPSM can implement this operation rather well.

In the example shown in Figure 5, three

independent

n-tuple

detectors are

applied to

the RGB planes. In the red plane, the detector looks for bright

spots surrounded by a darker area (pupil of eye). In the green

plane, the detector looks for a light area below a darker area

(lower eyelid). In the blue plane, the detector looks for a lightarea above a darker area (upper

eyelid). The combined grey-

scale intensity image (Figure

5e), because it merges the three

fairly good individual detectors,

results in not very good

detection of any of the three

entities, showing one of the

advantages of using separate

color planes for multiple

detection tasks. Note that

because the center spot also

meets the other two definitions,

it provides a strong response on

all three planes. Larger n-tuples could be used to eliminate these ambiguities.

Another interesting possibility for which SKIPSM is well adapted: grey-scale openings and

closings. Figure 6 shows a case in which grey-scale erosion of one or more of the color

variable is followed by grey-scale dilation of the result. Figure 7a) shows the application of

this sequence using a 5x5 circular structuring element to all three color planes (RGB) of the




x

y

image plane

intensity(or B)

erosion

x

y

hue(or R)

erosion

x

y

saturation(or G)

erosion

Figure 4. Using different operations on

the individual color-space variables.

Figure 5. a) 32x32 image of an eye. b) pupil detector - red.

c) lower eyelid detector - green. d) upper eyelid detector -

blue. e) combined detectors - intensity only.

upperdetector

lowerdetector

bluegreenspot

detector

red

positive weight

negative weight

center pixel

Key:

a)

b) c) d)

e)

7/28/2019 20 Color

4/6

eye image of Figure 5a). The resulting RGB results and the

grey-scale intensity-only information are shown in Figure 7

a real black eye. Figure 8 shows the RGB images and the

resulting grey-scale intensity-only image resulting from

subtracting these color images from the original color image.

Note the way that the pupil has been highlighted.

4. PROCESSING IN HSI SPACE

All the actual examples (as opposed to the discussion) in this

paper have involved processing in RGB space. This was done

primarily because inputs and outputs in RGB format (but

apparently not HSI) are available in Photoshop, the tool

used for preparing and publishing this paper. (Photoshop is

primarily designed for the publishing business.) Thus, even

though the actual image processing shown here was done in

Excel, which can handle any kind of computation that one

could desire to do, there was insufficient time to make the

complicated nonlinear conversions to and from HSI space.

Therefore, such examples are not provided here.

The fact remains, however, that for many image processing

applications, operations in HSI space can provide significant

advantages. Of course, operation in Intensity space is familiar

to all, because this is nothing other than the conventional

grey-scale processing used in most machine vision system.

For example, a line or feature on an object of one

predominant color but varying intensity and purity will appear

as a highly variable signal in two or perhaps all three of the

RGB planes, as well as the S (saturation) and I (intensity)

planes in HSI space. But in H (hue) space, it will show us as a

contiguous set of pixels with a small color range, making

detection and location of the feature relatively easy.

Because the goal of this paper is to present possible uses of

the kind of processing SKIPSM can provide, no more will be

said here on this subject. The author expects to explore

further the implications of SKIPSM for automated inspection

using color images in HSI as well as RGB space.




ximage plane

intensityetc.

erosion

intensityetc.

xy

dilation

originalcontour

erodedcontour

erodedcontour

dilatedcontour

originalcontour

y

firstoperation

secondoperation

Figure 6. Grey-scale erosion followed by grey-scale dilation

of an individual color variable.

Red

Blue

Green

Intensity

Red

Blue

Green

Intensity

Figure 7. Result of applying a 3x3 erosion diamond followed

by 3x3 dilation diamond to Figure 5a).

Figure 8. Result of subtracting Figure 7 from original image

shown in Figure 5a) to highlight things removed.

7/28/2019 20 Color

5/6

5 . SUMMARY AND CONCLUSIONS

This paper has presented ideas for using the high-speed and/or large-neighborhood capabilities of the SKIPSM paradigm to

facilitate automated inspection involving colored images. The emphasis has been on presenting possibilities for processing,

not on the advantages of one or another color inspection approach. This is a large field in its own right. It is hoped that

experts in this field will find reasons here to investigate SKIPSM implementations for operations that might otherwise be too

time consuming to be practical for on-line inspection.6. BIBLIOGRAPHY

1. F. M. Waltz, SKIPSM: Separated-Kernel Image Processing using finite-State Machines, Proc. SPIE Conf. on

Machine Vision Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 36,Boston, Nov. 1994

2. F. M. Waltz and H. H. Garnaoui, Application of SKIPSM to binary morphology, Proc. SPIE Conf. on Machine

Vision Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 37,Boston, Nov. 1994

3. F. M. Waltz and H. H. Garnaoui, Fast computation of the Grassfire Transform using SKIPSM, Proc. SPIE Conf. on

Machine Vision Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 38,Boston, Nov. 1994

4. F. M. Waltz, Application of SKIPSM to binary template matching, Proc. SPIE Conf. on Machine Vision

Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 39,Boston, Nov. 1994

5. F. M. Waltz, Application of SKIPSM to grey-level morphology, Proc. SPIE Conf. on Machine Vision Applications,

Architectures, and Systems Integration III, Vol. 2347, Paper No. 40, Boston, Nov. 1994

6. F. M. Waltz, Application of SKIPSM to the pipelining of certain global image processing operations, Proc. SPIE

Conf. on Machine Vision Applications, Architectures, and Systems Integration III, Vol. 2347, Paper No. 41,Boston,

Nov. 1994

7. A. A. Hujanen & F. M. Waltz, Pipelined implementation of binary skeletonization using finite-state machines, Proc.

SPIE Conf. on Machine Vision Applications in Industrial Inspection, Vol. 2423, Paper No. 2,San Jose, Feb. 1995

8. A. A. Hujanen & F. M. Waltz, Extending the SKIPSM binary skeletonization implementation, Proc. SPIE Conf. on

Machine Vision Applications, Architectures, and Systems Integration IV, Vol. 2597, Paper No. 12,Philadelphia, Oct.

1995

9. F. M. Waltz, Application of SKIPSM to binary correlation, Proc. SPIE Conf. on Machine Vision Applications,

Architectures, and Systems Integration IV, Vol. 2597, Paper No. 11,Philadelphia, Oct. 1995

10. F. M. Waltz, SKIPSM implementations: morphology and much, much more, Proc. SPIE Conf. on Machine Vision

Applications, Architectures, and Systems Integration IV, Vol. 2597, Paper No. 14,Philadelphia, Oct. 1995

11. F. M. Waltz, Automated generation of finite-state machine lookup tables for binary morphology, Proc. SPIE Conf.

on Machine Vision Applications, Architectures, and Systems Integration V, Boston,Nov. 1996

12. F. M. Waltz, Binary openings and closings in one pass using finite-state machines, Proc. SPIE Conf. on Machine

Vision Applications, Architectures, and Systems Integration V, Boston, Nov. 1996

13. F. M. Waltz, Implementation of SKIPSM for 3-D binary morphology,Proc. SPIE Conf. on Machine Vision

Applications, Architectures, and Systems Integration VI, Vol. 3205, Paper No. 13,Pittsburgh, Oct. 1997

14. F. M. Waltz, Binary dilation using SKIPSM: Some interesting variations, Proc. SPIE Conf. on Machine Vision

Applications, Architectures, and Systems Integration VI, Vol. 3205, Paper No. 15,Pittsburgh, Oct. 1997

15. J. W. V. Miller and F. M. Waltz, Software implementation of 2-D grey-level dilation using SKIPSM, F. M. Waltz

and J. W. V. Miller, An efficient algorithm for Gaussian blur using finite-state machines, Proc. SPIE Conf. on

Machine Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 18,Pittsburgh, Oct. 1997




7/28/2019 20 Color

6/6

16. R. Hack, F. M. Waltz, & B. G. Batchelor, Software implementation of the SKIPSM paradigm under PIP, Proc. SPIE

Conf. on Machine Vision Applications, Architectures, and Systems Integration VI, Vol. 3205, Paper No. 19,Pittsburgh,

Oct. 1997

17. F. M. Waltz, The application of SKIPSM to various 3x3 image processing operations, Proc. SPIE Conf. on Machine

Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 30,Boston, Nov. 1998

18. F. M. Waltz, R. Hack, and B. G. Batchelor, Fast, efficient algorithms for 3x3 ranked filters using finite-statemachines, Proc. SPIE Conf. on Machine Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 31,

Boston, Nov. 1998

19. F. M. Waltz, Automated generation of efficient code for grey-scale image processing, Proc. SPIE Conf. on Machine

Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 32,Boston, Nov. 1998

20. F. M. Waltz and J. W. V. Miller, An efficient algorithm for Gaussian blur using finite-state machines, Proc. SPIE

Conf. on Machine Vision Systems for Inspection and Metrology VII, Vol. 3521, Paper No. 37, Boston, Nov. 1998




20 Color

Documents

Transcript of 20 Color