Image processing primitives

Primitives for image and video signal processing.

Displaying images

DisplayImage

Accept a black-and-white input gray image represented by a float matrix and generate output in PGM (portable gray-map) format. Send the output to a user-specified command (by default, the internal viewer is used). The user can set the root filename of the displayed image (which will probably be printed in the image display window title bar) and can choose whether or not the image file is saved or deleted. The image frame number is appended to the root filename in order to form the complete filename of the displayed image.

DisplayRGB

This is similar to DisplayImage, but accepts three color images (Red, Green, and Blue) from three input float matrices and generates a .ppm (portable pixmap) format color image file. The image file is displayed using a user-specified command (by default, the internal viewer is used).

DisplayVideo

Accept a stream of black-and-white images from input float matrix, save the images to files, and display the resulting files as a moving video sequence. This primitive requires that programs from the Utah Raster Toolkit (URT) be in your path. Although this toolkit is not included with MLDesigner, it is available for free. The user can set the root filename of the displayed images (which probably will be printed in the display window title bar) with the ImageName parameter. If no filename is set, a default will be chosen.

The Save parameter can be set to YES or NO to choose whether the created image files should be saved or deleted. Each image's frame number is appended to the root filename in order to form the images complete filename.

The ByFields parameter can be set to either YES or NO to choose whether the input images should be treated as interlaced fields that make up a frame or as entire frames. If the inputs are fields, then the first field should contain frame lines 1, 3, 5, etc. and the second field should contain lines 0, 2, 4, 6, etc.

videodpy

Display an image sequence in an X window. This is simply the DisplayVideo primitive encapsulated in a module so that it can be easily used in other domains.

Reading images

ReadImage: Read a sequence of PGM-format images from different files and send them out in a float matrix. If present, the pound character (#) in the fileName parameter is replaced with the frame number to be read next. For example, if the frameId parameter is set to 2 and if the fileName parameter is dir.#/pic# then the file that is read and output is dir.2/pic2.
ReadRGB: Read a PPM-format image from a file and send it out in three different images; a Red, Green, and Blue image. Each image is represented in a float matrix. The same mechanism for reading successive frames as in ReadImage is supported.
videosrc: Read in an image from a specified file. This is simply the ReadImage primitive encapsulated in a module so that it can be easily used in other domains.
SunVideo: Reads frames from a sun video card and outputs them as 3 matrices: one for Y,U and V components. This primitive will only work in the Linux version of the MLDesigner.

Color conversions

RGBToYUV: Read three float matrices that describe a color image in RGB format and output three float matrices that describe an image in YUV format. No downsampling is done on the U and V signals.
YUVToRGB: Read three float matrices that describe a color image in YUV format and output three float matrices that describe an image in RGB format.

Image and video coding

DCTImage

Take a float matrix input particle, compute the discrete cosine transform (DCT), and output a float matrix.

DCTImageInv

Take a float matrix input, compute the inverse discrete cosine transform (DCT), and output a float matrix.

DCTImgCde

Take a float matrix which represents a DCT image, insert "start of block" markers, run-length encode it, and output the modified image.

For the run-length encoding, all values with absolute value less than the Thresh parameter are set to 0.0, to help improve compression. Run-lengths are coded with a "start of run" symbol and then an (integer) run-length.

The HiPri parameter determines the number of DCT coefficients per block that are sent to hiport, the high-priority output. The remainder of the coefficients are sent to loport, the low-priority output.

DCTImgCdeInv

Read two coded float matrices (one high priority and one low priority), invert the run-length encoding, and output the resulting float matrix. Protection is built in to avoid crashing even if some of the coded input data is affected by loss.

DPCMImage

Implement differential pulse code modulation of an image. If the past input is not a float matrix or has size 0, pass the input directly to the output. Otherwise, subtract the past from the input (with leakage factor alpha) and send the result to output

DPCMImageInv

This primitive inverts differential pulse code modulation of an image. If the past input is not a float matrix or has size 0, pass the diff directly to the output. Otherwise, add the past to the diff (with leakage factor alpha) and send the result to output.

MotionCmp

If the past input is not a float matrix (e.g. dummyMessage), copy the input image unchanged to the diffOut output and send a null field (zero size matrix) of motion vectors to mvHorzOut and mvVertOut outputs. This should usually happen only on the first firing of the primitive.

For all other inputs, perform motion compensation and write the difference frames and motion vector frames to the corresponding outputs.

This primitive can be used as a base class to implement slightly different motion compensation algorithms. For example, synchronization techniques can be added or reduced-search motion compensation can be performed.

MotionCmpInv

For NULL inputs (zero size matrices) on mvHorzIn and/or mvVertIn, copy the diffIn input unchanged to output and discard the pastIn input. (A NULL input usually indicates the first frame of a sequence.)

For non-NULL mvHorzIn and mvVertIn inputs, perform inverse motion compensation and write the result to output.

RunLenImg

Accept a float matrix and run-length encode it. All values closer than Thresh to meanVal are set to meanVal to help improve compression. Run lengths are coded with a start symbol of meanVal and then a run-length between 1 and 255. Runs longer than 255 must be coded in separate pieces.

RunLenImgInv

Accept a float matrix and inverse run-length encode it.

ZigZagImage

Zig-zag scan a float matrix and output the result. This is useful before quantization.

ZigZagImageInv

Inverse zig-zag scan a float matrix.

codef

This module encodes a sequence of images using motion compensation, a discrete-cosine transform, quantization, and run-length encoding. The outputs are split into high priority and low priority, where corruption of the low priority data will impact the image less.

codei

This module inverts the encoding of the codef block, and outputs a reconstructed image sequence.

videofwd

This module is obsolete and will probably disappear in the next release.

videoinv

This module is obsolete and will probably disappear in the next release.

Miscellaneous image blocks

AddMotionVecs

Over each block in the input image, superimpose an arrow indicating the size and direction of the corresponding motion vector.

Contrast

Enhance the contrast in the input image by histogram modification. Input image should be in an integer matrix. The possible contrast type are Uniform (default) and Hyperbolic.

Dither

Do digital halftoning (dither) of input image for monochrome printing. Input image should be in a float matrix. The possible dither methods are Err-Diffusion (default), Clustered, Dispersed, and Own. If you specify Own, then you can use your own dither mask.

EdgeDetect

Detect edges in the input image. Input image should be in a float matrix. The possible detectors are Sobel (default), Roberts, Prewitt, and Frei-Chen.

MedianImage

Accept an input gray image represented by a float matrix, median-filter the image, and send the result to the output. Filter widths of 1, 3, 5 work well. Any length longer than 5 will take a long time to run.

Median filtering is useful for removing impulse-type noise from images. It also smooths out textures, so it is a useful pre-processing step before edge detection. It removes inter-field flicker quite well when displaying single frames from a moving sequence.

RankImage

Accept an input gray image represented by a float matrix, rank filter the image, and send the result to the output. A common example of a rank filter is the median filter, for example, MedianImage, which is derived from this primitive. Pixels at the image boundaries are copied and not rank filtered.