American MemoryThe National Digital Library Program: 
Archived Documentation

The Library of Congress / Ameritech National Digital Library Competition (1996-1999)

QUALITY REVIEW OF DOCUMENT IMAGES

INTERNAL TRAINING GUIDE

National Digital Library Program, Library of Congress

September 1996

Contents:

Introduction

The purpose of this training guide is to describe for the image reviewer the imaging process, document the standards for quality review of images, provide step by step instructions for the quality review part of the process, and define terms related to the imaging process.

The National Digital Library Program Imaging Process Overview

Imaging is the process by which digital images of objects (such as books, maps, papers, and photographs) are created to reproduce the object. We create digital images so that they can be placed online. The images are created digitally by "scanning" the original. The scanning is done by a machine that works in a manner comparable to a copier; the object has a "picture" made of it. Objects can be scanned at different settings controlled in the scanning software and adjusted by the scan operator, depending upon the requirements for each image. Settings include choices like bitonal, halftone, and different levels of grayscale as well adjustments for brightness, density, and contrast.

The imaging for the Library is done by outside contractors. The contractors provide scanning equipment, scanning personnel, project managers, and other experts who design the programs or assist technically. Their job is to scan the original document, process it, do a thorough quality control check, place it on an optical disk as an image file organized by batch, and return it to us.

This is the point where you, the image reviewer, come into the picture. You will be responsible for checking the contractor's work for the types of errors that will be discussed in the imaging standards section of this document. If you find errors or have doubts about any image you should make a note of it and print out the image. Once you have completed reviewing a batch, prepare a memo for the quality review supervisor that lists the errors or questionable images and include the printout with your memo. The quality review supervisor makes a judgement based upon an examination of the original and if the contractor has made mistake, e.g., scanning a printed halftone without treatment (e.g., with a special pattern called diffuse dithering), the image is returned to the contractor, corrected to our specifications, and delivered to us as rework.

The Different Types of Document Images Produced for the NDLP (so far)

At this point in the development of the National Digital Library Program, printed documents like book pages are scanned to produce one of three types of images: bitonal, bitonal with diffuse dithering, and grayscale.

Bitonal images are produced when the original is simple text (typography) or line art, e.g., drawings or charts rendered without tones of gray. Such originals have only two shades, black and white, that need to be shown. Pure black and white can be encoded with a single bit for each dot (a dot may also be referred to as a pixel, short for picture element) where "0" is white and "1" is black. The NDLP typically renders bitonal images at a resolution of 300 dots per inch.

This may be contrasted with the handling of digital images of printed halftones. Printed halftones are also bitonal originals in that they are created by black ink on white paper. But the grid pattern of dots that permits the human eye to see them in shades of gray requires that they be treated differently. Halftones are the black and white photographs that you see in the newspaper. If you look closely, you will see they consist of black dots; densely packed to simulate black, sparsely packed to simulate gray, and absent to simulate white.

When the printed halftone's pattern of dots encounters the raster or grid used by the scanner, the result is an image with dark waves called moiré patterns, a form of aliasing. To inhibit such an outcome, one or more processes may be used. One example is diffuse dithering; a diffuse pattern of dots randomizes the scanner's pattern and thus smooths out the resulting image and suppressing the moiré pattern. Another example is a more complicated process of descreening and rescreening. The NDLP typically renders images including a printed halftone at a resolution of 300 dots per inch.

The third type of image you may encounter is called grayscale. In a typical grayscale image each dot may have two or more values (remember, in a bitonal image each dot can have only one value). If a 2-bit depth is selected, each dot is capable of representing four different levels of gray. If an 8-bit depth is selected, each dot is capable of representing 256 different levels of gray. The objects commonly scanned as grayscale in the National Digital Library Program are manuscript pages or photographs, and the spatial resolution may run from 100 to 300 dpi.

File formats and compression

Images are stored in files that adhere to certain format rules or guidelines. In addition, various compression algorithms may be used.

For National Digital Library Program document images, three file and compression formats predominate. Most bitonal images are stored in TIFF files with CCITT (fax) Group 4 compression. The filename extension is .tif. When the Xerox 5200 scanner is used to produce diffuse dithered imges of printed halftones, the PCX format (with its "native" compression) is used. However, a PCX file may contain an undithered image because we have permitted contractor to use PCX for all illustrations. The filename extension is .pcx. Uncompressed grayscale images are stored in the TIFF format; compressed grayscale images are processed with the JPEG (Joint Photographic Experts Group) algorithm and stored in the JFIF (JPEG File Interchange Format). The filename extension is .jpg.

Imaging Guidelines for the National Digital Library Program

The guidelines in this section will not be a detailed list of every possible problem you will encounter in the review of images because there are so many that any list would be very long, very dull, and, in the end, incomplete. Rather the goal is to provide you with a set of guidelines to use to assist you in making judgments on images while performing quality control.

The signs of poor imaging include:

Stark black and white contrasts on anything except simple line drawings. A simple line drawing is a drawing where the lines are not compact (less than one/24 inch apart) and any attempts by the publisher to have grayed shading in the illustration is minimal;

Loss of distinct lines, including hairlines, profiles, skylines, horizons, etc., that are a result of a lack of dithering or too light a scan.

Loss of features such as, in the case of an illustration of a person, eyes, ears, and eyebrows. If you are looking at the illustration of a building pay attention to the windows, the landscaping, etc.;

Skewing that is significant and not the result of the tight binding of the book;

Text that is cut off, including page numbers;

Illustrations without a caption.

Other obvious problems include alignment problems, compressed text, or lines or blotches that appear not to be a part of the illustration. What we call alignment problems are instances where an image has shifted so that the borders are curved or jagged. A good indicator that the blotches and lines are not part of the illustration is that they often appear numerous times over many successive images.

Examples of these problems are attached to the paper version of this document.

A Word About Software

Depending upon the image, we use one of three types of software to view images, ViewDirector, PhotoStyler, and HiJaak Pro. ViewDirector is the preferred software for viewing TIFF images. Both TIFF and PCX can be viewed in ViewDirector, but it does not scale images to 1:1 with reliability. For undithered images this is not a problem, the reviewer can still get a very good sense of the image quality. ViewDirector's advantage is that it is fast.

PhotoStyler is the software used to view PCX and JPEG images. PhotoStyler's disadvantage is its inability to allow the viewing of TIFF images when compressed with CCITT compression. Its advantage is that it does allow the viewing and the printing of an image at a 1:1 scale.

A third software package that is useful is HiJaak Pro. It allows the viewing of files in both the TIFF format and PCX format, and enables the reviewer to open many files at a time ( I've opened more than twenty at a time.). In addition, its resolution is sharp and clear for both file formats. HiJack's disadvantage is its lack of flexibility for manipulating the image and the effort it sometimes takes to print an image at 1:1.

Image Quality Review: Step-by-Step Instructions

This section will guide you step-by-step through the process of image quality review.

  1. When the optical disk and paperwork are received from the contractor , a file is created based on its batch number. Batch numbers tend to group together based on the collection being scanned. For example, the Margaret Mead collection we are creating starts with batch number 130. Other batches will be numbered differently, say batch 150 and above. The batch files are stored in accordion folders over my desk and are stored sequentially. The optical disks are stacked next to the folders and are numbered by disk, not by batch number.

  2. Get the file folder for the batch you want to work on from the accordion folder.

  3. Check the contents of the folder. You should find a directory list of the files in that batch, a letter telling you which shuttle disk the batch is on, and a log in which the contractor has indicated any problems they encountered during the scanning of the object. You should refer to the log when you encounter a questionable image to see if the contractor has noted the same image. If they have, note their log entry in your memo.

  4. Place the shuttle disk in the appropriate optical disk drive. The optical disk drives are at the side of the terminal and resemble 8-track tape players in appearance. The disks you will be working with can function in any of the drives we have, but sadly, none of your 8-track tapes will.

  5. From the window on the computer screen select the appropriate software icon for your viewing purposes and open it with a double click. This will vary between two different software packages depending on the types of images that you will be performing quality review on. If you are reviewing TIFF files, you will open the ViewDirector software. If you are looking at PCX files, you will open PhotoStyler software. You may also use HiJaak Pro for looking at both TIFF and PCX files.

  6. Once you have opened the appropriate software for your file type, choose the drive path that contains the optical disk. You can do this by pulling down the drive window in the dialogue box and clicking the letter of the optical drive that contains the optical disk. The letters should be on the optical drives over the mouth of the drive.

  7. Choose the batch you are reviewing from the directory and open it with a double click.

  8. After opening the batch, the files contained in that batch will display in the directory. Select the file you wish to review and open it with a double click.

  9. The individual image files will now be displayed for the file you selected. Make sure the filenames adhere to our filenaming convention, which means there should be eight characters in the filename. Select the first image file from the directory. For TIFF files this should be the target page, that is, the page that contains the title of the document you will be reviewing. Make sure that the target page matches the title page. The title page will usually be the second TIFF file you view, but it sometimes appears later. The important thing is to make sure they match. In the PCX format the first image file will be (hopefully) the first illustration in the document.

  10. Do a 100 percent quality review of all images that have a ". PCX" extension and a 10 percent check of all images that have a ".TIF" extension. Avoid just going to every tenth ".TIF" image file, but select image files as randomly as you can. If you see a problem with one file, check the surrounding image files. In many instances problems occur in groups and we are very concerned with finding out the extent of problems so that they can be fixed. You may also look at clumps of image files, but your 10 percent check should range over the whole document in some way or another.

  11. Look for image quality problems, following the approach described in the section "Imaging Guidelines for the National Digital Library." Note any problems you see, print out a copy of the image and make a note of the problem to be included in the memo that you will write describing your review. It is a good idea to zoom in and out as necessary and otherwise move around the image to get a better look at it. The more ways you can look at an image, the better your quality review will be.

  12. Check if the relationship between filename and pagenumber changes. If it does, find out why. For instance, if file xxx001.tif is page 1, this relationship would be expected to be maintained throughout the document. Also, check the last image in the directory just to see if it appears as though it is the last page of the document.

  13. Write your memo. The batch name is the subject. List any problems (including questions) you encountered by image filename and attach printouts to the memo. An example of a review memo is attached to this document.

  14. Deliver the batch folder and memo to my desk for my review.

  15. Your part is done unless I have questions or comments about one or more of the images you cited as problematic, in which case I will get in touch with you.

This list may seem long, but most of these steps you will know after your first time through the process.

Terms Defined

This section will acquaint you with some basic terms used in imaging. It's important to know what they are and when they apply to an image.

Algorithm
A rule (often mathematical) governing computer processes.
Artifacts
Visual effects (usually thought of as defects) introduced into a digital image in the course of scanning or compression that do not correspond to the image scanned.
Batch
In the imaging process as we do it, the grouping of books, photographs, or other objects into an organizational unit for our reference only. Each collection has its own distinct series of batch number assignments.
Binary image
A computer image where each dot may have just two values, either black ( 0', for the absence of light) or white ( 1' for the presence of light). Suitable for printed text or line art because only the values of black and white are needed for these types of images. Synonym for bitonal image.
Bit-mapped image
An image created from a series of bits that form pixels. Each pixel can vary in color or gray-scale value.
Compression
A process that reduces the file size for processing, storage, transmission, and display. Compression may be lossless (e.g., CCITT) or lossy (JPEG). The quality of the image may be affected by the compression techniques used and the level of compression applied.
Diffuse dithering (see dithering)
Digital image
An image composed of bits and bytes.
Digitizing
To convert an image into binary code. Visual images are digitized by scanning them and assigning a binary code to the resulting bit-mapped image data.
Dithering
A technique for creating an image with apparent levels of gray by grouping black and white dots into cells. According to the number of black and white dots and their arrangement in the cell, the eye perceives the cell as a single gray, not as a group of black and white dots. Commonly used by laser printers to simulate grey images. The NDLP use of diffuse dithering represents a special case in which the scanner's dot pattern is randomized.
DPI
Dots per inch. It is a measurement of the scanning resolution of an image or the quality of an output device. It expresses the number of dots a printer can print per inch, or a monitor can display, both horizontally and vertically. A 600-dpi printer can print 360,000 (600 x 600) dots on one square inch of paper.
Dynamic range
It is the number of possible colors or shades of gray that can be included in a particular image. 8-bit images can represent as many as 256 colors; 24-bit images can represent approximately 16 million colors.
Eight-bit image
An image where each dot may have any one of either 256 possible gray values or 256 possible color values. The value is calculated by raising 2 to the 8th power.
Grayscale
The range of shades of gray in an image. The grayscale of scanners and terminals are determined by the number of shades between black and white that they can recognize and reproduce.
Halftone image
An image in which combinations of dots are used to create an impression of grays or colors by grouping and density. For example, the eye will see shades of gray in black dots on a white background. Where the dots are large, dense and possibly overlapping, the eye sees dark gray or black; where the dots are small and sparse, the eye sees light gray or white.
Image file
The stand-alone file of each individual page image.
Optical Disk
Generic term for any disk read by a laser, including all CD-ROMs. In the NDLP production environment, the term is a shorthand for the write-once Panasonic disks sometimes used for contractor delivery, especially the "shuttle" disks used to transfer image files between us and the contractor.
PCX
For our purposes it is the file format in which we scan and store illustrated images. The images can either be dithered or undithered.
Pixel
The picture elements that make up an image, similar to grains in a photograph or dots in a halftone. Each pixel can represent a number of different shades or colors, depending upon how much storage space is allocated for it.
Raster
the grid pattern applied when an image is scanned.
Resolution (spatial)
Images measured in terms of pixels dots per inch or other linear measure. The higher the number of pixels, the higher the resolution.
Resolution (tonal)
another way of describing the numbers of bits of information per pixel, used of grayscale or color images.
Rework
The delivery of contractor work that responds to a quality review memo sent by us. It should address all the problems indicated in the review memo.
Twenty-four bit image
a color image where each dot may have any one of 16.7 million color values. The 24 bits are typically made up of three eight-bit values, with each eight bit value representing a separate color (usually red, green, and blue). Very high tonal resolution.