LESSON 5 - DISPLAY DEVICES
VISUAL DISPLAY DEVICES
There are several types of display devices used in VR systems. These include projection screens, CRTs, LCDs, and LEDs.
In the simplest systems where recorded images are provided (as background or otherwise), filmed images can be projected onto flat or curved screens. These screens can be either reflective or rear projection transmissive. In either case the primary limitation is that the images can not be interactive and can not easily provide three dimensional viewing. A recorded sequence of images can not be easily altered by the user. And unless each of two stereoscopic images can be directed to the two individual eyes, a 3-D image cannot be obtained.
The cathode ray tube, or CRT, currently can provide the highest resolution of any of the display devices used to create a head mounted display and 3-D images. This is the same device used in computer monitors and television sets.
The CRT consists of a glass envelope, which narrows from the width of the display to a small neck at the rear. The inside of the display screen is coated with a chemical, called a phosphor, wh0ich gives off light when struck by electrons. At the rear of the neck of the tube is a heater which warms a piece of metal called the cathode. When this cathode is heated electrons are loosened from the solid metal and can be easily moved by an electric field. The cathode is attached to the negative side of a high voltage power supply; the face of the tube is connected to the positive side. When electrons are released by heating, they are repelled by the negative cathode and attracted to the positive face. These electrons then move toward the face where they produce light when they strike the phosphor. Charged electrical elements in the tube next control the quantity and speed of these electrons.
To steer the electrons to different locations on the tube face one of two methods is used. In electrostatic deflection two sets of slanted plates are used. One set, placed on the right and left sides of the tube neck, is used to move the beam of electrons right and left. If the left plate is connected to a positive voltage source and the right plate to a negative source, the electrons will move left and strike that side of the tube face. How far they move is determined by the strength of the voltage source. A second set of plates located above and below the beam controls the up and down movement of the electron beam. The electrostatic method of deflection was used in early TV sets and is currently used in oscilloscopes and some other types of CRTs. In magnetic deflection, two coils of wire are located around the outside of the tube neck. By controlling the current through these coils, the strength of vertical and horizontal magnetic fields can be controlled. Electrons are influenced by magnetic fields. (You can see this by placing a small bar magnet near the face of a TV set and see the resulting distortion of the picture.) By controlling these magnetic fields, the electron beam can be deflected up and down, right and left. Magnetic deflection is used in current TV sets.
To produce color images it is necessary to use three types of phosphor, each of which will produce light of a different primary color when struck by an electron. The electron beam is controlled to strike only the red phosphor dots when the red image components are being transmitted, the blue dots when the blue image is being sent, and the green dots for the green image. The result is a picture which can contain any visible color from black to white, the combination of all three primary colors.
Another method of producing color uses only a monochrome CRT along with color filters. In this method, a red filter is placed in front of the CRT while an image containing the red information is transmitted; then a blue filter is used during the transmission of the blue information, followed by a green filter during the green image information. (It should be noted that the primary colors do not have to be red, blue, and green, but any such primary combination.) This method was used with early weather satellites to transmit color images of the earth. It was also used in an early version of color television transmission, the CBS system. In this system three images were transmitted for each frame. The receiver had a large color filter wheel, driven by an electric motor, which rotated a color filter in front of the screen during each transmission. Today some systems - VR and otherwise - use this method by placing a color LCD filter in front of the monochrome tube. Usually the tube is relatively small - one to two inches. These filters can change color in response to electrical signals and can be used to produce full color images. Such systems are used because current small sized CRTs aren’t made with the three color phosphor screens needed for a color CRT.
CRTs require extremely high voltages to operate. A twenty-five inch TV will have a typical voltage of 25,000 volts. Even one or two inch CRTs require 800-1500 volts. When used in a head mounted display this requires the voltages be near the user’s head, a potentially dangerous situation. The relatively high weight of CRTs means that when used in an HMD they are usually mounted along side the wearer’s head and have their image reflected in mirrors. Another potential problem is that the CRT has a high vacuum. Should the tube be cracked there is danger from the pressure driven glass. CRTs do, however, provide a bright image which can be viewed in daylight.
Liquid Crystal Displays (LCDs) consist of two glass plates sandwiching a thin layer of chemical called a liquid crystal. This chemical has the property of changing its appearance when subjected to an electric field. Thin wires embedded in the glass plates enable an electric field to be placed across the chemical. When such a field is energized the liquid crystal turns opaque. Light from the back is blocked and the area in the field takes on a black color. Alternatively, the back of the display can be silver coated making it a mirror. In this case light from the front of the display is reflected back out toward the front except where it is blocked by the opaque crystal. If the network of wires is arranged to provide a matrix of dots, an image can be produced in the same way as on a CRT. Some liquid crystal chemicals turn to a transparent color under the field. These can be used to produce color displays.
LCDs are very low powered and require only low voltages. One of the problems with LCD displays is that they produce no light of their own, but require an outside source. An advantage of the transmissive nature of liquid crystal displays is that they can be used as a transparent display to mix real world images with computer generated data. Such a display can be incorporated into a pair of glasses or the windshield of an aircraft as a heads up display (HUD) and used to enable the pilot to read instruments without taking his eyes from the outside view.
Another type of display uses light emitting diodes (LEDs). These are point sources of light produced when current passes through a particular type of diode. LEDs emit only a single color of light, but are available in red, green , yellow, and blue. Combinations could thus theoretically produce any color desired. They can produce a fairly bright display but not one easily read in sunlight. LEDs are relatively high powered. A typical LED may require about 10 mA of current to be easily seen. If a display were to have a 1000 x 1000 array, a total of 10,000 amps would be required for full time illumination. This, of course, is not practical. One company has been experimenting with a single column LED display which utilizes a series of vibrating mirrors to produce a full two dimensional image. This display is monochrome. A full color display would be much more complex. The filter technique cannot be used with an LED display because only a single color is produced and a filter of any other color would leave nothing to transmit.
One of the important characteristics of a visual display is its resolution. This is the smallest separation which two objects can have and still be seen as two distinct objects. CRTs can have a very high resolution. Current computer monitors have resolutions of approximately 1000 x 1000 dots (or pixels). This is the total number of row or columns so that a thirty inch display would have the same number as a one inch display. LCDs are usually much more limited, although there are currently available VGA quality color LCD displays. Most LCD displays used in virtual reality HMDs have significantly less resolution, typically limited to about 100,000 elements. Color displays, however, require three elements (one of each primary color) to produce a single pixel. Thus only about 30,000 pixels are available in the HMD compared with nearly 1,000,000 in a super VGA monitor.
If two displays of any of the above types are used to display two stereoscopic images, a 3-D head mounted display (HMD) can be fabricated. As the displays must be attached to the user’s head, they need to be mounted near the eyes. Wide angle optics are then needed to allow the user to focus on the two images without great eye strain. The optics can also allow the user to effectively look at divergent angles at the two displays. Crossing your eyes to look inward is relatively easy, but trying to look outward in two different direction is far more difficult. The human muscle system is not arranged for this. By using wide angle optics (lens and/or prisms) it is possible for the user to focus straight ahead while the view is bent inward from divergent directions.
Because of the limitations of keeping the displays close to the head and relatively close together, the size as well as the weight of the displays is important. Displays must be able to fit closely side by side because there is a limit to how much the optics can bend the view outward. This limits the size, not only of the display itself, but also of its housing. Weight is limited to prevent strain on the user’s head and neck in supporting it. If the user is to be able to move his head in a natural manner the HMD should cause only a minor distraction. An extremely large or heavy HMD would not fit this description.
When CRTs are used for the display, they are often mounted along side the user’s head and mirrors used to reflect the images into the viewing optics in front of his eyes. This also places the centers of mass of the relatively heavy CRTs close to the center of mass of the user’s head, thus minimizing the effect when the head is turned. The mirrors may be solid for a totally immersive image or half-silvered to allow the user to see the real world as well as the displayed image, as in a heads up display. In either case the image would need to be generated as reversed on the CRTs so that it would appear correct when reflected. It is, however, just as easy for a computer to produce the image reversed as correctly.
LCD displays are usually light weight and can be directly mounted in front of the eyes. Wide angle optics are still used to bring the focus point to a comfortable distance. Usually the problem with LCD displays is the size of the housing. Particularly if the resolution is high a large number of connections and support circuits are required as part of the display. The housing size limits how close the displays may be brought together.
Another method to overcome the limitations of size, weight, and spacing of the displays utilizes a counterbalanced or spring loaded support to give the unit a zero (or very small) weight. An advantage of this arrangement is that the supporting booms can have shaft encoders to measure the position. Such encoders can produce a very rapid response and eliminate any noticeable lag in responding to head movement. Large CRTs or other displays can be used. A difficulty in using these units is that the user must hold on to handles or supports to keep the display against his head and to move it. This is not natural (unless you are a submarine captain used to looking through a periscope) and also does not leave the hands free for interactive control. Also if the unit is large or heavy, even though it has an effective weight of zero, it still has mass and inertia effects will influence it movements.
One more method of making an HMD which has been occasionally used is to conduct the display from CRTs to the viewer’s eyes by a fiber optic cable. A fiber optic cable is basically a bundle of small glass fibers bound together. Light enters one end and is reflected along the inside walls of each fiber, around bends and twists. The light then leaves the other end of the fiber. If each fiber maintains the same spatial relationship at both ends, an image projected on one end of the bundle will be transmitted to the other. However, it is necessary that each fiber be in the same relative position to every other fiber at each end. Such systems are used in medicine to enable a surgeon to view the interior of a joint or other parts of the body by inserting a small cable through a tiny cut or opening. Such a cable can be threaded through a vein and used to obtain pictures from inside the heart itself. When used with an HMD the cables can carry an image from a somewhat larger CRT to the user’s eyes. If high resolution is required there must be a very large number of fibers in the bundle. To obtain a super VGA quality, say 1000 x 1000 pixels, a total of approximately 1,000,000 fibers, each maintaining the exact relationship with its neighbors at both ends of the cable, would be required. Such cables are complicated to manufacture and are extremely expensive.
At the high end HMDs have been made using high resolution (1000 line) CRTs and a switchable LCD filter to provide color. At the low end displays are often made using the LCD displays (usually about 2 inch) available as part of a commercial portable television. These are often limited in resolution to the order of 100 x 100 pixels.
Some systems use single or multiple screen displays rather than head mounted goggles. At the simple end, a single computer monitor can provide a 2-D display. More realistic displays may be constructed using multiple CRTs or rear projection units to provide different views depending upon which way the user looks. These can be particularly effective when used in a closed situation, such as a cockpit of an aircraft simulator. In this type of location the user is normally accustomed to looking out of a "window" so that the counded view of a CRT does not appear unnatural. Head tracking is not required because multiple display devices are used for different views. (This does, however, require that all views be computed at all times and can thus increase computation requirements. Usually in these situations the view is distant enough that 3-D steroscopic imaging is not important. It can, in fact, provide a more realistic simulation because the user is not required to don any abnormal hardware such as an HMD. If stereo vision is a requirement it can be implimented with shutter glasses as in the system pictured above. Shutter glasses alternately blank the right and left eye in sync with changing left and right displays presented on the same screen.
AUDIO DISPLAYS
Audio reproduction devices are usually of one of two types: voice coil speakers and piezoelectric devices.
The common loudspeaker and good quality headphones work on the same principle. A cylindrical permanent magnet is surrounded by a hollow cardboard tube around which is wrapped a coil of fine wire. A cone is attached to the tube. When a current is driven through the coil, a magnetic field is produced. This field interacts with the field of the permanent magnet to cause the tube to move back and forth along the axis of the cylinder. As the current changes, the same changes are duplicated as movement of the tube and cone. The cone in turn pushes air in front of it, resulting in pressure waves (sound) of the same frequency and amplitude characteristics as the current.
Some materials - notably quartz or similar crystals - will produce an electric voltage when squeezed. Likewise when a voltage is placed across the crystal, the crystal will distort its shape. If a small amount of this material is attached to a thin metal disk, the disk can be made to flex in response to voltages supplied to the material. This results in changing pressure waves, driven by the flexing disk, which are the same as the electrical input signals. Such devices are used in inexpensive computer speakers and cheap earphones.
Most audio display devices associated with virtual reality systems are of the higher quality voice coil headset type. Two earphones are used to provide the stereophonic signals necessary to produce high quality sounds and 3-D effects. The only real drawback to using headphones is that there is then something "unnatural" attached to the user. Speakers mounted near the users ears, but not attached to a HMD will eliminate this problem but are more difficult to use in producing realistic sounds which move in response to the user’s head movements.
TOUCH AND FORCE DEVICES
Force feedback, when it has been provided, has been the result of electric or hydraulic motors and actuators acting on control input devices, such as joysticks. Force, in these cases, is transmitted through the user’s hands and arms in a natural manner. One of the problems in using force feedback is in determining the level necessary. If a person is controlling a robot hand which lifts a one pound wrench, the actual force which the user would feel in lifting the wrench may be used. However, if the robot is lifting a 1000 pound mass, the force must be scaled in such a way as to still provide information about the weight of the object being lifted but not overload the user. Usually direct proportion is used. Thus a 1000 pound weight would provide 10 times the feedback of a 100 pound weight. However, if the robot is designed to handle a wide range of weights - for example, from one ounce to 1000 pounds - direct proportion feedback would be unable to provide a detectable level of stimulation at the low end and still not overload the user at the high end. Much work still needs to be done in this area.
Exoskeletons are sometimes used to sense hand and arm motion. These can be driven by electric and/or hydraulic motors to provide a variety of force feedback. The hand and arm have a total of 10 degrees of freedom (DOF). The skeleton necessary to sense this can weigh fifty pounds or more. However, the skeleton can be counterbalanced and computer controlled to move with little effort on the part of the user. In this state the user can easily move his hand or arm as though it were empty. But suppose this sensing device is used to control the robot described above in moving thousand pound crates. While the user can rapidly move the skeleton, the robot cannot move the large masses as quickly, or if it could, it would probably damage the crates. By driving the skeleton with forces proportional to those experienced by the robot the system can limit the user’s movements in a manner consistent with normal use.
Another method of force feedback has used inflatable bladders, for example the pilot’s seat in a flight simulator. If the aircraft goes into positive G motion the bladder is inflated giving the impression of increased weight.
Touch feedback has been provided by several methods. These include the use of vibrating mechanical stimulators, both electromagnetic and piezoelectric, direct electrical stimulation, and inflatable bladders.
An electromagnetic stimulator can be constructed using a solenoid such as is used in a dot matrix print head. In this, a coil of wire surrounds a tube in which an iron rod can move. As a current is pulsed into the coil the rod moves in response. Usually a spring is used to return the rod to its original position when the current pulse is removed. This unit is mounted so that the tip of the rod can strike a point on the user’s finger or hand. Touch is usually indicated by a changing amplitude. In other words, the harder the touch the harder the rod impacts the finger. However, sometimes a mapping into frequency is used: The harder the touch the higher the frequency of vibration. These devices are relatively high powered and bulky.
A piezoelectric vibrator, similar to the piezoelectric speaker described above, can be used instead of the electromagnetic device. Piezoelectric devices require less power and can produce less force.
Both of the above devices are relatively large when compared to the size of the actual touch receptors of the human skin. We have on the order of 100 such receptors per square cm. Actuators are usually limited to a maximum of one per finger.
Another approach to providing touch is the use of small bladders, or air pockets, which can be filled or emptied. The pressure is varied in response to computer generated commands and can be proportional in nature. The size of the bladders limits the number of such actuators which may be placed on, for example, a glove. Typically one to three such pockets may be used on a single finger.
Some work has been done with materials which deform under electrical stimulation and which can be manufactured as a more or less continuous sheet of material. It is hoped that these materials can eventually be used to produce a glove which can be computer driven to provide touch feedback with a much higher resolution (perhaps 0.1 inch) over the entire hand.