LESSON 6 - VR SYSTEM COMPONENTS
VR SYSTEMS
A virtual reality system as we think of it today consists of several hardware subsystems: input sensors, output actuators, and a computer based reality simulator. Additionally, two software items are needed: an application and geometry.
Sensors provide information from the user and can consist of both devices which he consciously uses (joystick) and devices of which he is unaware (head position tracker). Actuators, or effectors, are devices which change the generated information into physical actions. These, like the sensors, can be of a knowingly used device (robot hand) or an unaware item (exoskeleton force feedback drivers). The heart of the system is the reality simulator. This consists of a computer and other hardware, such as sound or graphics boards. This system is the hardware which performs the necessary calculations to produce the visual, audio, or other sensory effects in response to inputs from the user and his environment.
The application software is the set of rules which describes the simulation, its limits, and interactions. These laws and interactions can vary from one system to another and from one time to another. For example, in one simulation of the real world, one rule might be that when a virtual object is released it falls in the direction of the floor or ground. In another setting a released object might remain where it is released. The geometry software describes the objects involved in the simulation, their physical shapes, colors, textures, etc. This acts as a data base upon which the application can draw to build the virtual world.
SENSORS
Sensors, or transducers, are devices which change one form of energy into another, usually into electrical signals. In VR systems a variety of types of these sensors may be used. Most commonly they fall into the following categories: position, force, visual, sound, and biological. The largest group of these is position sensors.
Sensors are used for three purposes in VR systems: navigation, interaction, and command. To move about in a virtual world requires the ability of the user to navigate. This includes changing the point of view, as when turning the head. Interaction involves the ability of the user to influence the objects and actions within the virtual world. A subset of interaction - sometimes listed as a separate category - is selection of an object within the virtual world without influencing it.
An example which demonstrates these groupings is the walk through of the virtual house. The user needs a method to select the route through the house and the ability to change his point of view when turning his head. This is navigation. Turning on a table lamp with the resulting change of illumination would be an example of interaction as would being unable to proceed through the wall of a room. In order to try different cabinet styles or finishes, the user needs to be able to select a particular object. Selection can be done in a number of ways from looking toward to pointing at to touching the virtual object. Note that selection of an object must precede interaction with it but that once selected interaction is not necessarily required. In the example with the cabinet doors we can see the difference between interaction and command. If a door is selected and opened, it is an interaction because the door follows the rules of the application. If you wish to change the color or texture of the door it is necessary to give a command to the VR system because a color change is not a normal reaction to touching or moving a door. (It would, of course, be possible to define an application where this is normal. In that case the color change would be an interaction, rather than a command.) Commands are given to the VR system to change the application rules rather than to follow them.
All of these four activities can be accomplished through the use of overt or hidden sensors. The easiest method of implementing navigation is probably a joystick. To move, the user changes the position of the joystick. A position sensor measures the degree of movement. In this instance the user is well aware that he is telling the computer to move him through the virtual world; he does not normally use a joystick to move around in the real world. If the user is wearing an HMD and turns his head, the view presented will shift. To accomplish this, position sensors associated with the HMD measure movement and send the information to the computer. The user is unaware that he is sending information; he merely moves his head in a natural fashion and the expected result - a change of view - takes place.
POSITION SENSORS
A number of types of position sensors are used in VR systems. These may be used to
measure either linear displacement or angular rotation. Angular measurements are commonly
made using variable resistors, optical encoders, synchros or fiber optics. Variable
resistors, or potentiometers (pots), are tapped resistors where the resistance between one
end and the tap varies with the position of the tap. An optical encoder (shaft encoder)
consists of a light source and a light sensor. These are placed on either side of a
transparent glass plate which has been ruled with regularly spaced lines. As the plate
moves (rotates) relative to the sight source and sensor, the lines cause the light to be
alternately blocked and transmitted. By counting the number of lines crossed, the amount
of rotation can be determined. These encoders can also be made to indicate linear motion.
A synchro is a rotating transformer which can yield an output whose phase relationship
varies with the angle of rotation.
An optical fiber is a flexible glass or
plastic strand which reflect light internally along its length. If the strand is bent, the
amount of light transmitted through the fiber will lessen. By placing a light source at
one end and a light sensor at the other, the angle of the bend can be measured. This is
the principle used in the VPL data glove.
Linear displacement is commonly measured using variable resistors, optical encoders, or linear variable differential transformers (LVDTs). An LVDT is a transformer with three windings and a moving core. The phase and amplitude of the output signal are proportional to the displacement of the core.
For measurement of head or body position, non-contact methods of measurement are sometimes used. These include ultrasonic, electromagnetic, optical, and visual methods.
If an ultrasonic (sound at a frequency above the normal hearing range) signal source is attached to an HMD and several receivers placed around the user, the angle between the transmitter and each receiver can be measured and used to calculate the location of the transmitter. This method, called triangulation, has been used to locate radio transmitters for over 60 years. In three dimensional space the calculations required to locate a rapidly moving source in real time are quite demanding. However, current computer speed makes such calculations possible. A number of commercial position trackers are available.
The same principle can be used with an electromagnetic source or an optical source. A variation is to place a reflector on the HMD instead of the source itself and then to use a movable source and measure the angles when the signal strikes the reflector.
Infrared light can be reflected from the eyeball and the measured reflection angle used to track the movement of the eye. Such systems have largely been used for research in eye movements but could be used to navigate, select, or even interact with a virtual world.
In some systems a TV camera is used to photograph the user. Computer software techniques are used to determine the outline and position of the user in the resulting picture.
FORCE SENSORS
Force sensors can be used to measure the amount of pressure placed upon an object. Such sensors are incorporated into isometric mice-like controllers. Such a device might consist of a ball which is gripped by the user. By squeezing or twisting the ball the user can provide inputs for 6 DOF control. Additionally a number of buttons or switches can be provided for entering commands into the system.
Force sensors may be incorporated into a wired glove to measure amounts of force applied to a stationary object which is then used by the system to apply a virtual force or movement to objects within the virtual world. Force sensors can also be used to measure forces applied by a system controlled output device such as a robot hand. This information is then used to provide force or tactile feedback to the user.
Force sensors may be of a number of types. Commonly used methods are piezoresistive, piezoelectric, and strain gage. Piezoresistive material changes it electrical resistance with changes in deformation. The applied force causes the material to deform and thus to alter its resistance. This material may be used in small amounts to provide a measure of localized force or in a sheet with a matrix of wires embedded on either side to provide a measure of force applied at an entire array of locations. Strain gages consist of a length of fine wire, looped and bonded flat on a piece of plastic. The plastic is bonded to a structure which will accept the applied force, for example a lever handle. When the gage is bent, the wire is stretched and its resistance changes slightly. Such changes are very small but with appropriate amplifier circuits can be used to accurately measure extremely small forces.
VISUAL
Visual sensors usually consist of charge coupled device (CCD) camera sensors of the type used in video cameras. Using such a device, a picture of the user, his surroundings, a model, or some other scene can be captured. Normally a video camera will record 30 such pictures, or frames, per second. Any one or all of these may be captured and placed into the memory of a computer where computations allow the scene to be altered, objects added or deleted, and non-real computer-generated objects to be incorporated into the picture. Such sensors can thus be used to mix real and virtual worlds into a single system.
SOUND
The primary use of sound sensors in virtual reality systems - other than in ultrasonic position measuring systems - is as an input to voice recognition operations. When wearing an HMD the user is effectively blinded to the real world, making use of a keyboard or mouse very difficult at best. One method of providing user information, or commands, to the VR system is through the use of voice commands. In the last few years many strides have been made in such systems and their use is becoming more practical all the time. While currently commands are limited to a relatively simple list for any one system, it is likely that in the not-too-distant future, use of a natural language interface for computer command input will be practical and even common.
BIOSENSORS
Biological signals, such as ECG and even EEG (brain waves), have been used as inputs to VR systems. Electrical signals generated by the muscles and nerves of the body can be used to send information directly to the system computer without waiting for actual body movements. McDonnell-Douglas has already done some preliminary work with the idea of using biological signals - including EEG signals - as a direct input for flight control of high performance military aircraft. Other biological signals have been used as inputs from users who are paralyzed.
ACTUATORS
Actuators, or effectors, perform the reverse function of sensors. Actuators allow the virtual world to interact with the real. An action created in the virtual world can be translated into real movement or events which affect real world objects.
Displays - both visual and audio - are a form of effector. They present information from the virtual world in a form which the user can interpret. Through these, an event in the virtual world can influence actions of the user in the real world. A user driving a virtual racecar will react to an obstacle by attempting to move to another location.
Other actuators can be used to provide force feedback, tactile feedback, motion of platforms, air currents and air temperature changes, or other mechanical movements or sense inputs. Actuators are used to provide the force and movement of robot hands or other mechanical devices. Most of the mechanical actuators are either electric or hydraulic in nature. Hydraulic devices can provide greater force but usually are slower acting than electric and thus are often used on large, heavy items, such as rotating a 40 foot dish antenna. Electric actuators can be either electric motors or solenoids. Motors provide rotational movement while solenoids provide linear motion. Of course gears can always be used to change one form of motion to another. The drawback to this method is that response time is increased with gearing.