LESSON 10 - VR APPLICATIONS:
MEDICAL, EDUCATION
MEDICINE
VR systems have made large inroads into the medical field under other names. CAT scans, MRIs, and ultrasound all provide a non-real reality in the form of images which could not be viewed by the normal senses. As the power of computers has increased and their graphics capabilities expanded, the use of these tools has multiplied. An important factor is that the current generation of physicians has grown up with computers and adapts well to their use.
MEDICAL IMAGES
Arthroscopic surgery is a form of VR. In knee surgery it is now common to use only three small incisions. A fiber optic cable attached to a TV camera is inserted into one of the incisions; this is then used to display a picture on a monitor. Lights and instruments are inserted through the other incisions allowing the physician to operate without exposing the patient to major trauma. Recovery times have been reduced from several weeks to a few days. The instruments used are really remote controlled devices but are linked by mechanical means rather than electrical signals. In the future it may be possible to use miniature telerobotic instruments which can be released or guided into remote areas of the body and controlled externally. This is not as far out as it may sound. One Japanese company has produced an electric motor 1/80 inch across. It will actually fit through the eye of a needle.
Another widely used form of VR is the telepresence systems used with paramedics. An emergency medical technician can use telemetry and even TV to send information from a remote location to a hospital where a physician can aid or direct treatment. In the near future this will probably be enhanced by the use of HMDs and 3-D displays.
The next step beyond medical telepresence systems would logically be medical telerobotic systems with which a surgeon could operate on a patient at a distant location. Such systems are probably some distance in the future, right? WRONG! The US Army is developing such a system for battlefield use and has already used it successfully to perform operations on dogs. The system is unique in that the operator stands and looks down at a display in the way a surgeon normally looks at a patient. Special manipulators at used by the surgeon to control remote instruments which perform the actual operation. At the patient's end of the system the only human support is a medical corpman to assist.
Medical information systems also abound. Most now are merely retrieval systems for written data. However, the possibility exits for presenting the data in forms which allow the medical worker to instantly interpret complex meanings. For example, suppose a system monitors a patient’s vital signs during surgery. This information could be combined with patient history, lab test results, and a library of past medical experience. The result could be presented to the surgeon - on a heads up display, for example - as a guide to performing the operation. Looking down at the patient, the surgeon could view a segmented orange line outlining the proper incision; then he could cut on dotted line. Even if he was completely familiar with the particular operation, certain obscure combinations of conditions might indicate various alterations to the standard procedure. These could be presented dynamically in real time as warning lights, colors, shapes, or sounds. Alternate procedures - "what ifs" - could be shown, again in real time.
Another area where VR is being applied to medicine is in simulation. If an accurate simulator is made, inexperienced doctors can learn new techniques and procedures without the risk to patients. Telepresence can be adapted to this use also. Instead of one or two young doctors observing at the side of an experienced surgeon (or even six or eight observing from a distance), a hundred could follow the procedure through 3-D real time images and eventually with tactile feedback they could even feel with their own fingers.
VR systems and their component sensors and other elements are used in rehabilitation and therapy. Persons who have lost some of their natural sensory or motor abilities can be aided by the use of VR sensors and actuators. The ability of computer systems to transform something from one form into another can allow a visual image to be reproduced as a matrix of tactile stimulations on a person’s hand or back. Spoken language can be converted into typed copy. Visual sensors can read printed copy and speak the words aloud for a blind user. All of these systems exist today and more uses are being added constantly.
Here a musician who has become disabled, is able to play a synthesizer using equipment which maps residual electrical signals in his disabled muscles into control of note selection and volume.
An accident has limited this child's movement to her head. The equipment translates eye movements into controls for an external display. With very little prompting, she learned to move icons around the display and soon began using the system to communicate.
EDUCATION
The use of VR in education falls into two major categories: simulation and data systems.
Simulators are used to speed the learning process and to increase its convenience. This was the original purpose of flight simulators. By reproducing the real situation, the student can experience "first hand" what would be difficult, dangerous, or prohibitively expensive. Simulators are routinely used in school driver education courses to lower the time needed behind the wheel. And of course flight simulators are used to train student pilots. But simulators are not limited to training the student in how a device operates. They can be used to learn and test procedures. The army has set of tank simulators, described in the text, which are used, not to train soldiers to drive tanks, but rather to teach them the type of tactics and operations which they will need to use. An advantage of the simulator for such uses is that it can easily be altered. During the Gulf War these simulators were changed from training men for a war in Europe to providing them with experience in the desert around the actual areas where they would be fighting. Even cities and buildings were recreated so that the area would be familiar when the troops arrived.
Such simulators can be linked. In the case of the tank simulators, two groups in Germany and Kentucky were linked by satellite so that troops could learn to operate in concert or against other individuals and groups. The versatility of such systems is one of its greatest strengths.
Other simulators
are used for training operators of ships and helicopters. Use of texture mapping
enables the ship simulator shown to provide a high degree of realism.
At the very
high end, the MH-53J helicopter simulator is able to produce high quality, extremely real
images at a rate of 60 frames per second. The images are created in response to
operator actions from a database using map data and satellite, aerial, and hand-held
camera photographs. The data base covers an area of 60 square miles over which the
operator can "fly" the helicopter,
The other major application of VR to education involves data presentation. It is now possible to explore a virtual representation of an ancient historical site. Such systems are the logical extension of photoplates supplied in a standard text. Like the photographs they show actual places and items. However, with the use of movement and/or 3-D viewing capabilities they are more immersive and realistic. They can also be multimedia, containing sounds. Additionally, some systems are interactive to differing degrees. One CD ROM set of ancient art allows the user to view from different angles as well as summon information on the work, its history, location, and circumstances of discovery. This program uses only a standard 2-D VGA monitor. Imagine what could be done if 3-D vision and sound were added.
The
virtual Egyptian Temple of Horus described in the text is another example. Here the
student can view and study both still and action visuals and hear audio input from the
characters on the screen.
Egypt seems
to be a favorite subject for these virtual tours. They include a virtual balloon
trip around the pyramids. This simulation is relatively low resolution and should
not require high end hardware.
The scenes above were created using the Sense* Corporation WorldToolKit software. They are part of a real-time tour of the ancient Egyptian Fortress of Buhen. Detailed archaelogical data from the Boston Museum of Fine Arts' Egyptian Department was used to produce the model.
Data programs can also present realities that normally can’t be reached. For example, some provide a tour inside a molecule. Or a program can alter time and show the evolution in the shape of the continents, condensing hundreds of million of years into seconds. In the other direction, the first ten nanoseconds of an atomic interaction can be made to last for minutes while the student studies each detail.
Data can also be presented in a non-real form. In the same way that is done in the business and financial application programs, the flow and change of economics can be presented as a dynamic happening. With an interactive system the student could study the effect of changing interest rates or money supply as the system progresses over time.
Eventually virtual libraries will allow the student to access information in a variety of forms and to search interactively in many different directions. Hypertext provides a crude form of this now and is used by most on-line help programs.
Another advantage to virtual data programs and libraries is that they are easily duplicated or, rather than just being duplicated, can be accessed over a communication network and used anywhere in the world.
Whiteboards are now available which couple to a PC. These devices, like large writing pads or blackboards, allow the user to draw upon them with felt markers and transfer the drawing in real time to a PC. Hand drawn notes or figures can be mixed with computer-generated or stored graphics. Moreover these drawings can then be transferred over telephone lines to another PC anywhere in the world. If a whiteboard is available at the other end, figures drawn on the two boards can be combined and used by people at either end.
Consider a 3-D (vision and sound) system which allows a student of German to be transported to locations within Germany, speak in that language, and interact with computer-generated characters. Everyone agrees that the best way to learn a country’s language and culture is to live there. Such a system allows students to live there for an hour a day without the expense or time of travel.
In the not-too-distant future, the interactive movie technology discussed previously will allow a student to become a part of a history lesson. A framework of historical events can be programmed and the student allowed to interact with historical characters in a variety of contexts. Perhaps a debate with Plato. Or a discussion of physics with Einstein. Given the state of the art and the rate of development, such systems are neither impossible nor too far off.
SUMMARY
Virtual reality is still an infant technology. No one knows what its capabilities or uses will ultimately be. Like other emerging technologies, it is most likely that the truly major uses have not even been thought of yet.
In 1968 John Campbell, publisher of Astounding, was asked to write an editorial depicting the world as it would be in the year 2000. Replying that predicting the future was a losing proposition, he wrote an essay to prove his point.
The year was 1968, 32 years before the year 2000. Suppose that one of the military remote control drone aircraft flew through a French atomic bomb test to gather data. Somehow - don’t worry about how - the aircraft is transported back in time 32 years and lands at Wright Air Field in the year 1936. When examined, components with the names Westinghouse and General Electric indicate that the drone came from the future. The best scientists examine it to determine how far in the future.
First, the shape of the drone is a delta wing, wasp-waist design suitable for supersonic flight. The best aeronautical engineers are certain no such shape could fly. Remember, the speed of sound has yet to be passed. The engine, a ram jet, appears to be a hollow tube and the fuel kerosene. But when ignited it only burns. Ram jets operate at speeds in excess of 500 mph and the JATO rockets used to launch the drone are no longer there.
Now the chemists examine the aircraft. They find traces of half the elements in the periodic table - but they are radioactive! Not only that but there are elements that aren’t even in the periodic table. Nuclear fission has yet to be discovered.
The electronics experts examine the communication equipment. Only one thing resembles a vacuum tube, a micro Klystron. But no instrument of 1936 can detect the frequencies emitted by this device. The majority of the electronics looks to be pure silicon because 1936 chemical analysis cannot detect the levels of impurities which make semiconductor transistors operate. The only thing resembling a known device is a laser diode. This looks like a crystal rectifier, but no signals can be detected. This is because the laser diode requires pulses of many amps for durations of microseconds. Nothing in 1936 can even measure such times much less switch currents in them.
Campbell continues along these lines and finally suggests that the scientists, as a conservative guess, would conclude that the drone had come from at least 500 years in the future.
If we look at the changes which have taken place between 1968 and the present, they dwarf those of the previous 32 years. One of the most difficult tasks of the original Star Trek series was keeping ahead of existing technology. A look at the technology in the old episodes now is like looking at ancient history. Since 1968 we have seen the development of digital computers into miniature, low cost devices which pervade our everyday life. Video cameras and recorders are common place. Vast amounts of information can be transmitted through phone lines or satellite systems. No thought is given to a live news report from around the world. Trips into space often don’t even make a mention on the nightly news. Satellite weather photographs are expected. Cellular phones are only a little over three years old, yet don’t rate a second look. One- fifth of our electrical power comes from atomic power. An automobile engine is expected to last 100,000 to 150,000 miles instead of the 50,000 of 1968. Electronic calculators were not invented until 1968 and no one of that time period would have known what a digital watch was.
It is estimated that the amount of new information and discoveries is doubling every five years. By the time a child starts school, more knowledge has been added to the world than had been learned in all of history prior to his birth. Most of the jobs in today’s world did not exist 30 years ago. Most of the jobs in 30 years probably don’t exist today.
Virtual reality is an emerging technology. We have no idea what it will be used for eventually; but we can be sure that, like with any emerging technological tool, new and unexpected uses will arise. As Warner von Braun said, "To say impossible always puts you on the losing side."