1.  Following article from the London Times February 4, 1994 by Matthew May

Despite primitive graphics, spinning images if you move your head too fast and a distinct feeling of nausea if you do it too long, virtual reality headsets are on the way.

Known as immersion virtual reality - because a headset showing computer images and sound from built-in headphones should immerse you in an alternative computer generated world - the effect can be enhanced by adding a "data glove" or tow that you can pick up a virtual teapot or have a virtual flight.

The possibilities for this technology have been well explored and predicted leisure uses range far beyond advanced video games to virtual holidays    No self-respecting science fiction novel of the near future seems able to ignore the idea that an alternative virtual world will become commonplace.

In the way that desktop computer screens are starting to display high quality moving pictures, virtual reality  graphics are also likely to improve quickly. Eventually, it is argued, this facility will develop to the extent where famous, and even long-dead actors will be able to be "sampled" - as music is today - so that they can be reconstructed in a virtual world to do and say anything.

But for now, you can stick the word "virtual" in front of anything and let your imagination do the rest.  Perhaps a spot of virtual painting or a trip through a virtual museum, or how about virtual acting with some virtual actors?

This last possibility could become a reality today when Sun Microsystems, the computer firm, demonstrates a prototype example of a virtual theatre.

The system creates a virtual model of Shakespeare's Globe peopled with virtual actors performing extracts  from the Bard's plays, using the recorded voices of real actors.  You can choose to replace any of the actors and take their place on a virtual stage.  And, if you do not know your lines, they can be superimposed at the bottom of the screen.  Karaoke Shakespeare is on the way.

Like its contemporaries in other areas of virtual reality, the system is primitive, but its devotees argue, it should be compared to the early days of cinema.   For the moment, the virtual actors are represented visually only by mannequins that will move around the virtual stage but are otherwise not animated.

According to its inventor, Andrew MacRae, the theatre is an ideal for transformation into a virtual world because both media share certain attributes, not least that each requires a suspension of disbelief.  "Both take place within a bounded self-contained area, both restrict the user's field of view and movement, both employ actors whose movements are initially preplanned and both make use of props and sound effects, " he says, 

MacRae has tried to portray an accurate representation of the original Globe Theatre based on drawings dating from about 1600 and the recent plans drawn up for the rebuilding of the theatre in London's Bankside.  Background noise has also been added to try to represent the Globe in Shakespeare's day - when the theatre was on open-air stage playing to 3,000 people.

"All kinds of noises should be present as well as the dialogue of the play," says Mr. MacRae, who in his less-theatrical moments is the manager for Sun's industrial virtual reality division in California.

Though the developers wanted to choose familiar scenes from plays for the prototype, Shakespeare's most famous monologues are, almost by definition, not well-suited for the virtual theatre because of the lack of interaction.

The result is that two scenes from Romeo and Juliet have been chosen for the prototype - the balcony scene and where Mercutio is killed by Tybalt - as well as a scene from  MacBeth in which he and Banquo meet the three witches.

"When a user steps into a scene and assumes a role", Mr. MacRae says, "the software continues to play the scene, silencing the lines of that character."

He admits that the present version is "only the first, and as such, is very rudimentary". Obvious improvements, he says, are turning the mannequins into proper virtual actors, with individual physical attributes and costumes, who are able to follow and react to the movements and actions of the human participant.  Another useful modification would be the ability to handle several real players, each adopting a different part and able to  interact within the play.

As well as improving the quality of the pictures displayed on immersion virtual reality systems generally, researchers are eager to try to provide physical feeling.  Some data gloves attempt to give an idea of the texture of different virtual objects by inflating tiny air  sacs against the fingertip to indicate firmness.

2. Paper - The Virtual Globe Theatre by Andrew MacRae

Abstract

This paper describes a project in which a virtual reality model of Shakespeare's Globe Theatre is created and peopled with VR actors performing familiar scenes from the bard's plays.   In addition, a user may take the place of one of the VR actors within the scene and play that part.   Movement and speaking cues for the human actor are provided concurrent with the action of the scene.  Use is made of multidimensional sound placement in order to better achieve a willing suspension of disbelief.  The project is described in its current form followed by   a discussion of possible future directions it may take.

I) Why a Virtual Theatre

Most work in the field of Virtual Reality is involved in creating as realistic a world as possible.  The supposition is that only when VR worlds becomes indistinguishable from the real world will users accept them.  This paper uses the model of staged theatre to demonstrate that all that is actually needed is to provide an environment wherein the participant willingly suspends their disbelief for the duration of the experience.  To this end, the author claims that Virtual Reality equals Theatre.  (Note the use of the term "equals", this is used in the LISP sense, that is VR is not the same entity as staged theatre, but that they are equal in value.)  In  addition, virtual worlds share these other attributes with stage theatre; both take place with a bounded, self contained area, both restrict the user's field of view and movement, both employ actors whose movements are initially pre-planned according blocking instructions, both make use of props and sound effects.

II) Historical Accuracy of the Virtual Globe Theatre

A great danger whenever one presents a model of a historical entity or event is that of inaccuracy.  While multiple minor inaccuracies may not have much effect, a single major inaccuracy can invalidate the entire premise that the model was intended to prove.  On embarking upon this project, the author found that very little is known for certain about the physical structure of the Globe Theatre.  Few drawings of the theatre and its surroundings exist and those that do are contradictory in many important details.  John Nordon, a surveyor, drew a believably accurate depiction of both the South and North banks of the Thames in 1600, showing the Globe and its rival theatres in what is believed to be their correct locat6ions, and showing them to be hexagonal in shape,  Yet, in an insert in that same drawing, the artist shows a more detailed view of the  South bank area,   this time showing the theatres to be round.  The most influential view of the London theatres of Shakespeare's  time was drawn by Claes Jan Visscher in 1616. (see fig. 1 - below)    This drawing show the Globe clearly as a polygonal structure.  However, this drawing, as well crafted as it is,  was drawn three years after the Globe burned to the ground.  Furthermore, there is no evidence that the artist ever visited London and considerable evidence that it is based completely on other drawings of London by other artists, including the above mentioned Nordon.

Fig. 1. Visscher's engraving of the Globe Theatre and the Bear Garden

How does one go about creating a model of Shakespeare's Theatre?   This is a problem that has occupied scholars for centuries and has created much controversy.  Methods include careful reading of the plays known to have been produced there compiling the meager stage instructions.  For example, one play may call for a king to enter through one door while other characters enter through a second door while yet another character withdraws behind a curtain,  Based on such evidence,  Shakespeare scholars have come to the conclusion that there were three doors at the rear of the Globe's stage.  Such is the tenuous nature of what is thought to be known about Globe Theatre.

This much is known about the Globe Theatre, it was an open air theatre built in 1599 that held up to 3000 spectators who stood in the yard    before the stage or sat in one of two galleries.  A balcony stretched    across the the back of the stage from which actors could deliver lines, perhaps a dozen feet from the stage floor.  There was a  roofed shed-like structure above the stage called the "heavens".  There was probably a small balcony jutting from that structure where a trumpeter could sound fanfares and a small cannon be fired during battle scenes.  Behind the stage was the tiring house where the actors changed into costume and props where kept until needed on stage.  Little or no scenery was used in Shakespeare's time. The roof over the stage was thatched as was the roof over the galleries.  It was the roof thatching that caught fire on June 30th 16123 during a performance of Henry VIII when a discharged cannon performed its role all too well.

Those interested in learning more about the Globe Theatre are referred to "Rebuilding Shakespeare's Globe" by Andrew Gurr and John Orrell.   Published in 1989, this book provided most of the historical information used in this project.

III) Implementation of the Virtual Globe Theatre

IIIa) The Theatre

The model of the Globe Theatre as it is believed to have existed in Shakespeare's day was built using the AutoCAD drafting software package.   Existing plans developed for the re-building of the Globe Theatre by the Globe Theatre Museum of London were used as the basis of the model. Most of the plans date from 1989, but the model has been updated to use more recent information about the Globe, mostly information concerning the outer structure, gleaned from the discovery of the footing posts of the original theatre in 1989.  The most major change this discovery has caused has been the revision of the number of sides of the theatre from 24 to 20.   Once completed, the model was exported via DXF files and imported into Sense8's   World ToolKit VR software.  The model was exported as a number of separate files so as to reduce the complexity of the model as it is rendered in real time.  No effort was made within AutoCAD to create a realistic surface to the objects within the model.  The model is viewed using a set of Kaiser Electro-Optics VIM Virtual Reality headgear.  This set provides a viewing angle of 100 degrees.  Using the system's software, a user may explore the interior and exterior of the Globe Theatre, traversing the model by means of a three dimensional navigation device.  For this version a Polhemus tracker was used.  The entire hardware and software system is hosted on a Sun SPARCstation 10 UNIX workstation equipped with a ZX high performance graphics   board.

IIIb) Virtual Actors

The actors within the scenes depicted are objects just as are the elements that make up the theatre itself.  Existing models of mannequin were used.  As in staged theatre, each character has pre-determined movements defined during the course of the scene, i.e. blocking.  No attempt is made to animate the actors beyond their movement on the stage.

IIIc) Depicted Scenes

Three representative scenes from Shakespeare's plays have been recorded for use with the Virtual Globe Theatre.  The scenes were chosen   for a number of reasons.  First, scenes were chosen  that were familiar to the general public.  Second, the scenes were considered on the basis of the dynamics of the scene's dialog.  While Shakespeare's monologues contain some of his most famous prose, such speeches are not well suited for a virtual environment.  Third, scenes were selected based on the complexity of the action required in the scene.  As the current rendition of the Virtual Globe Theatre does not allow animation of the mannequins portraying the characters on stage, it is not possible to show any  action beyond basic traversals of the stage.

The first scene chosen was the famous balcony scene from Romeo & Juliet.  While it does contain a couple of long speeches, most of the dialogue is one or two sentences per line.  It also serves well as a beginning scene as it employs only two actors (and the offstage voice of Juliet's nurse).  Finally, the little action required for the scene does not involve any coordination between the two actors.

The second scene chosen is from MacBeth, or "The Scottish Play" as some would have us call it.  The scene is the one in which MacBeth and Banquo meet the three witches whose prophesies make up the events of the play.   This is a more complex scene with seven speaking roles, but again,  does not require coordinated interaction between the actors.

The third recorded scene is another from Romeo & Juliet.  It is the scene in which Mercutio is killed by Tybalt.  Four speaking roles are involved plus physical interaction between the players.

IIIc) Dialogue coach

The dialogue coach is the software process that loads, schedules and plays the prerecorded dialogue for each scene.  Each line of dialogue is recorded and saved into a separate computer file.  These files are arranged in a hierarchy of directories.  This hierarchy is designed to allow a user to chose between different recordings by different (human) actors, casting the scene if you were.

When a user steps into a scene and assumes a role, the dialog court continues to play the scene, silencing the lines of that character.  (The length of time that is silenced is derived from the time the replaced recorded line would have taken.  Alternately, the user may indicate the end of their speech by pressing a key.)  As an option, the user may elect to ask the dialogue coach to display the text for each line in the lower portion of the display screen  This allows a person to take part in a scene without having previously memorized the lines for that role.   The dialogue  coach allows a user to dynamically change parts within a scene, jumping from one character to another at will, or even retiring completely from the cast to only observe the virtual actors playing their roles.

IIId) Foley Function

Theatre is more than a visual medium, and it is more than simple recorded lines of dialogue.  We live in a complex audio world and we gain many cues from what we hear as "background noise."  The Globe Theatre in Shakespeare's day was an open-air stage playing to a crowd of three thousand people.   All kinds of noises should be present in addition to the dialogue of the play itself.   The Foley function is an attempt to fill the space of the audio space of the Virtual Globe Theatre with the background noises and sound effects necessary to create the theatre experience.  The current implementation uses an E-Mu sampled sound system driven by software from Visual Synthesis Corporation.  A recording of lute music from that era has been used to provide additional background.

IV) Future Implementations

The current implementation of the Virtual Globe Theatre is only the first and as such very rudimentary.  Some obvious areas where it may be improved and enhanced are:

Articulated and Reactive Mannequins: the ideal virtual theatre should have mannequins with articulated joints.  Further, the virtual actors should be capable of following and reacting to the movements and actions of the human actors.

Set Design: While the Globe Theatre in Shakespeare's time  did not use scenery, later stages have.  A role for the creation, placement, and management of stage sets would be of value.

Character Design: The present system uses only stock mannequins to portray the actors.  In the future, each character's  mannequin should be designed specifically for that role, including physical attributes as well as costumes.

Multiple Users: The Virtual Globe Theatre should be capable of hosting multiple users interacting within the same staged scene.  This suggests a theatrical version of the Turing test for artificial intelligence i.e. "true" artificial reality has been achieved when a user down not know if they are acting opposite a virtual actor or another user.

Lighting Design: As ray tracing and other rendering techniques become ever faster, one can begin to experiment with dynamic lighting of scenes.

V) Acknowledgments

The following people and organizations have been instrumental in the building of the Virtual Globe Theatre:

Professor Andrew Gurr, University of Reading, England

David E. Hinkle, Sense8, Sausalito,, CA

Cynthia Traeger, Visual Synthesis, Fairfax, VA

Thomas W. Jones, Polemus, Colchester, VT

Anne Greene, E-Mu Systems, Scotts Valley, CA

James W. Shuck, Kaiser Electro-Optics, Carlsbad, CA

Tim Crawford, King's College, London, England

The Actors:

Rhonda Abrams, Los Altos Hills, CA

Matthew Abergel, Chicago, IL

Patrick J. McEvoy, Palo Alto, CA

K.V.G. Dickinson, Palo Alto, CA