Multi-Touch Game
Table Project Specification
Aditya Mittal, James Wallace, Albert You, Paul Ferrara, Luke Behm
Microsoft, ScientificChess, East Coast Painters,
Date
03/14/2007
Document Version Control Information
|
Rev. no. |
Date Issue |
Author(s) |
Brief Description of
Changes |
|
0.1 |
Nov. 2, 2006 |
J. Wallace, A. Mittal, P.
Ferrara |
1st draft. |
|
0.2 |
Mar. 14, 2007 |
A. Mittal |
Full Revision. All content is new and much more detailed
and sensible. |
|
|
|
|
|
Contents
Purpose
of Multi-Touch Game Table Specification
1.1. Scope
of Multi-touch Game Table Specification
2.1. Purpose
of the Multi-Touch Game Table
3.1.4. Persons
and Skills Constraints
3.1.5. Accessibility
Constraints
3.4. Relevant
Facts and Assumptions
3.5. Similar
Multi-Touch Game Table Information
4.2. Software
Specific Functions
4.2.1. Multi-touch
and Multi-User
4.2.2. Generic
Application Interaction
4.3. Hardware
Specific Functions
4.3.1. Total
Internal Reflection
4.3.2. Reflected
Rear-View Projection
4.4.1. Graphical
User Interface
4.4.2. Application
Programming Interface
4.4.4. Communications
Interfaces
5. Non-Functional
Requirements
5.5. Maintainability
and Portability
6.1. Software
Specific Functions Accomplished
6.1.1. Multi-touch
and Multi-User
6.1.2. Generic
Application Interaction
6.2. Hardware
Specific Functions Accomplished
6.2.1. Total
Internal Reflection
6.2.2. Reflected
Rear-View Projection
7.7. User
Documentation and Training
11.1. Definitions,
Acronyms, Abbreviations
The purpose of Multi-touch
Game Table Specification is to provide technical details of form and function
in order to enable the engineering development team to create the Multi-touch
Game Table. Multi-touch Game Table Specification will also provide guidelines
for the intended use of the Multi-touch Game Table as well.
Repair Technicians, Manufacturer, Design Engineers
This version of the document is
authored by Aditya Mittal. Aditya, along
with Jim Wallace is the principle software engineer for designing the
driver. Jim Wallace is the primary
designer of the table hardware, who also constructed three prototypes, along
with Albert You and Luke Behm who helped put together the final table
hardware. Aditya is the principle
software engineer for designing the games as well as interfacing them with the
driver. Paul Ferrara is the primary hardware electrical engineer. Albert You is
the primary digital hardware engineer.
James Wallace now works for Microsoft.
Aditya Mittal, Albert You, and Paul Ferrara are seniors at
The Multi-touch Display Game Table will allow many people to interact with the system at the same time enabling many multiplayer games as well as allow awesome new applications that people can use their fingers as 10 mice instead of regular computer applications that have a single mouse.
Computer and television screens, keyboard, and mouse are not the best for our eyes and hands. They quickly tire us out and strain us as we work long hours on them. Multi-touch technology is an interactive, self obvious technology which serves as a new means of interaction with the computer. We extend this technology to create a Multi-Touch Game Table.
The Multi-Touch Game Table will give kids and adults a new means of game play beyond that experienced by the current video game systems with mice, keyboards, and hand held controllers. Users will be in all age ranges and will have a basic familiarity with standard board and card games. They may also have a basic understanding of video games however this is not a requirement.
There are costs constraints: Up to
$300 worth of hardware and parts can be purchased on
We are limited by the software available either on the University Computers, our Personal Computers, and free software available on the web. We do not have access to special video game and graphics software in order to develop more sophisticated applications for our Multi-touch Game Table.
We only have about four months of actual time in which we could work on the project within the two semesters, as much of the first semester included presentations and getting started. Along with the project, we also have all our other classes to attend to.
Our working team is small, and everyone was at a different level of education and skills. Jim Wallace, who is no longer with us as he is now working at Microsoft, got the project fired up by creating 3 small prototypes and doing a lot of background research work. He also began writing a simple driver for the hardware, which is now being completed by Aditya Mittal. Albert You got the Multi-touch Game Table hardware updated and more functional at the Physics Machine Shop. Luke Behm is playing a key role in reducing Accessibility Constraints and polishing and developing the aesthetics of the Multi-touch Game Table. Paul Ferrara is an extremely supportive team member. Every member of the small team is useful in reducing time constraints and certainly more skilled members could have helped speed up the process and increased our accessibility.
Often, in the development of the Multi-touch Game Table new materials need to be bought. Often permissions have to be sought. Many stores are closed in the weekends and other times. Lack of a vehicle can be an accessibility issue. We had to wait quite a long time in the past to get administrator privileges to the lab computer to install and develop our driver software. We also had longer waits on being able to get materials until Luke Behm joined the team.
All the parts of our project were bought in general stores or taken from the Syracuse University Lab, and we expect they all comply with the necessary standards. We do not believe we have assembled the parts in any way to create a harmful or dangerous whole. In fact, we believe we have created an improved product that complies with all health and safety standards. Since, Multi-touch Game Table is a new technology it does not have to comply with previously set standards on dimensions or other protocols. It does comply with standard communication protocols of the Windows operating system and computer and projector hardware. The table will comply with standard electrical power rules and regulations as well.
Our hardware is limited to the standard materials sold in the market and its assembly. We are constrained to using acrylic of a certain thickness, infrared LED of certain intensity and power consumption. Our table uses wood, mirror, wires, power supply, a table, an infrared webcam, and wheels.
Computer and projector will be needed separately. Quality and performance standards will be maintained. Suggested usage is assumed. No responsibility for other various uses is taken.
The Multi-touch Game Table is being built under a small budget, in a limited amount of time, by a small number of people. The best attempts to comply with all standards and measures of safety while providing the best of entertainment have been taken. Still, expectations should be limited to what is reasonable. However, the scope of application and possibilities of improvement are infinite.
The Multi-touch Game Table should allow the user to touch the screen and provide input to applications with his/her fingers or provide input to the screen through a laser beam when at a distance. The Multi-touch Game Table should also display on the screen the output from the computer. These input and output functions are discussed more specifically in the following sections.
The Multi-touch Game Table’s ability to detect multiple interactions at the same time gives rise to applications and games that are more interactive, and feature more sophisticated options for enhanced game play.
This ability also allows multi-user games to be played on this Multi-touch Game Table. The application software will be able to support this functionality. The driver software will support the interface as COM objects between the application software and the Multi-touch Game Table.
In our version of the Multi-touch Game Table, we not only produce a DLL from which any COM based application can borrow the input and gesture information but also an application built with AutoHotKey® that takes the COM inputs and creates mouse outputs for being able to run any generic application such as browsing the web.
Multi-touch systems are able to recognize a handful of predefined standard hand gestures allowing a more intuitive game play. This functionality is to be supported by the software.
The Infrared LED’s provide total internal reflection into the acrylic. During a touch, the infrared bounces out of the acrylic (commonly known as Plexiglas) and is detected by the infrared detecting webcam. A laser beam can also create detection at the webcam as it can pass right through the acrylic. The acrylic surrounding wood beams help get rid of the infrared escaping from the top and bottom of the acrylic.
Rear-view projection technology allows more comfort for the eyes, and bigger and better screens for cheaper. Rather than a cathode-ray- monitor that contains millions of tiny red, green, and blue phosphor dots that glow when struck by an electron beam that travels across the screen to create a visible image, a reflected rear-view projected image is created by a regular projector beam shining on a mirror which is then reflected onto a translucent sheet of tracing paper. This is far better than the metallic screens we have as our monitors. With this technology we experience much less strain on our eyes and body.
Proper wooden structures in the Multi-touch Game Table are to be used for supporting the mirror, webcam, acrylic, and the projector.
Our screen is horizontal and built into a sturdy table. This means one can sit comfortably and play it as if playing a regular board game. Do you prefer to read a book holding it straight up, or setting it down on a comfortable height table?
The interface consists of two parts, a projected image and a medium like acrylic to detect multiple interacts with those projected images.
The user interface is application dependent. Each game on the system will have the interface that is best for its application.
There are two APIs for this project; the first consists of a driver that reports back information about user interactions. The second API consists of the standard interfaces required for an application to take advantage of the new input technology.
All applications that support the Component Object Model (COM) technology will be able to borrow the driver parameters such as finger positions at any given time from the supplied DLL.
Webcam utilizes USB; System may use light weight network protocol for reporting driver information; the projector will be interfaced from the computer for reflected rear-view projection;
Lightweight OSC protocol over UDP
Specification of other particular non-functional attributes required by the Multi-touch Game Table:
The Multi-touch Game Table is a comfortable interactive game table around which kids and adults can sit and interact with computer applications and have special fun with games, like the old times with board games.
The Multi-touch Game Table can of course be used for gaming. In addition it can also be used for other computer applications such as painting, or browsing the internet as it can through the touch of the finger provide mouse clicks to the computer applications.
The Multi-touch Game Table is above and beyond the performance of other I/O devices such as the keyboard, mouse, and monitor. It is one device that can serve the purpose of all of these devices with far more comfort and we believe it is how everyone will interact with the machine in the future, along with voice command.
The Multi-touch Game Table can be operated in a variety of ways. The Multi-touch Game Table can be operated not only by hand and finger touches but also from a distance using lasers. Parental precaution is advised when playing with lasers on the table.
The Multi-touch Game Table has wheels to aid moving it around easily. The heavy parts such as the mirror and projector are removable and re-adjustable for portability. The Multi-touch Game Table has an acrylic screen that can easily be cleaned and scratches acquired over time and during game play can be removed with a plastic cleaner and scratch remover such as the Novus® 3 Plastic Heavy-Duty Scratch Remover.
The Multi-touch Game Table is created with the highest quality materials to help ensure the safety and security of children and adults playing games on it. While lasers can be used to interact with the table, it is advised that this be done with precaution. Lasers should never be pointed towards anyone face as they can permanently damage a person’s eyes. Special care should be taken when moving the table around, or when removing parts like the mirror for adjustment.
The Multi-touch Game Table group does not take any cultural or political responsibility. The particular applications run on the Multi-touch Game Table are at the sole discretion of the users.
The Multi-touch Game Table is suitable for adults and children under adult supervision. The designers and manufacturers of the Multi-touch Game Table are not responsible for any particular usage of the Multi-touch Game Table. The designers and manufacturers of the Multi-touch Game Table are also not responsible for the implications and effects of any particular application or computer program that is run on the Multi-touch Game Table. Any broken or malfunctioning parts should be immediately replaced to avoid physical injury. The Multi-touch Game Table does not come with a projector or a computer, required for proper usage of the Multi-touch Game Table. Extra caution should be taken when porting the Multi-touch Game Table.
This section of the document describes how the Multi-touch Game Table accomplishes its functional requirements. First how each specific functional requirement is accomplished is described, and then how performance requirements are accomplished is discussed.
The application software will be able to support this functionality. The driver software will support the interface as COM objects between the application software and the Multi-touch Game Table. Initially, Direct Show filters are used to filter the image for infrared spots created by hand and fingers.
We not only produce a DLL from which any COM based application can borrow the input and gesture information but also an application built with AutoHotKey® that takes the COM inputs and creates mouse outputs for being able to run any generic application such as browsing the web.
This functionality is to be supported by the software. Either we might incorporate certain hand gestures into the driver software itself, but this can only be done for a limited set of hand gestures. Simple gestures such as a mouse click can certainly be an integrated part of the Multi-touch Game Table driver. An interpretation of the rectangles and their positions by the application itself can allow for far greater interactivity and a world of possibilities.
Infrared LED’s embedded in a wooden frame mounted around a 24in x 36in piece of acrylic will provide this facility. Additional wooden beams surround the acrylic to help further eliminate the infrared escaping from the top and bottom of the acrylic.
A projector connected to the computer will shine the output onto a plane mirror settled at the bottom interior of the table. The output will then reflect up into the acrylic backed by a sheet of tracing paper, which is translucent, to permit a nice projected image to be created on the acrylic multi-touch screen.
Proper wooden structures in the Multi-touch Game Table are to be used for supporting the mirror, webcam, acrylic, and the projector.
Our screen is made out of acrylic and framed in a wooden frame fitted with a set of infrared LED strips and is horizontal and built into a sturdy table. This means one can sit comfortably and play it as if playing a regular board game.
The Multi-touch Game Table is required to perform as a good input and output device on which people can sit comfortably around and play a desired computer game. People should be able to touch the screen with their fingers and play games. The Multi-touch Game Table demo applications should demonstrate that it can be used with multiple inputs and properly project an output onto the comfortable acrylic screen.
This section describes any design or implementation issues involved in satisfying this requirement. As we build the project we encounter many issues from time to time in both software and hardware. While an extensive discussion of the issues would be time consuming and wasteful, some of the more important issues are discussed here.
The Lighting issue – the table functions a lot better in the dark, tube light and lamp light are much better than daylight. Bright daylight seems to interfere with the webcam’s ability to detect bright spots well.
The mounting of the projector – how can we mount the projector in such a way that we create a frame into which any household projector can be fitted and angled for proper projection?
Mirror mounting – how can we mount the mirror in a fashion in which we can easily adjust its angle and position for a proper rear-view projection?
Driver Software and Demo Applications – the driver and applications are still in writing so there are numerous open issues in terms of what gestures will eventually be supported intrinsically by the driver, how well the whole thing will function in various circumstances
Testing and Verification – the procedures and aspects that need to be tested are still an open issue
Some of our off-the-Shelf
solutions include the motion detection algorithm which helps us track the
bright spots on webcam output. Another
off-the-shelf solution is the idea of using infrared LED total internal
reflection and then reading it using an infrared camera from a published paper
from Microsoft research. Other solutions
such as using tracing paper and mirror for reflected rear view projection are
our own solutions to the Multi-touch Game Table.
Currently, we have thought up solutions to all our problems and are working on implementing the solutions.
Painting and aesthetics – Luke Behm has done a few nice coats of maroon spray paint on our table, however, more coats and some uncovered parts need to be coated and covered. Also, all the sides of the table need to be covered up.
Wood mounting – mount the mirror and projector and also to cover up open parts.
Driver Software – finish writing the driver software.
Demo Applications – create some demo applications to demonstrate the multi-touch principles.
Game Interfacing – try to interface the system with previously built applications like the web and games like Age of Empires.
As always this requirement might not be satisfied due to unwanted natural causes or acts of god. Additionally, we might not be able to fulfill our requirement in the event of an unwanted and unforeseen technical difficulty or if we shall not be able to finish the development on schedule. There are no major safety hazards involved other than general precaution. We make sound plans and keep our overall objectives in mind as we work through this project to reduce the probability of any preventable occurrence.
Up to $300 worth of hardware and
parts can be purchased on
Additionally, we spend countless hours of the week in lab dealing with everyday issues with hardware as well as software development, verification, and testing. We also bear the costs of going and getting the materials. Additionally, we bring to the project our expensive knowledge and training that we have paid for all these years of our college life.
The user will have access to a complete set of documentation that will help the user setup the device, test its functionality, and operate the device with ease. The users will also be able to refer back to the documentation for repair help and aid if and when they get stuck trying to understand our project.
The users will also be able to contact the manufacturer for additional help and aid with the product, provide us feedback, purchase or receive software updates, detect and repair minor hardware problems, or get information about the development of the product.
Two seventh graders are sitting at home one night playing chess against each other using the Multi-touch Game Table.
An artist is using the Multi-touch Game Table to paint new effects and visuals using his fingers.
An old man is browsing the stock prices online on the Multi-touch Game Table.
An Age of Empires® fan is playing the game using the Multi-touch Game Table rather than the old fashioned computer mouse and keyboard.
A DJ is using Multi-touch Game Table to run a turntables software with his hands to produce cool musical effects through his vibrant speakers.
A puzzle fan is assembling his new puzzle on the Multi-touch Game Table.
The project development consists of five stages. First, the project was thought of and proposed. This step was accomplished early last semester, as Aditya Mittal came up with the idea of an electronic board game, and James Wallace, who was at the time building a video game, approved and did some research and developed this idea into a multi-touch game table.
Second, James Wallace built three small (3in x 4in) prototypes starting with a flimsy first one and a smoothly operational third one. By the end of the first semester we had the third prototype that worked decently well. James Wallace had also begun writing a device driver, and ordered the physics shop to build the full size Multi-Touch Game Table.
Third, Albert You, Aditya Mittal, and Paul Ferrara returned from winter break. James Wallace was gone for working at Microsoft® and found the full size Multi-Touch Game Table built. However, the full size table needed much more refinement, and also the driver needed to be written. Aditya Mittal and Albert You got together as Aditya Mittal began working on the software and Albert You began working on cutting, soldering, and assembling the infrared LED strips into the table. Albert You also took the full size Multi-Touch Game Table to the physics machine shop and had improvements made to the wooden structure and had wheels added for portability. Aditya thought of the idea of using tracing paper for rear-view projection and Albert went out and brought drafting paper first, then tracing paper. Together Aditya and Albert also brought a mirror that broke.
Fourth, Aditya Mittal brought in his old friend Luke, a
graduate of
Five, the whole team shall test the Multi-touch game table
to ensure its functionality, performance, and aesthetics. Finally, the table will be ready for
competitions (April 20th for
$12,000 for 300 man hours. $400 for consumable material such as webcam, mirror, wood, table, power supplies. $4000 for computer, projector, and software for development, and hardware tools. $1000 for party. $15 and a pack of gum for testing. $200 miscellaneous expenses, broken and unused parts. $1,500,000 for manufacturing.
Other relevant documents include the User Guide, Project Timeline, Design Methodology, Concept Presentation, Project Demonstration Presentation, Technical Requirements Presentation, User Presentation, Technical Description, Multi-touch Game Table Specification (this), System Test Script, and User Test Script. We might add a business proposal and plan for manufacturing as time permits.
Multi-Touch Game Table – The name of the product
LED – Light Emitting Diode (infrared in our case)
COM – Component Object Model
DLL – Dynamic Link Library
AutoHotKey® - A macro program that generates WIN32 events so that our driver can interface with any other application
SpriteCraft® - A simple game engine used to develop the demo applications
OSC protocol - OpenSound Control ("OSC") is a protocol for communication among computers, sound synthesizers, and other multimedia devices that is optimized for modern networking technology
UDP - Universal Datagram
Protocol used for sending data on networked computers
DirectShow – Microsoft’s filter pack which is
part of DirectX and is used for filtering audio and video data from the webcam
Acrylic – commonly known as Plexiglas
Multi-touch Game Table Specification has been written in full by Aditya Mittal for the Multi-touch team using the document template provided by our Writing 407 professor Michael J Frasciello. Working experience is the only reference.
We would like to acknowledge all of our Senior Design professors starting with the main instructor of the course Dr. Frederick W Phelps for his motivating guidance and great stories to all our projects, Dr. Duane L Marcy for his technical guidance and keeping us on track, our technical writing instructor Michael J Frasciello for helping us create documentation like this, and instructional lab supervisor William Choate Tetley for helping us get to the materials we needed to begin and also for being very interested in our work and helping us out as needed. We would like to thank all of our professors for being greatly motivating and promoting our projects to success.
We would also like to thank Dr. Carlos R P Hartmann and Dr. Shiu-Kai Chin for inviting the executive vice president of Lockheed Martin Corp, Robert B. Coutts who is responsible for Lockheed Martin’s $9 billion Systems Integration business area. We thank Robert B. Coutts for coming and talking with us.
We would also like to thank Jeff Hahn for starting the revolutionary multi-touch technology.
Additionally, we would like to thank ourselves, the senior design team, Aditya Mittal, James Wallace, Albert You, Paul Ferrara, and the keenest Luke Behm for joining us and helping us greatly.
We would also like to thank all our classmates for the great inputs in the various problems we encountered as we worked in the lab through the months and Syracuse University ECS for supporting such great project work and providing us with such opportunities.