In this game, the player helps Isaac Newton avoid cranial trauma by using a gesture controller to dodge falling apples in this endless platformer game that is just as addictive as it is infuriating. 

In this project, we create a gesture-controlled endless platformer in which the player avoids falling obstacles by moving their avatar left and right using the left and right motion of their hand.

Gaming has been found to hone players’ hand-eye coordination by training players to quickly respond to events depicted on the game display typically through button presses and analog stick pushes on a handheld controller or button presses and mouse movements on a laptop. But handheld controllers and keyboard/mouse controllers come with a large set of problems: encouraging players to sit hunched over as they play, causing neck and shoulder strain, increased risk of overuse injuries in the hands (such as carpal tunnel syndrome, tendonitis, and synovitis), and decreased accessibility to gaming for those with joint, muscle, or tendon disorders who experience difficulty grasping controllers and making fast, precise movements using the fingers.

Gesture control is the obvious solution: by taking player input from more natural movements, games can avoid causing these issues. Innovations in virtual reality have attempted to implement it by tracing the movement of the player’s hands via probes that the player holds in both hands, and advancements in animation and special effects have employed the use of computer vision and green screening to trace the position and movement of an actor’s body. But VR controllers usually end up functioning like an upgraded version of the Wii as they still require the player to grasp the controller and (sometimes) push buttons or tilt/otherwise maneuver the controller in unnatural ways that cause the same problems, and programs that trace a person’s whole body using a green screen suit or similar wearable tags on a players’ body are cumbersome, expensive, and unenjoyable.

This project seeks to implement a different form of gesture control in an endless running platformer using two pairs of light emitters and receivers: a small IR LED will emit light to a mirror suspended above the area in which the player will move their hand, and the reflected light will be received by a phototransistor below. When the player’s hand moves above the LED to block the light from the phototransistor, Isaac Newton (the player’s avatar) will move in the corresponding direction (left/right across the screen) to dodge falling apples.