Conclusions

In this project, we create a gesture-controlled endless platformer in which the player avoids falling obstacles on an 8-bit color display by moving their avatar left and right using the left and right motion of their hand.

Building the controller hardware turned out to be much more involved than anticipated as the circuit took over three weeks to completely debug (using different types of phototransistors, modifying the phototransistors, calculating the desired angle for the mirrors to be affixed, creating different mirror configurations to increase the area at which the IR LED may hit the mirror, etc.) and the design of the controller housing had to go through several revisions to yield a viable product. While we also experimented with using a more permanent circuit (soldering to protoboards or placing the whole circuit on a custom PCB) to make the controller hardware more concealed and give the player’s hand more space, we determined that the permanent circuits could not be affixed to the controller housing as easily as the breadboards and the time required (additional shipping delays) would not be proportional to the added value or space conserved.

The program design was a productive exercise in exploring how images could be converted into pixelated sprites and drawn on a VGA display using a combination of C code, header files, and Python scripting. Our project emphasized the visual aesthetics of the game in this regard. Creating the internal logic of the game required accounting for all of the possible animation permutations and interactions between the different sprites; this required distributing the finite state machine across two threads on two cores in order to reduce the animation workload per core. Special care was taken to optimize the smoothness of control of the player avatar so that the Pico would respond promptly to the player movements with little to no delay; we believe that we have succeeded in achieving this. Enabling the hardware-software connection of what was essentially a manual input console was a valuable learning experience for the team.

This project did have a few limitations that would be interesting to add in an extended version of the project. As mentioned during the demo, it would be nice to have a second mode of play in which the player goes through distinct levels, sound effects, background music, and a menu to toggle different features on and off and switch between game modes. While sound effects and background music were not included due to time constraints and possible intellectual property concerns, implementing the menu and different game modes turned out to be exceedingly difficult without utilizing the ability to push buttons or grasp a mouse (the very tasks the project design was created to avoid) because while the player could move the cursor by moving their hand left and right using the existing interface, a third control would have to be implemented to allow the player to click on different items. Our rationale for why the third control point would require a near-complete redesign of the controller mechanism was discussed in Controller Design, so we decided to leave the game limited to two points of control.

There were other ideas related to the game design that we discussed in our proposal and during development that we were not able to implement: a scoring scheme, player shooting controls, and randomly/fixed spawned player power-ups. Our proposed scoring scheme was that in the endless mode, points could be accrued for each apple slice that hits the bottom of the screen, and in the levels, points could be accrued for successful completion and/or obstacles survived. The player shooting controls (allowing the player to shoot back at the apples and destroy them) ran into the same problem described in Controller Design about implementing a third control point in the circuit. If we were to repeat and extend the project to complete the required redesign of the controller circuit to allow for control through multiple hand gestures while moving the hand (discussed in Controller Design) or create other permanent hardware, there may be opportunities to patent the design of the controller, but since we have used a design that is available in the public domain, patenting the current hardware design would be impossible and entirely unethical.

Despite all of the difficult decisions we had to make in our design process, we are all very proud of the final result of our game and feel very satisfied with how this project was able to showcase the skillset that we honed over the course of this semester. This project was engaging, hands-on, very fun to use, and a pleasure to work on.