Watermelon Game
Implemented on RP2040
Yen-Hsing Li(yl2924), Yimian Liu(yl996)
Demonstration Video
Introduction
Our final project is to implement a interactive video game called “Watermelon Game” using the Raspberry Pico controller. The rule of the game is pretty simple, the player drops various kinds of fruits, which represent in different radiuses of balls, from the top of the container. The same kind of fruit will merge together when they collide with each other inside the container. The ultimate goal of this game is to get them combine to make the biggest possible fruit, which is the watermelon. The more watermelon you get, the higher score you earn in the game. If any single fruit cross the line at the top of the container and overflowing out, the game would end.
High Level Design
Our concept was inspired by lab2 and an online game sharing the same title. Building upon the code from lab2, we transformed the bounding box into a container to catch fruits falling from above. A top boundary check determines if the game ends.. We also changed the 2x2 pixel boids into different sizes of hollow ball plotted on the VGA screen. We developed 2D Elastic Collison and Gravity Effect on each fruits to simulate the physical collision effect in the game. Background music and sound effects are included using the DAC module, and calculated by the DDS algorithm by core 1 of the Pico.
We divide the game development progress into two several parts. First, we developed two user interfaces: one is the game menu and the other is the in-game interface. The game menu only has 3 options: mode1, mode2 and an option to toggle the background music. The in-game interface includes the boundary of the container, the circles represent the different sizes of fruits, and the score the player earned in the game. The anticipated in-game interface is shown in the following figure. The dotted line in the figure is the game over line, if any circle exceeds this line then the game is over.
Fig. The anticipated in-game interface. The circles represent different kind of fruits. The dotted line is the game end line.
Next, we need to implement the gravity simulation and the collision detection logic to update the circle constantly when circle falls. For the gravity simulation, we simply add a downward velocity at each time step to simulate this effect. Additionally, we implemented another checker to check whether the neighbor circle has the same radius. If so, they merge with each other and create a larger circle. Besides, we also integrate the joystick circuit with the RPi Pico to receive real-time signals from the joystick. This guarantees that the user's input will be accurately received and processed by the processor.
Background Math
For more detailed implementation of the physics in the game, please refer to the Software Design session further below.
Elastic collision
For simulating the 2D collision effect, the elastic collision formula is used to calculate the behavior of encounter between fruits in the container. The 2D elastic collision refers to the interaction between two objects in a two-dimensional space where both momentum and kinetic energy are conserved. After two objects collide, their original velocity changed. In an angle-free representation, the changed velocities are computed using the centers x1 and x2 at the time of contact as:
where the angle brackets indicate the inner product of two vectors.
Logical Structure
This project was divided into several key components. Each component has a specific role, and together they create a cohesive and functional application.
Initialization and Main Control Loop (main.c): This part serves as the entry point of the program. It initializes hardware components, sets up the game environment, and enters a control loop that manages the game's state. This involves setting up the RP2040's clock, initializing VGA for display, configuring SPI for the DAC (Digital-to-Analog Converter), and starting a core for audio processing. The main loop runs the game logic, updates the display, and checks for user input.
Graphics Handling and VGA Output: This part manages the visual representation of the game. This includes drawing balls, the game boundary, and any text or UI elements. The code in lib/vga_graphics.h and related components handles the drawing routines. These routines convert game state into pixels on the screen, managing aspects like ball positions, colors, and boundary lines.
Physics Engine: This part simulates real-world physics, particularly the dynamics of bouncing balls. This includes handling collisions between balls and with the game boundary. The files ball_physics.c and box_physics.c implement this. They calculate ball movements, detect collisions, and adjust ball velocities and directions based on physical laws. This component is crucial for making the game feel realistic.
User Input Processing: This part translates user actions from the joystick into game commands. The code in gpio.c and its interactions with other components manage this. It detects joystick movements or button presses and changes the game state accordingly, such as starting the game, moving objects, or navigating menus.
Audio Output via DDS: This part generates sound effects to enhance the gaming experience. It utilizes Direct Digital Synthesis (DDS) techniques for sound generation. The code modulates waveforms and outputs them through a DAC. This process is managed in the callback function in main.c and is synchronized with game events like collisions.
Menu and Game State Management: This part provides a user interface for starting the game, choosing settings, and displaying game-related information like scores. It is handled in menu.c and parts of main.c. It manages different states of the game, such as the main menu, game playing, and game over scenarios.
Memory and Resource Management: This part efficiently manages the limited resources of the microcontroller, like RAM and processing power. This is evident in the use of fixed-point arithmetic (for better performance than floating-point on microcontrollers) and dynamic memory allocation for game objects like balls.
Concurrency with Protothreads: This part manages concurrency without a complex operating system. The use of protothreads allows for writing non-blocking code that can handle multiple tasks, like updating the display, reading inputs, and playing sounds, seemingly in parallel.
Design Tradeoffs
There were some design trade offs we need to consider in this project. For example, we need to decide whether to control the game using the keyboard used in lab1 or purchase a dedicated joystick. We decided to go with the later one since we believed that a dedicated joystick would offer a more intuitive and immersive gaming experience for the users as contrast to simply pressing button on the keyboard. In addition, there is also a trade off in the data structure of the fruits. In lab2, we concluded that we can simulate at most around 550 boids running smoothly on the VGA screen. With this in mind and consider the possibility of over a hundred of fruits may present in the container, we decided to inherently use the fix15 type from lab2 for representing the position and velocity attributes for each fruit objects. This decision is to avoid resource starvation, which may occur by using other data types such as int or float.
Program/Hardware Design
Hardware Details
Our project hardware mainly consists of the Raspberry Pico controller, the MCP4822 DAC module connected with the speaker, a joystick, and a VGA screen for display. Below shows the hardware schematic diagram of the Pico controller. Each channel of the VGA and the joystick is connected with a 330 ohm resister. For GPIO pins, we used 16 and 17 for VGA Vsync and Hsync. GPIO 18/19/20 are for RGB channels, respectively. The joystick uses GPIO 10/11/12/13 to control four directions in the game and the menu. GPIO 4/5/6/7/8 are used for the MCP4822 DAC module. Finally, we uses GPIO 0/1 connected with the UART for putty debug purpose.
Fig. Hardware schematic diagram of the Pico controller
Program Details
Joystick Control
When the main program starts, the GPIO pins connected to the joystick are initialized and set as input pins (GPIO_IN). This is done using the gpio_init and gpio_set_dir functions. Internal pull-up resistors are enabled for these pins using the gpio_pull_up function. This ensures that the GPIO pins read a high value when the joystick is in the neutral position. For reading the GPIO inputs, the corresponding GPIO pin is connected to ground through the joystick when a direction is pressed, causing its state to change from high to low.
We also wrote a dedicate functions gpio_edge and gpio_value . They are used within the main thread (protothread_anim) to read the state of the joystick pins and determine if a direction is pressed or is currently being held down. gpio_edge(DIRECTION) return 1 if the GPIO rising edge of the corresponding direction is detected. gpio_value(DIRECTION) return 1 if gpio_get(DIRECTION) is 1. These functions are helpful for controlling the logics in the game . For example, in the protothread_anim thread, the joystick inputs are used to navigate through a menu (gpio_edge(RIGHT), gpio_edge(UP), gpio_edge(DOWN)) and control the movement of the balls in the game (gpio_value(RIGHT), gpio_value(LEFT)).
Balls Spawn
In our code, we define a ball type structure, which is similar to the boid structure we used in lab2. The ball has its own position, velocity, and type. The ball_type structs defines different types of balls, each with unique properties like radius, mass, color, and score.
To spawn a new ball in the program, initBallNode is used to create a new ball of a specified type and add it to the game. It creates a ball instance and inserts it into the linked list of balls. All spawned Balls in the programs are managed using a linked list, which allows dynamic addition and removal of balls from the game. insertBall adds a new ball to the beginning of the list, while deleteBall removes a specified ball from the list.
Gravity Simulation
In game, there are two modes: mode1 is called the “gravity mode” and mode 2 is the “non-gravity mode”, which means that the gravity simulation only affect on mode 1. In our code, gravity is simulated as a constant force that acts on the balls, affecting their vertical motion. Within the gravity_function , the vertical velocity of the ball is incremented by a global gravity value (g_gravity). This value represents the acceleration due to gravity. This function is called on each ball in every iteration of the game loop to continuously apply gravity.
Box Boundary and Bouncing Physics
The box boundary is implemented to simulate the container in the game. When a ball comes into contact with one of these boundaries, it bounces back. This bouncing behavior is defined in the bounce_function, which handles the collision of balls with the boundaries of the box. The function checks if the ball has hit any of the boundaries (bottom, left, or right) of the box. If the ball hits the bottom boundary (hitBottom), its vertical position is adjusted to be the bottom boundary minus its radius, ensuring it doesn't go past the boundary. Also, the vertical velocity is reversed and multiplied by a factor to simulate the loss of energy upon bouncing. If the ball hits either the left (hitLeft) or right (hitRight) boundaries, its horizontal position is updated using the same logic mentioned above, and the horizontal velocity is reversed without any reduction factor to perform an elastic collision in the horizontal direction. This function is called and checked for every frame of the game.
Friction Simulation
Friction acts in the opposite direction of movement, gradually reducing the speed of a moving object until it comes to a stop. In the context of the bouncing balls, friction affects their rolling movement on the game's surface.
\(v_{x_{new}} = v_x \times f\)
\({y_{new}} = v_y \times f\)
To implement it, each ball has a velocity vector \(\vec{v} = (v_x, v_y)\), where \(v_x\) and \(v_y\) are the velocities in the horizontal and vertical directions, respectively. During each update cycle, the frictional force is applied against the direction of the ball's velocity. This is done by slightly reducing the magnitude of the velocity vector. If \(\vec{v}\) represents the velocity and \(f\) is the friction factor (a value slightly less than 1), the new velocity is calculated as \(\vec{v}_{new} = \vec{v} \times f\). The friction factor \(f\) is a fixed-point number to maintain consistency with the project's computational approach.
The frictional adjustment is applied separately to both components of the velocity:
This reduction in velocity is applied in every frame, gradually slowing down the ball until its velocity is negligible. To prevent the balls from moving indefinitely due to very small velocities, a threshold velocity (\(v_{threshold}\)) is set. When the ball's velocity falls below this threshold, it's set to zero, effectively stopping the ball. For example, imagine a ball rolling to the right (positive x-direction). Initially, it has a velocity \(\vec{v} = (5, 0)\). With a friction factor \(f = 0.98\), after one update, the velocity becomes \(\vec{v}_{new} = (4.9, 0)\). This process continues, reducing the ball's speed until it stops.
By simulating friction, the game behaves more like a real physical system, where objects don't move indefinitely and eventually come to rest, enhancing the realism and immersion of the gameplay experience.
Merging Function
The merging function is used when a collsion between balls are detected, and the type of balls are the same. The function delete one of the ball node and change the type of another ball to bigger one.
Collide Detection and Avoiding Overlap
The trickest part is how to deal with the collision while avoid them overlap. In each frame, we traverse through the ball linked list to see if there is any collision occur between two balls. This is achieved by calculating the distance between the centers of two balls and compares it with the sum of their radii.
Once a collision is detected, we first check whether the two balls are the same type. If so, we call merge function mentioned previously. Also, an little animation so called “boids” effect is added during the merging process. We will introduce this later in the following session.
After that, the avoid_overlap function is used to adjust the positions of the colliding balls so that they no longer overlap. This function calculates the amount of overlap and adjusts the positions of both balls based on their masses. This adjustment ensures that the balls are repositioned as if they just touched each other at the moment of collision.
Finally, the collide_function handles the updated response of the balls to the collision. It calculates the new velocities by the elastic collsion formula we mentioned above in the Background section. These functions are called within the main game loop for each pair of balls.
Elastic Simulation
The project achieves an elastic simulation, particularly in the handling of collisions between balls and with the game boundaries, by carefully modeling the energy transfer and the elasticity of interactions. The key to this simulation is the use of an ELASTICITY constant, which plays a crucial role in determining how the balls behave upon impact.
Elasticity in physical terms refers to the ability of a body to resist a distorting influence and to return to its original size and shape when that influence or force is removed. In the context of the game, this concept is applied to the collisions. When two balls collide, or a ball collides with a boundary, they don't stick together or stop moving (which would be inelastic), instead, they bounce off each other, which is characteristic of an elastic collision.
The ELASTICITY constant in the game controls how much kinetic energy is conserved in these collisions. In a perfectly elastic collision, no kinetic energy is lost, and ELASTICITY would be 1. However, in most real-world scenarios and in the game, some energy is converted into other forms, like heat or sound, during a collision. Therefore, ELASTICITY is set to a value less than 1, typically reflecting this energy loss. This setting results in the balls not bouncing back with the same velocity they had before the collision, simulating a realistic physical response.
During the collision handling (as seen in functions like collide_function), when the balls collide, their velocities are adjusted based on their masses and the angle of impact, but they are also scaled by the ELASTICITY factor. This scaling down of velocity post-collision is what simulates the energy loss. The balls bounce back, but with less kinetic energy than they had before the collision, giving a realistic feel to the simulation.
Merging Spical Animation
To add animation when two balls merge, we designed a "boids" effect. For simplicity, the boids are small balls that follow the same physics laws as the real balls, but have a very short lifespan and don't interact with other real balls, akin to neutrinos.
When a merge occurs, 10 boids of the same color are generated from the merge point, each with a random initial speed and direction. Unlike regular balls, the boids have a "time to live" (ttl) property in their data structure to limit their lifespan. They also have a "collidable" property set to false to indicate they are different from regular balls.
Deal with BGM
When designing the background music (BGM), we aim to incorporate two types of sound. One is the background music and the other is a special sound triggered when balls merge. The background music is a sequence of notes selected from a chord and repeats indefinitely until the game ends. During gameplay, whenever balls merge, the special merging sound is played alongside the ongoing background music.
struct note { unsigned int frequency ; unsigned int duration ; struct note *next ; } ;
To tackle this, we've designed a unique data structure called 'note' that includes the frequency, duration, and a pointer to the next note. A repeated timer, dedicated solely to music synthesis, samples and plays the current note at a specific frequency every time it's triggered, and it decreases the duration by 1. When the duration reaches 0 and the next note isn't null, the next note will be played. Theoretically, with a series of notes, it can play any single note music.
graph LR
A[Note 1] -->|next| B[Note 2]
B -->|next| C[Note 3]
C -->|next| D[Note 4]
D -->|next| E[Note 5]
E -->|next| F[Note 6]
F -->|next| G[Note 7]
G -->|next| H[Note 8]
H -->|next| A
For the background music, we used eight notes to form a simple, infinitely repeating melody. To achieve this, we linked the eighth note back to the first note, forming a looped linked list. As the duration of each note decreases while playing, we added a mechanism to restore the original duration after each play, allowing the music to continue uninterrupted.
graph LR
A[Note 1] -->|next| B[Note 2]
B -->|next| C[Note 3]
C -->|next| null[null]
In addition to the background notes linked list, we have a second linked list with a higher priority for playing special merging sounds. It shares the same data structure as the background music list, but it doesn't loop. The last element's next pointer points to 1, causing it to stop once the sound has finished playing.
graph TB
switch(Switch)
bgMusic(Background Music Linked List) --> switch
mergingSound(Merging Sound Linked List) --> switch
switch --> musicOutput(Music Output)
Additionally, the frequency of the merging sound is related to the size of the merged balls. Specifically, this design aims to introduce variation and certainty to the game, thereby enhancing the user's gaming experience.
UI Design
The UI and menu design in this project are implemented to provide a user-friendly interface, allowing players to navigate through options like starting the game, selecting modes, and toggling background music. This aspect is critical for enhancing the overall user experience and making the game accessible and engaging.
The project utilizes a VGA display for its output. The display.c component is responsible for all the drawing functions on the screen, including text and graphical elements. This is where the menu items, game status, scores, and other relevant information are visually rendered.
The menu.c file contains the logic for the menu system. It defines a list of menu items (menu_list), each represented by a structure containing the item's text, color, background color, position, and size. This structure provides a flexible way to define and modify menu items.
The menu system is designed to be navigated using the joystick. Functions like menu_up, menu_down, and menu_select in menu.c handle the logic for moving through the menu items and selecting options. These functions change the visual appearance of the menu items to indicate the current selection and trigger actions like starting the game or changing settings.
graph LR
A[Start Main Program] --> B[Initialize Display]
B --> C[Display Main Menu]
C --> D{User Input}
D -->|Select Item| E[Perform Action]
D -->|Navigate| C
E -->|Start Game| F[Game Screen]
E -->|Change Settings| C
F --> G{In-Game User Input}
G -->|Play Game| F
G -->|End Game| C
The UI provides immediate visual feedback in response to user actions. For instance, when a menu item is selected or a setting is toggled, the menu visually updates to reflect this change. This feedback is essential for a good user experience, letting players know that their inputs have been recognized.
The menu system is integrated with the main game logic in main.c. The game's state (such as whether it's in the menu, playing, or game over) determines what's displayed on the screen and how user inputs are handled.
Referenced Library
- vga_graphics.h: VGA library provided in the class
- pt_cornell_rp2040_v1.h: protothreads librarys provided in the class
Results
The development and testing of the interactive ball game on the RP2040 microcontroller demonstrated successful implementation of both the menu UI and the game mechanics in different modes. The game's behavior aligns well with the intended design, offering an engaging and dynamic experience. Below are detailed observations from the testing phase:
Menu UI
The implementation of the Menu UI, as shown in the first GIF, demonstrates a successful and engaging user interface. The background animation features falling balls, adding an interactive and dynamic aspect to the menu. This design choice significantly enhances the visual appeal and keeps the user engaged even before the game starts.
The menu interaction is intuitive and responsive, controlled effectively using a joystick. Navigating through the menu items is smooth, with the selection changing as the joystick moves up or down. The implementation of the joystick control is well-executed, providing a natural and user-friendly experience.
The functionality to toggle game modes and background music (BGM) through joystick movements is a thoughtful addition. It allows players to customize their gaming experience directly from the menu. The option to start the game with a simple joystick movement to the right when 'START' is selected showcases a straightforward and efficient user interface design.
Game - Gravity Mode
In the Gravity Mode, as illustrated in the second GIF, the game mechanics are notably well-implemented. The representation of gravity and its effect on the ball's motion is realistic. The ball is initially positioned at the top of the screen with varying size and color, creating diversity and unpredictability in gameplay.
The implementation of gravity is evident in the increasing speed of the falling ball, simulating real-world physics accurately. The addition of air friction, which slows down the ball proportionally to its speed, further adds to the realism of the game. This thoughtful consideration in the physics engine significantly enhances the gameplay experience.
The collision dynamics, both with the boundary and between balls, are well-handled. The assumption of elastic collisions with boundaries (where only the direction of velocity changes) is clearly observed. The interactions between balls are also effectively managed, demonstrating momentum and energy exchange principles.
A particularly engaging feature is the merging of balls upon collision, if they are of the same type. The merging animation, with boids dispersing in all directions, adds a visually appealing effect to the game. The inclusion of sound effects, both for the background music and the merging of balls, further enriches the gaming experience.
Game - Non Gravity Mode
The Non-Gravity Mode (Mode 2), shown in the third GIF, offers a distinct gameplay experience from the Gravity Mode. In this mode, the absence of gravity combined with enhanced friction creates a different dynamic for the movement and interaction of the balls.
The collisions, both with the game's boundaries and between balls, function smoothly and as expected. The enhanced friction in this mode leads to a quicker stabilization of the balls, aligning with the intended design of the game.
The contrast between the Gravity and Non-Gravity modes provides a versatile gaming experience, catering to different player preferences. This versatility showcases the game's adaptability and the effectiveness of the underlying physics engine in simulating different scenarios.
Conclusion
Our project achieved all of our initial goals and expectations. Besides, we completed all additional feature requirements such as merging effect, background music and sound effects. We learned a lot from this project, one of the most significant learnings came from developing the 2D collision algorithms. This required a deep dive into physics principles, particularly elastic collisions, and gravity effects, enhancing our understanding and ability to apply these concepts in a practical setting. While developing the system, a major challenge we encountered and overcame was the overlapping problem during collisions. This pushed us to develop innovative solutions to solve this issue. We also developed an intuitive user interface, making the game easy to navigate and enhancing the overall user experience.
Looking forward, there are several areas where our project could be further improved. For example, we could optimize the computation flow for collisions. From the previous result, we observed that maintaining a steady frame rate becomes challenging when the container holds too many balls, likely due to the resource-intensive functions implemented for collision and overlap management. Simplifying these calculations could significantly boost the game's performance. Moreover, since core1 is currently underutilized—being used only for synthesizing music and sound effects—better performance could be achieved by more evenly distributing calculations across both cores.
Regarding intellectual property considerations, our project was designed to be open-source. The foundational code utilized the Protothreads library, known for its efficient and lightweight macros that aid in managing threads. This library is distributed under a BSD-style license by its creator. Such a license categorizes it as an open-source library.
Appendix
Appendix A: Permissions
The group approves this report for inclusion on the course website.
The group approves the video for inclusion on the course youtube channel.
Appendix B: Commented Code
The commented code can be found at https://github.com/IoTcat/ece5730/tree/final-project
Bill of Materials
A bill of materials for this project is as follows:
| Material | Source | Cost |
|---|---|---|
| RPi Pico | Lab | - |
| Joystick | Amazon | $28.95 |