Our final project, Temu Pebble Band, is an interactive musical rhythm game reimagined through the lens of the Raspberry Pi Pico RP2040 microcontroller, paying homage to the smash hit 2007 game Rock Band, while also acknowledging the homemade, well-loved feeling to our game that some may describe as “Temu”. Our game challenges players to follow along with falling notes on a VGA display while playing the corresponding keys on a mock piano we built ourselves from scratch. As players strike the correct notes at the right time, the piano notes play aloud over the speaker, and the player ends up playing the song for the level they selected. After the player’s run is over, the game gives them a score depending on the number of notes they’ve hit or missed, before returning to the menu screen where they can play other levels or try again. Our goal at the beginning of this project was to create a game that is technically robust enough to be immersive, accessible to recreate using just the materials at our disposal, and most importantly, end up with a game that is fun to play and build.
Our idea originated from a shared interest in music, gaming, and hands-on hardware projects. Taking from all the labs, we wanted to combine audio synthesis and interactive animation into a cohesive game experience, but interfaceable with custom hardware. Inspired by Rock Band and Piano Tiles, we wanted to make a handmade version, using a custom-built instrument.
At a high level, the game operates as follows:
The high level mechanics of our game can be understood in the block diagram included here. We used a miniature replica of a piano spanning one full octave of 13 keys (Middle C to the next C up), with each button mapped to either a specific note. The limited key count simplifies gameplay and allows us to scale popular or complex songs down to a playable format for our hardware constraints.
We pre-generated each note’s sound and hard-coded the sequence of notes for each song. We captured the pre-generated notes by recording ourselves playing the exact keys on a physical piano. Next, we ran these audio files through custom Python scripts to get them in a format compatible with our Pico and DAC to be played aloud, as described in great detail in the Software section of the report. Likewise, we simplified the instrument inputs to digital buttons rather than analog or capacitive sensors, reducing complexity while maintaining responsiveness.
Our game draws inspiration from existing games such as Rock Band and Guitar Hero, but we created all visuals, code, and audio mappings independently. The exception is the edited Temu logo in the menu screen, but this alteration falls into fair use as it is clearly a parody. We are not distributing this game commercially. We either recorded all sounds or took them from open-source/public domain collections. Regarding the songs in our game, both Twinkle Twinkle Little Star and the arrangement of Great Fairy Fountain are in the public domain. References and data sheets are documented in the appendices.
This project involved three main hardware components: building a miniature mechanical piano, connecting the circuitry of the microcontroller to the piano, and enabling the microcontroller to control graphics and audio according to user input. Here, we go into great detail regarding the implementation of each of these three undertakings.
When we set out to complete this project, we knew we wanted our piano interface to look like a piano, feel like a piano, and play like a piano, but be scaled down and have fewer keys for the purpose of our game. As a result, we decided to narrow our piano down to just one octave with thirteen keys, spanning from one C to the next C up. We knew a mini-piano following our specifications must have been built before, so we started researching other projects where people have constructed similar pianos to see if there were any helpful tips for building our desired product. We found a self-documented project on Autodesk Instructables where an individual 3D printed and built a 1-octave piano, found here. This 3D printed piano fit all of our specifications, although the documentation for the rest of the project was not the best. This piano had way more wires than we wanted in ours, many steps regarding its construction were not easy to follow or even discussed, and we faced challenges in finding the hardware necessary to recreate the piano that this individual made. As a result, we decided to attempt building our piano using the 3D printed pieces in this project, but we built and implemented everything beyond that from scratch. We did not use any code from this project and we implemented the mechanics of our keyboard using our own ideas and hardware we had access to. This source was useful because we used their published 3D printing files to print the keys and exterior of our piano, but all actual implementations were completed on our own.
After 3D printing the keys and shell of the piano, we found a metal rod in the lab to use as an axle for the keys to rest on. At this point, all of the piano keys could move up and down on the axle although they would rest on the table as if they were being pressed without any springs or elastic to offer support in the back. We searched through hardware stores like Lowe’s and Home Depot to find springs to use to hold our keys in proper position, however we could not find any good options that would not be very expensive. Thus, we opted to use rubber bands to hold our keys up and give enough force to spring back into position after being pressed. To anchor the rubber bands in place, we added M5x20mm screws through the sockets in the back of all 13 keys, held in place with 3 M5 nuts (with two on top of the key, and one underneath the key). Next, we tied rubber bands around each of the stumps made by each of the screws as seen here, and our piano keys were able to be pressed and released with a very similar feel to a real piano.
Figure 1: Interior mechanics of the piano using rubber bands
A new problem began to arise however, where the rubber bands would calcify and snap easily (because they were old), and it would be very difficult to replace a broken rubber band if it were not on end keys. Therefore, we decided to take a new approach, using strips of elastic to hold each of the keys in position. The elastic would be easy to replace, and would not have as much friction on the desk or piano shell, meaning they would break or snap a lot less frequently. Next, we used two clamps to hold our piano to the table and tested our interface. While this elastic allowed our keys to be pressed similarly to a real piano, a new and unrelated problem occurred in which some keys would get caught underneath other keys when they were pressed. This issue only happened on the white keys that neighbored black keys due to the overhang getting stuck on the next white key over. To correct this, we added plastic tabs between these keys to act as dividers and prevent the overhang from getting caught on the under-shelf of its neighbor. These tabs served as intended, and their functionality can be seen here:
Figure 2: Plastic dividers between keys with overhangs that would catch when pressed
Figure 3: Image of finished piano octave with keys held as rest and capable of being pressed
Now that our piano keys could be pressed and released similar to the real instrument, the next step was to find a way for these key presses to be registered by our microcontroller. To do so, we wanted our keys to act as buttons where each key would constantly be in contact with either power or ground and any change in this status could be detected as a press. This approach would require three main components for each key: a path to power, path to ground, and connection to the microcontroller to communicate its status. While the screws and bolts were originally introduced to each key as a way to anchor the elastic, we decided to take advantage of these conductive, metal components. By connecting wires to each key’s screw, we had built a connection to communicate the key’s status to the microcontroller if we could find a way to connect these screws to ground or power. To accomplish this, we decided to introduce one power rail and one ground rail to embed in the piano’s shell and use for all thirteen keys.
For the power rail, we used a solderboard and added a connection to 3.3V that extended the full length of the octave. We glued this “power strip” to the roof of the piano’s shell in a position the screws would not come into contact unless they were pressed pretty far down. This would avoid false signals and ensure key-motion was not restricted by coming into contact with a solid strip prematurely. In addition to the power rail, we need a ground rail to prevent our floating inputs since the mux does not have pull-down resistors. Without the ground rail, our piano would enter “seizure mode” as shown below:
For the ground rail, we could not use the same approach as the power rail because there was not enough space underneath the keys and elastic strips to fit another solderboard that would come in steady contact with all thirteen keys. So, we resorted to using tin foil. We took a big sheet of tin foil, rolled it into a cylinder that could fit into the small gap available, and then added additional pieces of compacted tin foil as needed to ensure each key's screw would rest against the tin foil while not pressed. We acknowledge that there are many ways this ground rail could have been executed better, but this was the best we could do at the time, given the resources and constraints we faced. Unfortunately, this is likely where a large percentage of errors encountered further on would originate, so I implore anyone considering recreating this project to find a better way of adding ground to their piano interface. Nonetheless, the result of adding these three components can be seen here:
Figure 4: Labeled diagram of electrical components added to piano
Figure 5: Side by Side depiction of how our keys work as buttons
|
List of Materials Used |
|
|
Components:
|
Tools:
|
Finally, here is the finished result:
Figure 6: Our completed, playable 1-octave piano
After completing the construction of our functional piano with button-keys, we needed to connect the signal carrying wires from each of the thirteen keys to the microcontroller to be processed in real time during gameplay. We did not want to use 13 separate GPIO pins on our microcontroller, because the Raspberry Pi Pico RP2040 only has 28 pins available, without additional hardware. Two crucial components of our project’s gameplay would be playing audio over the speakers and updating the game on the connected VGA screen, so we wanted to avoid giving up nearly half our available pins to the piano if we could come up with a more elegant solution. After consulting with Professor Bruce Land about where to go next, we decided to try using digital multiplexors, or “muxes”, to greatly simplify the number of pins needed. We were able to find two 74AC151 8-input multiplexors, which provided us with 16 inputs, more than enough for our 13 piano keys. We researched and read the datasheet of these muxes here, and found the following information regarding the connections and pins of these multiplexors:
Figure 7: Important information about the 74AC151 mux from datasheet
(Fairchild Semiconductor™, 1988)
These muxes seemed like a good option for our purposes, so we decided to integrate them into our design. We decided to split the 13 piano keys by putting 8 of them onto one mux, while the remaining 5 keys would be connected to the first 5 inputs of the second mux. Using another half-solderboard, we connected 3.3V from the RP2040 to power and grounded both the Ground and Enable pins on the muxes. This way, the muxes would be constantly enabled to have the selected input pin’s value read and passed to the RP2040 according to the value of the three Select Input bits discussed in the datasheet. The integration of these muxes is explained in greater detail in the program section of our report, but at a high level: All thirteen piano keys were connected to an input on the muxes, the microcontroller quickly cycles through and reads the value on all 8 pins during each cycle, taking note of which keys are pressed and playing the audio of that key over the speaker. When a piano key was pressed, it would pass a 1 to the mux (as explained in the previous section), so if one of the mux’s input pins passed on a 1 that meant its respective key on the piano must be pressed.
So, while 13 signals were being sent from the piano to the muxes, only 2 signals were sent from the muxes to the RP2040. Thus, only 2 GPIO pins needed to be initialized and used as inputs– one pin for each of the muxes (27 and 28). To cycle through the 8 inputs of each of the muxes, we used three Select Input bits in order to generate binary numbers from 0 to 7. We used 3 of the GPIO pins (2, 3, and 4) on the RP2040 to act as the 3 Select Input bits, and were able to recycle these values and connect them to both muxes. As a result, we were able to narrow the number of GPIO pins used from 13 to only 5 pins (3 output and 2 input). Here, we have an image of the solderboard used in our project, followed by a schematic of the wiring connecting the piano to the microcontroller via these two muxes can be seen in the following figure, which is then explained in even more detail in the subsequent table.
Figure 8: Image from lab of circuitry connecting piano interface to the microcontroller
Figure 9: Schematic of the circuitry involved with the piano key button interface
|
To Microcontroller |
||
|
Wire |
Purpose |
Function |
|
1 |
Power (3.3V) |
Supply power to multiplexors and power rail of mechanical piano |
|
2 |
Ground |
Act as ground for multiplexors, enable the multiplexors, and supply ground rail for mechanical piano |
|
3 |
MUX_1 (GPIO 27) |
Data Output of Mux 1 (left mux in diagram) |
|
4 |
MUX_2 (GPIO 28) |
Data Output of Mux 2 (right mux in diagram) |
|
5 |
MUX_SEL2 (GPIO 4) |
These three signals form an index between 0 and 7 that selects which input signal (I0, I1, … I7) to pass on through the Data Output (Z) of each multiplexor |
|
6 |
MUX_SEL1 (GPIO 3) |
|
|
7 |
MUX_SEL0 (GPIO 2) |
|
|
To Mechanical Piano Keys |
||
|
Wire |
Purpose |
Function |
|
8-15 |
8 Inputs to Mux 1/Output of Piano Keys 1-8 |
Connects each piano key to a multiplexor input (I0, I1, … I7) to be checked if respective key is pressed or not; 1 if key pressed, 0 if key at rest |
|
16-20 |
5 Inputs to Mux 2/Output of Piano Keys 9-13 |
Connects each piano key to a multiplexor input (I0, I1, … I4) to be checked if respective key is pressed or not; 1 if key pressed, 0 if key at rest |
|
21 (Not Pictured) |
Ground |
Connect tin foil rail to ground so keys are grounded at rest |
|
22 (Not Pictured) |
Power (3.3V) |
Connect solderboard rail to power so keys are powered when pressed down |
With that, our piano interface was fully completed, and able to be connected to the microcontroller for use in gameplay. While the piano was still being built, we used the keypad from Lab 1 as a placeholder for the inputs from the user pressing buttons. During this time, we reused the keypad code we wrote in Lab 1, but introduced a new thread to detect piano key presses as discussed in the Program Details section of our write-up.
|
List of Materials Used |
|
|
Components:
|
Tools:
|
Figure 10: Final piano interface
Our project made use of the VGA monitor and the speakers to display animations and play piano audio according to the keys pressed by the user, both of which were controlled by and connected to the Raspberry Pi Pico RP2040 microcontroller. Fortunately, the use of the speakers and VGA monitor were very similar to that of Lab 1 and Lab 2, so no additional wiring or hardware was needed. In fact, to add audio and visual capabilities to our project we re-built our breadboard from Lab 2, and made a copy of our code from previous Labs to use as a base for this product. Several of the peripherals previously used for this course were recycled for our final project, as described in the updated peripheral table from our Lab 2 report:
|
Peripherals |
Purpose/Implementation |
|
Raspberry Pi Pico |
Responsible for detecting piano key presses via the two 8-input muxes and then playing the corresponding piano notes by controlling the DAC. Also responsible for updating and animating the game screen seen on the VGA monitor. Also acts as a memory unit to store the audio to be used. |
|
MCP4822 (DAC) |
Inputs a 12-bit digital input from the Pico and outputs a corresponding analog signal to the audio socket. Here, it inputs a short pulse and outputs that to the speakers. The Pico communicates with the MCP4822 via SPI, sampling it at 50kHz. |
|
SJ1-355XNG 3.5mm Audio Socket & Speaker |
Converts the DAC’s analog waveform output into an audible note that is played each time the user presses a piano key. |
|
VGA Display Monitor |
Used to show the game, menu screen, and results. It has three color inputs (Red, Green, Blue) and two timing inputs (HSYNC, VSYNC). Green uses two GPIOs for higher resolution, because humans perceive more shades of green. VGA is implemented using PIO co-processors instead of the CPU for precise timing. The display updates at 25.172 MHz, but sending it values at 25 MHz is sufficient. Resistors act as voltage dividers to limit the display voltage to 0-0.7V. |
|
VGA Adapter (with 330Ω resistors & a 470Ω resistor) |
Used for connecting the Pico to the VGA monitor. The connector utilizes a VSync, HSync, GND, Red, Blue, and Green pin as shown in the wiring schematic. Since the RGB pins require a voltage from 0-0.7V, and the display has an internal 70Ω resistor, we require a 330Ω resistor between our 3.3V output and the pin to put us in a safe range. To give our green values a bit more range, we have another pin from our Pico going into the green VGA pin through a 470Ω resistor, allowing us to have more control over the voltage at that pin. |
The connections between these peripherals can be seen in the following image of our breadboard taken in the lab along with its corresponding schematic. The subsequent table discusses each of the GPIO pin connections to our RP2040 and what they each do.
Figure 11: Image of our RP2040 connected to VGA and Speakers
Figure 12: Schematic of microcontroller wiring (Piano, VGA, and Audio supported)
|
Piano Keys |
||
|
GPIO Pin |
Connection |
Purpose |
|
2 |
Mux Select Bit 0 |
Output: Selects the pin which the mux will read and pass to its output |
|
3 |
Mux Select Bit 1 |
|
|
4 |
Mux Select Bit 2 |
|
|
27 |
Mux 1 Output |
Input: Detects the keys pressed and released according to these values |
|
28 |
Mux 2 Output |
|
|
DAC |
||
|
GPIO Pin |
Connection |
Pin Function |
|
5 |
PIN_CS |
SPI |
|
6 |
PIN_SCK |
SPI |
|
7 |
PIN_MOSI |
SPI |
|
VGA |
||
|
GPIO Pin |
Connection |
Purpose |
|
16 |
HSync |
Ensure coherent and consistent image during animation |
|
17 |
VSync |
|
|
18 |
470Ω → Green 1 |
Allow for full color display on VGA Monitor |
|
19 |
330Ω → Green 2 |
|
|
20 |
330Ω → Blue |
|
|
21 |
330Ω → Red |
|
Our code is organized into three sections: user input handling, game logic and an animation loop, and audio generation. The code for this project runs on a single core. Initially, we decided to separate our audio code into a single core, but this proved unnecessary due to our pure DMA implementation of sound generation. An overview of our logic is shown below:
The user input for our project is done via a single thread. Given that we have 13 buttons, we decided to condense the GPIO required to leave more room for additional peripherals later on. To do this, we used two 8:1 muxes, reducing the 13 GPIO to only 5—two for mux outputs and three for mux selects. We scan the mux selects, checking if the outputs of either mux are one, allowing us to scan all 13 inputs. We scan the inputs every 30ms. An oscilloscope trace of our mux output can be seen below in Figure 13, showing how the key is read every 30ms.
Figure 13. Oscilloscope of mux output
We consider a button as pressed if an output equals one for two consecutive scans to combat switch debouncing. Likewise, when a button transitions from a one to a zero and stays zero for two scans, we consider that key released. Whenever a key is pressed or released, it calls the corresponding callback function. While on the menu screen, releasing a key causes the cursor to change items or selects the current item. On the game screen, pressing a key highlights the corresponding note, plays the corresponding sound, and checks if the note in that lane is in the hitbox and if so, it updates that note’s status to “hit.”
Our project consists of two distinct “modes:” a menu/credits mode and the game mode. The game mode is further separated into three different submodes: song 1 (Great Fairy Fountain), song 1 with lives, and song 2 (Twinkle Twinkle Little Star). In the mode with lives, you can only miss three notes before you hit the game over screen. The game over screen displays after you finish a song or lose, showing the number of hit and missed notes, your accuracy, and max combo. The menu screen allows you to select between the submodes and the credits screen. Figure 18 in the Results section shows the screens explained above.
The logic for animating and controlling the game is split into two threads, one for timing the spawning of notes and one for animating the objects to the screen. The spawning thread reads from the array, which is set by the menu screen selection, constantly spawning notes and yielding until the end of the sequence is reached. The notes are hardcoded into our program to follow a set song. The animation thread handles drawing and updating every element on the screen. Here, it handles descending the notes, highlighting the pressed keys in red or green, highlighting the lanes that contain notes in a thin magenta outline, and deleting/shrinking the notes that are hit. The thread that handles scanning our piano buttons and notifies the animation loop thread which keys are pressed and which keys have been struck correctly. When a key is pressed we highlight it (the highlighted color is green if a white key is pressed, red if a black key is pressed) and play the corresponding note. If the correct key is pressed while the note is in the hitbox, that note is marked as “hit” and once that note is updated again, it will either disappear or shrink depending on if the note is a “normal” note or a “long” note. “Long” notes must be held down until they disappear to simulate sustained notes. Creating “long” notes was one of the trickier aspects of this project since they need to persist after being hit, shrink while they are being hit, but stop shrinking once their lane’s key has been erased. Additionally, when a note is hit, it will either be rated as a good, bad, or perfect hit, whose status is displayed in the top right corner of the screen. A perfect hit is when the bottom of the note is within 20 pixels from the bottom of the hit area, a good hit is when it is between 20 and 30 pixels, and a bad hit is anywhere between 30 pixels and the top edge of the hit area.
The course's VGA graphics library facilitates the streaming of our game to a VGA screen, the fonts, and the shape primitives (lines and rectangles). The shape primitives form the core functionality of our game, allowing us to easily draw the falling notes, lines on the track, an animated piano on the bottom of the screen, and text. However, it was insufficient for our needs as we wanted to draw pictures on the screen for background art.
The backbone of our background visuals was a Python script. We wrote a script that inputs an image, resizes it to 640x480 pixels (the size of our VGA screen), and then converts it to a C header file that our program will read. Since each pixel is determined by a 4-bit value to represent one of 16 colors, the script also needs to find the closest color for each pixel in our given image. To do this, we first need to determine the RGB values of our 16 colors. Given that we have a single input to our red and blue VGA channels, the possible red and blue values are either 0 or 255. However, we have two inputs for the green channel, which gives more flexibility. The green channel inputs go through a 330-ohm and 480-ohm resistor each, both creating a voltage divider with the 70-ohm resistor inside the display. Since the Pico outputs a 3.3V signal and the display accepts a voltage range of 0-0.7V, we can calculate the voltage and thus green RGB value that these two inputs can create. Suppose we turn on the 330-ohm output:
. We put the 480-ohm and 70-ohm resistors in parallel because, despite
the 480-ohm Pico pin being off, it is still a low impedance path to
ground. Thus, when we turn on the pin connected to a 330-ohm resistor,
we get a green value of 187, and doing the same calculation but
switching 330 with 480, we get a value of 132. Thus, our green values
can either be 132, 187, or 255 in the case that both are on. Figure 14 shows a side-by-side comparison of a picture of the demo code given to us
by this class, and the result of uploading that picture to our Python
script and then decoding the file to re-display the image.
Figure 14. Side-by-side of demo code colors (left) and generated/decoded image (right)
After we determined the correct color values, we still needed to write the values to a readable header file. Because each pixel is defined by four bits, we can compact the header file by concatenating each pair of pixels into a single byte, resulting in each image taking up about 153.6 KB. This storage optimization was necessary due to the large number of audio files we use, which will be discussed later in this section. The result of this can be shown in Figure 15, showing the background images of our menu and game screens, drawn and edited by Dyllan. The jankiness was intentional, to play into the theme of “Temu.”
Figure 15: Menu screen and image of gameplay
Next was the audio of our project. We settled on utilizing DMA channels to read through an array that holds the data of a WAV or MP3 file. Again, this uses a Python script to generate the array and saves it to a header file. Similar to the picture encoding, we read each amplitude value of a WAV/MP3 file, scale it to be between -2048-2047 (since we use a 12-bit DAC), combine that with the DAC configuration bits, and then save that value to an array. The array is then saved to a header file uploaded to the board during flashing and read from whenever the DMA channel is activated. We can get away with only using two DMA channels despite our several samples by reconfiguring them to read from different arrays whenever we need to change sounds. This audio pipeline allowed us to record samples from a real piano and play them back whenever a key was pressed, as shown below:
Initially, we planned on synthesizing our piano audio, but this proved too difficult during the time frame. We first tried to synthesize a piano noise in Python and then port over our code. However, we were unsuccessful in creating a realistic piano sound that could rival our recordings. We originally tried to generate a piano sound via analyzing the spectrogram of a real piano and recreating it. However, this ended up sounding like a solid frequency instead of sounding like a real piano sound. Below in Figure 16, you can see a side by side comparison of the spectrogram of our generated sound, and a real piano spectrogram that we recorded. You can also hear the generated sound below:
Figure 16. Side-by-side comparison of generated piano noise (left) and recorded spectrograms (right) with generated piano audio.
We also tried to utilize the Karplus-Strong algorithm to create a piano sound since we know it can make a good string sound. However, we could only recreate a guitar string pluck, instead of a hammer hitting three strings for a piano sound. Below in Figure 17 is the generated audio and spectrogram from our Karplus-Strong implementation:
Figure 17. Karplus-Strong implementation of a piano noise
Lastly, we wanted to try streaming audio from a computer to have background music of different instruments, while the player was responsible for creating the piano sounds. Here, the process was more time-limited than limited by the methods we used. To stream sound from our computer, we need two features: a way to control when audio is played and a way to stream audio. We could use the Pico as a USB device and control audio playback, but we could not implement the Pico as an audio sink. We wanted to keep the whole project pipeline computer independent (besides power), or only have a USB connection, thus barring us from using an audio jack to stream audio through the Pico’s ADC. With more time, we would flesh this feature out and add background music.
All in all, our game runs quite smoothly. Our game starts with a menu screen where you can navigate around using key 1 and 2. From there, you can move to the credits screen or navigate to the game and play. While playing, you are given notes that fall down from the screen. Your goal is to hit the notes at the correct timing as they enter the hitbox, leading you to play a song by yourself. If you want an extra challenge, you can select the mode with lives where you can only miss two notes – if you miss a third, the song ends early and you lose. To have some variety we have implemented two songs – Great Fairy Fountain from The Legend of Zelda and the classic Twinkle Twinkle Little Star. A quick display of the different screens in our game are shown below:
|
Menu screen and full hardware setup |
|
|
Game Screen (no lives) |
|
|
Game screen (with lives [in top left corner]) |
|
|
Credits screen |
|
Figure 18. Summary of game screens
We optimized our VGA animation to minimize flickering from drawing too much in a single frame. Additionally, we optimized the memory required to store all of our images and audio samples, with each image taking about 153KB and each audio sample taking about 70KB, adding up to a total of about 1.2 MB. Using these audio samples, we were also able to create extremely realistic piano sounds as you play.
The hardware portion of our project may have introduced a couple of bugs or errors into our final product. As mentioned earlier in the hardware section, the ground rail that all of the piano keys connected to while at rest was made out of flimsy tin foil molded into the shape of our piano’s inside. Because of this, the foil moved around a bit as the piano was transported or the keys were pressed. Every time a key was pressed, the foil underneath it would shift slightly, and sometimes even the foil touching neighboring keys would be affected as well. When the foil moved the nuts and screws would sometimes lose contact with ground even at rest, which would skew the results and confuse the microcontroller. If a key was pressed and made contact with the 3.3V power rail, if it returned to resting position without touching the grounding foil enough, the charge would not dissipate fast enough for the muxes to register it as a zero. In this case, the voltage of the key may be 0.5V or more, which would be considered a press even when it was not intended to. This slight shifting paired with the high volume of key presses in our implementation process resulted in many keys mistakenly being registered as pressed during the game. This explains why some keys flicker red during the game even if they are not pressed, as shown below in Figure 19.
Figure 19. Flickering key
Another issue our group faced with the hardware was strange behavior in one of the muxes. We needed to use two different 8-input muxes, because we had 13 inputs, and while Mux 1 connected to the first 8 keys worked fine, Mux 2 connected to the remaining 5 keys had some inexplicably strange behaviors. While the tin foil ground rail was definitely not the most reliable way to ground all of the keys, these upper 5 notes seemed to flicker and detect phantom presses much more frequently than the notes connected to Mux 1. We used an oscilloscope in the lab to attempt to get to the bottom of why that may be occurring, but we could not resolve why exactly this was happening. The oscilloscope traces showed Mux 1 to be functioning as expected, but the traces for Mux 2 demonstrated some unexpected, undesired behavior. While each of the inputs to Mux 2 seemed to act as intended (updating when key pressed or not), the output of this mux was nearly always high. This was not correct because the mux should only pass through a high signal when keys were pressed. The Enable, Ground, Power, and 3 Select Input bits were wired the exact same way as Mux 1, and the inputs from the keys acted the same, but the output of Mux 2 seemed inexplicably inverted. In the software, both of the muxes were initialized and checked the exact same way, except that Mux 2 would not do anything if the three unconnected inputs were high (it would only play the piano audio for the keys that were actual valid inputs). Because of this, it seemed as if the upper 5 notes of the piano should not function correctly at all, however they did seem to correctly detect and play the audio when they were pressed by the user– they would just play the audio additional times even when it wasn’t supposed to. Ultimately, it seems as if this error may somehow be an internal hardware issue with the 74AC151 mux itself, as all of its inputs were acting the same as Mux 1, but giving the seemingly incorrect output. We were unable to debug the root of this issue, although the piano still worked as intended, if not a little buggily.
Additionally, because we are using DMA channels to display to the VGA and stream audio simultaneously, they compete, leading to the VGA screen shaking and getting distorted as shown on the below in Figure 20. Instead of attempting to remedy this, we decided it was a fun visual effect and decided it was a cool find, and kept it. On a similar note, instead of further pursuing synthesized piano notes as shown above in the What did not work? Section, we settled for a not-synthesized but still very realistic piano noise, because we are playing back a recorded piano sound. It still fulfilled its purpose of being an interactive experience based on user input, despite not being a synthesizer.
Figure 20. Distorted VGA display
Our game has no safety concerns, other than maybe the piano tipping over and falling on you. To prevent this, and to provide more stability, we clamped the piano to the table. Furthermore, our game is intended to be usable by everyone as long as you can reach and press the keys, our game is for you. Usability is somewhat affected by the ability to play the game well; for example, some group members are much worse at the timing of key presses than others, giving bad results. You do not need to read sheet music or anything like that, just press the key under the note on the screen.
While our initial expectations of having multiple instruments, a multiplayer mode, background music, and synthesized sound may have been a bit too ambitious, our finished project is still a respectable product. We implemented a version of Rock Band where the player creates their own music. By writing and utilizing Python scripts, we were able to overcome our lack of synthesis via audio playback for real piano sounds. We also created a working piano-like mechanism with realistic proportions and feel. There were some bugs with its responsiveness and finickiness. Still, given that it prioritized a cheap budget by using 3D printed parts, tinfoil, and leftover elastic, we did well for our constraints. Overall, we made a fun to play game using our tiny Pico.
However, if we had more time or if we were to reimplement our project, we would put more of an emphasis on grounding each piano key, making sure that it has optimal responsiveness. Additionally, we would create custom backing tracks via synthesis to accompany the music that we create while playing. We would also return to our original idea of having another instrument, i.e. drums, to enable more variety and a multiplayer mode where one can compete.
Regarding intellectual property, we did not use code from other people although we consulted AI for debugging problems. We did not use any of Altera’s IP, no code in the public domain, did not reverse-engineer a design or have to deal with patent/trademark issues. Did not sign any non-disclosure agreements to get sample parts, and we don’t have patent opportunities. We did however make a game which is similar to already copyrighted products, such as the game Rockband, guitar hero, and piano tiles. Additionally, a compressed version of the Super Mario Bros. death sound plays when you lose in the lives mode, and the Temu logo is on display in our menu screen. However, we are not redistributing this game for profit, so we do not need to worry about copyright infringement.
References:
Commented code:
/**
* Dyllan Hofflich
* Paige Shelton
* Edwin Chen
*
* This demonstration animates two balls bouncing about the screen.
* Through a serial interface, the user can change the ball color.
*
* HARDWARE CONNECTIONS
* - GPIO 16 ---> VGA Hsync
* - GPIO 17 ---> VGA Vsync
* - GPIO 18 ---> 470 ohm resistor ---> VGA Green
* - GPIO 19 ---> 330 ohm resistor ---> VGA Green
* - GPIO 20 ---> 330 ohm resistor ---> VGA Blue
* - GPIO 21 ---> 330 ohm resistor ---> VGA Red
* - RP2040 GND ---> VGA GND
#define PIN_CS 5
#define PIN_SCK 6
#define PIN_MOSI 7
#define SPI_PORT spi0
* # LDAC to gnd
#define MUX_SEL0 2
#define MUX_SEL1 3
#define MUX_SEL2 4
#define MUX_1 27
#define MUX_2 28
*
*
*
* RESOURCES USED
* - PIO state machines 0, 1, and 2 on PIO instance 0
* - DMA channels (3, by claim mechanism)
* - LOTS OF RAM (for pixel color data, backgground data, )
*/
// Include the VGA grahics library
#include "vga16_graphics.h"
// Include standard libraries
#include
#include
#include
#include
// Include Pico libraries
#include "pico/stdlib.h"
#include "pico/divider.h"
#include "pico/multicore.h"
// Include hardware libraries
#include "hardware/pio.h"
#include "hardware/dma.h"
#include "hardware/clocks.h"
#include "hardware/pll.h"
#include "hardware/spi.h"
#include "hardware/regs/rosc.h"
#include "hardware/adc.h"
// Include protothreads
#include "pt_cornell_rp2040_v1_3.h"
// include picture header
#include "gamebg.h"
#include "menubg.h"
// include dac header
#include "audio_download.h"
#include "C.h"
#include "CSharp.h"
#include "D.h"
#include "DSharp.h"
#include "E.h"
#include "F.h"
#include "FSharp.h"
#include "G.h"
#include "GSharp.h"
#include "A.h"
#include "ASharp.h"
#include "B.h"
#include "HighC.h"
#include "HighD.h"
#include "amplitude_envelope_mario.h"
// === the fixed point macros ========================================
typedef signed int fix15;
#define multfix15(a, b) ((fix15)((((signed long long)(a)) * ((signed long long)(b))) >> 15))
#define float2fix15(a) ((fix15)((a) * 32768.0)) // 2^15
#define fix2float15(a) ((float)(a) / 32768.0)
#define absfix15(a) abs(a)
#define int2fix15(a) ((fix15)(a << 15))
#define fix2int15(a) ((int)(a >> 15))
#define char2fix15(a) (fix15)(((fix15)(a)) << 15)
#define divfix(a, b) (fix15)(div_s64s64((((signed long long)(a)) << 15), ((signed long long)(b))))
// Wall detection
#define hitBottom(b) (b > int2fix15(360))
#define hitTop(b) (b < int2fix15(100))
#define hitLeft(a) (a < int2fix15(100))
#define hitRight(a) (a > int2fix15(540))
// max, min
#define max(a, b) ((a > b) ? a : b)
#define min(a, b) ((a < b) ? a : b)
// uS per frame
#define FRAME_RATE 33000
// DAC Pins
#define PIN_CS 5
#define PIN_SCK 6
#define PIN_MOSI 7
#define SPI_PORT spi0
// Built in LED Pin
#define LED 25
// VGA display parameters
#define SCREEN_WIDTH 640
#define SCREEN_HEIGHT 480
// ================================================================================================================
// =================================== BEGIN AUDIO SYNTHESIS CODE =================================================
// ================================================================================================================
// Number of samples per period in sine table
#define sine_table_size 22008
// Sine table
int raw_sin[sine_table_size];
// Table of values to be sent to DAC
// unsigned short DAC_data[sine_table_size];
// Pointer to the address of the DAC data table
const unsigned short *address_pointer2 = &DAC_data[0];
// A-channel, 1x, active
#define DAC_config_chan_B 0b1011000000000000
// Number of DMA transfers per event
const uint32_t transfer_count = sine_table_size;
// initializing dma channels
int data_chan;
int ctrl_chan;
dma_channel_config c2;
void play_sound()
{
// *thunk*
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
// Wait for the DMA channel to finish before starting a new transfer
// dma_channel_wait_for_finish_blocking(data_chan);
// dma_channel_wait_for_finish_blocking(ctrl_chan);
dma_start_channel_mask(1u << ctrl_chan);
}
}
void play_high_c()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_high_c, // The initial read address
40455, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_c()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_c, // The initial read address
28224, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_HighD()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_HighD, // The initial read address
23991, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_cSharp()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_cSharp, // The initial read address
35751, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_d()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_d, // The initial read address
36692, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_dSharp()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_dSharp, // The initial read address
39044, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_e()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_e, // The initial read address
37162, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_f()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_f, // The initial read address
39044, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_fSharp()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_fSharp, // The initial read address
38573, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_g()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_g, // The initial read address
33399, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_gSharp()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_gSharp, // The initial read address
31517, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_a()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_a, // The initial read address
35751, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_aSharp()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_aSharp, // The initial read address
29636, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_b()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_b, // The initial read address
31517, // Number of transfers
false // Don't start immediately.
);
if (!(dma_channel_is_busy(data_chan) || dma_channel_is_busy(ctrl_chan)))
{
dma_start_channel_mask(1u << data_chan);
}
}
void play_mario_death()
{
dma_channel_abort(data_chan); // abort the current transfer
dma_channel_abort(ctrl_chan); // abort the current transfer
// stop ping ponging
channel_config_set_chain_to(&c2, data_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// reconfigure Dma channel to play mario instead
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data_mario, // The initial read address
59806, // Number of transfers
false // Don't start immediately.
);
// Start the DMA channel
dma_start_channel_mask(1u << data_chan);
// wait for the DMA channel to finish
dma_channel_wait_for_finish_blocking(data_chan);
// write stuff to screen
fillRect(0, 0, SCREEN_WIDTH, SCREEN_HEIGHT, RED); // clear the screen
// bring back original config
// stop ping ponging
// channel_config_set_chain_to(&c2, ctrl_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data, // The initial read address
transfer_count, // Number of transfers
false // Don't start immediately.
);
}
// ================================================================================================================
// ====================================== END AUDIO SYNTHESIS CODE ================================================
// ================================================================================================================
// ================================================================================================================
// ====================================== BEGIN ANIMATION CODE ===================================================
// ================================================================================================================
// ===============================
// ======= menu code ============
// ===============================
static PT_THREAD(protothread_animation_loop(struct pt *pt));
// Menu state variables
int menu_state = 0; // 0 = main menu, 1 = game, 2 = credits, 3 = game over
int menu_selection = 0; // 0 = play endless, 1 = play song with lives, 2 = play song with no lives, 3 = credits
int current_menu_selection = 0; // current menu selection
int lives = -1;
int numLanes = 13;
void draw_piano(int lane, bool outline); // forward declaration of draw_piano function
// draw cursor on the menu given the current menu selection
void draw_cursor(int erase)
{
// Draw the cursor on the screen
int x = 80; // x position of the cursor
int y = 320 + (menu_selection * 40); // y position of the cursor
if (erase)
{
fillRect(x, y, 10, 10, BLACK); // erase the cursor on the screen
}
else
{
fillRect(x, y, 10, 10, WHITE); // draw the cursor on the screen
}
}
// ===========================
// ===== menu loop ===========
// ===========================
void draw_credits()
{
// Draw the credits on the screen
fillRect(0, 0, SCREEN_WIDTH, SCREEN_HEIGHT, BLACK); // clear the screen
setTextColor2(WHITE, BLACK);
setTextSize(5);
setCursor(100, 100);
writeString("Credits");
setTextSize(2);
setCursor(15, 160);
writeString("Made by: Dyllan Hofflich, Paige Shelton, Edwin Chen");
setCursor(15, 200);
writeString("Music by: YOU!");
setCursor(15, 240);
writeString("Press any button to go back to the main menu");
}
const int trackWidth = SCREEN_WIDTH / 3; // total width of the track
const int whiteHeight = 120; // height of the white key
const int hitHeight = whiteHeight + 80; // top height of the hit line from above the bottom of the screen
const int hitWidth = 40; // how tall the hit line is (hittable area)
int combo = 0; // combo counter for the number of notes hit in a row
int maxCombo = 0; // max combo counter for the number of notes hit in a row
typedef struct note
{
int lane; // which lane number it is in i.e. || 1 || 2 || 3 ||
float y; // y position of the note (top edge of rectangle)
int height; // height of the note -- used for sustaning notes
int color; // ye (in form 0-15)
bool hit; // if the note has been hit or not - used for erasing the note as its hit
bool sustain; // if the note is a sustained note or not
// Maybe add start time and
} note;
// number of lanes
const int gravity = 5; // The speed at which the notes fall -- can be changed to make it harder or easier
const int noteSkinniness = 2; // offset for the notes to make them look better and be in the center of the lane
volatile int numNotesHit = 0; // number of notes hit
volatile int numNotesMissed = 0; // number of notes missed
const bool pianoKeyTypes[13] = {1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1}; // 1 is a white key 0 is black -- used for drawing the piano keys on the screen
bool pianoKeysPressed[13] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
bool setup = false; // flag to check if the setup has been done
volatile note notes[13][50]; // 3 lanes of notes, 50 is the max number of notes in each lane at a single time (arbitary large number)
volatile int activeNotesInLane[13]; // number of notes in each lane
// draw the main menu
void draw_menu()
{
// fillRect(0, 0, SCREEN_WIDTH, SCREEN_HEIGHT, BLACK); // clear the screen
drawPicture(0, 0, (unsigned short *)vga_menu_image, 640, 480); // Draw the picture on the screen
setTextColor2(WHITE, BLACK);
setTextSize(2);
setCursor(100, 320);
writeString("Play Great Fairy Fountain");
setCursor(100, 360);
writeString("Play Great Fairy Fountain with Lives");
setCursor(100, 400);
writeString("Play Twinkle Twinkle Little Star");
setCursor(100, 440);
writeString("Credits");
}
void draw_piano(int lane, bool outline);
/**
* @brief Initializes the VGA display -- draws background.
*/
void draw_background()
{
// Draw vertical lines on the screen equally spaced away from the center
float offset = trackWidth / 2;
float singleTrackWidth = trackWidth / numLanes;
// drawVLine(SCREEN_WIDTH/2 + offset, 0, SCREEN_HEIGHT, WHITE);
// drawVLine(SCREEN_WIDTH / 2 - trackWidth / 2, 0, SCREEN_HEIGHT, WHITE);
// Draws the track lines -- lines in between the two outer lines dictated by numLines
for (int i = 0; i <= numLanes; i++)
{
drawVLine(SCREEN_WIDTH / 2 - trackWidth / 2 + (i * trackWidth / numLanes), 0, SCREEN_HEIGHT, WHITE);
}
// Draw the number of hit and unhit notes on the screen
setCursor(10, 10);
setTextColor2(WHITE, BLACK);
setTextSize(2);
writeString("Notes Hit: ");
setCursor(10, 25);
writeString("Notes Missed: ");
setCursor(10, 40);
writeString("Combo: ");
setCursor(10, 55);
writeString("Max Combo: ");
// Draw a piano diagram on the screen
for (int i = 0; i < numLanes; i++)
{
draw_piano(i, 0); // draw the piano keys on the screen
}
// Draw black keys on the screen
// for (int i = 0; i < 12; i++)
// {
// if (!pianoKeyTypes[i])
// {
// fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (i * trackWidth / numLanes), SCREEN_HEIGHT - hitHeight, trackWidth / numLanes, hitHeight / 2, BLACK); // draw the bottom of the note that moved down
// }
// }
}
void draw_piano(int lane, bool outline)
{
// Draw the piano keys on the screen
int blackHeight = whiteHeight / 2; // height of the black key
char keyColor;
char outlineColor;
if (activeNotesInLane[lane] > 0)
{
outlineColor = MAGENTA;
}
else
{
outlineColor = GREEN;
}
if (pianoKeyTypes[lane]) // white key
{
if (pianoKeysPressed[lane])
{
keyColor = RED;
}
else
{
keyColor = WHITE;
}
if (((lane != (numLanes - 1)) && (lane != 0)) && (!(pianoKeyTypes[lane + 1]) && !(pianoKeyTypes[lane - 1]))) // black key on the right and left
{
if (!outline)
{
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes) - 1, SCREEN_HEIGHT - whiteHeight, (trackWidth / numLanes) + 2, whiteHeight, keyColor); // draw the note and bleed it into the next lane a bit
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane + 1) * trackWidth / numLanes) - 2, SCREEN_HEIGHT - whiteHeight + blackHeight, (trackWidth / numLanes) * 1 / 2 + 2, blackHeight, keyColor); // draw the note and bleed it into the next lane a bit
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane - 1) * trackWidth / numLanes) + trackWidth / (2 * numLanes) - 2, SCREEN_HEIGHT - whiteHeight + blackHeight, (trackWidth / numLanes) * 1 / 2 + 2, blackHeight, keyColor); // draw the note and bleed it into the next lane a bit
}
drawRect(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane - 1) * trackWidth / numLanes) + trackWidth / (2 * numLanes), SCREEN_HEIGHT - whiteHeight, (trackWidth / numLanes) * 2, whiteHeight, outlineColor); // draw the outline
char rightCOlor = (pianoKeysPressed[lane + 1]) ? GREEN : BLACK;
drawVLine(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane - 1) * trackWidth / numLanes) + trackWidth / (2 * numLanes) + (trackWidth / numLanes) * 2 - 1, SCREEN_HEIGHT - whiteHeight, blackHeight, rightCOlor); // draw the outline
char leftCOlor = (pianoKeysPressed[lane - 1]) ? GREEN : BLACK;
drawVLine(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane - 1) * trackWidth / numLanes) + trackWidth / (2 * numLanes), SCREEN_HEIGHT - whiteHeight, blackHeight, leftCOlor);
}
else if (lane != (numLanes - 1) && !(pianoKeyTypes[lane + 1])) // black key on the right
{
if (!outline)
{
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes) - 1, SCREEN_HEIGHT - whiteHeight, (trackWidth / numLanes) + 2, whiteHeight, keyColor); // draw the note and bleed it into the next lane a bit
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane + 1) * trackWidth / numLanes) - 2, SCREEN_HEIGHT - whiteHeight + blackHeight, (trackWidth / numLanes) * 1 / 2 + 2, blackHeight, keyColor); // draw the note and bleed it into the next lane a bit
}
drawRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes), SCREEN_HEIGHT - whiteHeight, (trackWidth / numLanes) * 3 / 2 + 1, whiteHeight, outlineColor); // draw the outline
char rightCOlor = (pianoKeysPressed[lane + 1]) ? GREEN : BLACK;
drawVLine(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes) + (trackWidth / numLanes) * 3 / 2, SCREEN_HEIGHT - whiteHeight, blackHeight, rightCOlor);
}
else if (lane != 0 && !(pianoKeyTypes[lane - 1])) // black key on the left
{
if (!outline)
{
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes) - 2, SCREEN_HEIGHT - whiteHeight, (trackWidth / numLanes) + 2, whiteHeight, keyColor); // draw the note and bleed it into the next lane a bit
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane - 1) * trackWidth / numLanes) + trackWidth / (2 * numLanes) - 2, SCREEN_HEIGHT - whiteHeight + blackHeight, (trackWidth / numLanes) * 1 / 2 + 2, blackHeight, keyColor); // draw the note and bleed it into the next lane a bit
}
drawRect(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane - 1) * trackWidth / numLanes) + trackWidth / (2 * numLanes) - 1, SCREEN_HEIGHT - whiteHeight, (trackWidth / numLanes) * 3 / 2 + 3, whiteHeight, outlineColor); // draw the outline
char leftCOlor = (pianoKeysPressed[lane - 1]) ? GREEN : BLACK;
drawVLine(SCREEN_WIDTH / 2 - trackWidth / 2 + ((lane - 1) * trackWidth / numLanes) + trackWidth / (2 * numLanes) - 1, SCREEN_HEIGHT - whiteHeight, blackHeight, leftCOlor);
}
else // no black key on either side
{
if (!outline)
{
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes) - 1, SCREEN_HEIGHT - whiteHeight, (trackWidth / numLanes) + 2, whiteHeight, keyColor);
}
drawRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes), SCREEN_HEIGHT - whiteHeight, (trackWidth / numLanes) + 1, whiteHeight, outlineColor);
}
}
else // black key
{
if (pianoKeysPressed[lane])
{
keyColor = GREEN;
}
else
{
keyColor = BLACK;
}
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes), SCREEN_HEIGHT - whiteHeight, trackWidth / numLanes, blackHeight, keyColor); // draw the bottom of the note that moved down
drawRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes), SCREEN_HEIGHT - whiteHeight, trackWidth / numLanes, blackHeight, outlineColor); // draw the bottom of the note that moved down
}
}
void draw_hitLine()
{
// Draw the hit line -- the line that the notes must hit
int singleTrackWidth = trackWidth / numLanes;
drawHLine(SCREEN_WIDTH / 2 - trackWidth / 2, SCREEN_HEIGHT - hitHeight, trackWidth, WHITE);
drawHLine(SCREEN_WIDTH / 2 - trackWidth / 2, SCREEN_HEIGHT - hitHeight + hitWidth, trackWidth, WHITE);
}
/**
* @brief Spawns a note in the given lane
* @param lane The lane to spawn the note in
* @param color The color of the note
* @param height The height of the note
* @note Assumes that the lane is valid and that there is space in the lane
*/
void spawn_note(int lane, int color, int height, int sustain)
{
// Spawn a note in the given lane
if (activeNotesInLane[lane] < 50) // check if there is space in the lane
{
notes[lane][activeNotesInLane[lane]].lane = lane;
notes[lane][activeNotesInLane[lane]].y = -height; // spawn at the top of the screen
notes[lane][activeNotesInLane[lane]].height = height;
notes[lane][activeNotesInLane[lane]].color = color;
notes[lane][activeNotesInLane[lane]].hit = false; // not hit yet
notes[lane][activeNotesInLane[lane]].sustain = sustain; // not a sustained note (long press note)
activeNotesInLane[lane]++;
draw_piano(lane, 1); // draw the key on the screen
// printf("Spawned note in lane %d at y = %f, height = %d, color = %d\n", lane, notes[lane][activeNotesInLane[lane]].y, notes[lane][activeNotesInLane[lane]].height, notes[lane][activeNotesInLane[lane]].color); // print the note position for debugging
}
}
/**
* @brief Draws the falling notes
* @param erase 1 to erase the notes at their top edge, 2 to erase the note at the bottom edge, 3 to erase the whole note, 0 to draw them
*/
void draw_notes(int erase)
{
int singleTrackWidth = trackWidth / numLanes;
// Draws the falling notes -- lines in between the two outer lines dictated by numLines
for (int i = 0; i < numLanes; i++)
{
for (int j = 0; j < activeNotesInLane[i]; j++)
{
// Draw the note at its current position
if (erase) // erase the note if needed
{
if (erase == 1)
{ // erase only the top of the note that moved down
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (i * trackWidth / numLanes) + noteSkinniness, notes[i][j].y, trackWidth / numLanes - noteSkinniness, gravity, BLACK); // erase the top of the note that moved down
}
else
{ // erase only the whole note
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (i * trackWidth / numLanes) + noteSkinniness, notes[i][j].y, trackWidth / numLanes - noteSkinniness, notes[i][j].height, BLACK); // erase the whole note, subtract 10 to make it look better
}
}
else
{
// FOR PIANO THIS IS FINE BUT FOR DRUM WE WILL NEED TO DELETE THE WHOLE NOTE ONCE IT IS HIT
// if (!notes[i][j].hit) { // dont move the note down if its been hit
// fillRect(SCREEN_WIDTH/2 - trackWidth/2 + (i*singleTrackWidth), notes[i][j].y, singleTrackWidth-5, notes[i][j].height, notes[i][j].color); // draw the whole note
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (i * trackWidth / numLanes) + noteSkinniness, notes[i][j].y + max(notes[i][j].height - gravity, 0), trackWidth / numLanes - noteSkinniness, gravity, notes[i][j].color); // draw the bottom of the note that moved down
// }
// printf("Note %d in lane %d at y = %f, height = %d, hit_satus = %d\n", j, i, notes[i][j].y, notes[i][j].height, notes[i][j].hit); // print the note position for debugging
}
}
}
}
/**
* @brief Erase a single note from the screen as it falls
* @param lane The lane the note is in
* @param noteIndex The index of the note in the lane (in activeNotesInLane)
*/
void erase_note(int lane, int noteIndex)
{
int singleTrackWidth = trackWidth / numLanes;
// Erase the note at its current position
// fillRect(SCREEN_WIDTH/2 - trackWidth/2 + (lane*singleTrackWidth) + noteSkinniness/2, notes[lane][noteIndex].y + notes[lane][noteIndex].height - gravity, singleTrackWidth-noteSkinniness, gravity, BLACK); // erase the bottom of the note that moved down
if (!notes[lane][noteIndex].sustain)
{
fillRect(SCREEN_WIDTH / 2 - trackWidth / 2 + (lane * trackWidth / numLanes) + noteSkinniness, notes[lane][noteIndex].y, trackWidth / numLanes - noteSkinniness, notes[lane][noteIndex].height, BLACK);
} // erase the whole note if it is not a sustained note
activeNotesInLane[lane]--;
notes[lane][noteIndex] = notes[lane][activeNotesInLane[lane]]; // Move the last note to the current position
if (activeNotesInLane[lane] <= 0)
{
draw_piano(lane, 0); // draw the key on the screen
}
}
/**
* @brief returns if the note is in the hit area
*/
bool check_hit(note noteObj)
{
return ((noteObj.y + noteObj.height) > (SCREEN_HEIGHT - hitHeight)) && ((noteObj.y + noteObj.height) < (SCREEN_HEIGHT - hitHeight + hitWidth));
}
void draw_end_screen()
{
// Draw the end screen on the screen
fillRect(0, 0, SCREEN_WIDTH, SCREEN_HEIGHT, BLACK); // clear the screen
setTextColor2(WHITE, BLACK);
setTextSize(4);
setCursor(100, 100);
writeString("Game Over");
setTextSize(2);
char notesTextBuffer[4];
setCursor(50, 140);
writeString("You hit: ");
setCursor(150, 140);
sprintf(notesTextBuffer, "%d", numNotesHit);
writeString(notesTextBuffer);
setCursor(50, 180);
writeString("You missed: ");
setCursor(200, 180);
sprintf(notesTextBuffer, "%d", numNotesMissed);
writeString(notesTextBuffer);
setCursor(50, 220);
writeString("Your accuracy was: ");
setCursor(270, 220);
if (numNotesHit + numNotesMissed > 0)
{
sprintf(notesTextBuffer, "%.2f", (float)numNotesHit / (numNotesHit + numNotesMissed));
}
else
{
sprintf(notesTextBuffer, "N/A"); // Handle the case where no notes are hit or missed
}
writeString(notesTextBuffer);
setCursor(50, 260);
writeString("Your max combo was: ");
setCursor(300, 260);
sprintf(notesTextBuffer, "%d", maxCombo);
writeString(notesTextBuffer);
}
int songLength = 45;
static int twinkle_note = 0;
/**
* @brief Updates the falling notes
* Currently just moves them down the screen and deletes them once we hit the bottom
*/
void update_notes()
{
// Update the falling notes -- move them down the screen
for (int i = 0; i < numLanes; i++)
{
for (int j = activeNotesInLane[i] - 1; j >= 0; j--)
{
// Move the note down the screen
notes[i][j].y += gravity;
// if the bottom of the note is outside the hit area, make it smaller
if ((notes[i][j].y + notes[i][j].height) > (SCREEN_HEIGHT - hitHeight + hitWidth))
{
notes[i][j].height -= gravity; // make the note smaller
}
// If the note is off the screen, remove it from the lane
if (notes[i][j].y > SCREEN_HEIGHT - hitHeight + hitWidth)
{
// Remove the note from the lane
erase_note(i, j);
numNotesMissed++; // increment the number of notes missed
if (maxCombo < combo)
{
maxCombo = combo; // update the max combo counter
}
combo = 0; // reset the combo counter
if (lives != -1) // if we are playing a song with lives
{
lives--; // decrement the number of lives
// erase the last heart on the screen
drawCharBig(10 + ((lives) * 20), 70, 0x14, WHITE, BLACK); // erase the last heart
if (lives == 0)
{
play_mario_death(); // play the death sound
menu_state = 3; // go back to the main menu
draw_end_screen(); // draw the end screen on the screen
setup = false; // reset the setup flag
// clear notes
for (int i = 0; i < 13; i++)
{
activeNotesInLane[i] = 0; // clear the notes in the lane
}
// reinitialize the notes
for (int i = 0; i < 13; i++)
{
for (int j = 0; j < 50; j++)
{
notes[i][j].hit = false; // reset the hit status of the note
notes[i][j].sustain = false; // reset the sustain status of the note
notes[i][j].height = SCREEN_HEIGHT; // reset the height of the note
notes[i][j].y = 0; // reset the y position of the note
notes[i][j].lane = 0; // reset the lane of the note
notes[i][j].color = 0; // reset the color of the note
}
}
// stop dma channel
dma_channel_abort(data_chan); // abort the channel
dma_channel_abort(ctrl_chan); // abort the channel
return; // return to the main menu
}
}
// write miss on the screen
setCursor(SCREEN_WIDTH - 100, 10);
setTextColor2(WHITE, RED);
setTextSize(2);
writeString("MISS!!!!");
continue; // skip the rest of the loop
}
// If the note has been hit, make it smaller
if (notes[i][j].hit)
{
notes[i][j].height -= gravity; // make the note smaller
if (notes[i][j].height <= 0 || (!notes[i][j].sustain)) // if the note is gone, remove it from the lane
{
erase_note(i, j);
}
}
}
}
}
const unsigned int great_ff[35] = {11, 9, 8, 9, 9, 7, 6, 7, 7, 6, 5, 6, 6, 4, 3, 4, 11, 9, 8, 12, 11, 10, 11, 15, 12, 11, 12, 11, 9, 7, 6, 13, 13};
// const unsigned int twinkle_twinkle[96] = {0, 13, 0, 13, 7, 13, 7, 13, 9, 13, 9, 13, 7, 7, 7, 13, 5, 13, 5, 13, 4, 13, 4, 13, 2, 13, 2, 13, 0, 0, 0, 13, 7, 13, 7, 13, 5, 13, 5, 13, 4, 13, 4, 13, 2, 2, 2, 13,
// 7, 13, 7, 13, 5, 13, 5, 13, 4, 13, 4, 13, 2, 2, 2, 13, 0, 13, 0, 13, 7, 13, 7, 13, 9, 13, 9, 13, 7, 7, 7, 13, 5, 13, 5, 13, 4, 13, 4, 13, 2, 13, 2, 13, 0, 0, 0, 13};
const unsigned int twinkle_twinkle1[45] = {0, 0, 7, 7, 9, 9, 7, 5, 5, 4, 4, 2, 2, 0, 7, 7, 5, 5, 4, 4, 2,
7, 7, 5, 5, 4, 4, 2, 0, 0, 7, 7, 9, 9, 7, 5, 5, 4, 4, 2, 2, 0, 13, 13, 13};
const unsigned int twinkle_twinkle2[35] = {11, 9, 8, 9, 9, 7, 6, 7, 7, 6, 5, 6, 6, 4, 3, 4, 11, 9, 8, 9, 12, 11, 10, 11, 15, 12, 11, 12, 11, 9, 7, 6, 13};
bool songChoice = 0; // 0 = great fairy fountain, 1 = twinkle twinkle little star
// 6, 13, 20, 27, 34, 41
const unsigned int *twinkle_twinkle;
static PT_THREAD(protothread_twinkle_notes(struct pt *pt))
{
PT_BEGIN(pt);
const int maxHeight = SCREEN_HEIGHT / 4; // max height of the note
while (1)
{
while (menu_state != 1 || twinkle_note == -1)
{
PT_YIELD_usec(100000); // Yield for 100ms
}
if (twinkle_note < songLength) // reset the note index
{
// Spawn notes every 100ms
int lane = twinkle_twinkle[twinkle_note]; // Random lane
int color = rand() % 16; // Random color
// int height = rand() % (maxHeight); // Random height
int sustain = 0; // Random sustain (0 or 1)
if (songChoice == 1 && (twinkle_note == 0 || twinkle_note == 6 || twinkle_note == 20 || twinkle_note == 27 || twinkle_note == 34 || twinkle_note == 41)) // if the note is a long note
{
sustain = 1; // make it a long note
}
int height = hitWidth; // Fixed height for now
if (sustain)
{
color = YELLOW;
height = 2 * hitWidth;
}
if ((songChoice == 0) && (twinkle_note == 26))
{
play_HighD();
spawn_note(lane, color, height, sustain);
}
else if ((songChoice == 1) || (twinkle_note != 24))
{
spawn_note(lane, color, height, sustain);
}
}
// draw twinkle note number on the screen
// setCursor(10, 70);
// setTextColor2(WHITE, BLACK);
// setTextSize(2);
// char notesTextBuffer[4];
// sprintf(notesTextBuffer, "%d", twinkle_note);
// writeString(notesTextBuffer);
twinkle_note++;
if (twinkle_note >= songLength + 5)
{
menu_state = 3; // go back to the main menu
draw_end_screen(); // draw the end screen on the screen
setup = false; // reset the setup flag
}
PT_YIELD_usec(800000); // Yield for 100ms
}
PT_END(pt);
}
// ========================================
// ============ Spawning thread ==========
// ========================================
// ========================================
// ============ ANIMATION LOOP ============
// ========================================
static PT_THREAD(protothread_animation_loop(struct pt *pt))
{
PT_BEGIN(pt);
if (menu_state == 0)
{
if (!setup)
{
setup = true; // set the setup flag to true
draw_menu(); // Draw the menu on the screen
draw_cursor(0); // Draw the cursor on the screen
}
}
else if (menu_state == 2)
{
if (!setup)
{
setup = true;
draw_credits(); // Draw the credits on the screen
}
}
else if (menu_state == 3)
{
if (!setup)
{
setup = true;
draw_end_screen(); // Draw the credits on the screen
twinkle_note = 0; // reset the note index
numNotesHit = 0; // reset the number of notes hit
numNotesMissed = 0; // reset the number of notes missed
combo = 0; // reset the combo counter
maxCombo = 0; // reset the max combo counter
// reset the lives
lives = -1; // reset the lives
for (int i = 0; i < 13; i++)
{
activeNotesInLane[i] = 0; // clear the notes in the lane
}
// reinitialize the notes
for (int i = 0; i < 13; i++)
{
for (int j = 0; j < 50; j++)
{
notes[i][j].hit = false; // reset the hit status of the note
notes[i][j].sustain = false; // reset the sustain status of the note
notes[i][j].height = SCREEN_HEIGHT; // reset the height of the note
notes[i][j].y = 0; // reset the y position of the note
notes[i][j].lane = 0; // reset the lane of the note
notes[i][j].color = 0; // reset the color of the note
}
}
}
}
else if (menu_state == 1)
{
if (!setup)
{
setup = true;
drawPicture(0, 0, (unsigned short *)vga_image, 640, 480); // Draw the picture on the screen
draw_background();
if (lives != -1) // if we are playing a song with lives
{
for (int i = 0; i < lives; i++)
{
drawCharBig(10 + (i * 20), 70, 0x14, RED, RED); // draw the heart on the screen
}
}
}
draw_notes(1);
update_notes();
draw_notes(0);
draw_hitLine();
setTextColor2(WHITE, BLACK);
char notesTextBuffer[4];
setCursor(130, 10);
sprintf(notesTextBuffer, "%d", numNotesHit);
writeString(notesTextBuffer);
setCursor(170, 25);
sprintf(notesTextBuffer, "%d", numNotesMissed);
writeString(notesTextBuffer);
setCursor(80, 40);
sprintf(notesTextBuffer, "%d ", combo);
writeString(notesTextBuffer);
setCursor(130, 55);
sprintf(notesTextBuffer, "%d ", maxCombo);
writeString(notesTextBuffer);
PT_YIELD_usec(30000); // Yield for 30ms
}
PT_END(pt);
}
// ================================================================================================================
// ===================================== END ANIMATION CODE ======================================================
// ================================================================================================================
// ================================================================================================================
// ========================================== BEGIN INPUT CODE ====================================================
// ================================================================================================================
// Keypad pin configurations
#define BASE_KEYPAD_PIN 9
#define KEYROWS 4
#define NUMKEYS 12
unsigned int keycodes[12] = {0x28, 0x11, 0x21, 0x41, 0x12, 0x22, 0x42, 0x14, 0x24, 0x44, 0x18, 0x48};
unsigned int scancodes[4] = {0x01, 0x02, 0x04, 0x08};
unsigned int button = 0x70;
int prev_key = -1;
int key_pressed = 0;
void key_pressed_callback(int key); // forward declaration of the key callback function
void key_released_callback(); // forward declaration of the key released callback function
// ===========================================
// ============= KEYPAD CODE ================
// ===========================================
void key_pressed_callback_game(int key); // forward declaration of the key callback function
void key_released_callback_game(int key); // forward declaration of the key released callback function
void key_pressed_callback(int key); // forward declaration of the key callback function
void key_released_callback(int key); // forward declaration of the key released callback function
static PT_THREAD(protothread_keypad_scan(struct pt *pt))
{
// Initialize protothread and parameters
PT_BEGIN(pt);
static int i;
static uint32_t keypad;
// Main loop to handle keypad input
while (1)
{
// Scan the keypad for a keypress
for (i = 0; i < KEYROWS; i++)
{
gpio_put_masked((0xF << BASE_KEYPAD_PIN), (scancodes[i] << BASE_KEYPAD_PIN));
sleep_us(1);
keypad = ((gpio_get_all() >> BASE_KEYPAD_PIN) & 0x7F);
if (keypad & button)
break;
}
// If a key is pressed, find the key code
if (keypad & button)
{
for (i = 0; i < NUMKEYS; i++)
{
if (keypad == keycodes[i])
break;
}
if (i == NUMKEYS)
(i = -1);
}
// If no key is pressed, set i to -1
else
{
i = -1;
}
// If a new key is pressed, "i" will be different than "prev_key"
// if (i != prev_key) {
// blink the LED if a key is pressed
if (i != -1 && prev_key == i)
{
key_pressed_callback(i); // Call the key callback function
key_pressed = 1;
}
else if (prev_key != i && key_pressed)
{
key_released_callback(prev_key); // Call the key released callback function
key_pressed = 0;
}
prev_key = i;
// }
// Yield for 30ms
PT_YIELD_usec(30000);
}
// End the protothread
PT_END(pt);
}
// Mux Pins
#define MUX_SEL0 2
#define MUX_SEL1 3
#define MUX_SEL2 4
#define MUX_1 27
#define MUX_2 28
#define SIZE 13
#define MUX_1_CHECK 8
#define MUX_2_CHECK 5
static unsigned int prev_keys2[13] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
bool prev_prev = false;
static unsigned int prev_keys[13] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
static unsigned int curr_keys[13] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
static unsigned int new_keys[13] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
static PT_THREAD(protothread_piano_scan(struct pt *pt))
{
// Initialize protothread and parameters
PT_BEGIN(pt);
static int i;
static int val_1;
static int val_2;
static int ind;
gpio_init(MUX_SEL0);
gpio_set_dir(MUX_SEL0, GPIO_OUT);
gpio_put(MUX_SEL0, 0);
gpio_init(MUX_SEL1);
gpio_set_dir(MUX_SEL1, GPIO_OUT);
gpio_put(MUX_SEL1, 0);
gpio_init(MUX_SEL2);
gpio_set_dir(MUX_SEL2, GPIO_OUT);
gpio_put(MUX_SEL2, 0);
gpio_init(MUX_1);
gpio_set_dir(MUX_1, GPIO_IN);
gpio_pull_down(MUX_1);
gpio_init(MUX_2);
gpio_set_dir(MUX_2, GPIO_IN);
gpio_pull_down(MUX_2);
// Main loop to handle keypad input
while (1)
{
// scan through all of the options to see which one is pressed
// see if the pressed ones are not pressed in previous check
// if it is new, "key_pressed_callback(i);" on the new one
// if it is not pressed anymore, "key_released_callback(i);" on it
// Scan all of the keyboard keys if they are pressed
for (int i = 0; i < 8; i++)
{
gpio_put(MUX_SEL0, (i >> 0) & 1);
gpio_put(MUX_SEL1, (i >> 1) & 1);
gpio_put(MUX_SEL2, (i >> 2) & 1);
sleep_us(5);
int val_1 = gpio_get(MUX_1); // read the value of the first mux pin
int val_2 = gpio_get(MUX_2);
if (i == 6)
{
curr_keys[7] = val_1;
}
if (i == 7)
{
curr_keys[6] = val_1;
}
else
{
curr_keys[i] = val_1; // store the value in the current keys array
}
if (i < MUX_2_CHECK)
{
// int val_2 = gpio_get(MUX_2); // read the value of the second mux pin
int ind = i + 8;
curr_keys[ind] = val_2; // store the value in the current keys array
}
}
for (int i = 0; i < SIZE; i++)
{
if ((curr_keys[i] == prev_keys[i]) && (curr_keys[i] != prev_keys2[i])) // same for one scan but diff 2 scans ago
{
if (curr_keys[i] == 1)
{
key_pressed_callback(i + 1); // Call the key callback function
key_pressed = 1;
}
else
{
key_released_callback(i + 1); // Call the key released callback function
key_pressed = 0;
}
}
prev_keys2[i] = prev_keys[i]; // Copy the current keys to the previous keys for the next iteration
prev_keys[i] = curr_keys[i]; // Copy the current keys to the previous keys for the next iteration
}
PT_YIELD_usec(30000);
}
// End the protothread
PT_END(pt);
}
/**
* @brief callback for key release
*/
void key_released_callback(int key)
{
// printf("Key released: %d\n", prev_key); // Print the key released for debugging
if (menu_state == 1) // if we are in the game
{
key_released_callback_game(key); // Call the key released callback function
}
else if (menu_state == 0) // if we are in the menu
{
for (int i = 0; i < 13; i++)
{
activeNotesInLane[i] = 0; // clear the notes in the lane
}
// reinitialize the notes
for (int i = 0; i < 13; i++)
{
for (int j = 0; j < 50; j++)
{
notes[i][j].hit = false; // reset the hit status of the note
notes[i][j].sustain = false; // reset the sustain status of the note
notes[i][j].height = SCREEN_HEIGHT; // reset the height of the note
notes[i][j].y = 0; // reset the y position of the note
notes[i][j].lane = 0; // reset the lane of the note
notes[i][j].color = 0; // reset the color of the note
}
}
if (key == 1)
{
draw_cursor(1); // erase the cursor on the screen
menu_selection = (menu_selection + 1) % 4; // Move down the menu
draw_cursor(0); // draw the cursor on the screen
}
else if (key == 2)
{
draw_cursor(1); // erase the cursor on the screen
menu_selection = (menu_selection - 1 + 4) % 4; // Move up the menu
draw_cursor(0); // draw the cursor on the screen
}
else if (key == 3)
{
if (menu_selection == 0)
{
menu_state = 1; // Start the game
numLanes = 13;
twinkle_twinkle = twinkle_twinkle2; // Set the song to play
songChoice = 0; // Set the song choice to great fairy fountain
songLength = 35; // Set the song length
draw_background(); // Draw the background for the game
draw_hitLine(); // Draw the hit line for the game
setup = false; // reset the setup flag
}
else if (menu_selection == 1)
{
menu_state = 1; // Start the game with lives
lives = 3;
numLanes = 13;
twinkle_twinkle = twinkle_twinkle2; // Set the song to play
songChoice = 0; // Set the song choice to great fairy fountain
songLength = 35; // Set the song length
draw_background(); // Draw the background for the game
draw_hitLine(); // Draw the hit line for the game
setup = false; // reset the setup flag
}
else if (menu_selection == 2)
{
twinkle_twinkle = twinkle_twinkle1; // Set the song to play
songChoice = 1; // Set the song choice to twinkle twinkle little star
songLength = 45; // Set the song length
menu_state = 1; // Start the game with 12 lanes
numLanes = 13; // Set the number of lanes to 12
draw_background(); // Draw the background for the game
draw_hitLine(); // Draw the hit line for the game
setup = false; // reset the setup flag
}
else if (menu_selection == 3)
{
menu_state = 2; // Show credits
draw_credits(); // Draw the credits on the screen
setup = false; // reset the setup flag
}
}
}
else if (menu_state == 2 || menu_state == 3) // if we are in the credits
{
menu_state = 0; // Go back to the main menu
draw_menu(); // Draw the main menu
setup = false; // reset the setup flag
}
}
/**
* @brief callback for key press
*/
void key_pressed_callback(int key)
{
if (menu_state == 1) // if we are in the game
{
key_pressed_callback_game(key); // Call the key callback function
}
}
/**
* @brief callback for key press while playing the game
*/
void key_pressed_callback_game(int key)
{
key = key - 1; // convert to 0-indexed key
// Check if the key pressed is valid
if (key >= 0 && key < numLanes)
{
// Make key pressed true
pianoKeysPressed[key] = true;
// Draw the key pressed on the screen
draw_piano(key, 0); // draw the piano keys on the screen
if (key == 0)
{
play_c();
}
if (key == 1)
{
play_cSharp();
}
if (key == 2)
{
play_d();
}
if (key == 3)
{
play_dSharp();
}
if (key == 4)
{
play_e();
}
if (key == 5)
{
play_f();
}
if (key == 6)
{
play_fSharp();
}
if (key == 7)
{
play_g();
}
if (key == 8)
{
play_gSharp();
}
if (key == 9)
{
play_a();
}
if (key == 10)
{
play_aSharp();
}
if (key == 11)
{
play_b();
}
if (key == 12)
{
play_high_c();
}
// Check if there are any notes in the lane
if (activeNotesInLane[key] > 0)
{
// printf ("key pos: %f\n", notes[key][activeNotesInLane[key]-1].y); // Print the position of the key pressed
// printf ("Key is low enough %d\n", notes[key][activeNotesInLane[key]-1].y > (SCREEN_HEIGHT - hitHeight));
// printf ("Key is high enough %d\n", notes[key][activeNotesInLane[key]-1].y < (SCREEN_HEIGHT - hitHeight + hitWidth)); // Print the position of the key pressed
// Check if the note is in the correct position
for (int i = 0; i < activeNotesInLane[key]; i++)
{
note noteKey = notes[key][i];
// Check if the note is in the correct position
if (check_hit(noteKey))
{
notes[key][i].hit = true; // mark the note as hit
numNotesHit++; // increment the number of notes hit
combo++; // increment the combo counter
if (abs(notes[key][i].y + notes[key][i].height - (SCREEN_HEIGHT - hitHeight + hitWidth)) < 20) // if the note is hit perfectly
{
// write perfect on the screen
setCursor(SCREEN_WIDTH - 100, 10);
setTextColor2(WHITE, GREEN);
setTextSize(2);
writeString("PERFECT!");
}
else if (abs(notes[key][i].y + notes[key][i].height - (SCREEN_HEIGHT - hitHeight + hitWidth)) < 30) // if the note is hit well
{
// write GOOD on the screen
setCursor(SCREEN_WIDTH - 100, 10);
setTextColor2(WHITE, GREEN);
setTextSize(2);
writeString("GOOD!!!!!");
}
else // if the note is hit poorly
{
// write GOOD on the screen
setCursor(SCREEN_WIDTH - 100, 10);
setTextColor2(WHITE, RED);
setTextSize(2);
writeString("BAD!!!!!!!!");
}
}
}
}
}
}
/**
* @brief callback for key release while playing the game
*/
void key_released_callback_game(int key)
{
key = key - 1; // convert to 0-indexed key
// Check if the key pressed is valid
if (key >= 0 && key < numLanes)
{
// Make key pressed false
pianoKeysPressed[key] = false;
// Draw the key pressed on the screen
draw_piano(key, 0); // draw the piano keys on the screen
// printf("Key released: %d\n", key); // Print the key released for debugging
// Check if there are any notes in the lane
if (activeNotesInLane[key] > 0)
{
// printf ("key pos: %f\n", notes[key][activeNotesInLane[key]-1].y); // Print the position of the key pressed
// printf ("Key is low enough %d\n", notes[key][activeNotesInLane[key]-1].y > (SCREEN_HEIGHT - hitHeight));
// printf ("Key is high enough %d\n", notes[key][activeNotesInLane[key]-1].y < (SCREEN_HEIGHT - hitHeight + hitWidth)); // Print the position of the key pressed
// Check if the note is in the correct position
for (int i = 0; i < activeNotesInLane[key]; i++)
{
note noteKey = notes[key][i];
// Check if the note is in the correct position
// if (check_hit(noteKey))
// {
notes[key][i].hit = false; // mark the note as hit
// }
}
}
}
}
// ================================================================================================================
// ========================================== END INPUT CODE ======================================================
// ================================================================================================================
// ===========================================
// =============== LED BLINKY ================
// ===========================================
static PT_THREAD(protothread_blinky(struct pt *pt))
{
// used to tell if the program is running
// Initialize protothread and parameters
PT_BEGIN(pt);
static int i = 0;
// Main loop to blink the LED
while (1)
{
// Blink the LED every 50ms
gpio_put(LED, i % 2);
i++;
PT_YIELD_usec(50000);
}
// End the protothread
PT_END(pt);
}
///////////////////////////////////////////////////////////////
// MAIN FUNCTION
///////////////////////////////////////////////////////////////
int main()
{
// Initialize stdio/uart (printf won't work unless you do this!)
stdio_init_all();
printf("Hello, friends!\n");
// Initialize SPI channel (channel, baud rate set to 20MHz)
spi_init(SPI_PORT, 20000000);
// Format (channel, data bits per transfer, polarity, phase, order)
spi_set_format(SPI_PORT, 16, 0, 0, 0);
// Map SPI signals to GPIO ports
// gpio_set_function(PIN_MISO, GPIO_FUNC_SPI);
gpio_set_function(PIN_SCK, GPIO_FUNC_SPI);
gpio_set_function(PIN_MOSI, GPIO_FUNC_SPI);
gpio_set_function(PIN_CS, GPIO_FUNC_SPI);
// initialize VGA
initVGA();
// // Select DMA channels
data_chan = dma_claim_unused_channel(true);
;
ctrl_chan = dma_claim_unused_channel(true);
;
// Setup the control channel
dma_channel_config c = dma_channel_get_default_config(ctrl_chan); // default configs
channel_config_set_transfer_data_size(&c, DMA_SIZE_32); // 32-bit txfers
channel_config_set_read_increment(&c, false); // no read incrementing
channel_config_set_write_increment(&c, false); // no write incrementing
channel_config_set_chain_to(&c, data_chan); // chain to data channel
dma_channel_configure(
ctrl_chan, // Channel to be configured
&c, // The configuration we just created
&dma_hw->ch[data_chan].read_addr, // Write address (data channel read address)
&address_pointer2, // Read address (POINTER TO AN ADDRESS)
1, // Number of transfers
false // Don't start immediately
);
// Setup the data channel
c2 = dma_channel_get_default_config(data_chan); // Default configs
channel_config_set_transfer_data_size(&c2, DMA_SIZE_16); // 16-bit txfers
channel_config_set_read_increment(&c2, true); // yes read incrementing
channel_config_set_write_increment(&c2, false); // no write incrementing
// (X/Y)*sys_clk, where X is the first 16 bytes and Y is the second
// sys_clk is 125 MHz unless changed in code. Configured to ~22 kHz
dma_timer_set_fraction(0, 0x000B, 0xffff);
// 0x3b means timer0 (see SDK manual)
channel_config_set_dreq(&c2, 0x3b); // DREQ paced by timer 0
// chain to the controller DMA channel
// VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV
// channel_config_set_chain_to(&c2, ctrl_chan); // Chain to control channel COMMENT OUT TO PREVENT LOOPING
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dma_channel_configure(
data_chan, // Channel to be configured
&c2, // The configuration we just created
&spi_get_hw(SPI_PORT)->dr, // write address (SPI data register)
DAC_data, // The initial read address
transfer_count, // Number of transfers
false // Don't start immediately.
);
// Map LED to GPIO port, make it low
gpio_init(LED);
gpio_set_dir(LED, GPIO_OUT);
gpio_put(LED, 0);
// Initialize the keypad GPIO's
gpio_init_mask((0x7F << BASE_KEYPAD_PIN));
// Set row-pins to output
gpio_set_dir_out_masked((0xF << BASE_KEYPAD_PIN));
// Set all output pins to low
gpio_put_masked((0xF << BASE_KEYPAD_PIN), (0x0 << BASE_KEYPAD_PIN));
// Turn on pulldown resistors for column pins (on by default)
gpio_pull_down((BASE_KEYPAD_PIN + 4));
gpio_pull_down((BASE_KEYPAD_PIN + 5));
gpio_pull_down((BASE_KEYPAD_PIN + 6));
// Add core 0 threads
pt_add_thread(protothread_animation_loop);
pt_add_thread(protothread_blinky);
pt_add_thread(protothread_keypad_scan);
pt_add_thread(protothread_piano_scan);
pt_add_thread(protothread_twinkle_notes);
// Start scheduling core 0 threads
pt_schedule_start;
}