ECE 5730 Final Project: DJ Synth Pad

Project Introduction

DJ Synth Pad is an audio playground on a breadboard, letting users create, mix, and visualize music in real time.

This project implements a DJ-style synthesizer using the RP2040 microcontroller, enabling users to layer piano, guitar, and drum sounds over selectable backing tracks in real time. The system features a 3x4 keypad for triggering notes and samples, a potentiometer for pitch bending, and a live VGA interface for visual feedback. Audio is generated via direct digital synthesis for piano, Karplus-Strong string synthesis for guitar, and DMA playback of .wav samples for drums and backing tracks. The goal was to create an expressive, low-latency musical interface that leverages the full range of embedded tools explored in ECE 5730, from audio synthesis and real-time multithreading to DMA, SPI, and dual-core execution.

Watch our final project demo below!

High Level Design

Rationale & Motivation

The inspiration behind DJ Synth Pad stemmed from our shared passion for music and a desire to build an expressive, interactive musical interface from scratch. We both greatly enjoy playing instruments and listening to music, and we wanted to bring that creativity into the world of embedded systems. Inspired by commercial tools like MIDI controllers, drum machines, and DJ mixers, our goal was to create a performance platform that feels responsive and engaging while operating entirely on low-cost microcontroller hardware.

We also saw this project as a natural extension of the core themes in ECE 5730: real-time synthesis, multithreading, peripheral control, and user interface design. We aimed to build a system that not only sounds great, but feels great to use. The project challenged us to merge artistic goals with embedded engineering to create a musical experience that's both technically and creatively satisfying.

Background Math

Logical Structure

DJ Synth Pad operates as a tightly integrated embedded music system driven by the RP2040’s dual-core architecture, implemented primarily in dj.c. The system is split into two major domains:

• Core 0: Handles real-time audio synthesis and manages hardware input(s)
• Core 1: Manages the VGA display and UI state

Core 0 runs three main subroutines: one for keypad input handling and system state changes, one for potentiometer-based pitch bending, and one for guitar synthesis. The keypad is decoded to trigger piano notes, drum samples, scale and instrument switching, and backing track control. Piano notes are generated via DDS inside a high-frequency timer ISR, while guitar tones invoke a Karplus-Strong algorithm that uses a circular buffer and custom filtering in karplus_strong.c. Drum and backing tracks are streamed via dual-channel DMA using SPI, with separate channels (A and B) for each of the two speakers.

Core 1 is dedicated to VGA rendering, utilizing five protothreads for visual updates: title and instructions, piano key visualization, drum pad visualization, playback state (seek bar, labels), and a queue polling thread that receives updates from Core 0. Communication between cores is handled using the multicore_fifo system, where structured messages encode note changes, instrument mode, and drum presses. These messages are parsed on Core 1 and rendered as stateful UI feedback.

Audio output is handled by an MCP4822 SPI DAC, with Channel A used for backing tracks and Channel B for piano, guitar, or drum output. The DAC is clocked at 20 MHz and synchronized with DMA or ISR writes depending on the audio path. The potentiometer, connected to ADC0, continuously updates a fixed-point pitch multiplier that dynamically affects DDS frequency. The entire system is coordinated at high performance through interrupt timers, multithreaded state machines, DMA chaining, and SPI streaming, allowing for seamless musical interaction across all channels.

Hardware/Software Tradeoffs

The system design favors software implementations for flexibility and expandability. All audio synthesis, input handling, and UI control are implemented in software using protothreads and ISR routines. We leveraged hardware peripherals—namely the SPI controller, ADC, and DMA engine—for precise, efficient data transfer. While some audio routing could have been offloaded to dedicated audio ICs or hardware mixers, implementing everything in software on the RP2040 allowed us to finely control timing, layering, and resource sharing, especially for synchronized drum and backing track playback.

Relevant Patents, Copyrights, and Trademarks

All .wav samples used in this project, both for drum hits and backing tracks, were sourced from copyright-free libraries and are cleared for non-commercial use. We based our synthesis techniques on publicly available course materials and research papers, with citations provided in Appendix D. No proprietary trademarks or patented protocols were reverse-engineered or violated in the course of this project.

Hardware Design

Overview

All hardware used in DJ Synth Pad was sourced from prior ECE 5730 labs, so no additional components were purchased for this project. The system integrates a Raspberry Pi Pico (RP2040) microcontroller with the following peripherals: a 3x4 matrix keypad for user input, a potentiometer connected to the ADC for pitch bending, an MCP4822 SPI DAC for stereo audio output, and a VGA display driven via PIO and DMA. Connections were made on a standard breadboard with passive components like resistors for VGA signal conditioning and overcurrent protection on the keypad GPIOs. Pin assignments and electrical connections are documented in the schematic and wiring note below. Relevant datasheets and hardware resources are included in Appendix D.

Hardware Schematic

Physical Circuit

Detailed Wiring Description

Peripheral Pinouts

Keypad GPIO Connections

MCP4822 DAC Pinout

VGA Connector Pinout

Software Design

Overview

DJ Synth Pad’s software architecture is built around the protothreads cooperative multitasking model, distributed across the RP2040’s dual cores. Core 0 handles audio synthesis, DMA playback, keypad input, and potentiometer-based pitch bending, while Core 1 manages the VGA display and state-driven UI. All major components are implemented in C using fixed-point arithmetic for fast real-time performance. Audio routines use interrupt service routines (ISRs) or DMA, and UI state is synchronized between cores via multicore FIFO messaging.

The following software components draw heavily from ECE 5730 course examples, particularly for audio synthesis, protothreads, VGA rendering, and SPI/DMA configuration. All referenced course resources are listed in Appendix D.

FSMs and User Input Handling

A key aspect of our system is robust user interaction through a 3x4 matrix keypad, read via GPIO 9–15. The keypad is polled in a dedicated protothread with a four-state debounce finite state machine (FSM) adapted from Lab 1. Button events control all user actions: keys 1–8 trigger notes, 9 cycles the key signature, * changes the backing track, # switches instruments, and 0 toggles playback. When in drum mode, keys 1–6 trigger individual drum samples. Debounced inputs update shared state and, when needed, generate VGA updates sent over the multicore FIFO. The FSM is implemented in thread_keypad_input in dj.c, and transitions between RELEASED → MAYBE_PRESSED → PRESSED → MAYBE_RELEASED.

Below is a diagram of the debounce FSM used for keypad input.

In addition to the debounce FSM, we implemented simple FSMs to manage global instrument, key, and playback state. These are represented by the InstrumentState, KeyState, and PlaybackState enums respectively. Transitions between states occur in response to keypad inputs, and the updated state is used by audio threads and passed to the UI via FIFO.

Multithreading

The project uses eight protothreads, three on Core 0 and five on Core 1, scheduled cooperatively using the provided pt_cornell_rp2040_v1_3.h utilities:

• Core 0:
  • thread_keypad_input – scans and debounces keypad input; updates state, triggers sound, sends messages to Core 1
  • thread_pitch_bend – continuously reads ADC to scale piano frequencies via fixed-point multiplier
  • thread_guitar – detects active guitar notes and triggers Karplus-Strong synthesis
• Core 1:
  • thread_poll_queue – listens for state changes from Core 0 over multicore FIFO
  • protothread_vga_title – displays title screen and instructions
  • protothread_vga_state – renders playback state, key signature, instrument, and seek bar
  • protothread_vga_drums – shows drum pad visualization and highlight when pressed
  • protothread_keys – displays piano key state and visual feedback when pressed

Communication between cores is implemented using the RP2040’s multicore_fifo hardware queue. Messages are structured using a custom StateMessage format defined in shared.h, allowing compact, typed events to be sent from Core 0 to Core 1. Events like note presses, instrument changes, and drum triggers are pushed into the FIFO on Core 0 and parsed on Core 1 to drive UI updates.

Drums and Backing Tracks: DMA Playback

Both drums and backing tracks are streamed as 16-bit PCM .wav samples using dual-channel DMA (channels A and B) to the MCP4822 DAC over SPI. Each stereo output is mapped to a separate DAC channel, allowing mixed-layer playback. Samples were trimmed and resampled in Audacity and then converted into const unsigned short arrays using our custom convertWav.py script. This script reads a mono .wav file and generates C arrays that can be directly included and streamed from flash memory, by prepending the necessary DAC configuration bits (0b0011 for channel A, 0b1011 for channel B) to each 12-bit sample. The DMA timers are configured to output at 22.05 kHz, and the fraction 250 MHz × 2 / 22673 was used to match this rate, ensuring audio quality.

Each drum sound and backing track has its own .c file, which is linked into dj.c as an array and length variable. These are stored in structured arrays (AudioTrack tracks[] and DrumSample drums[]) and indexed during playback. To perform the actual transfer, we used DMA chaining: each DMA output uses a control channel and data channel. When the control channel completes a trigger, it automatically starts the corresponding data channel to send the next audio word. For example, CTRL_CHAN_A chains into DATA_CHAN_A for backing track playback, and CTRL_CHAN_B to DATA_CHAN_B for drums. This enables seamless looped playback with minimal CPU intervention. Playback is triggered in playback() and playDrum() functions and can be stopped or restarted cleanly via stopPlayback().

We initially explored applying real-time tempo or EQ adjustment to backing tracks and generating drum synthesis on-chip, but these proved too computationally demanding within the constraints of DMA-only playback and real-time SPI output. Using sample playback allowed us to preserve audio quality, minimize CPU load, and layer sounds reliably. These design decisions prioritized responsiveness and stability over complexity.

Piano: Direct Digital Synthesis

The piano tones are generated using Direct Digital Synthesis (DDS) inside a timer ISR running at 50kHz, implemented in the alarm_irq() function in dj.c. A 32-bit phase accumulator phase_accum_main_0 is incremented by a frequency-dependent value calculated using major_freqs[][] and the real-time pitch bend multiplier pitch_multiplier_fix15. The upper 8 bits of the accumulator index into a 256-entry sine lookup table sin_table, which stores fixed-point values scaled for the 12-bit MCP4822 DAC. The DDS equation effectively becomes: phase_accum += (frequency × pitch_multiplier × 2^32) / Fs, where all values are fixed-point.

The sine value retrieved from the table is multiplied by an amplitude envelope that modulates the output based on attack and decay slopes, defined by attack_inc and decay_inc. This amplitude ramping is handled within the ISR itself to ensure clean fades when keys are pressed or released. The final 12-bit sample is offset, prepended with the DAC control bits for Channel B, and transmitted via spi_write16_blocking(). This low-latency pipeline allows seamless synthesis of piano notes. Pitch bending is controlled in a separate protothread, thread_pitch_bend, which reads the potentiometer using ADC0 and adjusts the frequency increment multiplier dynamically to allow smooth half-step bends. The ADC is polled every 30ms, and if the change in reading exceeds a threshold (BEND_THRESHOLD), the value is converted to a fixed-point multiplier between 0.94 (frequency of a half step down) and 1.06 (frequency of a half step up) to avoid jitter and ensure smooth response.

Guitar: Karplus-Strong String Synthesis

Guitar tones use a software implementation of Karplus-Strong synthesis detailed in karplus_strong.c. The play_guitar_note function initializes a circular buffer with one of several pluck styles—random noise, low-passed noise, Gaussian, or sawtooth—stored in the init_string array. Each pluck style simulates a different excitation of the string. The core loop repeatedly calls compute_sample(), which applies a two-stage filter: a low-pass averaging filter for energy decay and a fractional-delay all-pass filter controlled by eta[] to fine-tune pitch. Filtered output is fed back into the string buffer, and read/write pointers ptrin and ptrout update with wrapping at the buffer length. Output samples are streamed to DAC Channel B using spi_write16_blocking(), producing expressive, physically-plausible guitar tones with minimal CPU load. The loop sustains for a fixed number of samples (~20k), and buffers are zeroed at the end to ensure silence.

VGA User Interface

The VGA interface was designed for clarity, feedback, and aesthetics. It displays the current instrument, key signature, and backing track, along with live visuals of key presses and drum hits. Core 1 runs five dedicated protothreads that call helper functions from vga16_graphics.c to render text, shapes, and color changes to the screen. The piano UI includes a 14-key layout with black and white keys that highlight pressed notes in orange. Notes vary depending on the selected key signature, and the correct labels are rendered dynamically. The drum pad UI features a 2x3 grid with labeled pads that momentarily turn white when hit, providing a clear flash for visual emphasis before reverting. A seek bar updates every frame, advancing a small circle whose speed is scaled to the total frame length of the current track, stored in track_total_frames[]. This results in smooth scrolling tied to playback duration. While we initially planned a real-time waveform visualizer, we ultimately focused on interactive elements to maximize usability within VGA’s resource and timing constraints.

Below is a picture of the VGA UI on startup. The default drum backing track is stopped, and the default instrument is the piano in the key of C.

Below is a picture of the VGA UI while the user is playing guitar in the key of A over the jazzy backing track.

Results

Overview

DJ Synth Pad successfully demonstrated real-time audio synthesis, layered playback, and interactive control using low-cost embedded hardware. All system components, including dual-core protothread scheduling, DMA playback, and visual feedback, functioned as intended without hangs or crashes. The final product supported smooth mode switching, pitch bending, and musical improvisation using only a keypad and potentiometer, with engaging visual feedback on VGA.

Below is a video of the C, A, and G piano scales being played via direct digital synthesis:

Below is a video of the C, A, and G guitar scales being played via Karplus-Strong synthesis:

Check out the bonus demo clips in Appendix E for a couple of cool musical performances!

Speed

The system exhibited fast response and seamless concurrency. DMA playback from flash to DAC channels had no perceptible latency, and multiple drum samples could be triggered back-to-back or layered over backing tracks with no hesitation. VGA UI refreshes were distributed across five protothreads and updated without flicker at a stable ~30 FPS. To support this, we overclocked the RP2040's CPU to 250 MHz in main(), which drastically improved VGA rendering smoothness and reduced visual tearing. Audio ISR routines ran at 50 kHz for DDS and 22.05 kHz for DMA, and all protothread execution completed comfortably within timing margins. Notably, when a backing track is playing and multiple drum sounds are overlaid, minor DMA contention can cause a slight flicker or vibrato in the VGA output—an effect we came to view as a visual enhancement or "feature" rather than a flaw.

Accuracy

Piano notes generated by DDS were audibly and numerically accurate to within ±0.3 Hz of target frequencies. Guitar notes generated using Karplus-Strong were tuned via buffer length and filtered for pitch stability. DAC output remained within 12-bit precision, and pitch bending scaled correctly with potentiometer input. The VGA interface was stable with no artifacts, and the scroll bar accurately tracked playback duration using a timing divisor matched to track length in frames.

Safety

All GPIO lines used for the keypad were protected with current-limiting resistors to prevent overdraw or pin damage. VGA outputs were conditioned with resistive dividers where necessary. No high-voltage or high-current peripherals were used, and the entire system was powered from a USB supply with built-in current limiting. There were no observed electrical issues during operation.

Usability

The system was highly usable in both structured tests and free-form musical play. Keypad inputs were responsive and clearly mapped to user actions, with visual confirmation via drum pad highlighting and key color changes. Switching instruments or backing tracks was intuitive, and the pitch bend control allowed expressive variation. In user testing, the system was playable with no instruction after a brief demo and was enjoyed by both team members and observers alike.

Conclusions

DJ Synth Pad met and exceeded our expectations as a low-cost, interactive music system built entirely on the RP2040 platform. The final design supported concurrent audio synthesis, sample playback, and responsive VGA visuals—all managed across dual cores with minimal latency. Our project goals were ambitious, and although we didn’t fully implement our stretch goal of a physical kick drum pedal with voltage-triggered DMA playback, the core functionality was robust and musically expressive. If we had more time, we would explore enhanced piano synthesis techniques to generate more natural, layered sound, closer to a real instrument than the current single-layer DDS tone.

Our design adhered to all safety, interface, and timing constraints expected for embedded musical systems. We adapted publicly available resources provided on the ECE 5730 course webpage, including protothread infrastructure, VGA and SPI drivers, and Karplus-Strong and DDS examples to our specific application. All audio files were sourced from copyright-free libraries, and no IP-protected code, datasheets, or proprietary protocols were reverse-engineered or NDA-restricted. While our project doesn'tdirectly present a patent opportunity, it serves as a platform for future enhancements, especially for hardware-triggered percussion or advanced audio layering. Overall, we’re proud of the technical depth, musical flexibility, and creative satisfaction this project delivered.

Appendices

Appendix A: Permissions

The group approves this report for inclusion on the course website.

The group approves the video for inclusion on the course YouTube channel.

Appendix B: Code

Our project code can be found at this GitHub Repository.

Appendix C: Work Distribution

Much of the project code was written, debugged, and tested collaboratively, via pair programming. However, during asynchronous development, each member had different areas of focus.

Sivaraman was primarily responsible for:

• Integrating the DDS algorithm for piano synthesis
• Integrating the Karplus-Strong algorithm for guitar synthesis
• Designing and implementing the original VGA user interface
• Handling multithreading and FIFO messaging

Nicholas was primarily responsible for:

• Hardware setup and implementation
• Integrating DMA playback for the drums and backing tracks
• Processing user keypad and potentiometer inputs
• Expanding and refining the final VGA user interface

Nicholas drafted the report and built the website, and Sivaraman created the schematics/diagrams for the report and refined its content.