Christmas Caroling Muscle Machine¶
Kidus Zegeye (kmz25), Sofia Echavarria (see62)
Project Introduction¶
We used surface electromyography, EMG, to measure muscle response through microvolt changes seen when flexing a muscle that fed into a Markov model trained on Christmas music that created instrument melodies influenced by the strength of muscle tension. We wanted to implement this project because of our interest in human-interaction with technology and sensors to create art, in this case music.
High-Level Design¶
Background¶
For our signal conditioning system from the electrode input, we used two sources primarily to look at example systems that worked previously in conditioning electrode input: Bruce Land's Physiological AC Preamplifier and a DIY muscle sensor/EMG circuit for an Arduino designed by Advancer Technologies. In Bruce's physiological amplifier, he utilized a passive lowpass filter with the incoming electrode input then passed it through an INA121 differential amplifier (to determine the voltage difference between the two working electrodes) then passed it through a highpass filter, and finally an LM358 opamp to amplify the signal. Then Bruce adjusted the highpass and lowpass filter based on the type of physiological sensor being used. In addition, Bruce split a 9V battery so there was a +/- 4.5V line in addition to the ground line. The DIY muscle sensor circuit utilized an INA106 to determine the difference between the two working electrodes, then passed the signal through three TL072 dual-op amps to amplify, smooth, and rectify the signal. This circuit was powered by two 9V batteries with a +/- 9V rail.
Overall Design¶
In the program, we use two Markov models to determine what note is played and the length of the note. These Markov models allow us to stochastically determine the next value based on the previously selected value. Each value, be it a note or a note length, has a set of transition probabilities for each subsequent value from it. In this project, we went through a list of Christmas songs and counted up the frequencies of note transitions and note length transitions, which we used to create our probability distributions for each note/note length. With this, we were able to bias our model to choose notes and note lengths that sound somewhat Christmasy, to fit the season.
Then, we integrated the voltage measurements from the arm's muscle responses to bias the probabilities based on the magnitude of the muscle voltage. Since the notes we chose are in order of increasing pitch, and the note lengths are in order of decreasing length, larger values of muscle voltage will create higher-pitched, fast-paced songs. In addition to this, we also increase the tempo of the music with the measured voltage, causing the music to be increasingly frenetic as your arm is flexing. To add to the Christmas cheer, we used FM synthesis to modulate the sounds played for each note, which mimics different musical instruments. These instruments include plucked strings, drums, snares, chimes, and more. To allow the user to play with the different instruments when making their song, we included the ability to change the instrument via a keypad.
Hardware Design¶
Bio Instrumentation Circuit¶
For our hardware design, we were influenced by many sources to come up with our design and testing strategy.
First, we started with a three-electrode setup to measure the muscle voltage changes effectively. This is a traditional surface electrode setup to use three different electrodes as ground, and two as the testing voltage difference inputs. We determined that this would be the most effective system after talking with Bruce Land about our overall goals and development ideas.
Then, based on multiple experiments conducted by other researchers (see references) we came up with a plan for making our signal conditioning system. The surface electrodes pick up EMG signals in the millivolt range and we wanted to determine the difference between the two working electrodes using the ground electrode as a reference. Then we needed to amplify and condition the signal so that the RP2040 could understand it later.
Using the two examples described in our background as starting points for our hardware, we worked with Bruce Land to come up with a good testing method using the oscilloscope and function generator before hooking it up to the electrodes and isolating the system. To simulate our EMG biological input of a few mV pulsing, we set the function generator to output a 1V sine wave and sent it through a voltage divider to have a 0.01 mV sine wave coming into our first opamp, the INA121. We decided to use the INA121 because it is a standard differential op amp that also has an amplification component so we figured it would be effective to determine the difference between the working electrodes. We also decided to power our system with the lab DC power source with a +/- 9V rail and a common ground. Then, to configure the INA121, we had to connect the positive and negative inputs from our voltage divider into the opamp, connect our +/- 9V rail to the V+ and V- components, add a reference ground which was connected to the power supply ground and function generator ground and include a gain resistor. We determined that a gain of 10 would be sufficient since we didn't want to amplify too much noise since we were not filtering the input signal before putting it in the opamp. To implement the gain of 10, we utilized a 5.1 kOhm resistor. Finally, to test that our opamp was working, we connected the oscilloscope to the output and measured it to ensure that it was conditioning the signal as expected. For our .01 mV sine wave input, we expected the output of the op-amp to look like a sine wave with an amplitude of around 0.1 mV because of the gain of 10. Below is a diagram of our first tester circuit as described above.
Figure 1: Test Circuit 1
When testing this first circuit, there were a couple of issues we ran into. First, we were getting strange outputs because the ground we connected to the op-amp was not connected throughout all the devices. In addition, when we were looking at the sine wave on the oscilloscope, there was a good amount of noise making the signal look fuzzy. To try to reduce the noise we were seeing, we wanted to implement a bandpass filter in our signal conditioning stage.
To start, we implemented a highpass filter after the INA 121 to try to remove some of the noise getting picked up from the signal acquisition. We utilized a capacitor-resistor combination to make a high-pass filter before passing it to our next op-amp, dual op-amp LM358. We used a 1 uF capacitor and 100 kOhm resistor to filter the lower frequency noise. Then we inputted the filtered output into the next op amp, an LM258 dual op-amp with 100 gain. We wanted to do 100 gain to have a system with 1000 gain overall to get it in the effective voltage range instead of 100 gain overall. We tested the system using the same method but with the oscilloscope at the output of the second op-amp.The circuit we tested is included in the figure below.
Figure 2: Test Circuit 2
The outputted waveform was in the 1V range since our system had a gain of 1000 at this point. The waveform outputted by the oscilloscope can be seen below.
Figure 3: Oscilloscope Output of Signal Conditioning Circuit
The next step in our hardware design was to implement the electrode system and give the system a floating ground to make it safe for testing on humans. We went through some trial and error to determine how to best read the signal being outputted by the signal conditioning system. We couldn't use the VGA, lab oscilloscope, lab monitors, or anything connected directly to ground.
First, we connected the system to Sofia's laptop using an audio jack connected to an audio adapter. Then we put the input into an oscilloscope app. We tested flexing our muscles with that system but the oscilloscope output was very sensitive to audio noise like clapping our hands near the speaker and it was unclear if the input from the electrodes was really working.
So we pivoted to using a mini oscilloscope LCD device given to us by Bruce Land. Once we wired the system to that device, it seemed to be working better but still had a lot of noise. Because it seemed to have a lot of distortion on a pretty large scale, we dropped the circuit gain on the second op-amp to just 10 instead of 100. This made our overall system have a gain of 100 instead of 1000. While we were testing it, we also saw a lot of extra noise added when we tried to plug it into the RP2040 GPIO 26 which is an ADC (analog to digital) input. After a lot of searching, we found that the source of the noise was that the system was not properly grounded throughout. This happened because we were using one of the breadboards that had a split connection in the middle rails so our ground rail was not connected all the way through. Another issue we saw was that there wasn't enough of a signal coming through unless we had really big muscle contractions. When we removed the low-pass filter, it fixed this issue and the signal got through. After we fixed this grounding issue and high-pass filter issue our electrodes were outputting expected signals but we did see that we needed to strategically place the electrode pads on large muscles to pick up the best signal spikes.
To connect the electrode output to the RP2040 for use in the Markov model, we connected it with GPIO 26 which we configured as an ADC input. This helps take in the analog signals being outputted by the electrodes and our signal conditioning circuit and converting it to digital signals the RP2040 can process. We used this information in the Markov model as described in the program design.
Audio Circuit¶
To make our audio circuit, first, we looked at how the MCP4822 Digital-to-Analog Converter works and how to integrate the pin-out with our Pico microcontroller. The DAC is a 12-bit analog-to-digital converter that is compatible with any 2.7V to 5.5V supply with Serial Peripheral Interface (SPI). Since the Pico is 3.3V powered, we connected the Vdd pin on the DAC to the 3.3V output pin on the Pico. The CS pin is the chip select input which we connected to the GPIO5 pin which we configured as an active-low pin to enable serial clock and data functions which is part of the PSI channel in the GPIO definition section of our code and in the main() section of the code (see Figure 2 below). The SCK pin on the DAC controls the Serial Clock Input which we connected to the GPIO6 pin which we configured as part of the SPI channel below. The SDI pin on the DAC controls Serial Data Input which we connected to the GPIO7 pin which we configured as part of the SPI channel below. We have a configured MISO GPIO pin for the SPI channel on the Pico to work but we do not connect it to our DAC because it does not have the functionality to communicate from the DAC to the Pico. We only need the communication from the Pico to the DAC so we only utilize the MOSI GPIO pin.
The Vss pin is the ground pin for the DAC so we connect that to one of the GND pins on the Pico. The LDAC pin on the DAC is the synchronization input that transfers the DAC settings to the VOUTB which we connected to GPIO8 on the Pico. The LDAC is set as a GPIO output pin on the Pico that is held low to update both VOUTA and VOUTB so either can be utilized.
Finally, we had to connect the VOUTB to the audio jack that will connect to the speaker. The configuration of these analog output pins is configured in our code and sent to the DAC through the LDAC pin. For our system, we utilized the VOUTB pin to output the correct voltage to configure for the correct frequency. The Markov model code would tell the DAC what note and at what frequency to output it using the FM synthesizer code. Then the output of the DAC was connected to an audio jack connected to a speaker to effectively produce sounds when the RP2040 asks. The circuit can be seen below.
Figure 4: Audio Circuit
Keypad Circuit¶
For the keypad circuit, we used a 12-button keypad that we configured to the RP2040 to change instruments when pressing the different buttons within 1-8.
For the 3x4 matrix keypad, it had 7 pinouts with 4 being connected to each row and 3 being connected to each column. When we pressed a key, this would short one of the row pins and one of the column pins to demonstrate to the Pico which key was pressed. To make the shorting process easy for the Pico to read, we initialized the row pins to be GPIO outputs and the column pins to be GPIO inputs. To reduce any floating issues where the input or output pins are stuck in between states (on or off), we have to implement pull-up resistors (330 ohms as seen in hardware diagram) for the GPIO outputs and pull-down resistors, which are internal within the Pico, for the GPIO inputs. This helps all the GPIO pins be either 0 or 1 and not get stuck in the in-between.
We define the first GPIO keypad pin (GPIO9), the number of rows, and the number of keys for the scanning function. In the main() of our program, we initialize GPIOs 9, 10, 11, and 12 as outputs and GPIOs 13, 14, and 15 as inputs. We also turn on the internal pulldown resistors for the input pins as described above.
Figure 5: Keypad Circuit
Program Design¶
Markov Model Program¶
We implemented two Markov models, one for choosing notes and one for choosing note lengths. The notes that the program can choose from range from middle C to A5. This range includes sharp notes as well, including A5 sharp. We chose this range because we wanted to stay in the treble clef and this range encompassed all of the notes we encountered in our music sheet training data. We store these notes as an array of 23 frequencies in Hz, sorted from lowest to highest. The note lengths that are available range from sixteenth notes to full notes, including dotted quarter notes and dotted half notes. We store these note lengths as an array of integers: 16, 12, 8, 6, 4, 2, 1. These represent the lengths as multipliers of sixteenth notes, so 16 represents a full note, 12 represents a dotted half note, 8 represents a half note, 6 represents a dotted quarter note, and so on. These are sorted from longest to shortest length.
Figure 6: Range of notes from middle C to A5
These Markov models are represented as matrices with probabilities for their entries. Each row represents a note/length, and each value on the row represents the probability of transitioning from the row's note/length to the column's note/length. In practice, we actually used matrices of the same row/column structure but instead stored cumulative probabilities (that sum to 1) for each row so that we could use an input value (in our project's case, the readings from the electrodes on the arm), normalize the value to be less than or equal to 1, and then see which note/length the model will transition to next.
The code for running our Markov model is in a protothread that runs whenever a sound is not playing. When choosing our initial current note and current note length, we pick a random note and random note length. From there, we start an infinite while loop for the Markov models to run, playing a note for each iteration. We did this by looping through each entry of the current note/length row in the matrix and selecting the first transition that had a cumulative probability greater than the normalized input value. This would mean the input value fell into this transition's "bucket," and more common transitions would have larger "buckets" than less common ones. Once we found our next note and our next note length using this method, we played the sound and set the note/length that was played to be the current note/length. We then repeat the outer while loop to continue choosing another note/length. In order to populate the probabilities in these matrices, we counted the notes and note lengths in Christmas songs and normalized them using two Python scripts, note_agg.py and length_agg.py. The songs used were O Holy Night, Silent Night, Jingle Bells, 12 Days of Christmas (up to the fifth day), Deck the Halls with Boughs of Holly, Hark the Herald Angels Sing, Ring the Bells, I Heard the Bells on Christmas Day, and The Nutcracker March. The Python scripts we made take in notes or note length values via terminal input, and keep track of the amount of transitions. Once complete, it generates a matrix of probabilities and a matrix of cumulative probabilities that we were able to copy into our C code.
FM Synthesis¶
To play the sound characterized by the note and length chosen by the Markov model, we used FM Synthesis. We decided to use FM Synthesis to play sounds rather than just using sine tables because FM Synthesis allows us to modulate the frequency of the sound, making it possible to mimic the sound of instruments. We used Bruce Land's FM Synthesis code from here: https://people.ece.cornell.edu/land/courses/ece4760/PIC32/Sound_synth/FM_DAC_4_brl4.c
The FM Synthesis code runs on a repeating timer ISR that plays a portion of a sound every 50 microseconds. In the Markov model protothread, the protothread yields until tempo_v1_flag==1. This occurs when the sound being played ends. Once the sound ends, the Markov model protothread would setup variables for the ISR to play the note with the given note length, and then set play_trigger=1 and tempo_v1_flag=0. Like in lab 1, the FM Synthesis code utilizes a sine table that it indexes into via an incrementing index. Based on this index, we can change the frequency of the sound to be the sound of the chosen note. Via the array of note frequencies, we are able to take our chosen frequency and multiply it with the instrument's synth frequency and 2^32 divided by the sampling frequency (4000 Hz) to get our phase increment. This was fairly simple to implement, as it was just replacing a value in the FM Synthesis code. However, implementing note lengths was trickier.
Sounds played by the ISR only ended when tempo_v1_count became greater than the tempo. This would indicate that a different note would need to be played. At first, when trying to change the note length, we tried multiplying anything from the attack, decay, and envelope variables in an attempt to elongate the sound's envelope for longer note lengths. However, we realized that notes of different lengths don't have to have different attack or decay rates. Instead, we considered the current sound envelope to be a "full note." If the note is shorter than a full note, we would cut it off early by modifying the tempo variable by the current note length. Instead of dividing the tempo, we assumed that the current tempo would be the length of a sixteenth note, and multiplied it by the note length instead. This is why we made the note length values integer multipliers of a sixteenth note. This was successful in making the note lengths apply to the played sound.
One additional thing we did was modify the temp based on the voltage measurements of the arm. This would make larger arm voltage readings increase the tempo, causing notes to play faster. The tempo variable is an integer in terms of ISR ticks. To increase the tempo, we decrease the value of this integer so that it takes fewer ISR ticks to finish a note. So we subtract a base tempo of 5000 ISR ticks by the voltage reading from the arm (normalized to 1) times 1500. We felt that this made the change in tempo very noticeable but not too hard to manage when flexing.
Arm Voltage Reading¶
We use the RP2040's onboard ADC to get the voltage input from the bio instrumentation circuit. The output values range from 0 to 4096, which has the side effect of not measuring negative voltage readings which do get read by the oscilloscope. While we could have built a voltage rectifier, so that the voltage readings from the arm are always positive and the RP2040 would thus read more signals from the arm, we did not do so. This is due to time constraints and the lack of value, as we are able to get a readable signal from the ADC only using the positive voltage inputs.
One tricky issue was determining the method of reading the input from the ADC. The voltage measurements from the electrodes tend to spike and fall instantaneously when the arm is flexed. This means that when using voltage readings for choosing the transition for the Markov model, we can't just take a measurement when we need to play a note. If we did this, it would be difficult to get higher voltage values as input for the Markov model because the user would have to time the arm movement the instant the RP2040 takes the measurement.
Instead of using the raw ADC input value, we instead made a protothread that polled the ADC every millisecond. We then divide it by 4096, and add it to a variable called integral_sum. This builds the value of intergal_sum if a user is flexing their muscle repeatedly, like they are charging a battery. After adding to integral_sum, we decrease the value of integral_sum by 0.1%, so that the value can wean off over time if the muscle is not flexed. To remove noise from the ADC input, we only add to integral_sum if the ADC input is greater than 1000. We also clamp the integral_sum variable to be at integral_size=5 at most. We chose a number greater than 1 for the size so that the intergal_sum won't be maxed out by one strong flex. For example, if the muscle flexing maxes out the ADC input at 4096, it will only add 1 to the integral_sum, which would be only 1/5 of the integral_sum's capacity if integral_size==5.
When we use integral_sum in the Markov model protothread, we divide it by integral_size so that it is normalized to 1, and we can use that value in our cumulative distribution functions in a row of a Markov model.
Keypad Program¶
To implement button pressing, we had to use a finite state machine after scanning the keypad to ensure that for each button press, there is a change in instrument and that the microcontroller doesn’t detect multiple button presses each time you press or release a button. To prevent this, after each keypad scan was implemented, we check the state of the button pressing and see if we need to change states. We start in the not-pressed state and repeatedly scan the keypad for anything that is not a -1. If anything is returned on the i-value, then the state machine goes to the maybe-pressed state. After the thread scans the keypad again, if the possible value is the same as the i-value then it should transition into the pressed button state. In this transition, it will set the beeping to 1 which will enable the sound to be produced in the ISR. If the i-value doesn’t equal the possible value then it goes back to the not pressed state. During the transition from the maybe-pressed state to the pressed state, we change the instrument to the specified i-value if it is between 1-8. Once in the pressed state, if the i-value doesn’t equal the possible value when it moved to the pressed state then another button might be pressed or the current button was released and it moves to the maybe-not pressed state. Until either of those actions happen, it will remain in the pressed state. Finally, in the maybe-not pressed state if the i-value is just the same possible value then it should go back to the pressed state since the button wasn’t actually released. If any other i-value is returned, then it should go to the not pressed state. This finite state machine can be seen in the image below.
Figure 7: Keypad Button Pressing Finite State Machine
Results of the Design¶
Our final system included three electrodes connected to someone's skin (in our demonstration video it's Kidus) that creates a caroling beat that speeds up and goes higher when the user flexes their muscles. In addition, the user can change the instrument type using the keypad.
Final Hardware Diagram¶
Figure 8: Final Hardware Diagram
Safety¶
Since our project uses human testing, we had to be very careful with our system. To ensure we were safe, when we connected the electrodes to our skin, we needed the system to be connected to a floating ground as a regular grounded system could cause us to get electrocuted.
Usability¶
The usability of the project depends on the build of the person who is using it and the quality of the electrode connections. The amount of fat or muscle in the arm and the specific muscle being measured can affect the voltage reading. However, using high-quality electrode pads was a very important factor for reading stronger voltage measurements. The differences introduced by these factors can make getting a voltage reading that causes a meaningful change in the audio harder for some people than others.
Conclusions¶
For the hardware design, we would make some changes to make our system more reliable. We would have liked to include a signal rectifier op-amp to rectify the signal since we found that sometimes the signal would spike with a muscle contraction and then go below 0. To be able to gather all the data from the muscle contraction, I would have liked to include a rectifier but it would only increase the resulting system's effectivity a little bit.
Appendix A (permissions)¶
The group approves this report for inclusion on the course website.
The group approves the video for inclusion on the course YouTube channel.
Additional Appendices¶
Work Distribution¶
Sofia adapted the FM synthesis code developed by Bruce Land for the PIC32 to the RP2040. They also did all of the hardware testing and working on the bioinstrumentation
Kidus helped debug the bioinstrumentation circuit with the electrodes attached and developed and trained the Markov model.
References:¶
EMG Circuit Advancer Technologies: https://www.instructables.com/Muscle-EMG-Sensor-for-a-Microcontroller/
Physiological AC Preamplifier: https://people.ece.cornell.edu/land/PROJECTS/preamp2/index.html
RP2040 Datasheet: https://datasheets.raspberrypi.com/rp2040/rp2040-datasheet.pdf
FM Synth to SPI Example Code: https://people.ece.cornell.edu/land/courses/ece4760/PIC32/Sound_synth/FM_DAC_4_brl4.c
Physics of Music- Notes: https://pages.mtu.edu/~suits/notefreqs.html
INA121 Datasheet: https://www.ti.com/lit/ds/symlink/ina121.pdf?ts=1700234146755&ref_url=https%253A%252F%252Fwww.google.ca%252F
Sheet Music Training Data:
- https://www.8notes.com/scores/13800.asp
- https://www.8notes.com/scores/11339.asp
- https://www.8notes.com/scores/11338.asp
- https://www.8notes.com/scores/15318.asp
- https://www.8notes.com/scores/621.asp
- https://www.8notes.com/scores/13801.asp
- https://www.8notes.com/scores/20567.asp
- https://www.8notes.com/scores/23863.asp
- https://musescore.com/user/101118/scores/151348
Code¶
Schedule_FM_DAC.c¶
// clock AND protoThreads configure!
// threading library
// for sine
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "pico/stdlib.h"
#include "pico/multicore.h"
#include "hardware/spi.h"
#include "hardware/sync.h"
#include "hardware/irq.h"
#include "hardware/adc.h"
#include "pt_cornell_rp2040_v1.h"
// A-channel, 1x, active
#define DAC_config_chan_A 0b0011000000000000
#define DAC_config_chan_B 0b1011000000000000
#define Fs 4000.0
#define two32 4294967296.0 // 2^32
// === GCC s16.15 format ===============================================
#define float2Accum(a) ((_Accum)(a))
#define Accum2float(a) ((float)(a))
#define int2Accum(a) ((_Accum)(a))
#define Accum2int(a) ((int)(a))
// the native type is _Accum but that is ugly
typedef _Accum fixAccum ;
#define onefixAccum int2Accum(1)
//#define sustain_constant float2Accum(256.0/20000.0) ; // seconds per decay update
// === GCC 0.16 format ===============================================
#define float2Fract(a) ((_Fract)(a))
#define Fract2float(a) ((float)(a))
// the native type is _Accum but that is ugly
typedef _Fract fixFract ;
fixFract onefixFract = float2Fract(0.9999);
#define sustain_constant float2Accum(256.0/20000.0) // seconds per decay update
// === 16:16 fixed point macros ==========================================
typedef signed int fix16 ;
#define multfix16(a,b) ((fix16)(((( signed long long)(a))*(( signed long long)(b)))>>16)) //multiply two fixed 16:16
#define float2fix16(a) ((fix16)((a)*65536.0)) // 2^16
#define fix2float16(a) ((float)(a)/65536.0)
#define fix2int16(a) ((int)((a)>>16))
#define int2fix16(a) ((fix16)((a)<<16))
#define divfix16(a,b) ((fix16)((((signed long long)(a)<<16)/(b))))
#define sqrtfix16(a) (float2fix16(sqrt(fix2float16(a))))
#define absfix16(a) abs(a)
#define onefix16 0x00010000 // int2fix16(1)
// actual DAC data
uint16_t DAC_data;
uint16_t bio_DAC_data;
// the DDS units: 1=FM, 2=main frequency
volatile unsigned int phase_accum_fm, phase_incr_fm=1.8*261.0*two32/Fs ;//
volatile unsigned int phase_accum_main, phase_incr_main=261.0*two32/Fs ;//
//volatile int spiClkDiv = 2 ; // 20 MHz max speed for this DAC
// DDS sine table
#define sine_table_size 256
volatile fix16 sine_table[sine_table_size];
// envelope: 1=FM, 2=main frequency
// envelope: FM and main frequency
volatile fixAccum env_fm, wave_fm, dk_state_fm, attack_state_fm;
volatile fixAccum env_main, wave_main, dk_state_main, attack_state_main;
// define the envelopes and tonal quality of the instruments
#define n_synth 8
// 0 plucked string-like
// 1 slow rise
// 2 string-like lower pitch
// 3 low drum
// 4 medium drum
// 5 snare
// 6 chime
// 7 low, harsh string-like
// 0 1 2 3 4 5 6
volatile fixAccum attack_main[n_synth] = {0.001, 0.9, 0.001, 0.001, 0.001, 0.001, 0.001, .005};
volatile fixAccum decay_main[n_synth] = {0.98, 0.97, 0.98, 0.98, 0.98, 0.80, 0.98, 0.98};
//
volatile fixAccum depth_fm[n_synth] = {2.00, 2.5, 2.0, 3.0, 1.5, 10.0, 1.0, 2.0};
volatile fixAccum attack_fm[n_synth] = {0.001, 0.9, 0.001, 0.001, 0.001, 0.001, 0.001, 0.005};
volatile fixAccum decay_fm[n_synth] = {0.80, 0.8, 0.80, 0.90, 0.90, 0.80, 0.98, 0.98};
// 0 1 2 3 4 5 6 7
float freq_main[n_synth] = {1.0, 1.0, 0.5, 0.25, 0.5, 1.00, 1.0, 0.25};
float freq_fm[n_synth] = {3.0, 1.1, 1.5, 0.4, 0.8, 1.00, 1.34, 0.37};
// the current setting for instrument (index into above arrays)
volatile int temp_synth=0;
volatile int current_v1_synth=0;
// pentatonic scale
// Transposing the pitches to fit into one octave rearranges
// the pitches into the major pentatonic scale: C, D, E, G, A, C.
// C4 262 Hz (middle C)
// C4S 277
// D4 294
// D4S 311
// E4 330
// F4 349
// F4S 370
// G4 392
// G4S 415
// A4 440
// A4S 466
// B4 494
// C5 523
// C5S 554
// D5 587
// D5S 622
// E5 659
// F5 698
// F5S 740
// G5 784
// G5S 831
// A5 880
// A5S 932
// see: http://www.phy.mtu.edu/~suits/notefreqs.html
float notes[23] = {262,277,294,311,330,349,370,392,415,440,466,494,523,554,587,622,659,698,740,784,831,880,932} ;
float notes_model[23][23] = {{0.2391304347826087, 0.021739130434782608, 0.043478260869565216, 0.021739130434782608, 0.10869565217391304, 0.1956521739130435, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.25, 0.022727272727272728, 0.045454545454545456, 0.022727272727272728, 0.09090909090909091, 0.022727272727272728, 0.022727272727272728, 0.1590909090909091, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.045454545454545456, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728, 0.022727272727272728}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.012048192771084338, 0.012048192771084338, 0.1566265060240964, 0.012048192771084338, 0.3132530120481928, 0.13253012048192772, 0.012048192771084338, 0.13253012048192772, 0.012048192771084338, 0.03614457831325301, 0.012048192771084338, 0.012048192771084338, 0.012048192771084338, 0.012048192771084338, 0.024096385542168676, 0.012048192771084338, 0.012048192771084338, 0.012048192771084338, 0.012048192771084338, 0.012048192771084338, 0.012048192771084338, 0.012048192771084338, 0.012048192771084338}, {0.047619047619047616, 0.011904761904761904, 0.08333333333333333, 0.011904761904761904, 0.16666666666666666, 0.25, 0.011904761904761904, 0.17857142857142858, 0.011904761904761904, 0.047619047619047616, 0.011904761904761904, 0.03571428571428571, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904, 0.011904761904761904}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.019230769230769232, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616, 0.10576923076923077, 0.14423076923076922, 0.009615384615384616, 0.15384615384615385, 0.009615384615384616, 0.36538461538461536, 0.009615384615384616, 0.009615384615384616, 0.04807692307692308, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616, 0.009615384615384616}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.010101010101010102, 0.010101010101010102, 0.030303030303030304, 0.010101010101010102, 0.010101010101010102, 0.09090909090909091, 0.010101010101010102, 0.21212121212121213, 0.010101010101010102, 0.1111111111111111, 0.010101010101010102, 0.29292929292929293, 0.09090909090909091, 0.010101010101010102, 0.010101010101010102, 0.010101010101010102, 0.010101010101010102, 0.010101010101010102, 0.010101010101010102, 0.010101010101010102, 0.010101010101010102, 0.010101010101010102, 0.010101010101010102}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.01282051282051282, 0.01282051282051282, 0.01282051282051282, 0.01282051282051282, 0.02564102564102564, 0.01282051282051282, 0.01282051282051282, 0.14102564102564102, 0.01282051282051282, 0.3076923076923077, 0.01282051282051282, 0.0641025641025641, 0.21794871794871795, 0.01282051282051282, 0.02564102564102564, 0.01282051282051282, 0.01282051282051282, 0.01282051282051282, 0.01282051282051282,
0.01282051282051282, 0.01282051282051282, 0.01282051282051282, 0.01282051282051282}, {0.011494252873563218, 0.011494252873563218, 0.011494252873563218, 0.011494252873563218, 0.022988505747126436, 0.034482758620689655, 0.011494252873563218, 0.12643678160919541, 0.011494252873563218, 0.034482758620689655, 0.011494252873563218, 0.14942528735632185, 0.3563218390804598, 0.011494252873563218, 0.09195402298850575, 0.011494252873563218, 0.011494252873563218, 0.011494252873563218, 0.011494252873563218, 0.011494252873563218, 0.011494252873563218, 0.011494252873563218, 0.011494252873563218}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608,
0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.2391304347826087, 0.13043478260869565, 0.021739130434782608, 0.17391304347826086, 0.021739130434782608, 0.043478260869565216, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608, 0.021739130434782608}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.10344827586206896, 0.034482758620689655, 0.13793103448275862, 0.06896551724137931, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655, 0.034482758620689655}, {0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.08333333333333333, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664, 0.041666666666666664}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}, {0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216, 0.043478260869565216}};
float notes_cdf[23][23] = {{0.2391304347826087, 0.2608695652173913, 0.30434782608695654, 0.32608695652173914, 0.43478260869565216, 0.6304347826086957, 0.6521739130434783, 0.6739130434782609, 0.6956521739130435, 0.717391304347826, 0.7391304347826086, 0.7608695652173912, 0.7826086956521738, 0.8043478260869564, 0.826086956521739, 0.8478260869565216, 0.8695652173913042, 0.8913043478260868, 0.9130434782608694, 0.934782608695652, 0.9565217391304346, 0.9782608695652172, 0.9999999999999998}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.25, 0.2727272727272727, 0.3181818181818182, 0.3409090909090909, 0.43181818181818177, 0.4545454545454545, 0.4772727272727272, 0.6363636363636362, 0.659090909090909, 0.6818181818181817, 0.7045454545454544, 0.7272727272727271, 0.7499999999999998, 0.7727272727272725, 0.8181818181818179, 0.8409090909090906, 0.8636363636363633, 0.886363636363636, 0.9090909090909087, 0.9318181818181814, 0.9545454545454541, 0.9772727272727268, 0.9999999999999996}, {0.043478260869565216, 0.08695652173913043,
0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.012048192771084338, 0.024096385542168676, 0.18072289156626506, 0.1927710843373494, 0.5060240963855422, 0.6385542168674699, 0.6506024096385543, 0.783132530120482, 0.7951807228915664, 0.8313253012048194, 0.8433734939759038, 0.8554216867469882, 0.8674698795180725, 0.8795180722891569, 0.9036144578313255, 0.9156626506024099, 0.9277108433734943, 0.9397590361445787, 0.951807228915663, 0.9638554216867474, 0.9759036144578318, 0.9879518072289162, 1.0000000000000004}, {0.047619047619047616, 0.05952380952380952, 0.14285714285714285, 0.15476190476190477, 0.3214285714285714, 0.5714285714285714, 0.5833333333333333, 0.7619047619047619, 0.7738095238095237, 0.8214285714285714, 0.8333333333333333, 0.869047619047619, 0.8809523809523808, 0.8928571428571427, 0.9047619047619045, 0.9166666666666664, 0.9285714285714283, 0.9404761904761901, 0.952380952380952, 0.9642857142857139, 0.9761904761904757, 0.9880952380952376, 0.9999999999999994}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.019230769230769232, 0.028846153846153848, 0.038461538461538464, 0.04807692307692308, 0.15384615384615385, 0.2980769230769231, 0.3076923076923077, 0.46153846153846156, 0.4711538461538462, 0.8365384615384616, 0.8461538461538461, 0.8557692307692307, 0.9038461538461539, 0.9134615384615384, 0.923076923076923, 0.9326923076923076, 0.9423076923076922, 0.9519230769230768, 0.9615384615384613, 0.9711538461538459, 0.9807692307692305, 0.9903846153846151, 0.9999999999999997}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.010101010101010102, 0.020202020202020204, 0.05050505050505051, 0.060606060606060615, 0.07070707070707072, 0.16161616161616163, 0.17171717171717174, 0.38383838383838387, 0.393939393939394, 0.5050505050505051, 0.5151515151515151, 0.8080808080808081, 0.898989898989899, 0.9090909090909091, 0.9191919191919191, 0.9292929292929292, 0.9393939393939392, 0.9494949494949493, 0.9595959595959593, 0.9696969696969694, 0.9797979797979794, 0.9898989898989895, 0.9999999999999996}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.01282051282051282, 0.02564102564102564, 0.038461538461538464, 0.05128205128205128, 0.07692307692307693, 0.08974358974358974, 0.10256410256410256, 0.24358974358974358, 0.2564102564102564, 0.5641025641025641, 0.5769230769230769, 0.641025641025641, 0.8589743589743589, 0.8717948717948717, 0.8974358974358974, 0.9102564102564101, 0.9230769230769229, 0.9358974358974357, 0.9487179487179485, 0.9615384615384612, 0.974358974358974, 0.9871794871794868, 0.9999999999999996}, {0.011494252873563218, 0.022988505747126436, 0.034482758620689655, 0.04597701149425287, 0.06896551724137931, 0.10344827586206896, 0.11494252873563218, 0.2413793103448276, 0.25287356321839083, 0.2873563218390805, 0.2988505747126437, 0.4482758620689655, 0.8045977011494253, 0.8160919540229885, 0.9080459770114943, 0.9195402298850575, 0.9310344827586207, 0.9425287356321839, 0.9540229885057471, 0.9655172413793103, 0.9770114942528735, 0.9885057471264367, 0.9999999999999999}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.021739130434782608, 0.043478260869565216, 0.06521739130434782, 0.08695652173913043, 0.10869565217391304, 0.13043478260869565, 0.15217391304347827, 0.17391304347826086, 0.19565217391304346, 0.21739130434782605, 0.23913043478260865, 0.47826086956521735, 0.608695652173913, 0.6304347826086956, 0.8043478260869564, 0.826086956521739, 0.8695652173913042, 0.8913043478260868, 0.9130434782608694, 0.934782608695652, 0.9565217391304346, 0.9782608695652172, 0.9999999999999998}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.034482758620689655, 0.06896551724137931, 0.10344827586206896, 0.13793103448275862, 0.1724137931034483, 0.20689655172413796, 0.24137931034482762, 0.2758620689655173, 0.31034482758620696, 0.34482758620689663, 0.3793103448275863, 0.41379310344827597, 0.44827586206896564, 0.4827586206896553, 0.5862068965517243, 0.6206896551724139, 0.7586206896551726, 0.8275862068965519, 0.8620689655172415, 0.8965517241379312, 0.9310344827586208, 0.9655172413793104, 1.0}, {0.041666666666666664, 0.08333333333333333, 0.125, 0.16666666666666666, 0.20833333333333331, 0.24999999999999997, 0.29166666666666663, 0.3333333333333333, 0.375, 0.4166666666666667, 0.45833333333333337, 0.5, 0.5416666666666666, 0.5833333333333333, 0.6249999999999999, 0.6666666666666665, 0.7499999999999999, 0.7916666666666665, 0.8333333333333331, 0.8749999999999998, 0.9166666666666664, 0.958333333333333, 0.9999999999999997}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}, {0.043478260869565216, 0.08695652173913043, 0.13043478260869565, 0.17391304347826086, 0.21739130434782608, 0.2608695652173913, 0.30434782608695654, 0.34782608695652173, 0.3913043478260869, 0.4347826086956521, 0.4782608695652173, 0.5217391304347825, 0.5652173913043477, 0.6086956521739129, 0.652173913043478, 0.6956521739130432, 0.7391304347826084, 0.7826086956521736, 0.8260869565217388, 0.869565217391304, 0.9130434782608692, 0.9565217391304344, 0.9999999999999996}};
// full, half dot, half, quarter dot, quarter, eighth, sixteenth
int lengths[7] = { 16, 12, 8, 6, 4, 2, 1} ;
float lengths_model[7][7] = {{0.125, 0.125, 0.125, 0.125, 0.25, 0.125, 0.125}, {0.05263157894736842, 0.05263157894736842, 0.2631578947368421, 0.10526315789473684, 0.21052631578947367, 0.2631578947368421, 0.05263157894736842}, {0.03333333333333333, 0.03333333333333333, 0.06666666666666667, 0.1, 0.6333333333333333, 0.1, 0.03333333333333333}, {0.015873015873015872, 0.015873015873015872, 0.015873015873015872, 0.1111111111111111, 0.2857142857142857, 0.5238095238095238, 0.031746031746031744}, {0.009966777408637873, 0.04983388704318937, 0.04983388704318937, 0.06976744186046512, 0.574750830564784, 0.23920265780730898, 0.006644518272425249}, {0.003236245954692557, 0.016181229773462782, 0.022653721682847898, 0.08090614886731391, 0.2621359223300971, 0.5760517799352751, 0.038834951456310676}, {0.017857142857142856, 0.017857142857142856, 0.017857142857142856, 0.017857142857142856, 0.03571428571428571, 0.23214285714285715, 0.6607142857142857}};
float lengths_cdf[7][7] = {{0.125, 0.25, 0.375, 0.5, 0.75, 0.875, 1.0}, {0.05263157894736842, 0.10526315789473684, 0.3684210526315789, 0.47368421052631576, 0.6842105263157894, 0.9473684210526314, 0.9999999999999998}, {0.03333333333333333, 0.06666666666666667, 0.13333333333333333, 0.23333333333333334, 0.8666666666666667, 0.9666666666666667, 1.0}, {0.015873015873015872, 0.031746031746031744, 0.047619047619047616, 0.15873015873015872, 0.4444444444444444, 0.9682539682539683, 1.0}, {0.009966777408637873, 0.059800664451827246, 0.10963455149501661, 0.17940199335548174, 0.7541528239202657, 0.9933554817275747, 1.0}, {0.003236245954692557, 0.019417475728155338, 0.042071197411003236, 0.12297734627831715, 0.3851132686084142, 0.9611650485436893, 1.0}, {0.017857142857142856, 0.03571428571428571, 0.05357142857142857, 0.07142857142857142, 0.10714285714285714, 0.3392857142857143, 1.0}};
// note transition rate in ticks of the ISR
// rate is 20/mSec. So 250 mS is 5000 counts
// 8/sec, 4/sec, 2/sec, 1/sec
int base_tempo = 5000;
volatile int tempo = 5000;
int tempo_v1_flag, tempo_v2_flag ;
int current_v1_tempo=1, current_v2_tempo=2;
int tempo_v1_count, tempo_v2_count ;
// beat/rest patterns
#define n_beats 11
int beat[n_beats] = {
0b0101010101010101, // 1-on 1-off phase 2
0b1111111011111110, // 7-on 1-off
0b1110111011101110, // 3-on 1-off
0b1100110011001100, // 2-on 2-off phase 1
0b1010101010101010, // 1-on 1-off phase 1
0b1111000011110000, // 4-on 4-off phase 1
0b1100000011000000, // 2-on 6-off
0b0011001100110011, // 2-on 2-off phase 2
0b1110110011101100, // 3-on 1-off 2-on 2-off 3-on 1-off 2-on 2-off
0b0000111100001111, // 4-on 4-off phase 2
0b1111111111111111 // on
} ;
// max-beats <= 16 the length of the beat vector in bits
#define max_beats 16
int current_v1_beat=1, current_v2_beat=2;
int beat_v1_count, beat_v2_count ;
//
// random number
volatile int rand_raw ;
// time scaling for decay calculation
volatile int dk_interval; // wait some samples between decay calcs
// play flag
volatile int play_trigger;
volatile fixAccum sustain_state, sustain_interval=0;
// profiling of ISR
volatile int isr_time, isr_start_time, isr_count=0;
volatile int current_note=0;
volatile int current_length=1;
//SPI configurations (for Pico hardware)
#define PIN_MISO 4
#define PIN_CS 5
#define PIN_SCK 6
#define PIN_MOSI 7
#define LDAC 8
#define SPI_PORT spi0
//Bio-instrumentation ADC input
#define BIO_IN 26
volatile int in=0;
volatile int in_avg=0;
volatile int samples=0;
uint16_t average_bio=0;
float integral_sum = 0;
float integral_size = 5.0;
// Keypad pin configurations
#define BASE_KEYPAD_PIN 9
#define KEYROWS 4
#define NUMKEYS 12
unsigned int keycodes[12] = { 0x28, 0x11, 0x21, 0x41, 0x12,
0x22, 0x42, 0x14, 0x24, 0x44,
0x18, 0x48} ;
unsigned int scancodes[4] = { 0x01, 0x02, 0x04, 0x08} ;
unsigned int button = 0x70 ;
char keytext[40];
int prev_key = 0;
// Button State machine variables
volatile int NOT_PRESSED = 1;
volatile int MAYBE_PRESSED = 0;
volatile int PRESSED = 0;
volatile int MAYBE_NOT_PRESSED = 0;
volatile int possible = -1;
volatile int beeped=0;
//timer ISR
bool repeating_timer_callback(struct repeating_timer *t)
{
// time to get into ISR
isr_start_time = time_us_32();
tempo_v1_count++ ;
if (tempo_v1_count>=tempo*(lengths[current_length])) {
tempo_v1_flag = 1;
tempo_v1_count = 0;
}
// FM phase
phase_accum_fm += (int)(phase_incr_fm );
// main phase
phase_accum_main += phase_incr_main + (Accum2int(sine_table[phase_accum_fm>>24] * env_fm)<<16) ;
// init the exponential decays
// by adding energy to the exponential filters
if (play_trigger) {
dk_state_fm = depth_fm[current_v1_synth];
dk_state_main = onefixAccum;
attack_state_fm = depth_fm[current_v1_synth];
attack_state_main = onefixAccum;
play_trigger = 0;
phase_accum_fm = 0;
phase_accum_main = 0;
dk_interval = 0;
sustain_state = 0;
}
// envelope calculations are 256 times slower than sample rate
// computes 4 exponential decays and builds the product envelopes
if ((dk_interval++ & 0xff) == 0){
// approximate the first order FM decay ODE
dk_state_fm = dk_state_fm * decay_fm[current_v1_synth] ;
// approximate the first order main waveform decay ODE
dk_state_main = dk_state_main * decay_main[current_v1_synth] ;
// approximate the ODE for the exponential rise FM/main waveform
attack_state_fm = attack_state_fm * attack_fm[current_v1_synth] ;
attack_state_main = attack_state_main * attack_main[current_v1_synth] ;
// product of rise and fall is the FM envelope
// fm_depth is the current value of the function
env_fm = (depth_fm[current_v1_synth] - attack_state_fm) * dk_state_fm ;
// product of rise and fall is the main envelope
env_main = (onefixAccum - attack_state_main) * dk_state_main ;
}
// === Channel A =============
//sending the DAC sound data
// CS low to start transaction
gpio_put(PIN_CS, 0);
spi_write16_blocking(SPI_PORT, &DAC_data, 1);
wave_main = (sine_table[phase_accum_main>>24]) * env_main ;
// truncate to 12 bits, read table, convert to int and add offset
DAC_data = DAC_config_chan_A | (Accum2int(wave_main) + 2048) ;
// test for done
while (spi_is_busy(SPI_PORT)); // wait for end of transaction
// =======Channel B =======
//bio_DAC_data = DAC_config_chan_B | (raw_in & 0b111111111111);
//spi_write16_blocking(SPI_PORT, &bio_DAC_data, 1);
//while (spi_is_busy(SPI_PORT)); // wait for end of transaction
// CS high
gpio_put(PIN_CS, 1);
//mPORTBSetBits(BIT_4); // end transaction
// time to get into ISR is the same as the time to get out so add it again
isr_time = fmax(isr_time, time_us_32()+isr_start_time) ; // - isr_time;
return true;
} // end ISR TIMER2
// === Thread 1 ======================================================
//markov music model instantiate
static PT_THREAD (protothread_cmd(struct pt *pt))
{
static float value;
static float f_fm, f_main;
PT_BEGIN(pt);
// markov
static int i;
static int n;
static int l;
float note_roll;
float length_roll;
static int found;
tempo_v1_count = 0;
tempo_v1_flag = 0;
current_note = rand() % 23;
current_length = rand() % 7;
int start=1;
while(1){
//int in = (int)adc_read();
//printf("%d \n", in);
note_roll = integral_sum/integral_size; //((float) rand() / (RAND_MAX));
length_roll = integral_sum/integral_size; //((float) rand() / (RAND_MAX));
//reduces part of the integral sum
printf("integral_sum_over_size %f\n",length_roll);
found=0;
for (n=0; n<23; n++){
if(found==0 && notes_cdf[current_note][n]>=note_roll){
for (l=0;l<7;l++){
if(found==0 && lengths_cdf[current_length][l]>=length_roll){
printf(" #found#\n");
printf(" note: %f\n",notes[current_note]);
printf(" length: %d\n",lengths[current_length]);
printf(" base_tempo: %d\n", base_tempo);
printf(" tempo_change: %f\n", (integral_sum/integral_size)*1500.0);
printf(" calculation: %d\n", (int)(base_tempo * (1/(1.1-(integral_sum/integral_size)))));
if(start!=1){
PT_YIELD_UNTIL(pt,tempo_v1_flag==1);
}
else{
start=0;
}
current_note=n;
current_length=l;
current_v1_synth=temp_synth;
phase_incr_fm = freq_fm[current_v1_synth]*notes[current_note]*(float)two32/Fs;
phase_incr_main = freq_main[current_v1_synth]*notes[current_note]*(float)two32/Fs;
tempo = (int)(base_tempo - (integral_sum/integral_size)*1500.0);
if (tempo<2000){
tempo = 2000;
}
printf(" tempo: %d\n\n", tempo);
play_trigger = 1;
tempo_v1_flag = 0;
found=1;
}
}
}
}
if(found==0){
printf("none found");
current_note = 7;
current_length = 4;
}
}
PT_END(pt);
} // thread 1
// === Thread 2 ======================================================
// update the adc electrode input integral
static PT_THREAD (protothread_adc(struct pt *pt))
{
PT_BEGIN(pt);
while(1) {
// yield time 0.001 second
PT_YIELD_usec(1000) ;
// samples+=1;
in = (int)adc_read();
//in_avg += in;
//printf("in %d\n",in);
//determine integral proportional sum to try to keep it <1
if (in>1000){
integral_sum += in/4096.0;
}
integral_sum -= integral_sum*0.001;
//printf("integral %f\n\n",(integral_sum));
if(integral_sum> integral_size){
integral_sum=integral_size;
}
// NEVER exit while
} // END WHILE(1)
PT_END(pt);
} // thread 2
//keypad thread to change instruments
//instrument options:
// 0 plucked string-like
// 1 slow rise
// 2 string-like lower pitch
// 3 low drum
// 4 medium drum
// 5 snare
// 6 chime
// 7 low, harsh string-like
static PT_THREAD (protothread_keypad(struct pt *pt))
{
// Indicate thread beginning
PT_BEGIN(pt) ;
// Some variables
static int i ;
static uint32_t keypad ;
while(1) {
// Scan the keypad!
for (i=0; i<KEYROWS; i++) {
// Set a row high
gpio_put_masked((0xF << BASE_KEYPAD_PIN), //GPIO 9,10,11,12
(scancodes[i] << BASE_KEYPAD_PIN)) ;
// Small delay required
sleep_us(1) ;
// Read the keycode
keypad = ((gpio_get_all() >> BASE_KEYPAD_PIN) & 0x7F) ;
// Break if button(s) are pressed
if (keypad & button) break ;
}
// If we found a button . . .
if (keypad & button) {
// Look for a valid keycode.
for (i=0; i<NUMKEYS; i++) {
if (keypad == keycodes[i]) break ;
}
// If we don't find one, report invalid keycode
if (i==NUMKEYS) (i = -1) ;
}
// Otherwise, indicate invalid/non-pressed buttons
else (i=-1) ;
//iterate through the button finite state machine
//if the button hasn't been pressed
if (NOT_PRESSED==1){
//if it seems like it could have been pressed go to maybe pressed
if (i!=-1){
//set the i value to possible
possible=i;
MAYBE_PRESSED=1;
NOT_PRESSED=0;
}
}
//if it is the maybe pressed state check if it is pressed
else if (MAYBE_PRESSED==1){
//if the i value is the same as last state transition then pressed
if(i==possible){
PRESSED=1;
//button_number = i;
MAYBE_PRESSED=0;
}
//if the new i is not the possible i then button not really pressed
else{
MAYBE_PRESSED=0;
NOT_PRESSED=1;
}
}
//if in pressed state
else if (PRESSED==1){
//stay in state unless another different button is maybe pressed or button is let go of
//change instrument to button that is pressed
if(i>=1 && i<=8){
temp_synth=i-1;
}
if (i!=possible){
MAYBE_NOT_PRESSED=1;
PRESSED=0;
}
}
//if in maybe not pressed
else if (MAYBE_NOT_PRESSED==1){
//check if there is a new button pressed
if (i==possible){
MAYBE_NOT_PRESSED=0;
PRESSED=1;
}
//if nothing is pressed then go back to not pressed
else{
MAYBE_NOT_PRESSED=0;
NOT_PRESSED=1;
}
}
PT_YIELD_usec(30000) ;
}
// Indicate thread end
PT_END(pt) ;
}
//thread 4
// === Main ======================================================
// set up UART, timer, threads
int main(void)
{
srand((unsigned) time(NULL));
// === config the uart, DMA, vref, timer5 ISR =============
stdio_init_all();
// Format (channel, data bits per transfer, polarity, phase, order)
spi_init(SPI_PORT, 20000000);
spi_set_format(SPI_PORT, 16, 0, 0, 0);
// Map SPI signals to GPIO ports
gpio_set_function(PIN_MISO, GPIO_FUNC_SPI);
gpio_set_function(PIN_SCK, GPIO_FUNC_SPI);
gpio_set_function(PIN_MOSI, GPIO_FUNC_SPI);
gpio_set_function(PIN_CS, GPIO_FUNC_SPI) ;
// Map LDAC pin to GPIO port, hold it low (could alternatively tie to GND)
gpio_init(LDAC) ;
gpio_set_dir(LDAC, GPIO_OUT) ;
gpio_put(LDAC, 0) ;
//bio instrumentation ADC initialization
adc_init();
adc_gpio_init(BIO_IN);
adc_select_input(0);
//initialize timer
// Negative delay so means we will call repeating_timer_callback, and call it
// again 50us (40kHz) later regardless of how long the callback took to execute
struct repeating_timer timer;
add_repeating_timer_us(-50, repeating_timer_callback, NULL, &timer);
// init the threads
// turn off the sustain until triggered
sustain_state = float2Accum(100.0);
// build the sine lookup table
// scaled to produce values between 0 and 4095
int i;
for (i = 0; i < sine_table_size; i++){
sine_table[i] = float2Accum(2047.0*sin((float)i*6.283/(float)sine_table_size));
}
////////////////// KEYPAD INITS ///////////////////////
// Initialize the keypad GPIO's
gpio_init_mask((0x7F << BASE_KEYPAD_PIN)) ;
// Set row-pins to output
gpio_set_dir_out_masked((0xF << BASE_KEYPAD_PIN)) ;
// Set all output pins to low
gpio_put_masked((0xF << BASE_KEYPAD_PIN), (0x0 << BASE_KEYPAD_PIN)) ;
// Turn on pulldown resistors for column pins (on by default)
gpio_pull_down((BASE_KEYPAD_PIN + 4)) ;
gpio_pull_down((BASE_KEYPAD_PIN + 5)) ;
gpio_pull_down((BASE_KEYPAD_PIN + 6)) ;
pt_add_thread(protothread_cmd);
pt_add_thread(protothread_adc);
pt_add_thread(protothread_keypad);
pt_schedule_start;
} // main
note_agg.py¶
# // C4 262 Hz (middle C)
# // C4S 277
# // D4 294
# // D4S 311
# // E4 330
# // F4 349
# // F4S 370
# // G4 392
# // G4S 415
# // A4 440
# // A4S 466
# // B4 494
# // C5 523
# // C5S 554
# // D5 587
# // D5S 622
# // E5 659
# // F5 698
# // F5S 740
# // G5 784
# // G5S 831
# // A5 880
# // A5S 932
def build_cdf(model):
cdf = []
for i in range(len(notes)):
cdf.append([0]*len(notes))
for i in range(len(model)):
cumsum = 0
for connection in range(len(model[i])):
cumsum += model[i][connection]
cdf[i][connection] = cumsum
return cdf
notes = ["C4", "C4S","D4","D4S","E4","F4","F4S","G4","G4S","A4","A4S","B4","C5","C5S","D5","D5S","E5","F5","F5S","G5","G5S","A5","A5S"]
notes_model=[]
for i in range(len(notes)):
notes_model.append([0]*len(notes))
prev_idx=None
matrix_start_state = input("paste in matrix start state")
if matrix_start_state:
arr = matrix_start_state[2:-2].split('], [')
for i in range(len(arr)):
sp = arr[i].replace(' ','').split(',')
for j in range(len(sp)):
notes_model[i][j] = int(sp[j])
while True:
note = input("Note: ").upper()
if note in notes:
idx = notes.index(note)
if prev_idx is not None:
notes_model[prev_idx][idx]+=1
prev_idx = idx
elif note == "QUIT":
notes_norm = [list(map(lambda x: 0 if sum(note)==0 else x/sum(note),note)) for note in notes_model]
print(notes_model)
print('\n')
print(notes_norm)
print('\n')
print(build_cdf(notes_norm))
break
else:
print("invalid--skipped")
length_agg.py¶
def build_cdf(model):
cdf = []
for i in range(len(lengths)):
cdf.append([0] * len(lengths))
for i in range(len(model)):
cumsum = 0
for connection in range(len(model[i])):
cumsum += model[i][connection]
cdf[i][connection] = cumsum
return cdf
# full, half dot, half, quarter dot, quarter, eighth, sixteenth
lengths = ["16", "12", "8", "6", "4", "2", "1"]
lengths_model = []
for i in range(len(lengths)):
lengths_model.append([0] * len(lengths))
prev_idx = None
matrix_start_state = input("paste in matrix start state")
if matrix_start_state:
arr = matrix_start_state[2:-2].split("], [")
for i in range(len(arr)):
sp = arr[i].replace(" ", "").split(",")
for j in range(len(sp)):
lengths_model[i][j] = int(sp[j])
while True:
length = input("Length: ").upper()
if length in lengths:
idx = lengths.index(length)
if prev_idx is not None:
lengths_model[prev_idx][idx] += 1
print(f' {lengths[prev_idx]} -> {length}: {lengths_model[prev_idx][idx]}')
prev_idx = idx
elif length == "QUIT":
lengths_norm = [
list(map(lambda x: 0 if sum(length) == 0 else x / sum(length), length))
for length in lengths_model
]
print(lengths_model)
print("\n")
print(lengths_norm)
print("\n")
print(build_cdf(lengths_norm))
break
else:
print("invalid--skipped")