**Beat Revolution** Our project Beat Revolution is a mix between the games Beat Saber and Dance Dance Revolution (DDR). Beat Saber is a virtual reality game where players hold “lightsabers” to hit incoming blocks to the beat of the song. DDR is a classic arcade game where players stomp or “dance” on directional arrow pads according to the given arrows on the monitor. Beat Revolution combines the lightsaber idea from Beat Saber and the directional arrows from DDR.    Game Functionality =============================================================================== The game allows users to play arbitrarily many songs in one sitting as well as store scores across multiple systems or system restarts. Username Selection ------------------------------------------------------------------------------- When the system starts up, the player is prompted to enter a username. Short-pressing the left and right buttons scrolls through the letters of the alphabet, and long-pressing the left button confirms the current character. Long-pressing the right button confirms the selected username and moves the user to the song selection stage. Song Selection ------------------------------------------------------------------------------- Again, the left and right buttons are used to scroll through the list of songs. A list of high scores for the currently selected song is shown on the right display, and long-pressing the right button confirms the song and enters the gameplay state. During song selection, a user can quit the game by long-pressing the left button. This returns the player to the username selection stage. Gameplay ------------------------------------------------------------------------------- Once the user enters the gameplay state, the device pulls the arrow chart for the corresponding song from the server. The song is played from the SD card, and the user is given a few seconds to prepare before arrows start moving from the bottom of the screens. When the colored arrows reach the row of dark gray arrows at the top of the screen, the user is to wave the sabers in the direction corresponding to the arrow head. An arrow on the left display is to be hit by the left (red) saber, and an arrow on the right display is to be hit by the right (blue) saber. If a perfect hit is detected, the arrow flashes white, and if the user is only slightly off, the arrow flashes green. The player can quit in the middle of a game by long-pressing the left button, which returns them to the song selection stage. When the song is over, the user's score is displayed along with stats on the user's longest combo, number of perfect hits, number of decent hits, and number of misses. The user's score is uploaded to our server. The user can then long-press either button to return to the song selection stage. Scoring ------------------------------------------------------------------------------- The base scores for perfect and decent hits are 100 and 70, respectively. The combo number is the number of consecutive hits at that point in time, and on each hit the user also earns a score bonus equal to the current combo. Documentation =============================================================================== System Diagram -------------------------------------------------------------------------------  System Layout -------------------------------------------------------------------------------  In the system layout, the two IMUs and two displays are simplified to a single block, as most pins are the same on both left and right versions of each. The distinguishing pins are labeled NCS left and NCS right for the IMU, and CS left and CS right for the display. The different pins allow us to use our "chip select" method of reading/writing to both left and right IMUs and displays. State Machine Diagram -------------------------------------------------------------------------------  **State Alpha**: During this state, the device connects to WIFI and initializes the IMUs, MP3 player, and displays. This also acts as a calibration state where the IMU gets the normal values for acceleration and force. These will be the new “zero”. Then, it enters state 0. **State 0**: This is the state for username selection. Short presses left and right scroll through the potential username characters. A long left press locks in the selected character and moves to the next character in the username. A long right press moves us into the state 1. **State 1**: This state allows the player to select the song that they want to play. The list of available songs are on the left display, and high scores for that song are on the right display. A long left press brings you back to state 0 for username selection. A long right press selects the current song and brings you to state 2. **State 2**: State for the actual game play. During the game, we handle updating the saber movements, displays, and score as the song progresses. Players can stop the game at any point by long pressing the left button. When the song is complete, we move to state 3. **State 3**: State to display the user’s score breakdown and update the scores database. The user can see how they performed on the song. Press any of the buttons to move onto the song selection screen in state 1. Parts List ------------------------------------------------------------------------------- [Longruner Dfplayer Mini MP3 Player Module for Arduino LK01 (1)](https://www.amazon.com/dp/B01MXOFAE4) * Total price: $5.52 * Description: Outputs audio from ESP32 together with speakers. * Use case: Plays the songs. [SanDisk 16GB Ultra microSDXC UHS-I Memory Card with Adapter (1)](https://www.amazon.com/Sandisk-Ultra-Micro-UHS-I-Adapter/dp/B073K14CVB/ref=sr_1_2?keywords=sandisk+micro+sd+card+16&qid=1555254275&s=gateway&sr=8-2) * Total price: $5.79 * Description: SD card storage. * Use case: Stores mp3 files. [ELEGOO 6PCS 170 tie-Points Mini Breadboard kit for Arduino (1)](https://www.amazon.com/ELEGOO-6PCS-tie-Points-Breadboard-Arduino/dp/B01EV6SBXQ/ref=sr_1_3?crid=1GRM1DQ6YAO3K&keywords=tiny+breadboard&qid=1555460752&s=gateway&sprefix=tiny+bread%2Caps%2C146&sr=8-3) * Total price: $6.99 * Description: Where the imu sticks to, at the end of each of our sabers (sticks). * Use case: Use with imu to detect motion. Charts ------------------------------------------------------------------------------- The ESP32 needs a list of notes in order to work. These notes have a time associated with them, a direction, and whether they're on the left screen or the right screen. These lists of notes are in the form of song charts. Each song chart is a `.txt` file under `/charts/`. The syntax of a song chart was designed by hosed & empathetic humans and thus should be straightforward to work with. For example, take a look at the head of an iteration of `bloody_purity.txt`: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ plaintext offset 1503 bpm 155 duration 147000 L 6193 d R 6193 d L 6774 u ... ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * `offset 1503` - The first note starts at 1503 milliseconds. * `bpm 155` - The song runs at 155 beats per minute. This is important because BeatRevolution colors the notes according to the beat they land on. * `duration 147000` - The song ends after 147 seconds (2:27). The player's score appears after the song ends. * `L 6193 d` - This is a note that needs to be hit at 6193 milliseconds relative to the offset. The `L` means this note is for the *left* saber. `d`, *downward*, is the direction of the note. Naturally, the letters are: * `u` = up * `d` = down * `l` = left * `r` = right All note charts were produced by hand. To create the charts, we recommend (1) using a simple audio processing tool such as [Audacity](https://www.audacityteam.org/) and (2) taking note of the song's beats per minute. The notes are colored according to the beat they land on, like in DDR. Quarter notes are colored orange, eighth notes blue, twelfth notes purple, sixteenth notes green, and all other notes red. Making the chart for a song involved listening to small portions of the song on repeat and determining the beats where it felt appropriate to include hand motion (e.g. on beat $2$ of measure $4,$ the left hand moves to the left). Something we've looked into are programs that help create charts. Shown below, the list of beats, directions, and the corresponding saber side was passed into a Python script along with the tempo to determine the time, in ms, of each note relative to the beginning of the song.  For future improvements, we imagined that a more advanced script would play the song, shows its waveform or frequency spectrum along the way, and listens for the player to input their notes; specifically, it would be cool if `asdf` can input the left saber's notes and `up`/`down`/`left`/`right` can input the right saber's notes. Design Challenges and Decisions ------------------------------------------------------------------------------- A major challenge in this project was figuring out how to use two completely independent IMUs and LCD screens. Aside from the technical challenges, the components necessary for this project were all relatively clear. The arrow display, motion detection, and sound system are important parts of any rhythm game, and connecting to the internet was needed for persistence of high scores across multiple resets of the system. Storing and pulling information from the server also helped overcome the problem of limited space on the ESP32. ### Displays The independent displays problem was solved by digging into the TFT library and commenting out the line fixing the chip select pin. The chip select for each display was then wired to a different output pin on the ESP32; when the output pin for a display is written to low, that display receives the incoming information, and otherwise does not change. There were multiple options for giving visual feedback on the accuracy of note hits during gameplay. All of these involved an indicator object (e.g. a score) appearing on the display and then disappearing after some amount of time. To avoid the issue of having moving arrows prematurely erase the object, the top row of stationary arrows was chosen as an indicator since these were constantly redrawn over the moving arrows. The fade-out effect was added for a smoother visual transition, and coincidentally the sudden change from darkly colored arrows to brightly colored arrows followed by an immediate color fade produced the desired "glowing" effect. ### IMUs The IMUs continuously caused problems throughout the project. We started by using I2C communication with the IMUs, using the default 6.08 IMU library. We made several changes to the library, such as increasing the acceleration range, to suit our purpose, and we also struggled with mysterious memory problems with the library (fixed by enabling PSRAM on Arduino 1.8.8, or doing nothing on 1.8.9; still don't know why). When we put in a second saber, we initially went with the approach of setting the AD0 to HIGH/LOW depending on which IMU we wanted to read from. However, we realized that I2C commucation did not support fast enough switching between two IMUs. We were getting mixed data from the two IMUs when we tried to read from each at a frequency of 250Hz. Therefore, we switched to communication using SPI, which also required us to use a new library (discarding all our progress made with the previous library). We switched to the new MPU9250 library, which supports SPI and allows us to choose between our devices using the chip select pin. We ended up digging quite deep into the MPU9250 library because the IMUs were giving us trouble again. First, we noticed that we would randomly get readings of 0. This was resolved after noticing that the library took care of writing HIGH/LOW to the chip select pins, and removing `digitalWrite` statements before and after reading sensor data. Second, we noticed that the IMU was giving repeat readings -- we would get the same values in four or five consecutive `readSensor` calls. This was resolved after setting the sampling rate from a supposed default 1000Hz (the actual default appeared to be 50Hz despite what the docs claimed) to 250Hz using `setSrd(3)`. Third, we noticed that the IMUs would randomly break in the middle of games, by apparently disconnecting and giving readings of all 0. Looking further, we found that before the IMUs completely disconnected, they would return strange readings far from what we expect (for instance, reporting horizontal acceleration of 3m/s/s when an IMU is placed, unmoving, on a table). To try and resolve this problem, we dug into the source code of the MPU9250 library. In the process, we discovered and fixed a bug in `calibrateAccel()` that would reset the acceleration range to 2G. However, we could not spot the source of our problem. To try and put a bandaid over the bullet hole, we wrote our own versions of `begin`, `calibrateAccel`, and `readSensor`, which ignore the magnetometer and gyroscope. These functions are much faster than their original versions, which allowed us to reconnect and recalibrate the IMUs in around 60ms as opposed to several seconds. With this tool, we now: 1. reconnect the IMUs before every game, and 2. reconnect the IMUs during a game if we notice that the past 35 readings on the IMU have been identical, indicating that the IMU is dead. This solution isn't perfect. Readings are still often off during the game, sometimes reporting accelerations as high as 30m/s/s for an unmoving IMU. Unfortunately we can't do much more. Detailed Code Layout =============================================================================== The Game Class ------------------------------------------------------------------------------- The `Game` class handles the entire gameplay state. It is initialized with pointers to its `Display` and `Saber` objects along with pointers to the mp3 player and a list of the song names. This class mostly serves as a wrapper to combine `Display` and `Saber`. Important functions: * `Game::load(int song_index)` initializes all the score information (score, combo, number of perfect/decent/missed notes) to $0.$ It then downloads the note chart from the server and parses it, splitting times, directions, and handedness into the appropriate locations in storage. The function then loads both the `Display`s and the `Saber`s. * `Game::start(int song_index)` tells the mp3 to begin playing song number `song_index` and calls the `start` functions for `Display` and `Saber`. This also sets the starting time of the song to the current elapsed `millis()` timer of the ESP32. * `Game::process` checks for completion of the song and otherwise calls the `process` functions for `Display` and `Saber`. It also computes the new maximum combo and calls `update_score` on the right `Display` to visually indicate changes in score. * `Game::stop` pauses the mp3 player. Server Side ------------------------------------------------------------------------------- ********************************************************** * Server-side * * .------------------------------. * * | / | * * | | | Front * * | +-> +-- BeatRevolution.db | .-----. * * | +-> +-- scoreboard <---------+-+------->| ESP32 | * * | +-- html/table.html --+ | | '-----' * * | +-- chart.py <-+ | v | .----------. * * | +-- charts/ \ +->*-+----->| Webbrowser | * * | | | | '----------' * * | +-- asterisk.txt | * * | +-- bad_apple.txt | * * | +-- bloody_purity.txt | * * | +-- ... | * * '------------------------------' * ********************************************************** On an abstract level, the ESP32 plays the songs and runs the game, and the server saves scores and provides a song's list of notes (AKA the song chart). This section details the server side. At the top of file tree, there is **(1)**, a SQLite database called `BeatRevolution.db`, **(2)**, a scoreboard script `scoreboard` written in Python that saves & handles all the scores, and **(3)**, an endpoint `chart.py` where the ESP32 can GET a song chart from. The file structure is shown above as a block diagram. ### `BeatRevolution.db` The database contains a table called `new` whose purpose is to save scores. It has these columns: * `t`, a timestamp: used when sorting by most recent scores * `song`, an integer: the song index * `user`, text: the player's username * `score`, an integer: self-explanatory For example: `t` | `song` | `user` | `score` -----------------------|--------|----------|--------- `2019-05-17 20:06:43` | `3` | `DDRFAN` | `63000` `2019-05-17 19:54:05` | `4` | `DDRFAN` | `50000` ... | ... | ... | ... ### `scoreboard` The `scoreboard` endpoint is a Python script that saves and serves scores in the database with POST and GET requests, repsectively. The first responsibility of `scoreboard` is to accept POST requests that contain score information. Three parameters are expected: * `song`, the song index * `user`, the player name * `score`, the score the player achieved for the song If a POST request is sent to the endpoint with the above form parameters properly defined, then that score will be recorded into the database. The second responsibility is to get a list of high scores that BeatRevolution can display at the Song Selection screen. `&esp32` must be appended to the url when fetching score lists from the microcontroller. The scores are returned playername first, then score, separated by comma; further pairs of player and score are delimited by semicolon. For example, here is a list of three players' scores returned in the body of the response to the ESP32: ~~~~~~~~~~~~~~~~~~~~~~~~~~ plaintext Z,938000;QQ,844740;ALMONDS,691050 ~~~~~~~~~~~~~~~~~~~~~~~~~~ The reason the `esp32` argument is required is for the third responsibility: the `scoreboard` endpoint may be accessed from a webbrowser to view the scores. The endpoint will return the scores formatted in chic HTML & CSS. The HTML is rendered dynamically from the template located at `/html/table.html`. Go ahead and take a look inside. The Python method `render_template` in the scoreboard script takes a list of keyword arguments and looks for the syntax `{{ foo }}`, and then replaces each instance of `{{ foo }}` with the value associated with `foo` according to the keyword args. It resulted from the constraints of the 6.08 sandbox server and was inspired by [jinja templates](http://jinja.pocoo.org/). For both of the two GET requests described above, one may use the following parameters to change how the scores are sorted: * `song`: only get scores for a certain song index * `user`: only get scores for a certain user * `sortby`: can be `top` or `new`, returning the high scores or the most recent scores, respectively ### `chart.py` When `Game::load` is called, the microcontroller talks with `/chart.py` by giving it the following parameters: * `song`: The song index of the song just chosen by the player. The list of songs needs to be maintained in `charts.py` as it does on the ESP32. * `side`: Either `0` or `1`, which represent the left side and the right side respectively. The reason these are numbers rather than `l` and `r` or `left` and `right` is that it enables us to abstract the code easily with a simple for loop. `chart.py` will take the list of notes saved in one of the `.txt` files (see Charts for more information) and return the song duration, the offset, and the beats per minute, followed by a list of all the notes. With the rest of the notes left out, a typical example would be: ~~~~~~~~~~~~~~~~~~~ plaintext 147000,1503,155:6193,d;6774,u;8129,d;... ~~~~~~~~~~~~~~~~~~~ The song duration is 147 seconds, the offset is 1503 ms, and the BPM is 155. You can see that the notes are time-direction pairs. Display ------------------------------------------------------------------------------- The `Display` class handles the arrow display during the gameplay state. Each LCD screen is associated with its own `Display` object to separate the charts for the left and right hands. On initialization, a `Display` object is passed a pointer to a tft object along with a chip select pin indicating the screen corresponding to that object. For convenience, each `Display` object also contains a bijective map between directions stored in `char` form and their corresponding `int` forms for easy indexing. The `Display::load` function passes all the information specific to a song. This includes the following: * tempo, in beats per minute * offset, indicating the time at which the first note of the song occurs relative to the beginning of the track * rate of arrow movement, in pixels per ms * a pointer to the list storing the correct times of all arrow hits for that song * a pointer to the list containing the correct directions of all arrow hits for that song * a pointer to the list where index $i$ indicates whether note $i$ was hit along with whether it was a perfect or decent hit * a pointer to the variable storing the current score * a pointer to the variable storing the current combo length Subsequent display changes during the gameplay state require all of this information. When the song starts, `Display::start` is called to set the start time of the song to the device's internal `millis()` clock. This allows the device to make calculations relative to the starting time of a particular game. `Display::print_song` is called to print the name of the song on the bottom of the left screen, and `Display::update_score` is called to print updated scores and combo lengths on the bottom of the right screen. There are two `Display` objects within the `Game` class, and exactly one of the above functions is called on each `Display` to distinguish between the left and right screens. Helper functions for `Display::process` are as follows: * `Display::draw_arrow(char dir, int x, int y, uint16_t color)` draws an arrow centered at pixel $(x,y)$ pointing in the direction indicated by `dir` and in color `color` * `Display::calc_center(int dir, uint32_t beat, uint32_t timer)` calculates the current $y$ coordinate of an arrow given its time (`beat`) in the chart and the current time `timer`. If the note corresponding to this arrow is too far in the future to be displayed, or if the note has already passed and was marked as a missed note, this function returns negative values to indicated that behavior. * `Display::translate_arrow(char dir, int x, int y, uint16_t color)` translates the arrow of direction `dir` centered at $(x,y)$ one pixel up the screen. It blanks the lower one-pixel-thick boundary of the arrow and draws the one-pixel-thick upper boundary of the arrow shifted one unit up, minimizing the number of pixels changed at each step. This eliminates the flicker seen when calling `Display::draw_arrow` to blank out the previous arrow location and completely redraw the arrow in its new location. * `Display::find_color(uint32_t beat)` returns the color of a note occurring at time `beat` according to the DDR color scheme. It uses the bpm (tempo) of the song along with the offset time to determine whether `beat` falls on the beat, the half-beat, the quarter-beat, or the triplet subdivision in the song. These have specific corresponding colors in DDR for the player's ease in anticipating the note, and the same system is used here. * `Display::glow_arrow(uint32_t full_bright, int8_t accuracy)` is used to create the white or green glow-fade arrow when a note is hit. `full_bright` indicates the time at which the hit was detected, or the starting time of the glow-fade. `accuracy` indicates the accuracy of the hit (perfect, decent, miss). The function returns the current color of the glow-fade arrow based on a linear scale with endpoints at pure white/green and the default dark gray. If too much time has elapsed and the arrow has completely faded to gray, the function returns $0.$ `Display::process` runs through the list of notes in the chart and displays up to $10$ corresponding arrows on the screen at a time. The arrow disappears from the screen if it has been hit, and a glow-fade arrow flashes on the screen to give visual indication on the quality of the hit. To ensure that the left screen displays only the information for the left saber (and same for the right side), this function first writes the chip select output pin to low before making any changes to the display, and it sets the pin back to high after making all the necessary changes. Motion Detection ------------------------------------------------------------------------------- Motion detection in game is handled by the `Saber` class, which reads in and processes IMU acceleration data. The `Saber` class uses the modified MPU9250 library class and the chip select pin. `Saber::load` accepts pointers to times and directions of notes in the chart just like `Display` so that the two classes can share information. `Saber` also accepts pointers to detailed score and combo information for ease of sharing information between classes. `Saber::start` starts the internal ESP32 timer for note detection. Most of the work is then performed by `Saber::process`, which reads new acceleration data into a cyclic buffer array every $4$ ms. This acceleration data is used to match detected player movement to the expected movement at that point in time. Key helper functions for `Saber::process`: * `Saber::match(uint32_t expected_time, char expected_dir)` attempts to perform the actual matching of detected movement to expected movement. It acts like a state machine with four states. For the sake of example, suppose that `expected_dir` is up. In this case, we expect to first observe a large acceleration in the positive z direction at the beginning of the movement, then a large acceleration in the negative z direction at the end of the movement. We start in the rest state. The function scans through the acceleration array starting at the oldest data point. If acceleration of sufficient magnitude in the expected direction is detected (the $z$ component is above the threshold), the function enters the second state. To minimize the effect of random spikes in data, sufficient time must be spent in the second state (more precisely, enough consecutive readings of the $z$ component must be above the threshold) before moving into the third state. If a reading below the threshold is detected too soon, then the function returns to rest state. Otherwise, after entering the third state, the function now checks for sufficiently large accelerations in the opposite direction (very negative $z$ accelerations). If such an acceleration is detected, the function proceeds to state four. Similar to before, in order to minimize the effect of random noise in data, sufficient time must be spent in the fourth state. If a reading below the threshold is detected too soon, then the function returns to state three. Otherwise, if enough time is spent in state four, the function reports a match. The type of the match will be determined by `Saber::hit_type`. * `Saber::get_movement_start_time(int correct_hit_index)` uses the current time, as well as the starting index in the cyclic buffer, to return the time that an acceleration in the input index was read. This function is used with an input of where `Saber::match` first enters state four. By experimentation, this index appeared to be the most accurate representation of the time of the intended human motion. * `Saber::hit_type` determines the accuracy of a hit after the hit time is calculated by `Saber::get_movement_start_time`. If the hit is within a small range (70ms in either direction) of the expected hit time, the hit is classified as perfect; otherwise, if the hit is within the range considered "decent" (120ms in either direction), the hit is classified as decent. If the hit is marked as "too early" (more than 120ms before the expected time), it does not count and the note is still available to be hit. Otherwise, if the hit is too late, it is recorded as a miss. The output of `Saber::hit_type` is then entered into the array storing hit quality, and this information is shared with the `Display` class. Score and combo information are also updated appropriately at this time. An additional function `Saber::calibrate` was written in an attempt to fix the IMU crashes. This calls edited library functions to quickly begin/restart IMU communication and calibrate for bias. User Interface ------------------------------------------------------------------------------- The `Interface` class handles interaction with the entire game, such as username selection, song selection, gameplay, high score display, and moving between those states. The main state machine is handled by `Interface::process`. It begins with the username state. When the username selection is finished, it moves into the song select state. From the song selection, the player can choose to go back to the username selection or forward to playing the song they had selected. In the middle of gameplay, players can choose to return to song selection, voiding their score. When they finish the song, players can see a breakdown of their score, include number of perfects, decents, misses, and max combo. From there, they return to the song seelction screen upon a button press. The username selection is handled by `Interface::select_username`, based off of the Wikipedia search interface. Username selection occurs on the left display, whereas instructions for selection is displayed on the right. Short button presses indicate moving from character to character based on indexing through a char array of potential characters. A long left press locks in the currently selected button, and increases the username character index. We continue to fill the username character slots up to a maximum of 8 characters. A long right press indicates that the player is finished selecting their username. The username is stored in a private variable to be displayed on other screens. Song selection is handled by `Interface::update_song_display` on the left display. It handles how the song list is displayed as the player scrolls through the list. Our implementation displays the selected song in a bright white color in the middle of the display and in size 2 font, whereas other songs above and below are displayed in a faded color (hand-picked grays) and in size 1 font. On the right display of song selection, `Interface::display_high_scores` shows a list of high scores for the selected song. It pulls the list of high scores, then separates it into players and scores, displaying them side by side. After the player finishes the song, `Interface::upload_score(uint16_t score)` sends a post request to our server to send the score. Then, `Interface::display_play_data` handles how the player's score and score details are displayed. On the left display, it shows the player's score. On the right display, it shows a score breakdown of the max combo, the number of perfects, decents, and misses. The helper functions in the `Interface` class are as follows: * `Interface::select_username` uses two helper functions. `Interface::display_warning(char* warning_message)` displays a warning to the player if they don't adhere to the username selection rules. It contains two warnings: "Maximum character limit!" and "Can't have an empty username!". `Interface::clear_warning` overwrites the warning message with a background colored rectangle to remove it. * `Interface::update_song_index(int flag1, int flag2)` handles button presses changing the selected song for `Interface::update_song_display`. It doesn't allow the player to go past the first or last songs. * `Interface::get_all_high_scores` calls `Interface::get_high_scores(int index)` for all song indexes and stores them into a high scores variable. * `Interface::get_high_scores(int index)` is a helper for `Interface::get_all_high_scores` and is used separately after a player finishes a song to update the new high scores of that song. * `Interface::clear_screens` paints fills both screens with the background color to easy switch from state to state. Button ------------------------------------------------------------------------------- The `Button` class was adapted from the classy button exercise in 6.08, with some edits made to include a "timeout". It handles the state machine of how the button operates and accounts for debounce. Button operations consist of either no press, a short press, or a long press. We use two buttons in our game to handle username selection, song selection, and moving between states of the game. The two functions within `Button` are as follows: * `Button::read` reads the highs and lows of the pin, then indicates whether or not the button is currently being pressed. * `Button::update` has 5 states: resting, starting debounce, short press, long press, and ending debounce. We first `Button::read` the current press state of the button (whether or not it's being pressed). We then determine if the button state has recently been updated, and if not, reset its state to resting. The reason for this timeout is that we're not necessarily reading the buttons all the time, and we want to prevent strange behavior if a button enters non-resting state and then isn't read again for several seconds. In the rest state, if the button is pressed, then we go to the starting debounce or tentative press state. If the button is held for long enough, we move to the short press state. If the button is let go in this state, we move to the ending debounce state. In this state, if the button is let go for long enough, we return a short press, otherwise, return to the short press state. If the button is held for long enough in the short press state, we move into the long press state. If the button is let go for long enough afterwards, we return a long press, otherwise we return to the long press state. Power Consumption =============================================================================== Power was not much concern as we intended or system to remained plugged in. If future work would like to create a battery-powered system, the following pilot information may be useful. Using a USB current measuring device, we crudely estimated current and power consumption of the components. All numbers are approximate. * ESP32 SoC: $0.52 W$ * Speaker: $0.26 W$ * Two TFT displays: $0.36 W$ * MP3 player and IMUs: $\leq0.06 W$ Team Members =============================================================================== * Qi Qi * Wanlin Li * almonds@mit.edu * Diana Nguyen