**Team 65: Flappy Chicken (aka Flap Harder)** System Overview =============================================================================== Our objective was to build upon the hit game, Flappy Bird, to create a multiplayer, real life game: Flappy Chicken. Players can physically control the chicken with two different methods: a flapping motion, and a joystick option. The multiplayer option allows you to compete against friends. The current top ten players of all time get a spot on the leaderboard, but you can always see your personal high score and most recent score on your own controller, displayed on the LCD at the end of each game session. ![Figure [granada]: Overall System Design](./figures/Overall_System_Diagram.png width="400px" border="0") Our system consists of three layers of gameplay: server, console, and controller. The “game server” runs on a single laptop, to which multiple players’ laptops can connect via IP address and port as “game consoles.” Each console then acts as a proxy server with a socket listening on its own public IP address. Each player’s controller, consisting of an ESP32 microcontroller and other parts (see Hardware Documentation section) on a breadboard, then talks to that player’s console via the socket, sending up data when the controller detects a flap. The consoles, in turn, send and receive information from the main game server, which oversees aspects of the multiplayer game that are shared among all players. In addition to these three layers, each player’s console and controller must talk to the database that stores the scores associated with each user. This database is hosted on the 6.08 server; game consoles use POST requests to post the score associated with the current username to the database, and game controllers use GET requests to get the most recent and high score of the current username to display on the LCD. Software Documentation =============================================================================== Our codebase consists of three main divisions, as well as the *hiscores.py* file. This file lives on the 6.08 server and contains a request handler for GET and POST requests, as well as methods for interfacing with the user score database. The three main divisions of the codebase are the *game* folder, the *sp_game* folder, and the *controller* folder. The *controller* folder contains all of the client-side C++ code running on each player’s ESP32 controller - this code is independent of whether the player is participating in a single or multiplayer game. The *sp_game* and *game* folders contain the Python server-side code for running single and multiplayer games, respectively. We chose to leave the *sp_game* folder in the codebase because the code for running a single player game is a lot simpler and was used as an intermediate step to developing the multiplayer version, but it is also possible to run a multiplayer game with only one player. In fact, the *game* folder version will work for an arbitrary number of players. At a high level, to run a single player game, one must run *sp_game/server.py* with Python 3 from a terminal on the laptop being used as a console. They must then upload the code in *controller.ino* to their correctly wired controller (see Hardware Documentation section) and traverse the menu displayed on the LCD monitor until they see a game window pop up on their console. To run a multiplayer game, one must instead make sure *game/mp_server.py* is running on the laptop designated as the game server. Then one must run *game/mp_client.py* from a terminal on the laptop being used as a console. As soon as one player’s controller, with the *controller.ino* code uploaded, connects to their console, the game will be playable. An arbitrary number of players can join the game on their own consoles, and when the first player flaps their controller (or flicks their joystick, if in joystick mode), the multiplayer game will begin. Game Console GUI (Single and Multiplayer) ------------------------------------------------------------------------------- Both the *sp_game* and *game* folders contain the *block.py*, *constants.py*, and *background.py* files as well as a folder called *sprites*. These files are responsible for visualizing the main components of the game window, including birds, pipes, and background, as well as providing a framework of constants for both games to work with. The *block.py* file contains sprite classes for different objects within the game. The Pipe class initializes scaled pipe objects of equal widths but random heights. Each pipe is placed on the right edge of the screen and moves to the left at a speed defined in *constants.py*. The Bird class initializes a bird object with a random chicken sprite from the *sprites* folder (examples pictured below) and assigns its position. The bird is able to move vertically based on gravity and the player’s controller flaps. However, it cannot move horizontally. When it collides with a pipe or goes out of bounds of the window, it dies and the game ends. The *constants.py* file defines game constants to define colors as well as dimensions and speeds of sprites. The *background.py* file provides an initial version of the game window background that we used to test horizontal scrolling functionality, although the final version of the game simply uses a still image behind the pipes. The classes in these files comprise the basic components of the game, but they interact with *game.py* for single player, and *mp_server_game.py* and *mp_client_game.py* for multiplayer, the details of which will be further discussed in the following sections. ![Figure [granada]: Example Chicken Sprites](./figures/Chickens.png width="400px" border="0") ![Figure [granada]: Bonus Chicken Sprites](./figures/Special_Chickens.png width="400px" border="0") ![Figure [granada]: Example Game Window](./figures/flappy_game.png width="400px" border="0") Single Player Server and Console ------------------------------------------------------------------------------- ![Figure [granada]: Single Player State Machine](./figures/Sp_Console_State_Machine.png width="400px" border="0") The state machine of the single player game outlined in the figure above is implemented in *sp_game/game.py*. This file uses the Pygame library as well as the previously discussed code in *block.py*, *constants.py*, and *background.py* to implement a Game class with an internal loop during the Gameplay state. While this Gameplay loop is running, events are triggered by flaps, whether from a test client using keyboard input or from a player using their ESP32 controller, and allow the game to progress until the bird dies. The *server.py* file instantiates a Game, and listens for flaps on a socket listening on the laptop’s public IP address. As an intermediate stage, we used *mac_client.py* (for Mac OS) and *windows_client.py* (for Windows) to test the server-side game functionality using keyboard input. Running the appropriate Python client at the same time as the server will allow the up-arrow key to substitute for flapping. When using the ESP32 controller as a client instead, flaps are sent to the socket using the Wifi.h library, triggered by joystick flicks in joystick mode or physical flaps of the breadboard in flapping mode. When a player dies, triggering the transition from state 1 to state 2, the *game.py* file uses helper functions in *sp_game/scores.py* to POST the username and score to the appropriate URL on the 6.08 server (in this case http://608dev.net/sandbox/sc/barryxu2/hiscores.py), which according to the request handler and helper functions in *hiscores.py* stores the user and score pair with a timestamp in the database. Another helper function in *scores.py* is then used upon entering state 3, sending a GET request to the same *hiscores.py* file, which searches the database and returns the top ten user scores for display on the leaderboard. Multiplayer Server and Console ------------------------------------------------------------------------------- Multiplayer functionality can be divided into two components: the game server, or the server-side, and the game console, or the client-side. We use PodSixNet (https://github.com/chr15m/PodSixNet), a wrapper around Python’s built-in asyncore library, to handle multiple connections. Both the server- and client-sides share a similar structure. The server is implemented across two files, *mp_server_game.py* and *mp_server.py*. The *mp_server_game.py* file processes information received from clients. It handles all in-game events that concern all players, such as updating the bird positions and handling collisions. It contains the MPGame class, which implements the server’s representation of a multiplayer game in the form of the state machine below: ![Figure [granada]: Multiplayer Server State Machine](./figures/Mp_Server_State_Machine.png width="400px" border="0") In the Start state, the game waits for clients to connect to the server. When any client flaps, this signals the start of the game. To transition into the Gameplay state, it invokes the *load_game* method, which sets up gameplay elements such as sprites and scores for the start of a fresh game. In the Gameplay state, all game events like collisions and flaps are processed at each timestep, where timesteps run at 60 Hz. Sprite positions are also broadcasted to clients so that they can update their local representations on the console. This allows for client-side GUI, though depending on the strength of WiFi, there may be lag between server and client sprite representations. When a player collides with a pipe or goes out of bounds and dies, the server tells the corresponding console to end its local game, which triggers a state transition in the console state machine (explained later) as well as a POST request to *hiscores.py* on the 6.08 server storing the user’s score, with the help of helper methods in *scores.py*. When all players in the multiplayer game have died in this manner, the server moves to the Leaderboard state, telling all consoles to move to their own Leaderboard states in which they display the current top ten scores (retrieved via a GET request to *hiscores.py* in *scores.py*) for a short period. After this delay, the server returns to the Start state and the game cycle begins again. Note that during the delay, the server will ignore all client flaps so as to avoid a buildup in the queue at the start of the next game. The *mp_server.py* file handles connections, and contains the ClientChannel and MPServer classes. The ClientChannel class is the server’s representation of a single connected client console. It contains various callbacks which are invoked upon certain events, such as a client connecting, disconnecting, or sending information up to the server. The MPServer class inherits from the PodSixNet.Channel.Channel and MPGame classes. It runs the server to which clients connect and is in charge of all communications between server and client. It handles client connections and disconnections via network callbacks, while client messages are handled through event callbacks. Similarly, the client console is implemented across two files, *mp_client_game.py* and *mp_client.py*. The *mp_client_game.py* file contains the ClientGame class, which uses Pygame to render the game similarly to the single player version detailed in prior sections. It implements the console’s representation of a multiplayer game in the form of the state machine below: ![Figure [granada]: Multiplayer Console State Machine](./figures/Mp_Console_State_Machine.png width="400px" border="0") State transitions in this state machine are triggered by game server, as previously explained. In the Start state, a starting image is displayed. In the Gameplay state, the server updates sprites on each timestep and generates new pipes as needed, sending data down to the console. Pipe movement is computed locally, but constants are chosen to stay synchronized with the server. This is the cleanest solution, but weak connections can cause the console to become desynchronized from the server and thus other players’ consoles. All bird positions are computed by the game server and sent down to the console. In the End Screen state, an end image is displayed informing the player that they have died. This remains in place until all players have died, at which point the Leaderboard state is triggered and the current top ten scores are displayed for a short time. As explained above, flaps sent to the server are not processed during this short delay. After the delay, the Start state is reentered and a new game begins. The *mp_client.py* file contains the Client class, which inherits from the PodSixNet.Connection.ConnectionListener and ClientGame classes. It handles communications to and from the game server via callbacks; information received from the server is sent to ClientGame for processing such as updating sprites or displaying high scores, and flaps received from the ESP32 game controller are passed up to the game server. The console Client also instantiates a socket listening at the console’s public IP address, which blocks until a controller has connected. The details of the controller software will be explained in the following section. ESP32 Game Controller ------------------------------------------------------------------------------- ![Figure [granada]: Controller State Machine](./figures/Controller_State_Machine.png width="400px" border="0") The client-side controller code, written in C++ for the ESP32 microcontroller, is contained in the *controller* folder. The *controller.ino* file contains the main *setup()* and *loop()* functions that implement the LCD display state machine pictured above, while all of the other header and C++ files implement modularization in the form of our own custom libraries. The button handler library, consisting of *buttons.h* and *buttons.cpp*, contains the Button class we have been using for the majority of the semester, providing debouncing and short/long press functionalities. The value of our button is updated in each loop of *controller.ino*, for up-to-date information in the state machine switch statement. The display library, consisting of *display.h* and *display.cpp*, contains all of the helper functions for drawing the LCD display screens at different states and transitions of the menu state machine, using the TFT graphics library. The flap detection library, consisting of *flapDetection.h* and *flapDetection.cpp*, is inspired by the step detection lab from early in the semester. It uses the IMU to detect whether a flap has occurred during a given loop iteration based on the x, y, and z-axis accelerations and certain empirically determined thresholds. These thresholds could easily be tuned to provide more or less flapping sensitivity as desired. To ensure that noise didn’t interfere too much with the acceleration magnitude readings, we averaged the current acceleration with the past two measurements at every timestep. This flap detector is called during every *controller.ino* loop that is in the Gameplay state and in flapping mode at the same time. The HTTP request library, consisting of *getRequest.h* and *getRequest.cpp*, provides methods for sending a GET request with a username to the *hiscores.py* file on the 6.08 server and storing the response containing the high score and most recent score for that user. This library is used on the transition from state 3 to state 4; when the user quits the Game state, the response to the GET request is used in displaying the stats for that user in the Stats state. The joystick handler library, consisting of *joystick.h* and *joystick.cpp*, interfaces with our joystick pins to read joystick movements along the LR (left-right) and UD (up-down) axes. This is accomplished with a JoyAxis class that handles low-level readings on a single axis of the joystick, and a JoyStick class with a LR JoyAxis and an UD JoyAxis, used for communicating with the main loop in *controller.ino*. The directions of each joystick axis are updated in each loop of *controller.ino* for up-to-date information in the state machine switch statement, similar to the button value updates. The username creation library, consisting of *username.h* and *username.cpp*, implements a UserNameMaker class, with methods for scrolling up/down through the letters of the alphabet at a single index of the username as well as scrolling left/right through the indices of the username. Because of this left/right and up/down editability, any of the five characters in the username can be edited at any time while in the Username state. Hardware Documentation =============================================================================== The physical components of Flappy Bird include an ESP32 microcontroller, IMU, LCD display screen, button, joystick, power management board and battery. The only new piece of hardware outside of those provided in the class was the joystick. The joystick we used can be found at https://www.adafruit.com/product/245, the documentation for which can be found at https://www.parallax.com/sites/default/files/downloads/27800-2-Axis-Joystick-Documentation-v1.2.pdf. ![Figure [granada]: Example Wiring](./figures/Hardware.png width="400px" border="0") ![Figure [granada]: Functional Block Diagram](./figures/Functional_Bock_Diagram.png width="400px" border="0") Power Management =============================================================================== In order to calculate our power management, we make a few assumptions. First, we assume that the power consumed by the button is negligible compared to the other components of our system. Secondly, we are assuming that the current draw and voltages will remain roughly constant for every component. Based off the datasheets for each component, the following voltages and currents were found and listed in the table below along with their power consumption. ![Figure [granada]: Power Consumption by Component](./figures/Power_Table.png width="400px" border="0") We will assume that the battery we are given is an ideal battery, and remains at a constant voltage and energy capacity throughout its lifetime. The battery is rated for 1500 mAH and its nominal voltage is about 3.7 V. Our battery is connected to a boost converter which then gives us a clean 5V for our ESP. We will assume that the boost converter is also ideal, and consumes no power when converting voltage. Taking all of this into account, our current system should ideally run for 2.466 hours before the battery dies. This calculation was done by simply finding the power consumption of all the components and adding those together. We then found the power the battery provided, and divided this by the total power consumption of our components. Designing a power management system that would show substantial benefits is difficult. The ESP32 is constantly running, constantly performing calculations and doing work, the LCD is constantly on, and the IMU is constantly being used throughout the game to detect flaps with its accelerometer. One way to save energy is to turn off the IMU if the user is playing in joystick mode. However, the power consumption from the IMU is very small compared to the power consumptions of the ESP32 and LCD, so the lifetime of the system would change at a negligible rate (less than 32.5 seconds total). One could utilize a photovoltaic resistor, similar to what we did in lab 7, in order to lower the power consumption of the LCD using pulse-width modulation to dim the screen based on the ambient lighting. The users do not use the LCD during gameplay, so another thing we could have implemented is dimming the display during the time the user is playing. Challenges and Debugging =============================================================================== One of the design challenges we faced over the course of this project was differences in operating system compatibilities - some of our initial implementations worked only on the operating system on which they were developed. Making our system compatible with both Mac and Windows operating systems was the rationale for a few of our decisions: the use of the os.path library in loading image files and the use of two separate keyboard event detection libraries for *mac_client.py* and *windows_client.py*. We also found that a Mac computer could host our system on its public IP address and allow clients to connect without a problem, while a Windows computer needed to have a firewall setting turned off first. Another design challenge was that the LR axis of our joystick seemed more inaccurate than the UD axis in correctly identifying direction of movement. This was the rationale behind our decision to use only the UD axis for flaps and to save the LR axis for less time-sensitive things like username letter scrolling. We would also probably not recommend the use of this particular joystick model for future teams that depend on reliable 2-axis functionality. The biggest design challenge we faced, one that we didn’t have time to fully solve, was that our system is not very robust to changes in how powerful of a computer is being used or how fast the WiFi is. Less powerful machines and slower WiFi caused some lag in the rendering of the game, and some latency in posting and retrieving scores from the database. Final Product: Example Video =============================================================================== Below is an example of a multiplayer game in which the player on the left uses joystick mode and the player on the right uses flapping mode. ![Figure [granada]:](./figures/video.mp4 width="400px" border="0")