Gesture recognition

Spring 2019

The questions below are due on Sunday March 31, 2019; 11:59:00 PM.
 
You are not logged in.

If you are a current student, please Log In for full access to the web site.
Note that this link will take you to an external site (https://oidc.mit.edu) to authenticate, and then you will be redirected back to this page.

Back to Exercise 07

Music for this page

In this design exercise you are to build upon the cross-corelation infrastructure we developed in Week 06 in order to create a system that can recognize user gestures in a 2D plane.

The system should allow the user to record and remember at least three gestures and then compare those gestures to later unknown input gestures and determine which one is the most likely. For example, you teach your system what the Letter's A, B, and C look like and then later you make an "A" symbol, and it an recognize this.

To help you get started/motivated, one design for this system is shown below and could recognize four different gestures:

  • up, a one-dimensional slide "up" along the x-axis
  • down, a one-dimensional slide "down" along the x-axis
  • up-down, a back-and-forth motion
  • up-down-up, a back-and-forth-and-back motion

The state machine of the system is shown below, and allowed a user to both "teach" and "compare" gestures. Data was stored in a database on the server for persistence and so that the system didn't need to be retaught every time.

State Machine for this System

We'd recommend using just one axis of the accelerometer in order to keep the correlation simple and one-dimensional.

While we don't require you to use the state machine diagram above, this approach has worked well in the past.

In Summary, the system should:

  • Allow teaching and long-term (i.e. database) storage of three or more gestures based on 2D movements, though you only need to use one-dimension of accelerations for comparison
  • Allow comparison of uknown User-input gestures to the stored library of programmed-gestures using cross-correlation and returning of the result with the highest probability
  • Exist in a fully-operational single script (can't have a script for teaching gestures and one for comparing gestures)

This design exercise has come out over spring break to allow people more time to think it through. As always, please feel free to post stuff on Piazza.

The video should demonstrate teaching of gestures as well as comparison of gestures.

Any late factors are based on the timestamp for the last time that the URL is entered on the page or when the code is uploaded or when a comment is added/changed. If you change your URL after the deadline you will incur a late penalty. If you upload code after the deadline, you will incur a late penalty. If you comment after the deadline, you will incur a late penalty. One-Week Extensions DO NOT APPLY Design Exercises.

You must upload ALL CODE that you used in your design in one compressed folder This includes both server-side and embedded side. IN ONE COMPRESSED FOLDER. The system only takes the most recent upload, so if you upload a server file and then your embedded file, that will cause issues. Failure to upload code will result in signifcant point loss. This includes if you just forgot. Screenshots of your file's last modified date in My Computer are not evidence that the file has not been changed. They must be uploaded.

Problem Status:
A Python Error Occurred:

Error on line 2 of python tag (line 77 of file /S19/ex07/gestures):
    kerberos = cs_user_info['username']

KeyError: 'username'

Enter the url for the video

SUBMIT ALL YOUR CODE AS A ZIP FILE BELOW (DOUBLE CHECK THAT YOU HAVE ZIPPED IT CORRECTLY). YOU WILL LOSE POINTS IF ALL FILES ARE NOT PRESENT!

 No file selected

Enter any comments you may want us to know about. For example, if you started this exercise but don't want it graded, make a note here. Please hit submit on this question even if you don't have any comments.

Back to Exercise 07



This page was last updated on Monday March 18, 2019 at 11:53:22 AM (revision f18ecfd).
 
Course Site powered by CAT-SOOP 14.0.4.dev5.
CAT-SOOP is free/libre software, available under the terms
of the GNU Affero General Public License, version 3.
(Download Source Code)
CSS/stryling from the Outboxcraft library Beauter, licensed under MIT
Copyright 2017