The Server Side

Error on line 8 of Python tag (line 9 of source):
    kerberos = cs_user_info['username']

KeyError: 'username'

1) Overview

So far in 6.08 we've been working solely on our microcontroller. In Lab 02B, we started to work a bit with one of our class servers, but again, it was only from the perspective of our microcontroller. Today that changes. We'll take a look at how we can deploy scripts on a 6.08 class server and in future weeks use this quite a bit.

2) Python on the Server

We have a separate class server from the one we host the course website on, called 608dev-2.net. We used its sister machine 608dev.net in Lab 02B, you might remember. We've set up a sandbox on this server where you can place Python files to handle incoming HTTP requests. The way this will work is that you'll create a Python file containing a definition for a specific function called request_handler, which will take in one argument called request, a Python dictionary which will contain information about the specific HTTP request. The job of your request_handler function is to respond as needed to each request—and this will vary from exercise to exercise and project to project.

For example, if you'd like to simply echo back the structure of the request dictionary provided you could create the following function:

def request_handler(request):
    return str(request)

This function needs to be placed inside a Python file and placed onto the server. We'll get into how to do that below, but assuming you do place this file onto the server in the correct way, it will now be accessible to the outside world—and when you visit it, the server will use request_handler as you defined it to process the incoming HTTP request.

For example, if you placed the above function definition in a file called test.py, saved it onto the server, and then went and visited the appropriate URL in your web browser, you'd get the following data returned to you:

{'method': 'GET', 'values': {}, 'args': []}

If instead you visited it in the browser and appended some query arguments to the URL such as ?x=5&y=4 you'd get instead:

{'values': {'y': '4', 'x': '5'}, 'method': 'GET', 'args': ['y', 'x']}

If you were to POST to the site, other different keys would be available in the request dictionary for your use. We'll get more practice with the exact structure of this request dictionary in coming exercises and labs. Now, though, we want to focus on how we can place files onto the server.

3) The Sandbox

Each student in 6.08 has had an account on 608dev-2.net made for them.

• Your username on 608dev-2.net is: <div><font color='red'><b>A Python Error Occurred:</b><p>

  Error on line 1 of Python tag (line 75 of source):
      print('%s' % (kerberos,))
  
  NameError: name 'kerberos' is not defined
  

  <p></font></div>
• Your password on 608dev-2.net is: <div><font color='red'><b>A Python Error Occurred:</b><p>

  Error on line 1 of Python tag (line 76 of source):
      print('%s' % (cs_user_logos[kerberos],))
  
  NameError: name 'kerberos' is not defined
  

  <p></font></div>

When you log in to the machine, it will put you into your personal home directory. From here you are capable of carrying out a limited number of operations—such as building and editing the file structure, creating and moving files, and other things. There is no GUI to do this however, so you'll need to do this through the terminal. The course staff feels getting some exposure to working in a *Nix-type shell is a useful skill and so we'll expect you to gain some fluency in this in order to carry out homeworks and labs. If you've never done this before, it isn't too bad. We're really only going to be using around four or five commands total and we'll introduce them all today (for the most part).

3.1) Getting a Terminal Working

For Mac and Linux folks, you'll be using your terminal. It should already exist. Either control-alt-T or search for it in the Task bar to bring it up. (For Mac users, you can search for 'Terminal' in spotlight (command-space).

Windows users, unless you already have an alternative you like to use, we are going to recommend that you download and install Cygwin with openssh support. It provides a *nix like interface that will enable you to conveniently sftp and/or ssh into our class server as needed. In general you can follow the instructions on the site, but we'll need to make sure you install openssh support, so:

1. Download the setup.exe for your machine as it says on the site (linked above).
2. Follow the instructions, choosing "Install from Internet" and place in your "C:\cygwin" directory, allowing all users to have access and support for Linux defaults
3. Select a download mirror (I usually just pick the first one and it works at least when on MIT's campus)
4. When it prompts you for additional packages, type in "openssh" into the search field of the package selection window. On the left side of the things that come up, click on the "skip" icon to change the two openssh packages to checkboxes. Then press Next.
5. Wait for installation to finish and then you should be good!

Once it is installed, proceed to the following section. If you have installation issues, please post on Piazza and we'll help you figure it out.

3.2) How to Think About the Terminal

If you're comfortable working in a terminal, feel free to skip this section. For those of you who are not (and that is totally fine, 6.08 is a judgement-free zone), we're going to talk about the basics.

First when you're in a terminal, just remind yourself that it is simply a textual representation of the GUI file-managers we've all seen on Windows, Mac, and *Nix. When you open up a terminal, it will often place you into you home directory¹. To see where you are in the terminal you can always type the following command (followed by hitting enter), which asks for Present Working Directory:

pwd

On my Mac, for example, it spits back the following: /Users/jodalyst. The response you get will probably be slightly different. To see what files and folders are present in your current directory you can type the following (again, followed by hitting enter):

ls

Depending on how clean/organized and/or full your current directory is, either a little or a lot of stuff might fly by. These are all the files and folders currently present in the directory you're in. This just showed you in text what you'd see in your File Manager/Documents/My Computer GUI window.

To change into a directory you will use the Change Directory command or cd followed by the directory you want to move into. This is the same as when you double-click on a folder in a graphical file manager.

cd directory_to_move_into

My Documents folder is within my home directory, and within my Documents folder is my Arduino folder. I could therefore do the following:

cd Documents/Arduino

Following that if I then type pwd I'll get back /Users/jodalyst/Documents/Arduino.

If you want to go up one directory level you can type:

cd ..

If you want to go up two levels, you can type:

cd ../..

If you want to move back to the previous directory you were in (whatever that was), you can type:

cd -

If you just want to go/return to your home directory, you can type:

cd 

Or more explicitely (using the ~ symbol which means "home")

cd ~

If you're currently in /Users/jodalyst/Documents/Arduino and you want to move to a completely different directory you can type:

cd ~/other_folder/other_nested_folder

Please feel free to Google or ask on Piazza and/or in office hours for help with this. It can seem very weird at first.

3.3) Remote Server

Ok, now we've got a little idea about how to move around using the command line. So far we were doing it on our own local machine. Now we want to do it on a remote machine. There's several ways to do this. We'll go over how to do it with ssh and scp. You can also use sftp, but based on experience withs students the last two years, people have a hard time with sftp.

• ssh stands for Secure SHell, and is a way for you to get into the terminal on a remote machine. You can do anything you can do on your local machine on this remote machine through this (in theory...in practice we greatly limit you in 6.08 for the greater good). When logged in over ssh, you can make directories, cp files and even edit them (have to use a text editor, and your options are nano or vim)
• scp stands for SSH COPY and is a means of transfering files to and from one machine to another.

Let's get some experience with this.

4) SSH and SCP

One way to work with the server remotely is using the programs ssh and scp. You can think of ssh as a fully-fledged way of working directly on the server (as if it was your own computer). You can then use scp to move files between the server and your local machine. It acts like the standard cp command (which stands for "copy"), but can act between machines.

When using ssh and scp at the same time, I usually have two terminal windows open. One logged in using ssh to the remote machine, and one active in my local machine...that way I can move around one each and see what files are where.. With that in mind in one terminal window, let's log into the server using ssh. To do that you'd do enter the following:

ssh <div><font color='red'><b>A Python Error Occurred:</b><p>
Error on line 1 of Python tag (line 185 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined
<p></font></div>@608dev-2.net

It will prompt you for your password. As a reminder, your specific password is: <div><font color='red'><b>A Python Error Occurred:</b><p>

Error on line 1 of Python tag (line 188 of source):
    print('%s' % (cs_user_logos[kerberos],))

NameError: name 'kerberos' is not defined

If you successfully logged in, you'll get something similar to the following message:

Last login: Mon Feb 17 20:06:36 2020 from 73.69.114.36
<div><font color='red'><b>A Python Error Occurred:</b><p>
Error on line 1 of Python tag (line 194 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined
<p></font></div>@608dev2:~$

You are now in the server. For reference, you can exit at any time from this login session using the following command, which will return you back to your home computer.

exit

If you are disconnected from the internet while logged in over ssh, you'll get a "broken pipe" and get disconnected. You'll have to log in again.

Within the ssh environment on the server, all the commands we previously talked about will work similarly. For example, typing pwd right when you log in will spit back where you are in the server. Similarly, you could type ls, although as right now your remote home directory is empty, the command will return nothing.

For practice, let's make a directory by using another command, mkdir (Make Directory). Go ahead and run the command mkdir testdir. This will make a directory called "testdir" inside of where you are currently located in your file structure (your home directory on 608dev-2.net). Then, if you run ls, you'll see it show up. You've just made a directory! You have some power. Now let's move into it using the cd command....so do cd testdir. Then verify where you're at by running pwd (you should see /home/

Error on line 1 of Python tag (line 206 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined

We now know how to move around on our remote server and make directories. This will be critical to setting up our file structure for integrating with the web sandbox. Next, we'll need the ability to move files onto this remote directory, such as Python files. Back on your local computer, create a Python file called testo.py (use your favorite Python editor, we have no biases in 6.08) and inside of it create the following request_handler function definition:

def request_handler(request):
    return "Hello world."

Save testo.py to your home directory ON YOUR LOCAL COMPUTER. Windows-Cygwin users, you'll need to figure out where your *Nix-like home directory is. On my Windows machine it is under C:\cygwin64\home\jodalyst since my name is jodalyst, but that's me and your situation might be different.

Next navigate to where you placed this file on your local machine (use cd, etc.). Once there, we're now going to copy the file testo.py into the server. To do that run the following command over scp:

scp testo.py <div><font color='red'><b>A Python Error Occurred:</b><p>
Error on line 1 of Python tag (line 220 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined
<p></font></div>@608dev-2.net:~/

What this has just done is move the file testo.py from its specified location on your home computer into the home directory on the server. Move back into the ssh terminal on the server and move to your home directory (cd ~ for example is one command that will do this for you). When in your home directory type ls. You should see:

testo.py
testdir

This is telling you that you have a file and a folder in your home directory. Move that python file into testdir using the mv command:

mv testo.py testdir/

Pressing ls again from the home directory will reveal:

testdir

Running ls testdir to show what's inside testdir will show:

testo.py

Awesome. Now believe it not this file exists is world viewable. If you go to the following site in your web browser: http://608dev-2.net/sandbox/sc/

Error on line 1 of Python tag (line 247 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined

Hello World.

That is, incredibly, coming from the function you put on the server. Let's try another one. On your local machine, go ahead and make a second file called testo2.py. Inside of it place the following request_handler` definition:

def request_handler(request):
    return "Request Dictionary: " + str(request)

Save it locally then transfer it up to your server, placing it into the testdir directory. When done, you can visit it at http://608dev-2.net/sandbox/sc/

Error on line 1 of Python tag (line 257 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined

Request Dictionary: {'method': 'GET', 'values': {}, 'args': []}

This is that request dictionary we talked about earlier on this page. Right now you're just giving this thing a pretty simple GET request via the browser and it is responding with the resulting pretty simple dictionary, but we can spice it up a little bit:

If you added some query arguments at the end like so:
http://608dev-2.net/sandbox/sc/

Error on line 1 of Python tag (line 266 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined

Request Dictionary: {'method': 'GET', 'args': ['yo', 'hey'], 'values': {'yo': '11', 'hey': '5'}}

If you see this, then congratulations, your winner. This is basically how we'll deploy Python on the server in this course. You now have access to web-based inputs (both POSTs and GETs) and can have your Python respond to it accordingly!

It is important to remember that you have complete control over what you return. If you want to change how your returned text looks, we could add some in-line css for styling. For example, edit the local file so that it contains the following:

def request_handler(request):
    return """<b style="color:green;">Live your best life!</b>"""

Re-copy it via scp to the server and revisit the page (careful you'll overwrite...it doesn't matter here, but be careful in the future). Now when you revisit the URL in the browser, you should see some nice green text, encouraging you².

If you end up modifying some stuff on the server and want to get it back onto your local computer you can also use scp. To do that in your local computer's terminal do the following:

scp 
A Python Error Occurred:
Error on line 1 of Python tag (line 286 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined
@608dev-2.net:~/testdir/testo2.py.py ./testo2_backup.py

What this will do is copy from the specified directory on the server the file testo2.py and name it testo2_backup.py in your local directory on your computer!

So there you have it! A quick summary of some things you can do with ssh is shown below.

And remember in order to use scp, when on a terminal in a local computer's environment to transfer from the local computer to a remote one you do:

scp file_location/local_file_name username@remote_host:remote_file_location/remote_file_name

and to transfer from a remote computer/server to your local machine you do:

scp username@remote_host:remote_file_location/remote_file_name file_location/local_file_name

4.1) Editing On the Server

The class server is a completely command-line based entity. While it might seem like you can't edit files on the machine directly, this is incorrect. We have two editors installed for you. Nano and VIM⁴. Nano is the easier command-line editor to get started with. If you haven't used vim before, probably don't try unless you want to spend some time ramping up. Regardless, I'd only suggest using these editors if you're comfortable with them. For those curious below is a tutorial on how to use nano:

• Nano Tutorial

5) File Structure on 608dev-2.net

Regardless of how you manipulate files, in general, any Python file you place on the server will be served from its location relative to your home directory. For example, if you place a file right in your home directory, called fun.py, you can access it at http://608dev-2.net/sandbox/sc/<div><font color='red'><b>A Python Error Occurred:</b><p>

Error on line 1 of Python tag (line 368 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined

And if you made a file structure such that there were a series of nested folders like so:

home > project > integration > messaging

and you put a file called response.py in it, then you could access it at:

`http://608dev-2.net/sandbox/sc/<div><font color='red'><b>A Python Error Occurred:</b><p>
Error on line 1 of Python tag (line 379 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined
<p></font></div>/project/integration/messaging/response.py

In future weeks we'll discuss how to set up databases and serve static files, but for today, we'll just worry about placing Python files that handle HTTP request dictionaries in a stateless manner.

6) Writing a Request Handler Function

To finish this exercise off, you're going to need to write a request_handler function and then place it in the correct location on 608dev-2.net.

6.1) The Specification

The file should be placed under a directory called ex03 in your home directory and be called initial_test.py. The particular implementation of request_handler you need to implement should take in two arguments and carry out some math operations on them. In particular determine and repor the Least Common Multiple and Greatest Common Divide. Specifically:

• The function should expect two query arguments, x and y
• If an x and y argument are not provided the function must return the following: "You must enter an x and y value."
• If either x or y is not a number, the function must return "Both x and y must be valid numbers." If you're curious about how to try do that in Python consider using Try-Except logic. Check out the Python 3 documentation on how to handle errors.⁵
• The system will only work with integers. Treat non-integers the same as non numbers above.
• If a POST request happens, the function must return "POST requests not allowed."
• If everything is good, return the least common multiple and the greatest common divisor of the two numbers in the following format: "The LCM of {} and {} is {}. The GCD of {} and {} is {}." where the {} will be replaced with actual appropriate values.

When put all together, if your function is correct and you place it correctly (in the right location) on the site and you visit the following URL with specified query arguments like so: http://608dev-2.net/sandbox/sc/

Error on line 1 of Python tag (line 399 of source):
    print('%s' % (kerberos,))

NameError: name 'kerberos' is not defined

The LCM of 20 and 30 is 60. The GCD of 20 and 30 is 10.

Use the code checker below to get your request_handler function working. Feel free to use the test cases provided for local debugging (don't just debug in the code checker here).

def request_handler(request): pass

Finally, we must make sure that you've placed your code at the proper spot on the server. It isn't that we don't trust you, but we just want to make sure you're doing things right. Run the check below which throws some HTTP requests at the location on 608dev-2.net that corresponds to where you should have placed your file. You can ignore the text input-it doesn't get used.

If it returns a 0.00% that means you do not have the working function from above in a file called initial_test.py within a directory called ex03 in your home directory on the server.

¹though not a guarantee (click to return to text)

²While a lot of what we put on the server is intended for consumption by another machine, and therefore doesn't require pretty-looking fonts in different colors, you can directly create html-text for rendering in a browser as needed here (click to return to text)

³Also the best text editor in all of humanity. JK obviously, you use what you like best. No emacs is not installed on the sandboxed environment. (click to return to text)

⁴Sorry emacs people, I don't have it on this machine. This isn't because I don't like emacs (I don't like it though), but it was harder to get working in the jailed environment of the class server. (click to return to text)

⁵In Python it is often easier to ask for forgiveness rather permission...by this we mean try to do something and catch the failure if it doesn't work out. (click to return to text)