Application Layer Protocols for the Internet

(selected notes)

HTTP : HyperText Transfer Protocol

• Controls communication between web clients and servers
• TCP is underlying transport protocol
• Port 80 is HTTP communication socket
• HTTP is stateless:
• server does not maintain state information between requests
• compare this with FTP, which does maintain state.
• weigh advantages and disadvantages
• HTTP 1.0 is non-persistent:
• GET Request results in at most one file transferred as response
• Connection closed at end of response.
• HTTP 1.1 supports persistence but can specify non
• Persistence with pipeline: parallel connections; parse incoming file and issue new request as soon as embedded reference encountered.
• Three HTTP request methods:
• GET - response contains header info plus file
• HEAD - response contains header info only
• POST - send form parameters to server for processing
• Request consists of:
• Request line (method, URL, HTTP version)
• (optional) Header lines (preferences and negotiations)
• (optional) Entity body (form parameters for POST)
• Response consists of:
• Status line (HTTP version, status code, description)
• Header lines (file characteristics, cookie info, etc)
• Entity Body (message or requested file)
• Authentication
• Response is status code 401 plus header line specifying authentication method
• Browser responds by requesting ID/password from user or sending cached one
• Browser then repeats request and includes authentication info.
• Each subsequent request includes authentication info automatically (stateless)
• Cookies
• Used to mimic state in stateless protocol
• At first request, response contains Set-cookie header line
• Cookie info stored on client machine
• Subsequently, cookie info included on request header line
• Caching
• Store recently fetched files on client or on network between client and server
• Subsequent request will get response from cache.
• Important for reducing bandwidth demand.
• Critical that cached version is most up-to-date (not stale)
• Local cache
• File stored on client machine with modification time (from response header)
• utilize conditional GET
• On GET request, see if cached locally.
• If so, get mod time and include in "If-modified-since:" header line
• Server response is code 304 if not modified and file not transmitted
• Web cache
• AKA proxy server
• May be on client's local network
• May be elsewhere on Internet (see NLANR system of backbone caches)
• May be a hierarchy of caches
• each responds if possible
• else passes on request to next hierarchy level
• when response received, pass it on plus make copy to cache
• May be a cluster of caches
• eliminate duplicate copies by hashing
• hash value (computed from URL) identifies cache that has file

Socket Programming

Sockets are communication portals between client and server

Sockets are API between OS/network and application

Internet sockets implemented using either TCP or UDP

We'll focus on TCP

Guaranteed error-free and sequenced delivery

Connection-oriented stream service

Basic protocol for TCP-based sockets

3-way handshake to establish client-server connection

two way communication through connection socket

close connection socket

Some links on sockets programming in C/C++:

Course notes on client-server interaction. (www.cs.smsu.edu/~pete/csc465/spring97/notes/Chap21.html)

The socket interface

(www.cs.smsu.edu/~pete/csc465/spring97/notes/Chap22.html)

Example socket applications

(www.cs.smsu.edu/~pete/csc465/spring97/notes/Chap23.html)

FTP : File Transfer Protocol

Old as the hills

RFC 959.

Utilizes 2 TCP connections:

• Port 21: control (commands)
• Port 20: data (file transfer)

Control connection is persistent

Data connection is non-persistent

FTP server maintains state throughout session (e.g. current directory)

Commands are plaintext

Common client commands are:

• USER (send user ID)
• PASS (send user password)
• LIST (list files in current directory)
• CWD (change working directory on server)
• PWD (print working directory on server)
• RETR (retrieve file from server, name is parameter)
• STOR (store file on server, name is parameter)
• QUIT (logout)

Each command results in response from server: 3 digit code followed by message

Try it out (can at least log in)

Email

Components are:

User agent (mail reader)

Mail server

· holds received messages (mailbox)

· sends messages (outgoing message queue)

Protocol between mail servers: SMTP

Protocol between user agent and mail server:

· POP

· IMAP

· HTTP

· ??

Electronic trail Alice to Bob:

• Alice invokes user agent to :
• Address and Compose message
• Give send command
• Alice user agent send message to mail server message queue
• Alice mail server becomes client in SMTP connection with Bob’s mail server (if latter is not available, hold in queue and try later).
• Alice mail server transmits message to Bob mail server via SMTP
• Bob mail server puts message in Bob’s mailbox
• Bob invokes user agent to read message

SMTP : Simple Mail Transfer Protocol

• RFC 821

· TCP connection Port 25

Example SMTP session

(my commands are bolded)

> telnet csc.smsu.edu 25

Trying 146.7.45.212...

Connected to csc.smsu.edu.

Escape character is '^]'.

220 csc.smsu.edu ESMTP Sendmail 8.11.1/8.11.1; Mon, 5 Feb 2001 08:44:37 -0600 ()

HELO csc.smsu.edu

250 csc.smsu.edu Hello csc.smsu.edu [146.7.45.212], pleased to meet you

MAIL FROM: <dps910f@smsu.edu>

250 2.1.0 <dps910f@smsu.edu>... Sender ok

RCPT TO: <pete@csc.smsu.edu>

250 2.1.5 <pete@csc.smsu.edu>... Recipient ok

DATA

354 Enter mail, end with "." on a line by itself

TO: Charlie Brown

FROM:Lucy

SUBJECT: football

Charlie, I promise I will not pull away the football when

you kick off.

.

250 2.0.0 f15Eiu537020 Message accepted for delivery

QUIT

221 2.0.0 csc.smsu.edu closing connection

Connection closed by foreign host.

>

Resulting message is:

Date: Mon, 5 Feb 2001 08:45:22 -0600 (CST)

From: Lucy

To: Charlie, Brown

Subject: football

Charlie, I promise I will not pull away the football when

you kick off.

NOTE CORRECTION ON PAGE 111 OF BOOK: at end of first paragraph, change "HELO crepes.fr" with "MAIL FROM:"

· Connection is persistent : can send multiple messages by starting each new one with MAIL FROM:

• Each command results in 3-digit code and message

• Header lines defined in RFC 822

• Commands and message are plaintext

MIME : Multipurpose Internet Mail Extensions

• Use to transmit binary data and/or multiple objects in one message
• RFC 2045, 2046
• Defines Content-type: header line
• Format is: Type / subtype ; paramaters
• One message can have multiple objects (message + attachments).
• All transmitted together using MIME Content-type multipart

· Parameter defines separator (which is a character string)

Mail Access Protocols

• interface between email user agent and mail server
• user access to private mailbox
• why would SMTP not suffice?
• POP3, IMAP are examples

POP3 : Post Office Protocol version 3

• RFC 1939
• port 110
• TCP connection
• Intended to be very simple
• Authorization commands are
• user : your user ID
• pass : your password
• Major transaction commands are
• list : numeric listing of message lengths from server
• retr # : retrieve message number # from server
• dele # : mark message number # on server for deletion
• quit : terminate POP connection (and delete marked messages)
• State maintained throughout POP session but not between sessions
• example conversation shown in text

IMAP : Internet Mail Access Protocol

• RFC 2060
• port 143
• TCP connection
• More complex than POP3
• User can organize mailbox into hierarchical folders
• State is maintained between sessions
• User can retrieve portions of message (e.g. header only)
• Each client command preceeded by a client-selected command ID.
• Server is in one of 4 states, and transitions specified in state transition diagram in RFC 2060 section 3, reproduced here.

 
3. State and Flow Diagram
 
 An IMAP4rev1 server is in one of four states. Most commands are
 valid in only certain states. It is a protocol error for the client
 to attempt a command while the command is in an inappropriate state.
 In this case, a server will respond with a BAD or NO (depending upon
 server implementation) command completion result.
 
3.1. Non-Authenticated State
 
 In non-authenticated state, the client MUST supply authentication
 credentials before most commands will be permitted. This state is
 entered when a connection starts unless the connection has been pre-
 authenticated.
 
3.2. Authenticated State
 
 In authenticated state, the client is authenticated and MUST select a
 mailbox to access before commands that affect messages will be
 permitted. This state is entered when a pre-authenticated connection
 starts, when acceptable authentication credentials have been
 provided, or after an error in selecting a mailbox.
 
3.3. Selected State
 
 In selected state, a mailbox has been selected to access. This state
 is entered when a mailbox has been successfully selected.
 
3.4.  Logout State
 
 In logout state, the connection is being terminated, and the server
 will close the connection. This state can be entered as a result of
 a client request or by unilateral server decision.
 
 +--------------------------------------+
 |initial connection and server greeting|
 +--------------------------------------+
 || (1) || (2) || (3)
 VV || ||
  +-----------------+ || ||
 |non-authenticated| || ||
 +-----------------+ || ||
 || (7) || (4) || ||
 || VV VV ||
 || +----------------+ ||
 || | authenticated |<=++ ||
 || +----------------+ || ||
 || || (7) || (5) || (6) ||
 || || VV ||  ||
 || || +--------+ || ||
 || || |selected|==++ ||
 || || +--------+ ||
 || || || (7) ||
 VV VV VV  VV
 +--------------------------------------+
 | logout and close connection |
 +--------------------------------------+
 
 (1) connection without pre-authentication (OK greeting)
 (2) pre-authenticated connection (PREAUTH greeting)
 (3) rejected connection (BYE greeting)
 (4) successful LOGIN or AUTHENTICATE command
 (5) successful SELECT or EXAMINE command
 (6) CLOSE command, or failed SELECT or EXAMINE command
 (7) LOGOUT command, server shutdown, or connection closed

Example IMAP session

(my commands are bolded)

> telnet csc.smsu.edu 143

Trying 146.7.45.212...

Connected to csc.smsu.edu.

Escape character is '^]'.

* OK [CAPABILITY IMAP4 IMAP4REV1 LOGIN-REFERRALS AUTH=LOGIN] csc.smsu.edu IMAP4r

ev1 2000.284 at Tue, 6 Feb 2001 20:53:10 -0600 (CST)

A002 LOGIN pete myPassword

* CAPABILITY IMAP4 IMAP4REV1 NAMESPACE IDLE MAILBOX-REFERRALS SCAN SORT THREAD=R

EFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND

A002 OK LOGIN completed

A003 SELECT INBOX

* 31 EXISTS

* 0 RECENT

* OK [UIDVALIDITY 915631092] UID validity status

* OK [UIDNEXT 4315] Predicted next UID

* FLAGS (\Answered \Flagged \Deleted \Draft \Seen)

* OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Draft \Seen)] Permanent fl

ags

A003 OK [READ-WRITE] SELECT completed

A004 SEARCH ALL

* SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

28 29 30 31

A004 OK SEARCH completed

A005 SEARCH UNSEEN

* SEARCH

A005 OK SEARCH completed

A006 FETCH 29 (FLAGS BODY[HEADER.FIELDS (DATE FROM)])

* 29 FETCH (FLAGS (\Seen) BODY[HEADER.FIELDS ("DATE" "FROM")] {89}

Date: Mon, 22 Jan 2001 16:17:26 -0500

From: Scott Grissom <grissom@GVSU.EDU>

)

A006 OK FETCH completed

A007 LOGOUT

* BYE csc.smsu.edu IMAP4rev1 server terminating connection

A007 OK LOGOUT completed

Connection closed by foreign host.

DNS : Domain Name System

• Translates system name into IP address
• Several RFCs (do a search on "Domain Name" at www.rfc-editor.org)
• Major RFCs are 1034 and 1035.
• UDP
• port 53
• You can query from command line: nslookup <name>

Resolver is library function that application calls to request translation of name to IP address. In Unix, this is “gethostbyname(string)” It issues original DNS request and application suspends until it returns with either pointer to data structure with requested info or null pointer.

Local name server is within organization (or ISP) of requester. This is who resolver asks first.

Root name server is at tip of hierarchy. There are currently 13 worldwide (but they are replicated). If local name server cannot map resolver’s request (either from its database or cache), it asks root server. Called “root” because it handles only top-level domain (TLD), e.g. the suffix that appears after the last period (.com, .edu, .fr, etc).

Domain names and numbers are controlled by ICANN, the Internet Corporation for Assigned Names and Numbers. See www.icann.org.

There are renegade “alternative root servers” out there, supporting ICANN’s TLDs plus a slew of others.

· Iperdome, www-iperdome.com, offers personal domain services under “.per” domain.

· TINC, The Internet Namespace Cooperative, www.tinc-org.com, also supports alternative root servers, and has a form that allows you to test it out (the proxy) by typing in an URL from among domains it supports (try www.b).

· OpenNIC is yet another alternative, www.opennic.unrated.net/.

· And others. See if you can find more.

Authoritative name server is the one whose database (not cache) contains DNS entry for a requested host.

Intermediate name server is one that root server (or another intermediate name server) knows about but which turns out not to be the authoritative server for a given request.

Caching is crucial to DNS performance. Info from DNS response (containing results) is put into name server cache as it passes through on return trip to resolver.

Resource Record is DNS database or cache entry. Fields are:

• name -- the name used for matching DNS request
• value -- result for DNS response, depends on record type (below)
• type – can have one of several values. Most frequent are:
• A -- (authoritative) value field contains IP address
• NS – (name server) name is domain name and value is its name server
• MX – (mail) name is alias for mail server and value is mail server name
• CNAME – (canonical name) name is alias and value is real name.
• TTL – Time To Live. for caching.

Example: A worst case for “www.cs.pitt.edu” is this:

1. Resolver asks local name server.
2. Local name server doesn’t know but asks root server for “.edu”.
3. Root server for “edu” doesn’t know but asks server for “pitt.edu”.
4. Server for “pitt.edu” doesn’t know (thus is intermediate server) but asks “cs.pitt.edu”
5. “cs.pitt.edu” keeps “www.cs.pitt.edu” translation in its database, so it is authoritative server.
6. Authoritative server issues DNS response to intermediate server, which relays response to root server, which relays response to local server, which relays response to responder.

Recursive query: each client in chain issues DNS request on behalf of original requester and gets definitive response - resource record(s) or error. The “www.cs.pitt.edu” example above is of this type.

Iterative query: client may get a “referral” response that contains address of a different DNS server for client to ask.

Same example as above, only using iterative approach:

1. Resolver asks local name server.
2. Local name server doesn’t know but asks root server for “.edu”.
3. Root server for “edu” doesn’t know but responds to local server with IP address of server for “pitt.edu”.
4. Local server asks server for “pitt.edu”
5. Server for “pitt.edu” doesn’t know but responds to local server with IP address of server for “cs.pitt.edu”
6. Local server asks server for “cs.pitt.edu”
7. “cs.pitt.edu” keeps “www.cs.pitt.edu” translation in its database, so it is authoritative server.
8. Authoritative server issues DNS response to local server, which relays response to responder.

(NOTE: steps 5-8 are unlikely to occur this way; more likely the “pitt.edu” server will ask the “cs.pitt.edu” directly. Thus a combination of iterative and recursive)

[notes | CSC 465 | Peter Sanderson | Computer Science | SMSU ]

Updated 7 February 2001

PeteSanderson@smsu.edu