Kenneth: /* Interactive telnet session with a Web server */

2025-02-28T17:51:23Z

Interactive telnet session with a Web server

← Older revision		Revision as of 17:51, 28 February 2025
Line 251:		Line 251:

	We will first explore the HTTP protocol by using a good old program called telnet, that can connect on any port service by just providing the port number as an argument on the command line. This will let us see what is normally hidden by a standard Web browser: actual request and server responses, before being handled by a graphical end-user browser.		We will first explore the HTTP protocol by using a good old program called telnet, that can connect on any port service by just providing the port number as an argument on the command line. This will let us see what is normally hidden by a standard Web browser: actual request and server responses, before being handled by a graphical end-user browser.

			% telnet example.com 80
			GET / HTTP/1.0

	==Exercise 1.1. Fetch a document==		==Exercise 1.1. Fetch a document==

Kenneth: Created page with "== Telnet == == Interactive telnet session with Mail server ==

 $ telnet auth.oeey.com 25 Trying 50.50.251.110... Connected to auth.oeey.com. Escape character is '^]'. ..."

2014-09-30T14:47:16Z

Created page with "== Telnet == == Interactive telnet session with Mail server == <pre> $ telnet auth.oeey.com 25 Trying 50.50.251.110... Connected to auth.oeey.com. Escape character is '^]'. ..."

New page

== Telnet ==

== Interactive telnet session with Mail server ==

<pre>
$ telnet auth.oeey.com 25
Trying 50.50.251.110...
Connected to auth.oeey.com.
Escape character is '^]'.
ehlo bob
220 ******************************
250-auth.oeey.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH CRAM-MD5 LOGIN DIGEST-MD5 GSSAPI NTLM PLAIN
250-AUTH=CRAM-MD5 LOGIN DIGEST-MD5 GSSAPI NTLM PLAIN
250 8BITMIME

</pre>

The contents of your message file should resemble this example:

<pre>
HELO host.example.com
MAIL FROM: <test@host.example.com>
RCPT TO: <bob@example.com>
DATA
From: [Alice] <alice@geek.com>
To: <bob@example.com>
Date: Mon, 12 Apr 2010 14:21:26 -0400
Subject: Test Message

Hi there! This is supposed to be a real email...

Have a good day!
Alice

.
QUIT
</pre>

Now feed message to netcat:
/usr/bin/nc smtp.domain.com 25 < /tmp/message

<pre>
# telnet auth.oeey.com 25
Trying 50.50.251.110...
Connected to auth.oeey.com.
Escape character is '^]'.
220 ******************************
EHLO trogdor
250-auth.oeey.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH CRAM-MD5 LOGIN DIGEST-MD5 GSSAPI NTLM PLAIN
250-AUTH=CRAM-MD5 LOGIN DIGEST-MD5 GSSAPI NTLM PLAIN
250 8BITMIME
AUTH LOGIN
334 VXNlcm5hbWU6 ## Username:
a2VubmV0aC5idXJnZW5lcg== ## kenneth.burgener
334 UGFzc3dvcmQ6 ## Password:
SW1CQjE5OTkh ## ****
235 Authentication successful
RCPT TO: <kenneth.burgener@oeey.com>
503 Error: need MAIL command
MAIL FROM: <kenneth.burgener@oeey.com>
250 Ok
RCPT TO: <kenneth@kennethburgener.org>
250 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
Subject: This is a test
From: Kenneth <kenneth.burgener@oeey.com>
To: Kenneth <kenneth@kennethburgener.org>

This is a test
.

250 Ok: queued as 8E2FD52095
500 Error: bad syntax
quit
221 Bye
</pre>

=== SMTP Authentication ===

* IndiMail: Authenticated SMTP tutorial - http://indimail.blogspot.com/2010/03/authenticated-smtp-tutorial.html
* Tutorial: SMTP Authentication - http://www.fehcom.de/qmail/smtpauth.html
* http://jetmore.org/john/code/gen-auth
* SMTP AUTH for sendmail 8.10: Realms and Examples - http://www.sendmail.org/~ca/email/authrealms.html

Use base64 tool to encode decode:
echo -n "[pass]" | /usr/bin/base64
echo -n "[encpass]" | /usr/bin/base64 -d
printf "\0postmaster@example.com\0pass" | /var/indimail/bin/base64
printf 'VXNlcm5hbWU6' | mimencode -u ; echo

=== LOGIN Authentication ===

This method accepts username and password as supplemental args. It simply
returns each string Base64 encoded. This provides only minimal advantages
over using ENCODE twice. One advantage is hiding the password if you
provide it on STDIN

LOGIN: (base64) [http://www.webpan.com/Customers/Email/SMTP_Authentication_Telnet_Test.htm] [http://technet.microsoft.com/en-us/library/aa995718%28EXCHG.65%29.aspx]
<pre>
EHLO trogdor
AUTH LOGIN
334 VXNlcm5hbWU6 ## Username:
a2VubmV0aC5idXJnZW5lcg== ## kenneth.burgener
334 UGFzc3dvcmQ6 ## Password:
SW1CQjE5OTkh ## ****
235 Authentication successful
</pre>

=== PLAIN Authentication ===

This type generates a PLAIN (RFC 2595) authentication string. It accepts
supplemental arguments of username and password. It generates a Base64
encoded string "\0<username>\0<password>".

PLAIN: (base64 on single line) [http://qmail.jms1.net/test-auth.shtml]
<pre>
EHLO trogdor
AUTH PLAIN
334
AGtlbm5ldGguYnVyZ2VuZXIASW1CQjE5OTkh ## \000kenneth.burgener\000****
235 Authentication successful
</pre>
perl -MMIME::Base64 -e 'print encode_base64("\000jms1\@jms1.net\000not.my.real.password")'
perl -MMIME::Base64 -e 'print encode_base64("\000kenneth.burgener\000****")'
AGtlbm5ldGguYnVyZ2VuZXIASW1CQjE5OTkh

This also works:
AUTH PLAIN AGtlbm5ldGguYnVyZ2VuZXIASW1CQjE5OTkh ## \000kenneth.burgener\000****

=== CRAM-MD5 ===

CRAM-MD5 (RFC 2195) accepts three supplemental arguments. The first is the username and
the second is the password. The third is the challenge string provided
by the server. This string can be either Base64 encoded or not. The RFC states
that all (unencoded) challenge strings must start w/ '<'. This is used to
whether the string is Base64 encoded or not.

CRAM-MD5 uses the challenge and the supplied password to generate a digest.
it then returns the Base64 encoded version of the string md5("<username> <challenge>")

* http://indimail.blogspot.com/2010/03/authenticated-smtp-tutorial.html

<pre>
3. AUTH CRAM-MD5

The CRAM-MD5 is a challenge-response method where the password is not sent over the network. It is expected that the password is stored in the clear in IndiMail's backend database MySQL.

% sudo /var/indimail/bin/vpasswd postmaster@example.com -e pass

Next step is to write a script named cram-md5

% cat > cram-md5 <<>"
sys.exit(1)
str=cram_md5_response(sys.argv[1], sys.argv[2], sys.argv[3]);
print "%s" %str
EOF

% sudo chmod +x ./cram-md5

Now when you do (see below) auth cram-md5, the server will issue a challenge
e.g. in the below example, the challenge is

PDIwMTM3LjEyNjc1ODUxMDBAaW5kaW1haWwub3JnPg==

if you decode this, i.e.

% echo PDIwMTM3LjEyNjc1ODUxMDBAaW5kaW1haWwub3JnPg== | base64 -d
<20137.1267585100@indimail.org>

The response for the challenge can be generated using the cram-md5 shell script which we created above. i.e.

% ./cram-md5 PDIwMTM3LjEyNjc1ODUxMDBAaW5kaW1haWwub3JnPg==
cG9zdG1hc3RlckBleGFtcGxlLmNvbSBjZWU4Mzk3YWIxMjNhMGQ0ZjNhN2ZkZGJiOWNiODcxOQ==

% telnet 0 smtp
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
220 indimail.org (NO UCE) ESMTP IndiMail 1.137 3 Mar 2010 08:28:17 +0530
auth cram-md5
334 PDIwMTM3LjEyNjc1ODUxMDBAaW5kaW1haWwub3JnPg==
cG9zdG1hc3RlckBleGFtcGxlLmNvbSBjZWU4Mzk3YWIxMjNhMGQ0ZjNhN2ZkZGJiOWNiODcxOQ==
235 ok, go ahead (#2.0.0)

Please do take a look at Erwin Hoffman's excellent tutorial on the same subject at
http://www.fehcom.de/qmail/smtpauth.html
</pre>

=== CRAM-SHA1 ===

This behaves the same as CRAM-MD5 but uses SHA1 digesting rather than MD5.

=== DIGEST-MD5 ===

* http://www.sendmail.org/~ca/email/authrealms.html

<pre>
250-AUTH LOGIN PLAIN CRAM-MD5 DIGEST-MD5
250 HELP
>>> AUTH DIGEST-MD5
334 bm9uY2U9IkFKUlVjNUp4MFVRYnY1U0o5Rm95VW5hWnBxWklIRGhMVFUrQXduL0swVXc9Iixxb3A9ImF1dGgsYXV0aC1pbnQsYXV0aC1jb25mIixjaXBoZXI9InJjNC00MCxyYzQtNTYscmM0LGRlcywzZGVzIixjaGFyc2V0PXV0Zi04LGFsZ29yaXRobT1tZDUtc2Vzcw==
>>> dXNlcm5hbWU9InRlc3QiLHJlYWxtPSJ3aXouZXhhbXBsZS5jb20iLG5vbmNlPSJBSlJVYzVKeDBVUWJ2NVNKOUZveVVuYVpwcVpJSERoTFRVK0F3bi9LMFV3PSIsY25vbmNlPSJBSlJVYzVKeDBVUWJ2NVNKOUZveVVuYVpwcVpJSERoTFRVK0F3bi9LMFV3PSIsbmM9MDAwMDAwMDEscW9wPWF1dGgtY29uZixjaXBoZXI9InJjNCIsY2hhcnNldD11dGYtOCxkaWdlc3QtdXJpPSJzbXRwL2xvY2FsaG9zdC5zZW5kbWFpbC5jb20uIixyZXNwb25zZT0wZTdjZmNhZTcxN2VlYWM5NzJmYzlkNTYwNmExMDgzZA==
334 cnNwYXV0aD03NDM5ODBjODQ0MmRiYjcxNmQ0ZWE5ZTQ5OTNiMDFkMA==
>>>
235 2.0.0 OK Authenticated
</pre>

Decoded:

<pre>
nonce="AJRUc5Jx0UQbv5SJ9FoyUnaZpqZIHDhLTU+Awn/K0Uw=",qop="auth,auth-int,auth-conf",cipher="rc4-40,rc4-56,rc4,des,3des",charset=utf-8,algorithm=md5-sess

username="test",realm="wiz.example.com",nonce="AJRUc5Jx0UQbv5SJ9FoyUnaZpqZIHDhLTU+Awn/K0Uw=",cnonce="AJRUc5Jx0UQbv5SJ9FoyUnaZpqZIHDhLTU+Awn/K0Uw=",nc=00000001,qop=auth-conf,cipher="rc4",charset=utf-8,digest-uri="smtp/localhost.sendmail.com.",response=0e7cfcae717eeac972fc9d5606a1083d

rspauth=743980c8442dbb716d4ea9e4993b01d0
</pre>

=== GSSAPI ===

* http://en.wikipedia.org/wiki/Generic_Security_Services_Application_Program_Interface

=== NTLM ===

Although it may be advertised as one of the above types, this method of authentication if refered to singularly as NTLM. This is a multi-step authentication type. The first 3 arguments must be supplied up front. They are username, password, and domain, in that order. These three strings are used to generate an "Auth Request" string. This string should be passed verbatim to the server. The server will then respond with a challenge. This challenge is the fourth argument. After receiving the server challenge, gen-auth will produce an "Auth Response". Posting this response to the server completes the NTLM authentication transaction.

This authentication method requires the Authen::NTLM perl module to be installed. See EXAMPLES for an example of this transaction. Note also that 'domain' is often blank from client or ignored by server.

* Complicated challenge response:
* http://curl.haxx.se/rfc/ntlm.html
* http://davenport.sourceforge.net/ntlm.html
* [http://www.google.com/url?sa=t&source=web&cd=2&sqi=2&ved=0CB4QFjAB&url=http%3A%2F%2Fdownload.microsoft.com%2Fdownload%2F9%2F5%2Fe%2F95ef66af-9026-4bb0-a41d-a4f81802d92c%2F%255BMS-SMTP%255D.pdf&rct=j&q=telnet%20smtp%20auth%20NTML&ei=gwxCTYTRCo2LswadqtSdDg&usg=AFQjCNFj1yvCOttxEsT1iRQzJ4Y8iT3TrQ&cad=rja]

==Interactive telnet session with a Web server==
[http://www.pasteur.fr/formation/infobio/web/cours/ch01.html Interactive telnet session with a Web server]

We will first explore the HTTP protocol by using a good old program called telnet, that can connect on any port service by just providing the port number as an argument on the command line. This will let us see what is normally hidden by a standard Web browser: actual request and server responses, before being handled by a graphical end-user browser.

==Exercise 1.1. Fetch a document==

Let us fetch a document from the Pasteur Institute Web server:

<pre>
% telnet www.pasteur.fr 80
Trying 157.99.64.12...
Connected to www.pasteur.fr.
Escape character is '^]'.
GET /formation/infobio/web/cours/data/page1.html HTTP/1.0

HTTP/1.1 200 OK
Date: Tue, 24 Feb 2004 18:01:05 GMT
Server: Apache/1.3.26 (Unix) mod_perl/1.24_01 mod_ssl/2.8.10 OpenSSL/0.9.5a
Last-Modified: Tue, 18 Feb 2003 13:40:14 GMT
ETag: "101e6a1-cd-3e5237be"
Accept-Ranges: bytes
Content-Length: 205
Connection: close
Content-Type: text/html; charset=iso-8859-1

<html>
<head>
<title>A sample Web page</title>
</head>

<body>
<h1>A first header</h1>
And some text...

</body>
</html>
Connection closed by foreign host.
%
</pre>

You can as well get important information from the Web server, before getting the entire document:

<pre>
% telnet www.pasteur.fr 80
Trying 157.99.64.12...
Connected to www.pasteur.fr.
Escape character is '^]'.
HEAD /formation/infobio/web/cours/data/page1.html HTTP/1.0

HTTP/1.1 200 OK
Date: Tue, 24 Feb 2004 18:01:05 GMT
Server: Apache/1.3.26 (Unix) mod_perl/1.24_01 mod_ssl/2.8.10 OpenSSL/0.9.5a
Last-Modified: Tue, 18 Feb 2003 14:38:31 GMT
ETag: "101e6a1-cd-3e5237be"
Accept-Ranges: bytes
Content-Length: 205
Connection: close
Content-Type: text/html; charset=iso-8859-1

Connection closed by foreign host.
%
</pre>

What could this kind of information be useful for?

==Exercise 1.2. HTTP headers==

Let us use date information from the server. For instance, say you do not want a too recently modified file: you can use an HTTP header for this purpose.

<pre>
% telnet www.pasteur.fr 80
Trying 157.99.64.12...
Connected to www.pasteur.fr.
Escape character is '^]'.
GET /formation/infobio/web/cours/data/page1.html HTTP/1.0
If-Modified-Since: Tue, 18 Feb 2003 14:38:31 GMT

HTTP/1.1 304 Not Modified
Date: Tue, 24 Feb 2004 18:01:05 GMT
Server: Apache/1.3.26 (Unix) mod_perl/1.24_01 mod_ssl/2.8.10 OpenSSL/0.9.5a
Last-Modified: Tue, 18 Feb 2003 14:38:31 GMT
Connection: close
ETag: "101e6a1-cd-3e5237be"

Connection closed by foreign host.
</pre>

Another useful header let you chain several requests. Try:

<pre>
% telnet www.pasteur.fr 80
Trying 157.99.64.12...
Connected to www.pasteur.fr.
Escape character is '^]'.
GET /formation/infobio/web/cours/data/page1.html HTTP/1.0
Connection: keep-alive
</pre>

What can you do after server's reposnse?

==Exercise 1.3. HTTP status code==

HTTP return codes help you know whether a request has been successful. Explain what happens in the following:

<pre>
% telnet bioweb.pasteur.fr 80
Trying 157.99.64.11...
Connected to rosalind.sis.pasteur.fr.
Escape character is '^]'.
GET /formation/infobio/web/cours/data/page1.html HTTP/1.0

HTTP/1.1 404 Not Found
Date: Tue, 24 Feb 2004 18:01:05 GMT
Server: Apache/1.3.20 (Unix)
Last-Modified: Tue, 18 Feb 2003 14:38:31 GMT
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>404 Not Found</TITLE>
</HEAD><BODY>
<H1>Not Found</H1>
The requested URL /formation/infobio/web/cours/data/page1.html was not found on this server.<P>
<HR>
<ADDRESS>Apache/1.3.20 Server at bioweb.pasteur.fr Port 80</ADDRESS>
</BODY></HTML>
Connection closed by foreign host.
%
</pre>

And here:

<pre>
% telnet www.pasteur.fr 80
Trying 157.99.64.12...
Connected to www.pasteur.fr.
Escape character is '^]'.
HEAD /formation/infobio/web/cours/data/page2.html HTTP/1.0

HTTP/1.1 403 Forbidden
Date: Tue, 24 Feb 2004 18:01:05 GMT
Server: Apache/1.3.26 (Unix) mod_perl/1.24_01 mod_ssl/2.8.10 OpenSSL/0.9.5a
Last-Modified: Tue, 26 Nov 2002 15:21:19 GMT
ETag: "ee325-c4c-3de3916f"
Accept-Ranges: bytes
Content-Length: 3148
Connection: close
Content-Type: text/html; charset=iso-8859-1

Connection closed by foreign host.
</pre>

==Exercise 1.4. HTTP GET: dynamic content==

A Web server does not only serve static document. You can also issue request to get dynamically computed document. At the HTTP protocol level, there are several ways to achieve this. Firstly, try:

<pre>
% telnet www.pasteur.fr 80
Trying 157.99.64.12...
Connected to www.pasteur.fr.
Escape character is '^]'.
GET /cgi-bin/biology/bnb_s.pl?query=biopython HTTP/1.0
</pre>

what do you get?

Try another one (put your own email in place of YOUR_EMAIL):

<pre>
% telnet bioweb.pasteur.fr 80
Trying 157.99.64.11...
Connected to rosalind.sis.pasteur.fr.
Escape character is '^]'.
GET /cgi-bin/seqanal/pdbsearch.pl?email=YOUR_EMAIL&query=1crn HTTP/1.0
</pre>

==Exercise 1.5. HTTP POST==

Describe the difference between Exercise 1.4 and the following (do not forget to reset the <variable>Content-length</variable> according to the total length of the request character string, e.g: ):

<pre>
% telnet bioweb.pasteur.fr 80
Trying 157.99.64.11...
Connected to rosalind.sis.pasteur.fr.
Escape character is '^]'.
POST /cgi-bin/seqanal/pdbsearch.pl HTTP/1.0
Content-type: application/x-www-form-urlencoded
Content-length: 35

email=YOUR_EMAIL&query=1crn
</pre>

Telnet - Revision history

Kenneth: /* Interactive telnet session with a Web server */

Kenneth: Created page with "== Telnet == == Interactive telnet session with Mail server ==
$ telnet auth.oeey.com 25 Trying 50.50.251.110... Connected to auth.oeey.com. Escape character is '^]'. ..."