Unix Programming - linux socket programming and HTTP Protocol Problem

Author

linux socket programming and HTTP Protocol Problem

PGHULME

2006-08-17, 7:00 pm

Hi

I have started writing an app(in c and compiled with gcc) that delivers

html pages (aptly named a web server). my current code for sending the
http header and code looks like this:
// send header
send(clntSocket, "HTTP/1.1 200 OK\n", 16, 0);
send(clntSocket, "Content-type: text/html\n", 25, 0);
send(clntSocket, "Content-length:\n\n", 18, 0); //extra line feed for
http spec
//html code
send(clntSocket, "<HTML>", 6, 0);
send(clntSocket, "<HEAD>", 6, 0);
send(clntSocket, "<TITLE>", 7, 0);
send(clntSocket, "A Small Hello", 13, 0);
send(clntSocket, "</TITLE>", 8, 0);
send(clntSocket, "</HEAD>", 7, 0);
send(clntSocket, "<BODY>", 6, 0);
send(clntSocket, "<H1>Hi</H1>", 11, 0);
send(clntSocket, "<P>This is very minimal hello world HTML
document.</P>", 55, 0);
send(clntSocket, "</BODY>", 7, 0);
send(clntSocket, "</HTML>", 7, 0);
//close socket
close(clntSocket);

this is just to test the socket connection. The problem is, i try to
use internet explorer or firefox to connect to the server and retrieve
this test page. Both web browsers connect to the server but both fail
to load the page. Internet explorer complains that it cannot find
server. and firefox says that the connection was reset before the page
could be loaded. The thing is though, if i use wget to connect to the
server, it does manage to download the test page correctly. I have read

through the http specs, and they say that the header should be sent
followed by a new line (/n) then the data/html, once this is sent the
connection should then be closed. I have followed those specs but i am
unable to see why the code works when wget is connecting but not when
web browsers try.... anyone got any ideas????

boa

2006-08-17, 7:00 pm

* PGHULME wrote, On 17.08.2006 21:27:
> Hi
>
> I have started writing an app(in c and compiled with gcc) that delivers
>
> html pages (aptly named a web server). my current code for sending the
> http header and code looks like this:
> // send header
> send(clntSocket, "HTTP/1.1 200 OK\n", 16, 0);
> send(clntSocket, "Content-type: text/html\n", 25, 0);
> send(clntSocket, "Content-length:\n\n", 18, 0); //extra line feed for
> http spec
> //html code
> send(clntSocket, "<HTML>", 6, 0);
> send(clntSocket, "<HEAD>", 6, 0);
> send(clntSocket, "<TITLE>", 7, 0);
> send(clntSocket, "A Small Hello", 13, 0);
> send(clntSocket, "</TITLE>", 8, 0);
> send(clntSocket, "</HEAD>", 7, 0);
> send(clntSocket, "<BODY>", 6, 0);
> send(clntSocket, "<H1>Hi</H1>", 11, 0);
> send(clntSocket, "<P>This is very minimal hello world HTML
> document.</P>", 55, 0);
> send(clntSocket, "</BODY>", 7, 0);
> send(clntSocket, "</HTML>", 7, 0);
> //close socket
> close(clntSocket);
>
>
> this is just to test the socket connection. The problem is, i try to
> use internet explorer or firefox to connect to the server and retrieve
> this test page. Both web browsers connect to the server but both fail
> to load the page. Internet explorer complains that it cannot find
> server. and firefox says that the connection was reset before the page
> could be loaded. The thing is though, if i use wget to connect to the
> server, it does manage to download the test page correctly. I have read
>
> through the http specs, and they say that the header should be sent
> followed by a new line (/n) then the data/html, once this is sent the
> connection should then be closed. I have followed those specs but i am
> unable to see why the code works when wget is connecting but not when
> web browsers try.... anyone got any ideas????
>

The fields are supposed to be terminated with \r\n, not just a \n. Don't
know if that's your problem or not...

Boa

PGHULME

2006-08-17, 7:00 pm

the problem doesnt seem to be with the CR or LF, wget seems to be ok
with \n. The problem is that the web browsers seem to get annoyed when
the connection is closed. even though all the data has been sent.

boa wrote:
> * PGHULME wrote, On 17.08.2006 21:27:
>
> The fields are supposed to be terminated with \r\n, not just a \n. Don't
> know if that's your problem or not...
>
> Boa

Alex Fraser

2006-08-17, 7:00 pm

"PGHULME" <project_os@hotmail.com> wrote in message
news:1155846429.814660.302270@m73g2000cwd.googlegroups.com...
> I have started writing an app(in c and compiled with gcc) that delivers
> html pages (aptly named a web server). my current code for sending the
> http header and code looks like this:
> // send header
> send(clntSocket, "HTTP/1.1 200 OK\n", 16, 0);

That should be "...\r\n", same applies everywhere.

> send(clntSocket, "Content-type: text/html\n", 25, 0);
> send(clntSocket, "Content-length:\n\n", 18, 0); //extra line feed for
> http spec

You didn't actually send a value for the content-length.

Though it does not affect correctness, repeated send() calls like you are
using is bad for performance.

> //html code
[snip]
> //close socket
> close(clntSocket);

I think HTTP/1.1 clients assume persistent connections by default, but can
request the connection be closed. You should parse the request properly, and
tailor the response accordingly.

[snip]
> I have read through the http specs, and they say that the header should be
> sent followed by a new line (/n) then the data/html, once this is sent the
> connection should then be closed. I have followed those specs but i am
> unable to see why the code works when wget is connecting but not when
> web browsers try.... anyone got any ideas????

You did not read the specifications carefully enough and wget is more
forgiving of the errors than the browsers you tried.

Alex

Robert Harris

2006-08-17, 7:00 pm

PGHULME

2006-08-17, 7:00 pm

ok, so i have compared what another server sent out. after each line
it sends out 0d 0A (CR LF) like my code now does. i have even copied
what that server sends out to what my server sends out. they are both
the same. However. my code still flops somewhere. i think it has
something to do with how the connection is closed

Robert Harris wrote:[color=darkred]
> PGHULME wrote:
>
> You need to send the actual content length here. Use tcpdump to see what
> a real HTTP server sends back to your browser.
>
> Robert
>

Barry Margolin

2006-08-17, 7:00 pm

In article <QI5Fg.4660$fV1.4635@fe1.news.blueyonder.co.uk>,
Robert Harris <robert.f.harris@blueyonder.co.uk> wrote:

> PGHULME wrote:
>
> You need to send the actual content length here. Use tcpdump to see what
> a real HTTP server sends back to your browser.

Or omit the Content-Length header entirely, which implies that the
document ends when the connection is closed.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

Logan Shaw

2006-08-17, 7:00 pm

PGHULME wrote:
> I have started writing an app(in c and compiled with gcc) that delivers
>
> html pages (aptly named a web server). my current code for sending the
> http header and code looks like this:
> // send header
> send(clntSocket, "HTTP/1.1 200 OK\n", 16, 0);
> send(clntSocket, "Content-type: text/html\n", 25, 0);

Hang on, there. You're using send(), which doesn't guarantee that
it sends all the bytes. It will return the number of bytes sent,
and if that's not your full string, you need to call send() again
with the remainder.

I haven't done that much socket programming, but I believe you should
be able to solve that problem pretty easily by associating a file
stream with the file descriptor, then use fprintf() and friends to print
and let it take care of buffering and doing the necessary looping and
so on to make sure the whole string that you pass is sent:

const char *HTTP_NL = "\r\n";

FILE *clntStream = fdopen (clntSocket, "w");

fprintf (clntStream, "%s%s", "HTTP/1.1 200 OK", HTTP_NL);
fprintf (clntStream, "%s%s", "Content-type: text/html", HTTP_NL);

fputs ("<HTML>", clntStream);
fputs ("<HEAD>", clntStream);
/* etc., etc., etc. */

fclose (clntStream); /* should close clntSocket too */

Hope that helps...

- Logan

saju

2006-08-17, 9:59 pm

PGHULME wrote:
> Hi
>
> I have started writing an app(in c and compiled with gcc) that delivers
>
> html pages (aptly named a web server). my current code for sending the
> http header and code looks like this:
> // send header
> send(clntSocket, "HTTP/1.1 200 OK\n", 16, 0);

HTTP 1.1 assumes that you will use Persistent Connections by default. If
you don't want to reuse the transport for the next HTTP request you must
send a "Connection: close" header back to the client.

> send(clntSocket, "Content-type: text/html\n", 25, 0);
> send(clntSocket, "Content-length:\n\n", 18, 0); //extra line feed for

The Content-length is not set properly, the browsers don't know when you
have finished sending data. wget likely works because it can get away
with dumping bits to disk but a browser needs to read in enough HTML to
render it in a performant way. Most likely your browser is waiting for
more data and gets

when you close the connection abruptly.

Try sending "Connection: close" and omit the Content-length header
altogether for testing.

_snip_

regards
srp

--
I will never get off this planet
- Luke Skywalker

Nils O. Selåsdal

2006-08-18, 4:00 am

PGHULME wrote:
> the problem doesnt seem to be with the CR or LF, wget seems to be ok
> with \n. The problem is that the web browsers seem to get annoyed when
> the connection is closed. even though all the data has been sent.
send(clntSocket, "Content-length:\n\n", 18, 0); //extra line feed for
This does not look like a valid Content-length: header.

There are http server libraries around, better use those
http has many quirks.

Rainer Temme

2006-08-18, 4:00 am

PGHULME wrote:
> The problem is that the web browsers seem to get annoyed when
> the connection is closed. even though all the data has been sent.

You give an HTTP 1.1 answer, therefore you should indicate that
you want to close the connection.

Try to give an HTTP 1.0 answer.

BTW...creating the answer with multiple calls to send()
is very very ineffective. Better collect the complete answer in
a buffer and call send() only once.

Rainer

David Schwartz

2006-08-18, 4:00 am

PGHULME wrote:
> ok, so i have compared what another server sent out. after each line
> it sends out 0d 0A (CR LF) like my code now does. i have even copied
> what that server sends out to what my server sends out. they are both
> the same. However. my code still flops somewhere. i think it has
> something to do with how the connection is closed

My guess: You don't wait until you receive the entire request before
you send the query. As a result, you may be closing the connection
before the other side has even sent its query.

DS

David Schwartz

2006-08-21, 7:01 pm

Barry Margolin wrote:

> Or omit the Content-Length header entirely, which implies that the
> document ends when the connection is closed.

Forgive me for posting without looking up the answer, but if this were
true, chunked encoding would be much less useful. I believe this is
false.

DS

Henry Townsend

2006-08-21, 7:01 pm

David Schwartz wrote:
> Barry Margolin wrote:
>
>
> Forgive me for posting without looking up the answer, but if this were
> true, chunked encoding would be much less useful. I believe this is
> false.

I haven't looked in any docs recently either but I thought this was a
difference between HTTP 1.0 and 1.1; it would be legal if you announced
1.0 but illegal with 1.1.

Robert Harris

2006-08-21, 7:01 pm

Henry Townsend wrote:
> David Schwartz wrote:
>
> I haven't looked in any docs recently either but I thought this was a
> difference between HTTP 1.0 and 1.1; it would be legal if you announced
> 1.0 but illegal with 1.1.

I haven't looked in any docs recently and I couldn't possibly remember.
Oh for the days when:

1. Everything you wanted to known about a computer (i.e. the instruction
set) fitted in one book, i.e. before the PDP11 came out.

2. You could code everything you wanted in assembler (but you never
really wanted very much).

James Antill

2006-08-21, 7:01 pm

On Mon, 21 Aug 2006 09:02:20 -0700, David Schwartz wrote:

>
> Barry Margolin wrote:
>
>
> Forgive me for posting without looking up the answer, but if this were
> true, chunked encoding would be much less useful. I believe this is
> false.

You should only reply with a chunked transfer encoding with a HTTP/1.1
request. chunked encoding doesn't require you to kill keep-alive (which is
why it's useful on the server side).
Closing the connection isn't allowed from the client POV, so chunked
transfer encoding is the only option then.

http://www.w3.org/Protocols/rfc2616...ec4.html#sec4.4

--
James Antill -- james@and.org
http://www.and.org/and-httpd

Sponsored Links