Unix Programming - problem with wget

Author

problem with wget

olivier HARO

2004-10-15, 3:56 am

hello, i come here because I have a little problem with wget....

I use wget in a program written in C by the system(char *s) function.
The problem is that when wget has finished downloading the file, my C
program stops.

I have posted an HELP message on 'comp.lang.c' and they have redirected me
here ;)

The URL I want to download is a ASP document which content is 10 caracters
long
Has somebody an issue for me? (like using another program than wget... or
using a C souce function that can download a web document by URL)

Thank you by advance ;) (and sorry for the bad englesh)

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.775 / Virus Database: 522 - Release Date: 08/10/2004

Robert Harris

2004-10-15, 3:58 pm

olivier HARO wrote:
> hello, i come here because I have a little problem with wget....
>
>
> I use wget in a program written in C by the system(char *s) function.
> The problem is that when wget has finished downloading the file, my C
> program stops.
>
>
> I have posted an HELP message on 'comp.lang.c' and they have redirected me
> here ;)
>
>
> The URL I want to download is a ASP document which content is 10 caracters
> long
> Has somebody an issue for me? (like using another program than wget... or
> using a C souce function that can download a web document by URL)
Well, wget and system certainly work. (what does system return?)

In any case, if you want the document itself (and not just the return
code), you need to call popen(3) instead.

Robert
>
>
> Thank you by advance ;) (and sorry for the bad englesh)
>
>
> ---
> Outgoing mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.775 / Virus Database: 522 - Release Date: 08/10/2004
>
>

olivier HARO

2004-10-19, 3:58 pm

If thing you have not understood my problem

I want to download a document based on my web server, that's why I use
wget... (popen doesn't understand http urls, Am I wrong?)

The problem is that when I use the system(wget http://xxxxxxx.com), the C
program doesnt't wait the end of the system() function so that it implies
bugs.....
Is there any wait to download a http document in C language by an other way?
Is there a solution to my problem?

thanks! ;)

"Robert Harris" <robertdotfdotharris@blueyonder.co.uk> a écrit dans le
message de news:KeSbd.90226$BI5.34579@fe2.news.blueyonder.co.uk...[color=darkred]
> olivier HARO wrote:
me[color=darkred]
caracters[color=darkred]
or[color=darkred]
> Well, wget and system certainly work. (what does system return?)
>
> In any case, if you want the document itself (and not just the return
> code), you need to call popen(3) instead.
>
> Robert

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.778 / Virus Database: 525 - Release Date: 15/10/2004

Victor Wagner

2004-10-19, 3:58 pm

olivier HARO <o.haro@en-compro.com> wrote:
: If thing you have not understood my problem
: I want to download a document based on my web server, that's why I use
: wget... (popen doesn't understand http urls, Am I wrong?)
: The problem is that when I use the system(wget

But popen is able to run wget -O - <url>

http://xxxxxxx.com), the C
: program doesnt't wait the end of the system() function so that it implies

wget is able to detach itself from shell and put in the background.
Typically, it requires -b option in the command line.

Since system function does use shell, you could start wget in the
background with it.

Moreover, since system use shell, placing ampersand "&" at the end of
command line would place program into background. So, if your URL
contain some ampersands, it should be properly quoted.

try following code:

char wget_cmd [1024];
sprintf(wget_cmd,"wget '%s'",url);
system(wget_cmd)

(of course in real code you better use snprintf)

: bugs.....

Since for more than thirty years people used this function and it worked
for them, it seems that there is some misunderstanding on your part.

: Is there any wait to download a http document in C language by an other way?
: Is there a solution to my problem?
: thanks! ;)

There are numerous ways to do it. Note that wget itself as well as lynx
and many other web access tools are written in C. (Mozilla and Opera are
written on C++ although)

There are numerous C libraries which handle client part of http
protocol.

But for simple case it is enough just to open socket to port 80 of your
web server, send there simple HTTP protocol command

"GET /mydocument.doc HTTP/1.1\nHost: www.myserver.com\n\n"

if you are going for http://www.myserver.com/mydocument.doc
and read everything which comes from server.

--

Martin Carpenter

2004-10-21, 4:03 pm

"Victor Wagner" <vitus@45.free.net> wrote:

> There are numerous C libraries which handle client
> part of http protocol.

I'm playing with the cURL libs at the moment. Seem quite reasonable.
freshmeat.net will find them for you.

criterium

2005-04-11, 6:48 am

Hello all!

Does anaybody here have some experience using wget or curl to capture secure web pages (https)?
I have spent/lost hours and hours trying to dowbload https pages with these command line tools and I dont arrive to find the appropriate combination of options to achieve this.
The page is https://www.boursorama.com/palmares/palmares.phtml where my username and password are 22986473 and 660883.

Thousands thanks to who could provide the correct command line.

Sponsored Links