Code Comments

Programming Forum and web based access to our favorite programming groups.
For Programmers: Free Programming Magazines | New: Database administration forum
Registration is free! Edit your profileCalendarFind other membersFrequently Asked QuestionsSearch -> 
Post New Thread











Thread
Author

Web Spider Need Help!!!
i would like to create a web spider to search for rss feeds and input
the articles or data from those feeds into a mysql database. i have
tried many times to do this but for some reason when i try to insert
them into the mysql database it will only insert the last article on
the feed so what can i do to place them all?


Report this thread to moderator Post Follow-up to this message
Old Post
mysitesucks@gmail.com
03-16-06 11:56 PM


Re: Web Spider Need Help!!!
well my first question,(probably don't need to ask it) but is it
deffinatly returning the entire lot of RSS feeds to you, is it just the
entering data thats going wrong or is it that you are not actually
recieving the first feeds just the last one.


Report this thread to moderator Post Follow-up to this message
Old Post
sebdanger2@googlemail.com
03-16-06 11:56 PM


Re: Web Spider Need Help!!!
well my first question,(probably don't need to ask it) but is it
deffinatly returning the entire lot of RSS feeds to you, is it just the
entering data thats going wrong or is it that you are not actually
recieving the first feeds just the last one.


Report this thread to moderator Post Follow-up to this message
Old Post
sebdanger2@googlemail.com
03-16-06 11:56 PM


Re: Web Spider Need Help!!!
when i display them they all show up, its only the last one of those
that are displayed that will be inserted into the mysql database

here is te code
<?php
require_once '../Content/rss_fetch.inc';

$url = 'http://www.usatoday.com/repurposing/NBARss.xml' ;
$rss = fetch_rss($url);

echo 'Site: ' , $rss->channel['title'], ' <br / >';
if ( $rss and !$rss->ERROR) {
foreach ($rss->items as $item ) {
echo ' <p><a href="' . $item[link] . '">' . $item[title] . '
</a><br / >';
echo 'Publish Date: ' . $item[pubdate] . ' <br / >';
echo $item[ description ] . ' </p>' ;
}
} else {
echo 'RSS Error: ' . $rss->ERROR . ' <br / ><br />' ;
}

include("connect.php");

//Item Info
$Added=date(Ymd);
$Item_Title=$item[title];
$Item_Link=$item[link];
$Item_Category=$item[category];
$Item_PubDate=$item[pubdate];
$Item_Description=$item[description];
$add = "INSERT INTO Articles VALUES (NULL, '$Item_Title',
'$Item_Category', '$SubCategory', '$Added',
'$Item_PubDate','$Item_Link', '$Item_Description')";
$result=mysql_query("$add");
mysql_close($connection);
?>


Report this thread to moderator Post Follow-up to this message
Old Post
mysitesucks@gmail.com
03-17-06 02:55 AM


Re: Web Spider Need Help!!!
when i display them they all show up, its only the last one of those
that are displayed that will be inserted into the mysql database

here is te code
<?php
require_once '../Content/rss_fetch.inc';

$url = 'http://www.usatoday.com/repurposing/NBARss.xml' ;
$rss = fetch_rss($url);

echo 'Site: ' , $rss->channel['title'], ' <br / >';
if ( $rss and !$rss->ERROR) {
foreach ($rss->items as $item ) {
echo ' <p><a href="' . $item[link] . '">' . $item[title] . '
</a><br / >';
echo 'Publish Date: ' . $item[pubdate] . ' <br / >';
echo $item[ description ] . ' </p>' ;
}
} else {
echo 'RSS Error: ' . $rss->ERROR . ' <br / ><br />' ;
}

include("connect.php");

//Item Info
$Added=date(Ymd);
$Item_Title=$item[title];
$Item_Link=$item[link];
$Item_Category=$item[category];
$Item_PubDate=$item[pubdate];
$Item_Description=$item[description];
$add = "INSERT INTO Articles VALUES (NULL, '$Item_Title',
'$Item_Category', '$SubCategory', '$Added',
'$Item_PubDate','$Item_Link', '$Item_Description')";
$result=mysql_query("$add");
mysql_close($connection);
?>


Report this thread to moderator Post Follow-up to this message
Old Post
mysitesucks@gmail.com
03-17-06 02:55 AM


Re: Web Spider Need Help!!!
mysitesucks@gmail.com wrote:
> when i display them they all show up, its only the last one of those
> that are displayed that will be inserted into the mysql database
>
> here is te code
> <?php
> require_once '../Content/rss_fetch.inc';
>
> $url = 'http://www.usatoday.com/repurposing/NBARss.xml' ;
> $rss = fetch_rss($url);
>
> echo 'Site: ' , $rss->channel['title'], ' <br / >';
> if ( $rss and !$rss->ERROR) {
>     foreach ($rss->items as $item ) {
>         echo ' <p><a href="' . $item[link] . '">' . $item[title] . '
> </a><br / >';
>         echo 'Publish Date: ' . $item[pubdate] . ' <br / >';
>         echo $item[ description ] . ' </p>' ;
>     }
> } else {
>     echo 'RSS Error: ' . $rss->ERROR . ' <br / ><br />' ;
> }
>
> include("connect.php");
>
> //Item Info
> $Added=date(Ymd);
> $Item_Title=$item[title];
> $Item_Link=$item[link];
> $Item_Category=$item[category];
> $Item_PubDate=$item[pubdate];
> $Item_Description=$item[description];
> $add = "INSERT INTO Articles VALUES (NULL, '$Item_Title',
> '$Item_Category', '$SubCategory', '$Added',
> '$Item_PubDate','$Item_Link', '$Item_Description')";
> $result=mysql_query("$add");
> mysql_close($connection);
> ?>
>

Of course it only puts the last one in the database.  You don't have the
db related code within your loop.

It goes through and displays everything.  Then after the last one is
displayed, it inserts the current (last) item in the database.

Put your insert code within the loop and it should work better.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

Report this thread to moderator Post Follow-up to this message
Old Post
Jerry Stuckle
03-17-06 02:55 AM


Re: Web Spider Need Help!!!
thanls alot i have tried doing this for a month now and every one i ask
they don't know, but thank you very much


Report this thread to moderator Post Follow-up to this message
Old Post
mysitesucks@gmail.com
03-17-06 08:55 AM


Re: Web Spider Need Help!!!
Would you know any place where i can learn how to make a web spider?
and some good code examples or scripts?


Report this thread to moderator Post Follow-up to this message
Old Post
mysitesucks@gmail.com
03-20-06 11:56 PM


Re: Web Spider Need Help!!!
O'Reilly Perl & LWP has some good scripts.
Use them as a starting point. Translating to php
wouldn't be required. But doing so not all that hard.


Report this thread to moderator Post Follow-up to this message
Old Post
Sandy.Pittendrigh@gmail.com
03-20-06 11:56 PM


Re: Web Spider Need Help!!!
ok thanks, do u know any books to do it in php?


Report this thread to moderator Post Follow-up to this message
Old Post
mysitesucks@gmail.com
03-21-06 08:55 AM


Sponsored Links




Last Thread Next Thread Next
Pages (2): [1] 2 »
Search this forum -> 
Post New Thread

PHP Programming archive

Show a Printable Version Send to friend Email This Page to Someone! subscribe to this thread Receive updates to this thread
Computer Consultants
Programming Jobs
Visual Basic Controls
SQL Server Programming
Webservices
Java Security
Visual Studio
C# Programming
Visual J++
Software engineering
Open source Software
Perl Programming
PHP Programming
ASP Programming
ASP .NET Programming
Visual Basic Programming
Windows Scripting Host
Java Programming
Java Help
Java Beans
VBScript
Cobol
MAC Applications
Unix Programming
Forum Jump:
All times are GMT. The time now is 05:05 AM.

 
Free MCSE Braindumps | Real Estate Topics

Programming forum archive

Copyrights CodeComments.com 2004 - 2006

Powered by vBulletin Copyright 2000-2006 Jelsoft Enterprises Limited.