Today's Posts Follow Us On Twitter! TFL Members on Twitter  
Forum search: Advanced Search  
Navigation
Marketplace
  Members Login:
Lost password?
  Forum Statistics:
Forum Members: 24,254
Total Threads: 80,792
Total Posts: 566,472
There are 1422 users currently browsing (tf).
 
  Our Partners:
 
  TalkFreelance     Design and Development     Programming     PHP and MySQL :

web crawling PHP

Thread title: web crawling PHP
Closed Thread    
    Thread tools Search this thread Display Modes  
11-03-2006, 12:00 AM
#1
ktsirig is offline ktsirig
Status: Junior Member
Join date: Oct 2005
Location:
Expertise:
Software:
 
Posts: 46
iTrader: 0 / 0%
 

ktsirig is on a distinguished road

  Old  web crawling PHP

Hello,
I am supposed to construct a page that searches in specific websites to extract information, like those sites from where you can rent a car for example. There is a form in the site where the user selects some fields (for instance departure and drop-off date), then the data are submitted to the other page that searches 2-3 sites and finds which cars are available on those dates.
I wanted to ask if there are ready scripts to do that, if not, some hints on how to start.
I am familiar with PHP forms and data extraction from mysql databases, but when you extract data from other sites, I have no clue how I can begin and deal with it...

11-06-2006, 03:16 PM
#2
KarlP is offline KarlP
KarlP's Avatar
Status: I'm new around here
Join date: Apr 2006
Location:
Expertise:
Software:
 
Posts: 24
iTrader: 0 / 0%
 

KarlP is on a distinguished road

  Old

Hey, you will probably want to use PHPs CURL functions to achieve this:
http://uk.php.net/curl

good luck!

11-07-2006, 05:10 AM
#3
iisbum is offline iisbum
iisbum's Avatar
Status: Member
Join date: Oct 2005
Location: Clifton Park, NY, USA
Expertise:
Software:
 
Posts: 273
iTrader: 4 / 100%
 

iisbum is on a distinguished road

  Old

Kinda depends on the specifics, but you'll probably want to get the file_get_contents function:

http://us2.php.net/file_get_contents

That will allow you to read the contents of a URL into a string. From there you'll have to parse the data you need out of the HTML.

HTH,
Mubs

11-11-2006, 02:01 AM
#4
smallbadad is offline smallbadad
Status: I'm new around here
Join date: Jul 2005
Location: wales
Expertise:
Software:
 
Posts: 17
iTrader: 0 / 0%
 

smallbadad is on a distinguished road

Send a message via MSN to smallbadad

  Old

well not long ago i made a script that scans through a site and checks if every link works, im sure you can use the same method here... you use file_get_contents or i think you should be able to to use http://us2.php.net/manual/en/function.fread.php then just make it look for similarities eg..

Code:
<table>
<tr>
<td>something</td>
</tr>
<tr>
<td>something</td>
</tr>
<tr>
<td>something</td>
</tr>
</table>
then just do strpos($mystring, "<td>");
then loop it ect..

or you could make a reg. ex.

or if the website header neaver changes then i suppose you could read from a line.

anyway theres a few little hinters

Closed Thread    


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 

  Posting Rules  
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump:
 
  Contains New Posts Forum Contains New Posts   Contains No New Posts Forum Contains No New Posts   A Closed Forum Forum is Closed