Development of simple crawler

$10-30 USD

Cancelled

Posted

almost 9 years ago

$10-30 USD

Paid on delivery

I have a lot of URLs saved in a database as the following: Colum 1: Member ID Colum 2: URL: Example: Colum 1: ID 1 Colum 2: [login to view URL] By following the link, results for Google search by images appear. ----------------------------------------- Now I am looking for a programmer who can build a scraper / crawler that visits all the URLs in the database. In case there are more results that could be displayed at one page, the crawler should follow the other sites of google result pages. The following data should be get from within the result page and saved in a database: - Member ID - URL of the website where the picture is displayed - URL of the picture (can be extracted from the linkage in the picture) - date of first finding - date of last finding For each URL out of the database those are scraped, the scraper should change the URL => proxy usage.

PHP

Software Architecture

Project ID: 7629145

About the project

2 proposals

Remote project

Active 9 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

2 freelancers are bidding on average $30 USD for this job

@denjohx

Hi, I have write some crawlers in python + selenium to crawl google before, and I'm sure I can get this done fast, but i need some clarification before: the script should be saving the results in another database? What do you mean for change url for proxy usage? I will only need a database example, or complete dump for testing, and can deliver the script really fast, you will only need python and selenium installed in your system.

$29 USD in 1 day