Classified ads scrapping

Completed Posted 7 years ago Paid on delivery
Completed Paid on delivery

- scrap classified ads from 2 web sites and write the data into sqllite db

- after initial run only scrap new records

- track changes of already scrapped ads and write changes into a history table

- prevent duplicates

No requirement on the scripting language used for the scrapping, but please mention what you would use.

I've decided to limit the project scope to only one web site, not 2 as originally planned.

The adds served on the page are of different categories, about 10 of them. There's a large overlap between different categories, but not all categories have all the data.

The scrapping must handle that. The schema needs to be capable of storing all different types, and the columns not needed by a certain type should be null.

Some columns have data which would be stored in many-to-may lookup tables in a proper normalised database, but we don't need a complex schema. Instead, a column can have schema within a column with key/value pars, stored in JSON.

Web Scraping

Project ID: #11328684

About the project

8 proposals Remote project Active 7 years ago

Awarded to:

Mustang1478

Hi Can you send me over the 2 URLs' of the 2 target web site and let me know which data to extract from [login to view URL] i will make a demo scrapper in PHP or Python. Once i have more details then i will be able to give More

€55 EUR in 1 day
(13 Reviews)
4.7

8 freelancers are bidding on average €47 for this job

seoguru17

Dear Hiring Manager, I have read your job description. I am Data Entry, Data Processing & Web Research expert who knows the value of time, very hard working and always delivers the work on time. I think More

€24 EUR in 1 day
(10 Reviews)
4.0
zubairwebexpert

===Automatic Web Scrapping=== ===Automatic Web Scrapping=== ===Automatic Web Scrapping=== if you like contact me. thanks zubair

€34 EUR in 1 day
(3 Reviews)
3.7
g0dman

Hi. I can write scraper for you with c# application. .

€20 EUR in 2 days
(3 Reviews)
1.8
codeique

Hi, With reference to your project posting, we would like to get on board and help you complete the project in time with the highest level of quality. We are a team of professional software engineers, having expe More

€166 EUR in 7 days
(0 Reviews)
0.0
ibikhan2234

I can do any kind of scrapping like, Emails, Phone Numbers, Articles, Web Reviews and Classified Ads Scrapping for your specific needs. I have very fast and highly flexible web scrapping agents that are developed by me More

€24 EUR in 1 day
(0 Reviews)
0.0