Find Jobs
Hire Freelancers

Wikipedia Crawl or DB project

$250-750 USD

Closed
Posted over 9 years ago

$250-750 USD

Paid on delivery
I need to gather information about all of the schools in the USA. I noticed wikipedia has a lot of the data we need. I know with wikipedia there are 2 approaches, 1 use their SQL releases to find and pull the data out or devise a way to crawl the site (scrapy pref if we crawl). here is an example of what i found schools by USA [login to view URL] discricts [login to view URL] then to a specific school [login to view URL] more school examples [login to view URL](South_Dakota) [login to view URL] I need the data you will mainly find in the box on the right, the most important would be the school website url, but ideally we can figure out how to store all of the data in there in some normalized name value pairs. I like the crawl idea but lets not be overly passive of the SQL dumps [login to view URL]:Database_download you dont need to scope the process of running the crawler, after you develop and test and confirm it works, i can load it up on a server and run it for the entire data set. if we use the SQL dumps then i would like the target db to be mongoDB and or some json structure per school and per district
Project ID: 6596766

About the project

11 proposals
Remote project
Active 10 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
11 freelancers are bidding on average $597 USD for this job
User Avatar
Hi, I'm web crawler developer and can develop crawler on wikipedia Please share data fields ( you just mentioned right side content) means i can start working on it and send sample data before you award this project Thanks
$500 USD in 10 days
5.0 (7 reviews)
4.5
4.5
User Avatar
Hello, I am interested in working on this project. The web crawling approach can surely be done. I like using my own scraping code to have a better control, instead of using an existing software. On the SQL approach, I have to check how complex is the database first, because I did not work with wikipedia database before and have not downloaded it yet. Please let me know if you want to collaborate with me. Thank you, pragmatechdev
$650 USD in 10 days
5.0 (4 reviews)
2.1
2.1
User Avatar
Greetings! I have my team ready to scrape all school data manually. I am giving you assurance of quality work. Best regards, -Arnob
$555 USD in 10 days
5.0 (1 review)
2.1
2.1
User Avatar
A proposal has not yet been provided
$526 USD in 10 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello Sir/Mam, Thank you very much for this great opportunity. According to your requirement. We have perfect match for this job. We have reviewed your project requirements that you have posted,We are confident to do this job. We are also ready to start your work ASAP. We can communicate online on GTAlK or SKYPE as and how you feel comfortable. I believe in long term relationship. I look forward to hearing from you. We have immense expertise with Android, Mobile Phone, Graphic Design, Logo Design HTML5, CSS3, Java Script, AJAX, JQuery, PHP, MySql, SEO, CodeIgniter, Google Analytics, Google Webmasters tools designing. Good knowledge of responsive designs, We would be glad to help you develop this website in order help your business grow. We will also come up with suggestions and solutions
$722 USD in 10 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Austin, United States
4.9
488
Payment method verified
Member since May 9, 2004

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.