Robust Data collection/scraping powered by AWS

$30-250 USD

Completed

Posted

almost 6 years ago

$30-250 USD

Paid on delivery

Requirements: 1. Continuously and reliably scrape and collect job posting data from websites like indeed, dice, careerbuilder, Monster, etc. (any one or two would be sufficient). The best solution would be rotating among those sites. 2. It queries jobs based on a randomly generated combination of keywords, such as "java, Dallas Texas". 3. It should be disruption-free and utilize AWS Spot EC2 instance to power the scraping. That means, the solution should include programmatically create a spot instance and start working there. 4. The collected data should be saved to a central server, in a format of zipped csv file or Mongodb.

Project ID: 16990869

About the project

4 proposals

Remote project

Active 6 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

Awarded to:

@zekovicm

Hi there,I am Miljan,web scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within the agreed deadline. Check out my profile, portfolio, and former clients feedback - that'll let you know everything about me. Please feel free to contact me so that we can discuss further details. Thank you for taking the time to read my proposal.I am looking forward to hearing from you. Kind regards

$155 USD in 3 days

4.9

(71 reviews)

6.7

4 freelancers are bidding on average $176 USD for this job

@mantislin

Hi sir, This is Lin and I am scraping expert, i have checked all details for your project. can we discuss more info then i can provide example data for you? Please message me then we can discuss more ASAP. Thanks, Lin

$172 USD in 5 days

5.0

(260 reviews)

7.5

@cyberskytech

We are a small team of experienced IT professionals who excel in System Engineering, DevOps, Cloud, Web Development and Cyber Security. Our primary goal is to provide the best solutions for the least cost. Managed Infrastructure Service (RedHat, CentOS, Windows, AiX) Cloud Infrastructure (AWS, Azure) Virtualization (VMWare, HyperV, KVM, Xen, RHEV) Web Development (Nginx, Apache, PHP, MySQL) Cyber Security DevOps (Docker, Kubernetes, Puppet, Ansible) Office 365, GSuite, Zimbra

$155 USD in 10 days