Find Jobs
Hire Freelancers

Python 3 Scrapy Web Scraper behind Flask (or equivalent) Rest API

$30-250 USD

Cancelled
Posted over 6 years ago

$30-250 USD

Paid on delivery
I created a scrapy web crawler that can be invoked a by script or through the command to scrape between 2-30 pages and turn a json file. I'm trying to put that behind a simple Rest API using Flask and I can't get any of these suggestions to work: [login to view URL] The API works once, and then I get a reactor error. I need to both read the results from a JSON file or object that I can pass into another function, before inserting the results into a mongoDB database. I need the code to be simple so I understand it (and scalable) so many people can use the API and utilize different threads/queue. The objective of this project is just to put the scraper behind simple rest API that can be secured, so that it can be called in a async non blocking manner (multiple times) and return the scraped json to python variable. Its not a requirement to use Flask as long as the approach is simple and uses well known and maintained libraries. I also need to pass arguments to the spider that will be in the POST json body.
Project ID: 15990542

About the project

17 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs

About the client

Flag of UNITED STATES
New York, United States
0.0
0
Payment method verified
Member since Jan 2, 2018

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.