Find Jobs
Hire Freelancers

Webscraping Subreddit

$250-750 USD

In Progress
Posted about 3 years ago

$250-750 USD

Paid on delivery
I have done my best to provide detailed information regarding the project below. Please read the description in full. When posting bids, please indicate which data fields you are able to scrape, which data fields you cannot, and which you are unsure of until you work on the project. A simple bid without any discussion of the feasibility of requested “additional information” will not be considered. I have some CS experience so am open to discussing the project in technical terms. I am looking for someone to scrape the complete history the r/wallstreetbets subreddit. The subreddit dates back to 2012 so will be scraping roughly 9 years of history. I see documentation for a python API wrapper, but you are obviously free to use any methods you deem fit. The following dataset must be provided for the full universe of reddit `posts’ (parent submission to the subreddit): • The date of the post • The score of the post • The username of the poster • The complete text of the post • Some sort of identifier to link posts to children comments in the second dataset The following dataset must be provided for the full universe of `comments’ on all posts: • The date of the comment • The score of the comment • The username of the commenter • The complete text of the comment • The identifier of the ultimate parent post that the comment belong to • An identifier for the comment • An identifier for the direct parent comment that this comment is responding to If possible, I would also like to capture additional information for each post: • Separate counts for upvotes vs downvotes • Any `flair’ tags to the post. E.g. some posts are tagged as `Discussion’ or `YOLO’ • The url of any linked image • A download of the image (estimates on the physical storage that image downloads would require should be included in the bid) • Any outside url linked • The text content of any linked tweets • The text of any linked URLs • The account `karma’ of the poster/commenter username I am flexible in the format in which datasets are delivered. I can easily work with delimited text files (assuming you reasonably handle delimiters in text), width based text files, or any data storage type that SAS has a built import procedure to handle. I would like to scrape a single day to check the data output before completing the final project. I am happy to discuss a portion of payment receivable upon delivery of the trial output with remaining payment made on delivery of the full datasets. Thanks for your time.
Project ID: 29088589

About the project

19 proposals
Remote project
Active 3 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
Hello, I would like to work on your project, I have experience in web scraping with python, selenium, scrapy and also have worked with reddit API for automation tasks. Let's discuss more details about it so we can get started.! Regards
$300 USD in 14 days
5.0 (2 reviews)
0.8
0.8
19 freelancers are bidding on average $563 USD for this job
User Avatar
Dear Employer, I am a scraping expert and have delivered over 1000 projects since over a decade now. I have checked your project details and will ensure highly professional delivery. Kindly check my previous reviews here for your kind consideration: https://www.freelancer.com/u/asin1987 Hope to connect soon!! Thanks, Ashish
$750 USD in 7 days
5.0 (228 reviews)
8.1
8.1
User Avatar
Hi there, I'm bidding on your project "Webscraping Subreddit" Being an expert in Python and matlab programming I can do this project for you. please leave a message on my chat so we can discuss the budget and deadline of the project. Also, have a look at my profile for my reviews and portfolios. Thanks Note: This is not an automated bid and i'm ready to discuss your project right now. .. .
$750 USD in 5 days
5.0 (11 reviews)
6.1
6.1
User Avatar
Hi, How are you? Upon reading brief details I believe its a doable job I have great experience doing projects with Python and Web Scraping. I am confident about getting this done. While I believe we need to have detailed discussion about it. I work as per Eastern time daily so that would be good for us to catch up on this job, Lets discuss this further and get started for you as soon as possible. Thanks! Awais
$750 USD in 20 days
5.0 (5 reviews)
6.1
6.1
User Avatar
Hello I have read your job description carefully. I am a web scrapping expert with 7 years of experience in python. Also, I have rich experience in these fields. So I am interested in and I am confident in your project. I always provide perfect results to my clients as you can see my profile and reviews. If you give me a chance, I will provide you the perfect result. I hope you send me a quick message. Regards Daniel Ramos Garcia
$1,000 USD in 7 days
5.0 (8 reviews)
5.4
5.4
User Avatar
I checked your requirement you need multiple technologies like: Firebase + Flutter + Web Scraper with Python and Web API. Docker Container and MongoDB. I want to tell you we are a proper running company and we have 30 developers in the house. we are working on different technologies: laravel, Codeigniter, PHP, ruby on rails. python, AI, ML, chatbot, Django, Mongo DB, web scraping. Android, IOS, flutter, xamrine, and others. We also have a proper design and QA department. We are leading the digital world. Thanks.
$750 USD in 7 days
5.0 (2 reviews)
4.4
4.4
User Avatar
hello sir .. i am a efficient coder in python and have good experience in automation and web scraping...i use python selenium framework for automation and beautifulsoup for web scraping .i have build scraper for financial data and wall street data data. i can build scraper for you in 7 days at very low price .hope you will contact me soon .thank you
$400 USD in 7 days
5.0 (12 reviews)
4.2
4.2
User Avatar
HII THERE, MY NAME IS PRATIK DOMADIYA AND I AM PROFESSIONAL WEB SCRAPER. I HAVE BEEN WORKING AS A WEB SCRAPPER FOR ABOUT 1.5 YEARS. I AM NEW TO FREELANCING BUT HAVE BEEN WORKING WITH PYTHON FOR ABOUT A YEAR AND WOULD LOVE TO DISCUSS THE PROJECT FURTHER. I HAVE DONE SIMILAR PROJECT BEFORE . IN WHICH I HAVE DEVELOPED AUTOMATED SCRIPT TO DOWNLOAD TEXTILE INDUSTRY'S DATA SUCH AS PRODUCT IMAGES, PRICE, CATALOG IMAGES, AND OTHER REQUIRED INFORMATION FROM 7 DIFFERENT WEB SITE AND UPLOADED TO MY CLIENT'S SERVER DIRECTLY. ALSO, I HAVE BEEN DEVELOPING AUTOMATION CODE USING PYTHON SELENIUM WEB DRIVER WHICH IS THE FASTEST RIGHT NOW. I CAN SHOW YOU SAMPLE IF YOU WANT. I HAVE ALREADY WORKED ON SEVERAL PROJECTS OF LOCAL CLIENTS OF MY CITY. I'D LOVE THE OPPORTUNITY WORK FOR YOU ON THIS, AND GET THE CHANCE TO PROVE MYSELF IN THE FREELANCING FIELD. PLEASE CONTACT ME SOON SO THAT WE CAN START WORK EARLY. THANK YOU. KIND REGARDS, PRATIK DOMADIYA
$300 USD in 7 days
5.0 (7 reviews)
4.0
4.0
User Avatar
Hello, Let me tell you how I will approach this project. I am planning to use PRAW to scrape the r/wallstreetbets subreddit. According to PRAW's docs you can easily get the date, the score, the username, text of the post, upvotes and downvotes and some user informations (so probably also the karma) but I have yet to find a resource in the API giving the tags data. Getting image urls in the post or comment content is pretty straight forward, all I have to do is scan the text using regular expressions to extract all the urls and check the type of the content that the urls points to. Downloading the image is also possible. Concerning the tweet content, I don't know exactly how to do it but I will try to figure out a way but I prefer not to say that I can 100% add that feature. "The text of any linked URLs", I didn't get this part, if you could provide me with some clarification that would be great. For the datasets I think that JSON is a good option as it is easily transportable and doesn't require setting up and managing a database. The program will export two JSON files one containing the Posts and the other the Comments of the posts. I will group all the comments related to a post using the post's ID. If you want me to use another structure feel free to tell me and I will update the structure to match your needs. I hope I didn't forget something, I would like to discuss more details with you in the chat. I am looking forward to your response. Thank you.
$650 USD in 15 days
3.9 (7 reviews)
4.7
4.7
User Avatar
Hello CPReilly. I have rich experienced web scraping with python. I am ready to start the work for now. I am an expert of web scraping with Scrapy and Selenium and BeautifulSoup. I am sure I will finish your project nicely. Looking for your reply. Best regards. From Kostiantyn.
$500 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi there, I am expert in PHP,Python,JavaScript,Web Scraping using(selenium,beautifulsoup,lxml),MYSQL. I have completed lots of similar projects with other employers. Lets have quick chat on this project. Thanks & Regards
$400 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello, How are you? I saw your job post and I felt that I can help you having required experience as of Python and Web Scraping Please check my portfolio. Regards, Ayesha
$750 USD in 26 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello, I have already worked as a full-stack(MERN stack) developer for AAIWay based in Cummings, Georgia, and scraped more than 20k house listings. I can be a perfect fit for this job. Example of my work - https://www.freelancer.in/u/RithikSachdev I have 3+ years of experience in Software Development and web scraping. 1000% work on time and within the budget. Recently, Covid affected my family to the point of going bankrupt so, I started to work freelance recently along with my final year at college. I can assure you of 1000% efficacy in my work. Looking forward to discussing this with you.
$300 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello I am so glad to see your Scraping project. I can create and update the scraping bot to get data as you explained in the description. I am developing a bot based on Spider, Selenium, Request packages. Please contact me and let's discuss more. Hoping to reply from you.
$550 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello, I would like to work on your project. I have experience in Web Scraping, automation, selenium, python and have worked on similar projects. The fields mentioned in the description are doable and would be able to provide you all the data. We can discuss more about the technical details. Looking forward for your reply. Regards.
$300 USD in 3 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Somerville, United States
0.0
0
Payment method verified
Member since Jan 27, 2021

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.