Create a script to take a list of domains and pull stats on them via API ([login to view URL])

Closed Posted 3 years ago Paid on delivery
Closed Paid on delivery

Project summary/goal. We are processing domains in bulk for spam, we check tens of thousands of domains per day. We want to quickly find which domains have spam sites linking to them by checking the incoming link anchors.

When we have that data, we want to check the anchors against a keyword blacklist and remove the domains that have anchors that contain keywords from the blacklist.

==========

We want to create a web script that will take a list of domains and send them to the [login to view URL] API for processing.

API Docs: [login to view URL]

We want to send the domains and pull back the anchors associated for each domain. Specifically this API call - [login to view URL]

Once we have the data back, we want the script to take a list of keywords we provide and filter the anchors to see which keywords.

We will want to use a simple web interface, where we can provide a list of domains and a set of blacklist keywords. The script will send the domains for processing and then process the results into 2 csv files.

Output 1: A csv file with all non blacklist keyword domains and anchors in 2 columns

Output 2: A csv file with all blacklisted keyword domains and anchors in 2 columns

The interface would have a project list where we can create a project and start it, and also a settings page where we can update the blacklist keywords.

We can use a web template like this: [login to view URL] - we’ve used it extensively in the past already.

Thanks!

Web Scraping Web API PHP Python

Project ID: #24692017

About the project

15 proposals Remote project Active 3 years ago

15 freelancers are bidding on average $537 for this job

mhmhz

Hi I can implement a desktop tool if this acceptable Thanks

$800 USD in 7 days
(179 Reviews)
7.9
fhasanbd

I am a professional web data scraper specialized using Python program, PHP script, .Net program, Crawler and Bot. My tool can search data and get information from Aa to Zz with an existing lists of english words.

$500 USD in 7 days
(219 Reviews)
7.3
AITSoft

Hi, How are you? Upon reading brief details I believe its a doable job I have great experience doing projects with PHP, Python, Web Scraping and Web API. I am confident about getting this done. While I believe we need More

$750 USD in 11 days
(45 Reviews)
6.8
meladbadder

Hi, I am an expert in web scraping. I can create a script to take a list of domains and pull stats on them via API through my experience of the past. I worked on projects like your project. I will finish more quickly t More

$250 USD in 1 day
(32 Reviews)
6.1
harunnizam

Hello, sir. I have read your project requirements carefully. i am very familiar with API so I can do your projects perfectly as you wish. I have ever done many api projects before like yours. So I have enough experienc More

$500 USD in 7 days
(26 Reviews)
5.9
webbakervij

I Hope you are doing well! Firstly, I'd like to make familiar with my background so you can check my work out. Expert in BOTH Backend and Front-end Programming, PHP - OOP, MVC, REST API, HTML/CSS/JAVASCRIPT, Codeigni More

$500 USD in 5 days
(3 Reviews)
3.6
valerykrapiv

hello. i am python and web scraping expert. I've read your project description and would like to know more about the project and goals. i can build your project with python. i want to work with u on your project. regar More

$750 USD in 10 days
(10 Reviews)
4.6
tulikapriyadars1

hi, we have 12+ years experienced in this so you can also check our website www.vrickson.com. hope you ll hire me . ty Tulika

$710 USD in 20 days
(0 Reviews)
0.0
AliHajiabadi

Dear madam/sir, I am an experienced programmer with more than 12 years of experience. I have developed several web-scraping projects, mainly on Python using Selenium. I can help you with the project. Here is the detai More

$499 USD in 7 days
(0 Reviews)
0.0
chadhabh

I am a 6+ years experienced python developer. I can get this up and running with tests and all in a weeks time.

$694 USD in 5 days
(0 Reviews)
0.0