Laravel: Write a script to crawl my website to find and list links

Closed

I need a freelancer to write a script that will crawl the entire blog section of my website and retrieve all of the links (broken and unbroken), then dump those links (along with the page origin and the request status) into a log file.

The page origin path must be relative to my domain. The page origin, link url, and request status must be on one line, seperated by a comma, and then a space ( ", ") Like this:

PAGE ORIGIN, LINK URL, REQUEST STATUS

======

for example:

/blog/my-awesome-blog-post, [url removed, login to view], 200

/blog/another-blog-post, [url removed, login to view], 400

/blog/another-blog-post, [url removed, login to view], 500

/blog/what-a-great-blog-post, [url removed, login to view], 301

The crawler should crawl all links at [url removed, login to view]*

Also, the crawler should EXCLUDE my search route: [url removed, login to view]*

Skills: Data Processing, Laravel, PHP, Shell Script, Web Scraping

See more: list links website, website crawl script, script list files website directory html, script find external links website, movie links website script, linux shell script find links website, software write mq4, software write chip epson, useful software write book, software write web specs, interspire website publisher list links, free software write user guide, software write edid, free software write company profile, write list links joomla, asp script list files website, list links html webpage text file using script, software write websites idea, write script extract information website, software write book images, write script macro copy data website excel, software write books, software write protection, free software write book, software write book

Project ID: #11707744

Awarded to:

sszelag

Hi, Can you give me a link to your blog? I'd like to know how much data needs to be extracted. Thanks

$80 USD in 1 day
(9 Reviews)
4.8

3 freelancers are bidding on average $88 for this job

Verz1Lka

Hello! I'm web-scraping and web-automation expert and i can done your project. I use python language and scrapy framework. My scripts works on windows, mac or linux, but linux is preferably. I can schedule scripts on More

$83 USD in 3 days
(69 Reviews)
6.2
raheelhassan14

Nice to see your project. I have read the details. .Ready to start my work. Looking forward. Thank You!!

$100 USD in 2 days
(25 Reviews)
5.8