Laravel: Write a script to crawl my website to find and list links
This project was successfully completed by sszelag for $80 USD in a day.Get free quotes for a project like this
Project Budget$10 - $30 USD
Completed In1 day
I need a freelancer to write a script that will crawl the entire blog section of my website and retrieve all of the links (broken and unbroken), then dump those links (along with the page origin and the request status) into a log file.
The page origin path must be relative to my domain. The page origin, link url, and request status must be on one line, seperated by a comma, and then a space ( ", ") Like this:
PAGE ORIGIN, LINK URL, REQUEST STATUS
/blog/my-awesome-blog-post, [url removed, login to view], 200
/blog/another-blog-post, [url removed, login to view], 400
/blog/another-blog-post, [url removed, login to view], 500
/blog/what-a-great-blog-post, [url removed, login to view], 301
The crawler should crawl all links at [url removed, login to view]*
Also, the crawler should EXCLUDE my search route: [url removed, login to view]*
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online