Laravel: Write a script to crawl my website to find and list links

This project was successfully completed by sszelag for $80 USD in a day.

Get free quotes for a project like this
Employer working
Project Budget
$10 - $30 USD
Completed In
1 day
Total Bids
Project Description

I need a freelancer to write a script that will crawl the entire blog section of my website and retrieve all of the links (broken and unbroken), then dump those links (along with the page origin and the request status) into a log file.

The page origin path must be relative to my domain. The page origin, link url, and request status must be on one line, seperated by a comma, and then a space ( ", ") Like this:



for example:

/blog/my-awesome-blog-post, [url removed, login to view], 200

/blog/another-blog-post, [url removed, login to view], 400

/blog/another-blog-post, [url removed, login to view], 500

/blog/what-a-great-blog-post, [url removed, login to view], 301

The crawler should crawl all links at [url removed, login to view]*

Also, the crawler should EXCLUDE my search route: [url removed, login to view]*

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online