Find Jobs
Hire Freelancers

A web crawler that can pick up JS onclick events

£250-750 GBP

Completed
Posted over 11 years ago

£250-750 GBP

Paid on delivery
I need a web crawler that can find links on a page and list them. Even links that are hidden by javascript onclick events. It must 1) log the status code of the url given and any urls redirected through - example if given a url that redirects to another url with a 301 status code I need the 301 code and the 200 that it redirects to. 2) List the urls in a redirect chain if there is a chain. 3) Get all the links on the page given even ones hidden in onclick divs or other methods. 4) list all the rel, anchor text and image url elements for each link if they exist 5) follow redirects if required by meta redirects or [login to view URL] and list the urls in the redirect 6) We need to be able to run this from command line on a linux machine. I don't care too much what language but we need to be able to use it with php. Previously we were running HTML unit through shell_exec in php and then capturing what was echoed to the command line. Continuing like this is fine. We had some luck with HTML unit but we have not got enough experience to get all our requirements.
Project ID: 4058819

About the project

8 proposals
Remote project
Active 11 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
I have lots of experience with writing web automation software, please see PMB for examples of my previous projects related to web automation. Available to start immediately and finish as soon as possible. Best Regards, Zeke
£500 GBP in 10 days
4.9 (20 reviews)
4.8
4.8
8 freelancers are bidding on average £390 GBP for this job
User Avatar
I can help in your project, please check PMB and our ratings/reviews to get idea of our experience. Please let me know if you have any queries.
£250 GBP in 3 days
5.0 (32 reviews)
6.3
6.3
User Avatar
I look forward to discuss further and can deliver the project
£520 GBP in 12 days
5.0 (29 reviews)
6.2
6.2
User Avatar
Hi, Ready to start your work. Eagerly awaiting for your positive reply. Please check your inbox for further details. Thanks, Shaik.
£250 GBP in 5 days
5.0 (25 reviews)
5.0
5.0
User Avatar
Hello, I can do this work for you and I'm ready to start. Please see pmb for details. Regards Raul
£250 GBP in 7 days
4.9 (11 reviews)
4.8
4.8
User Avatar
Scrapping/Parsing/Automated engine Experts here. Check the message with attached samples and contact us. SI Team.
£750 GBP in 10 days
2.0 (2 reviews)
4.2
4.2
User Avatar
Hi, Please check your PMB regards, Arun
£700 GBP in 5 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED KINGDOM
Stockport, United Kingdom
0.0
0
Payment method verified
Member since Dec 17, 2012

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.