Find Jobs
Hire Freelancers

Batch scrape and analyze web pages (Perl or Ruby)

$30-250 USD

Completed
Posted over 10 years ago

$30-250 USD

Paid on delivery
Build a script in Perl or Ruby. The purpose of this script is to determine if the web page has special logic to handle display on mobile phone screens (such as serving different HTML or using special CSS). The script will take input from STDIN. The input will be a list of target URLs (with one URL per line). For each target URL perform the following steps: 1) Download a temporary copy of the web page with the User Agent string set to emulate Internet Explorer 9 and follow redirects (if any) 2) Calculate the following information about the HTML file returned a) URL of the downloaded file (different from the target if redirects were encountered) b) Count of characters in the file c) Count of tags in the file d) Determine whether the file includes a line with the string "@media" and also contains either "max-device-width:" OR "max-width:" 3) Download a temporary copy of each of the CSS files referenced in the HTML file 4) Calculate whether the following is true for ANY of the CSS files a) Determine whether the file includes a line with the string "@media" and also contains either "max-device-width:" or "max-width:" (this is called a media query) 5) Download the target URL with the User Agent string set to emulate an iPhone 5 6) Calclulate the following information about the HTML file returned a) URL of the downloaded file b) Count of characters in the file c) Count of tags in the file 7) Write a line to STDOUT (see output format mentioned below) with the information that was calculated above. 8) Delete any temporary files downloaded ------- OUTPUT LINE FORMAT: The output file will have one line per target URL. The line consists of tab delimited (\t) text. Below is a list of the items on each row: 1) URL of the downloaded file when user agent is IE9 2) Count of characters in the HTML file when user agent is IE9 3) Count of tags in the HTML file when user agent is IE9 4) "MQ" if the file includes the media query line when the user agent is IE9 (otherwise blank) 5) "MQ" if ANY of the linked CSS files contained a media query line when the user agent is IE9 (otherwise blank) 6) URL of the downloaded file when user agent is iPhone 5 7) Count of characters in the HTML file when user agent is iPhone 5 8) Count of tags in the HTML file when the user agent is iPhone 5 I will provide URLs for three web sites to test that the script is working correctly. If it gives correct answers for all 3, I will consider the work complete.
Project ID: 4836890

About the project

8 proposals
Remote project
Active 11 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
Please see private message.
$150 USD in 3 days
5.0 (7 reviews)
3.9
3.9
8 freelancers are bidding on average $172 USD for this job
User Avatar
Hello, I am a Perl scripting expert prepared to complete this task for you. Please forward your list of test sites. I will open a PMB to discuss details.
$187 USD in 5 days
4.9 (99 reviews)
5.6
5.6
User Avatar
Dear Terralever1 , Perl + web professionals here. Please, accept this bid to get your work done relatively fast and nicely.
$138 USD in 5 days
4.5 (3 reviews)
3.5
3.5
User Avatar
Your description is clear. I could easily do this using Perl.
$111 USD in 3 days
5.0 (1 review)
2.8
2.8
User Avatar
Hi, I am scraping expert, More details in PM.
$250 USD in 5 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I can do this for you. See my PM for details.
$129 USD in 3 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I'm interested.
$300 USD in 5 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, I can do this with both hands tied behind my keyboard.
$111 USD in 5 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Mesa, United States
5.0
27
Payment method verified
Member since Jan 17, 2013

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.