In Progress

Web scraping and importing into MySQL DB

Simple scraping project.

1) At 09:15 AM IST, code will go to:

[url removed, login to view]

2) Under the "Equity" tab, it will select the drop down as "All"

3) It will then scrape all the columns in the table viz:

Symbol, Pre-Open, Price, Chng, % Chng, Prev. Close, Quantity, Value (in lakhs), FFM, Cap(crs.), NM 52w H, NM 52w L, Data Date (IST), UpdateTime (IST)

4) Note that Pre-Open is a picture and you need to store the picture inside blob structure in MySQL

5) Once the data is scraped for all entries of the table as per the columns mentioned above, the code will dump that into a MySQL DB's new table - lets say we will call it "Pre_Open" (ID and password will be provided)

6) In the MySQL DB, the Symbol will need to be searched with for Symbol ID from a master table and Symbol ID along with the other columns above will need to be stored in the new table. So basically, you would get the ID for the Symbol from "Equity_Master" table and that ID along with the columns above would need to be stored in the "Pre-Open" table

7) The next day the code should again automatically run at 09:15 AM IST and scrape this data and repeat the process.

8) Provide with an additional small UI that can prove that the blob data for the image described above ("Pre-Open" column), can be shown to the user in a image control after reading the blob data from the DB. This is very simple UI just to prove that the image can be generated form the blob data.

Some points to note:

1) The code can be in C#/VB.Net v5.0 or above.

2) It should not use web browser control for scraping.

3) While scraping, proper progress message should be shown to the user so that the user understands that scraping is in progress and what is the status of the data being scraped.

4) There should not be any need to run the code manually every day. The code once run should automatically "awaken" at 09:15 AM IST and scan.

5) All dates and datetime stamp to be stored in IST

6) For holidays, when the market is closed, when the code awakens at 09:15 AM IST, it should understand from the top of the page which states: "Pre-Open Order Collection CLOSEDAs on Oct 07, 2016 09:07:47 IST", whether a new set of data has come in for that day or its a holiday and the old data of previous date is being shown or not. If an old data is shown, the code will do nothing and sleep, else it will scrape

7) Your code will be tested for 1 week (to include a holiday testing) and then milestone will be released

8) No upfront milestone payment pls.

9) Finally, pls read all the text above and then only bid.

10) Delivery in 2 days if thats fine.

Skills: .NET, C# Programming, MySQL, VB.NET, Web Scraping

See more: web page gettin auto logout time vb net code, iphone web db mysql webapp, web scraping using web browser, web browser interface mysql, customer web browser, web browser saves settings, random web browser, save picture visual basic web browser, 2005 web browser automate, web browser tabs, mysql importing sql gzip file, simple web browser, web browser mmorpg, web browser application bots, automated web browser

About the Employer:
( 1 review ) Kolkata, India

Project ID: #11721661

Awarded to:


A proposal has not yet been provided

₹2250 INR in 3 days
(0 Reviews)

5 freelancers are bidding on average ₹6439 for this job


Hello I am very interested in your project。 I am an expert in C#, php,, web scraping, web automation, selenium and others. Scrape is my top skill. I am ready to start to work this job. And I am free in rece More

₹11111 INR in 2 days
(96 Reviews)

Hello, I can do the task in less time, and 100% accurate - I'm an experienced software developer specialized in desktop applications, tools, and scripts. - I write programs using .NET since 2012, and using classic More

₹6000 INR in 2 days
(12 Reviews)

Hi, I'm a Web Scraper. I have been scraping website since last year. I scrape the website of tripadvisor, mobofree and [login to view URL], facebook, amazon, youtube and many more. Give me a PM so that I can give you More

₹5555 INR in 3 days
(9 Reviews)
₹8388 INR in 3 days
(1 Review)

Dear Friend, I'm a 13+ year’s experienced windows and web application programmer and have been using Microsoft technologies in delivering solutions to my clients. I understand you would like to have auto-scrape More

₹10000 INR in 5 days
(1 Review)