Advertising Optimization and Comparison Engine

We are looking for world-class software engineering company with world-class expertise in designing online ad performance based advertising and comparison engines. Someone with prior experience with OpenX, Atlas, or other ad network will be a great plus. A couple of years experience building backend systems for an Internet startup, Internet advertising network, or Search Engine. This team will architect, design, and implement high-volume, scalable, and fault-tolerant back-end server systems. Whatever architecture is chosen it must be able to be hosted on the Amazon Cloud. Project includes: Datafeed processing. Product and pricing information is provided by partner stores through their affiliate program datafeeds. The system will need to be able to process and integrate / normalize feeds from Commission Junction, Linkshare, Google Affiliate Network, and Shareasale, and possibly Amazon and eBay's product API. These feeds are updated daily, and the files are quite large (70+ MB zipped). They should be parsed into products and non-products. Non-products are those that we don’t want appearing on the site. With each new upload, out of stock products are deleted and only the ones that did not appear in the previous file are added. Currently we use excel files and email delivery to manually review all the new product additions. It is very important that the uploading of these files does not impact the page load-times of the site. User-interface. Built like an ecommerce site, but product details pages showing a price comparison table of available retailers along with coupons and dynamic advertising using OpenX. Pages will also need to be created on the fly and cached so database queries MUST run fast. Admin interface. Back-end login for management of products, brands, stores, styles (product categories), coupons, on-site content management (FAQ, privacy policy, meta tags, etc), file upload rules, AND advertising etc. Web Scraping. Must have the understanding of how to scrape data and store in database using Python (for multithreading) and proxy servers. Scalability Requirements Product database = approx. 4,000,000 records Daily traffic = 200,000 – 1,000,000+ Tag Database: Estimated over 12,000,000 records We are looking for a company with a strong track-record of web-database projects like this, can start work immediately, and complete the project quickly. Companies that can provide ongoing support and hosting management strongly encouraged to submit a proposal. This is not a one-time project, but hopefully the start of a long-term mutually beneficial relationship. Background and Qualifications: - Super smart, great coders, motivated, fearless, self-starter, quick learner. - Proven track record designing, implementing, deploying, and shipping high-volume and scalable software. - Proficiency in PHP, MySQL, Javascript,C++, programming in Unix/Linux environment. - Experience with one or all of the following a plus: MySQL, Apache, AJAX, JavaScript, Python, etc. - Bachelor's degree in Computer Science or related discipline is a definite plus (Math, EE, etc.); MS or PhD always a plus. - Experience with building applications on the Amazon cloud Company/Individual MUST be willing to sign NDA before any more specs are given to them. Please bid with confidence.