Data crawling jobs

I am looking for a freelancer who can help me with a web crawling project. I will provide a list of websites that I want to crawl, and I do not need the data to be cleaned and structured as I will handle that myself. The intended use of the crawled data is for other purposes. Ideal Skills and Experience: - Proficiency in web crawling and data extraction - Knowledge of programming languages such as Python or Java - Familiarity with web scraping tools and libraries - Attention to detail and accuracy in data collection - Ability to work independently and meet project deadlines

...- You will have to provide us with the Text and we will update or if we will need to write the text content, that will be considered additional work and you will have to pay for that 8. Image Alt Attributes - that can be done on a partly basis 9. Canonical Tags - can be done 10. Anchor Text Optimization - on a partly basis 11. Page Depth - will depend on CMS and other parameters Indexing and Crawling 1. - we will review and if required re-write 2. - we will review and if required we will suggest changes and if CMS will allow we will create and upload a fresh one 3. Index Coverage Issues - will depend on the nature of the issue 4. Pagination - we will have to check the back-end code section 5. Orphaned Pages - in that you will have to help us, identifying and locating

Data Scraping Tool to Extract Contact Information from Apollo Skills and Experience: - Proficiency in data scraping and web crawling techniques - Experience with Apollo platform and familiarity with its data structure - Strong understanding of data quality and cleaning processes - Expertise in handling CSV files and manipulating data within the format - Attention to detail and ability to accurately extract contact information Project Description: I am looking for a skilled freelancer who can develop a data scraping tool to extract contact information from the Apollo platform. The tool should be able to scrape the required data efficiently and accurately. Specific Requirements: - Scraping Contact Information: The tool should be desi...

...Little Helpers" you will notice that the website home page is indexed, but the content is not displayed with the message" No information is available for this page. Learn why", which points to an issue with Robots.TXT. However all testing using tools provided by google as well as external tools shows that the is not blocking the site from being crawled and there should be no issue with crawling the site? I have attached Screenshots to highlight the problem. The problem seemed to have started around the beginning of July 2023. I had not made any changes to the website when the problem started happening. Info that may help : 1) Site is built in Shopify 2) Site has been operating for many years 3) The domain was changed from . to .com in June 2022 Please can you ...

Web crawling for conference I am looking for a freelancer who can perform web crawling for a conference website. The specific information that needs to be extracted is the schedule and agenda of the conference. Ideal Skills and Experience: - Proficiency in web crawling and data extraction - Experience with extracting data from conference websites - Familiarity with Excel and the ability to deliver the extracted data in an Excel spreadsheet format The web crawling task needs to be performed once, and the data should be delivered in an Excel spreadsheet. Attention to detail and accuracy are crucial for this project.

I am looking for a freelancer to create and deliver an Excel Spreadsheet of product names and their respective prices, volumes (if possible) and before discount prices (if possile) from a beauty e-commerce website. The needs are for a one-time task, meaning I do not need the web crawling to be done on an ongoing basis. If you feel that you possess the skills necessary to complete this task, please reach out and we can discuss the specifics in more detail.

...the website's performance on search engines. Project Scope: SEO Audit: Conduct a thorough audit of our website to identify strengths, weaknesses, and opportunities for improvement. Analyze current SEO performance, including website structure, content quality, backlink profile, and technical aspects. Identify areas for potential penalties, duplicate content, and issues affecting search engine crawling and indexing. Keyword Research: Perform in-depth keyword research specific to the insurance industry in the US. Identify high-volume and relevant keywords related to insurance services, policies, and coverage. Analyze keyword competitiveness and prioritize target keywords based on search intent and potential traffic. On-Page Optimization: Optimize existing website content fo...

...identify and SOLVE any potential issues that may have arisen during the migration process. Key requirements for the project include: - Conducting a thorough audit of the website to identify any technical SEO issues, including 301 redirects - Checking for any broken links or missing pages - Ensuring that the website's sitemap, and htaccess files are properly configured for optimal crawling and indexing by search engines - Ensuring that search console is properly set up for the 3 shops Ideal skills and experience for this job include: - Strong knowledge of WordPress and its SEO capabilities - Experience conducting SEO audits and implementing on-page optimization strategies - Familiarity with tools such as Google Analytics 4, Google Search Console and Semrush (all of them...

...Report progress to a webhook at intervals - Post the content in batches to a webhook - Must be written in javascript or typescript Specifications =========== It takes 5 input parameters: - starturl [required] - contentidentifier (default: “”, type:string) - maxpages (default: 10, zero = all pages, type:string) - issitemap (default: false, type:string) - batchsize (default: 25, type:integer) crawling behavior ============= -if the “issitemap” param is set to "true" only scrape the links on the sitemap. Otherwise, follow all links that point to the same domain as the starturl. -Respect - If maxpages has been set to 0 we must crawl all pages. Otherwise only scrape the number of pages set by the maxpages parameter. - I need the content of t...

I will optimize your website for SEO, resulting in a higher ranking on search engines once crawling is complete. Within 1-2 weeks of project completion, you will observe your website being indexed on the first page. Our primary and targeted keyword will be "HereToKnow". There are a total of 6 pages and 60 post pages that have been published. The charge per page will be 150 Indian Rupees. Therefore, the total number of pages is 66, resulting in a total amount of 9,900 Indian Rupees. To calculate this, we multiply 66 by 150, which equals 9,900. Our goal is to finish this project before the deadline.

...develop a data crawler and scraper for my project. The primary goal is to extract both real text feed and video data from some website . The live feed data and video live streaming data both I need to fetch from the other websites. Websites to scrape data from: - [Input field for client to manually input websites] - News websites - Social media platforms Real-time or periodic updates: - Real-time - Periodically - I'm not sure, please advise Frequency of data updates: - Hourly - Daily - Weekly Ideal skills and experience: - Proficiency in web scraping and data crawling - Strong knowledge of different scraping techniques and tools - Experience in extracting real text feed and video data - Familiarity with various w...

...be able to extract the following details from fashion brands websites: - All Brand data (Name, Contacts...) - All data of Shops/Boutiques where the brand is distributed (name, adress, website...) - All products details (product name, type, price, colors, images, product description...) - Social media netwrok data (followers names) 2) The crawler should be able to extract the following details from fashion boutiques websites: - All Boutique data (Name, Contacts...) - All data about brands the boutique is distributing (name, adress, website...) - All products details (product name, type, price, colors, images, product description...) - Social media netwrok data (followers names) 3) the data set must so that data in between table 1 ...

Project Title: Crawler stops running on Linux server Description: - I am looking for a skilled developer who can help...Although there is no specific error message provided, the crawler has been stopping unexpectedly on the Linux server. - I have access to the server logs, which may provide useful information for debugging and resolving the issue. - The ideal candidate will have experience in troubleshooting and debugging Python programs, particularly on Linux servers. - Additionally, familiarity with web scraping and crawling techniques would be beneficial for understanding the code and identifying potential issues. - The goal of this project is to identify and fix the issue causing the crawler to stop running on the Linux server, ensuring that it runs smoothly and continuously as...

1. Crawler for Real Estate Listings: Objective: To create a web crawler that fetches real estate listings from specific websites related to Mallorca (such as Engel, First Mallorca, Idealista, Foto Casa) and downloads 20 new listings daily. Features: Web Crawling: Crawl specified websites and extract real estate listing information. Filtering and Parsing: Parse the listings to extract key details like price, location, number of bedrooms, etc. Daily Updates: Automate the crawling to run daily and retrieve 20 new listings. 2. WordPress Plugin: Objective: To integrate the information obtained by the crawler into the back office of a WordPress site with clean design. Features: Display Listings: List properties in a clean and structured way. Drop-Down Menu and Filter: Allow users...

Crawling data from Remote website using php-webdriver I am looking for a skilled developer to help me crawl data from a remote website using php-webdriver. The specific data I need to crawl is the text content. Also remote website uses Jquery ajax to avoid from direct html parser. Skills and experience required: - Proficiency in php-webdriver - Strong understanding of web scraping and data crawling techniques Data storage: The crawled data should be saved into a database. Crawling frequency: The data should be crawled and updated on a weekly basis. Please include examples of previous projects or relevant experience in your proposal.

...Feeding Journey: Record of breastfeeding, formula feeding, or introducing solids. Baby's reactions to new foods. Weaning process and transitioning. Chapter 7: Developing Personality: Notable personality traits emerging over time. First laughs, cries, and expressions. Interactions with siblings or pets. Chapter 8: Physical Achievements: Rolling over, sitting up, crawling, and standing. Baby's reaction to different stimuli (toys, music, etc.). First attempts at grabbing, holding, and manipulating objects. Chapter 9: Communication Milestones: Babbling, making sounds, and first words. Attempts to mimic sounds or gestures. How the baby responds to different voices and languages. Chapter 10: First Adventures: Baby&#...

...Ongoing SEO maintenance needed: Yes, both on-page and off-page SEO - Desired timeframe for seeing results: More than 3 months I am looking for an experienced SEO specialist who can help optimize my website for better search engine rankings. The ideal candidate will have expertise in technical SEO, including optimizing website structure, improving page load speed, and ensuring proper indexing and crawling by search engines. In addition to technical SEO, I also require ongoing SEO maintenance, both on-page and off-page. This includes keyword research, content optimization, link building, and monitoring and adjusting strategies to improve search engine rankings over time. I understand that SEO is a long-term process and it takes time to see significant results. Therefore, I have ...

...issues that could impact search engine crawling and indexing. Implement structured data () to enhance search engine understanding of the site's content. Local SEO: Optimize the website for local searches, considering the nature of the medical tourism business. Manage and optimize Google My Business listing. Content Strategy: Content Marketing – Implement a strategy for marketing content developed for the site and distribute content through various channels such as blog posts, articles, and infographics related to medical tourism. Use keyword research to inform content topics and ensure content is original, informative, and engaging. Performance Tracking and Reporting: Analyze Google Analytics and Google Search Console data for improving ...

I will provide an excel file with links of the product detail pages of Link Format: I need a program which gets the content text and pictures of these pages into an MS access database. Every data row should include for every div class="prems--cms-container" - The item number "c01-108-b023eb" of the above sample link - ID - text - pictures Details of which div I want to get to be defined Prefered programing language: MS Access or VB.Net Please give me your price per hour and time estimate in hours.

Looking for an SEO Specialist, who will do onpage and off page optimization of my website to grow organic traffic to my website Keyword Research and Analysis: Conduct comprehensive keyword research to identify relevant, high-traffic, and low-competition keywords for our industry. On-Page SEO Optimization: Implement on-page SEO best practi...Research and Analysis: Conduct comprehensive keyword research to identify relevant, high-traffic, and low-competition keywords for our industry. On-Page SEO Optimization: Implement on-page SEO best practices, including meta tags, headings, URLs, and internal linking, to optimize website pages for targeted keywords. Technical SEO: Audit and identify technical issues that may impact search engine crawling and indexing. Off-Page SEO and Link ...

I am looking for a freelancer who can help me identify and resolve indexing issues on Google. Main Issue: Indexing Issue...who can help me identify and resolve indexing issues on Google. Main Issue: Indexing Issues I am unsure of the reasons why indexing is not working, so I need someone with expertise in troubleshooting and resolving these types of issues. Timeline: Not necessary in a rush. Skills and Experience: - Experience in identifying and resolving indexing issues on Google - Knowledge of Google's crawling and indexing algorithms - Familiarity with website optimization techniques - Strong problem-solving skills - Excellent communication and collaboration abilities If you have the skills and experience necessary to solve this indexing issue, please submit...

I want to ask an expert to review the Google Search Console and help with fixes to enable sooner crawling by Google. Project using live time traking monitoring.

Need help in crawling content from a blogspot which contains links, images and plain text. It is legal and no issues.

I am looking for a talented freelancer to create 2D illustrations for a...illustrations should be in line with this concept. The site searches for ads for used cars on other sites, allowing the user to gather various offers on the platform. The illustration should clearly show the idea of crawlers gathering car ads The links below show similar generic images, which can be adapted for the automotive market. The image will be insert in this template

I am looking for a freelancer who can develop an Search engine (Apache Nutch 1.0) Crawler system with an integrated AI backend. The project requires the following functionalities: - Optimized crawling and indexing - Advanced crawling and indexing with custom plugins - Crawling, indexing, and AI-driven data analysis - single page search engine The data analysis should be integrated with an existing database system. Additionally, the ideal candidate should have experience in working with Neural Networks as the AI algorithm.

I am looking for a soccer auto-betting software that has the following features: - Live match updates and statistics to stay updated on the latest information - Automated betting based on pre-set strategies for efficient and seamless betting - Integration with popular bookmakers for easy and convenient betting I already h...experience for this project include: - Proficiency in developing betting software with live match updates and statistics integration - Knowledge of automated betting strategies and experience in implementing them - Familiarity with popular bookmakers and their API integration for a seamless betting experience - Ability to create a user-friendly interface for easy usage of the software -Familiar with crawling technology, familiar with WINFORM, C# 4.8, WEBSOCKET,...

...architecture, crawling, and indexing mechanisms. Skills in identifying and fixing technical issues like broken links, crawl errors, page speed optimization, and implementing structured data (schema markup). Content Creation: The ability to produce high-quality, engaging, and valuable content that aligns with user intent and incorporates relevant keywords. Content can include articles, blog posts, infographics, videos, and more. Link Building: Knowledge of strategies to acquire high-quality backlinks from authoritative and relevant websites to improve a website's authority and trustworthiness. Analytics: Proficiency in using web analytics tools like Google Analytics to measure and analyze website traffic, user behavior, and conversions. Being able to draw insights and ...

6 Month old Headless CMS website with over 150K pages & growing fast. Similar to a (cars for sale) listing-style site with a blog. Current team consist of a head developer, front end designer, back end SEO structure. Attached are snapshots of the sites indexing issues (see attached), Needed is someone that can jump...issues (see attached), Needed is someone that can jump into the mix & fix the issues, (not an advisory role, but an actual do the work role). We require a proficient individual who can integrate with our team to address indexing issues. The candidate must be capable of troubleshooting existing problems and establishing systems for future content indexing. A comprehensive understanding of swiftly crawling and indexing sizable content sites is essential.

...can help me with a website scraping project. I need someone with experience in data scraping and web crawling to extract data from a website to extract data for 3 bed 2 bath townhomes which has no basement. 1. Extract for sold properties 2. Extract for listed properties . Here are the details of the project: Website: To be provided Data Refresh: - The data needs to be refreshed once daily. Data Storage: - I need the data to be stored permanently in Grafana And have a time series graph Skills and Experience: - Strong expertise in data scraping and web crawling. - Proficiency in programming languages such as Python. - Experience working with APIs and web scraping frameworks. - Familiarity with data s...

Hi Freelancers! Our website is terrible on loading. We have installed the Seraphinite Accelerator plugin but still it is not very fast loading. In our opinion there are a couple of unused plugins which do generate quite some server load. Our configur...mem: 512 MB PHP max message: 128 MB PHP time limit: 240 PHP max vars: 4000 SUHOSIN geïnstalleerd: – MySQL versie: 5.7.41 Max. Upload: 128 MB Current website: 10.8GB Also we experience a lot database connection error. Currently we host with Strato but not on a Pre-installed WP server. We might need to migrate to another server when the current server is not sufficient. Currently we have the crawling by search engines not enabled due to the slow loading issues. We need a WP expert on the server site. Thanks!

...visibility and rankings. Specific Requirements: - Optimize the URL structure of the website to make it more search engine friendly - Implement best practices for URL optimization, including using relevant keywords in URLs - Ensure proper website indexing and crawling by search engines Ideal Skills and Experience: - Proven experience in onsite SEO optimization - In-depth knowledge of URL structure and optimization techniques - Familiarity with meta tags and keywords for SEO purposes - Strong understanding of search engine crawling and indexing processes Restrictions on Keywords: - There are no restrictions on the keywords used for onsite SEO optimization Additional Services: - There are no additional services required to boost the SEO performance of the website If you...

I am looking for a freelancer who can help me prevent crawling of all URL's on GitLab in order to improve website performance. The ideal candidate should have experience in the following: - Knowledge of GitLab and web crawling prevention techniques - Understanding of website performance optimization - Familiarity with authentication methods, specifically username/password - Ability to assess and implement security measures, such as IP whitelisting The project may also involve defining a specific user agent, although I am not sure if this is necessary. Therefore, the freelancer should be open to discussing and determining the need for a specific user agent. If you possess the skills and experience mentioned above, please submit your proposal. Thank you.

We are crawling a cloudflare protected website. We need to make interactions with the website buttons, therefore we have a python code that runs chrome over Selenium. Our bot needs to check the website every 1 minutes, therefore we use 100 http proxies and rotate and shuffle these proxies when giving them to the bots. However, every 1 or 2 weeks, all of our proxies get blocked by the cloudflare systems. Is there any solutions around this problem?

...experienced C# developer to update our existing web crawling tool Ver2. The specific updates needed for the tool are bug fixes. ============== I need to crawl data from my tool is working fine before but they have updated the site so the data structure was changed. That is why I need someone modify to the correct field. No need to add more functions. JUST NEED TO map the correct field for a tool to get the right information. Product ID for testing: 3256804659474341 3256804340980991 Install MongoDB before running the tool. ============== I prefer working with a freelancer for this project. Previous experience with web crawling tools is required. Skills and experience required

...develop a website crawler for me. The purpose of this crawler is to gather data from various websites. Download and convert 5 website products and properties into xml or xls format per week. Skills and experience required: - Proficiency in web scraping and data extraction - Strong knowledge of programming languages such as Python or JavaScript - Familiarity with web development frameworks and libraries - Ability to implement authentication for the crawler - Experience in crawling and extracting text-only data The ideal candidate should have a good understanding of website crawling techniques and be able to handle authentication for the crawler. They should also have a strong attention to detail to ensure accurate data extraction. If you have...

I have a VPS and I need a python script that will take the contents of a text file, run it through Pinecone and then use the OpenAI Embeddings API to make it searchable, and finally allow me to call the chatgpt API to be able to query it. The instructions for doing this are here: (I don't need the web crawling or conversational interface). The final result will be a URL where I can send a question about my text file, eg. does the N1 section of the AQA Maths syllabus contain The script would then use the information in (attached) to answer the question. I would need to be able to change the content of the text file myself, either stored on the server or saving it as an external URL. I will provide:

I am looking for an experienced C# developer to update our existing web crawling tool. The primary focus of this project is to fix bugs and errors in the current tool. Specific updates needed for the web crawling tool include: - Fixing bugs and errors in the current tool Additional functionality is not required for this project. The client would like just a few updates to be made to the existing tool. The preferred timeline for this project is within 1 week. Ideal skills and experience for this job include: - Strong experience in C# development - Proficiency in web crawling and data scraping - Attention to detail and ability to fix bugs and errors - Ability to work within tight timelines

I have a pet e-commerce and need to crawl products from other competitors websites into my shopify store.

Job Description: We are seeking an experienced SEO expert with a strong background in resolving SEO crawling issues in Google Search Console. The ideal candidate will have in-depth knowledge of search engine optimization best practices, technical SEO, and website crawling and indexing processes. This role requires a proactive and analytical mindset, excellent problem-solving skills, and a thorough understanding of Google Search Console. Responsibilities: • Identify and diagnose crawling issues in Google Search Console affecting website visibility and indexing. • Analyze server logs, crawl stats, and error reports to determine the root cause of crawling problems. • Review and optimize files to ensure proper access for search engine crawlers. &bull...

I am experiencing an SEO crawling indexing issue in Google Search Console for my recently developed website. This issue is causing the website to not properly be indexed in search results, and I need help to resolve it. I am looking for research and investigation into the source of the issue, strategies to rectify it, and ongoing maintenance and checks to ensure the issue stays resolved. The website has gone live less than a month ago, so this issue is quite new. I need someone who can provide expertise and knowledge on how to resolve this issue.

...& Automation Expert, you will play a crucial role in developing and maintaining our web applications, implementing data scraping solutions, and automating various business processes. Responsibilities: Design, develop, and maintain web applications and APIs using modern programming languages and frameworks. Create data scraping and web crawling solutions to gather information from various online sources. Implement automation workflows and systems to streamline business processes and improve efficiency. Collaborate with cross-functional teams to understand project requirements and deliver high-quality solutions. Troubleshoot and resolve issues related to web development, data scraping, and automation. Stay updated with emerging technologies and industry t...

As a knife collector, I am looking for a freelancer to help me build a database web crawler using Scrapy, which can search for specific knives that I identify and want to buy. Specifically, I will need the crawler to gather brand, model and price informat...a freelancer to help me build a database web crawler using Scrapy, which can search for specific knives that I identify and want to buy. Specifically, I will need the crawler to gather brand, model and price information from e-commerce sites, e-bay, Etsy and other knife forums..Once collected, I need the data to be saved in a format that I can use and pursue purchase of the identified knives. I am looking for an experienced freelancer with knowledge on web crawling, Python, Scrapy and data processing. Thank you for...

...Skills and Experience: - Proficiency in custom-built website development and management - Expertise in website crawling and indexing - Knowledge of SEO best practices and search engine algorithms - Experience in working with files and resolving crawling issues - Familiarity with website backups and restoration processes Requirements: 1. Fixing : - Identify and resolve any errors or misconfigurations in the file - Ensure that the file allows Google and other search engines to crawl and index the website properly 2. Resolving technical issues: - Conduct a thorough website audit to identify any other technical issues affecting search engine crawling - Address and resolve these issues to improve website performance and visibility 3. Backup and restorat...

...that could impact search engine crawling and indexing. -Implement structured data () to enhance search engine understanding of the site's content. -Local SEO: -Optimize the website for local searches, considering the nature of the medical tourism business. -Manage and optimize Google My Business listing. -Content Strategy: -Content Marketing – Implement a strategy for marketing content developed for the site and distribute content through various channels such as blog posts, articles, and infographics related to medical tourism. -Use keyword research to inform content topics and ensure content is original, informative, and engaging. -Performance Tracking and Reporting: -Analyze Google Analytics and Google Search Console data for ...

...in-depth research and analysis. - The main objective of this project is to gather website URLs from the Tor Network. - The ideal candidate should have extensive experience in crawling the Tor Network and extracting valuable data. - Must have experience using Tor and Nginx. - The crawler should be able to navigate through the network, collecting website URLs for further analysis. - I am particularly interested in experts who have a strong understanding of the Tor Network's structure and protocols. - The selected candidate should be able to handle large amounts of data and deliver accurate and reliable results. - Familiarity with data analysis techniques and tools is a plus. - The project requires meticulous attention to detail and the ability to work inde...

Project already built mostly but need to be finished. need a smart python engineer that is able to guess the changes, understand and fulfill requirements. Not someone that stays and is delivering work that does not match what I have said multiple times. Project is already done but needs a perfectionist to do required changes. I am absolutely looking for along term partnership and strong incentives in the future.

I am looking for a skilled Wordpress bugfixer to help me enable my Wordpress website to be crawled and indexed by Google. The specific issue I am experiencing is that my website appears in Google search results, but is not fully indexed. I have an SEO plugin or tool already installed, and this is a new website so it has never been successfully indexed by Google in the past. The suc...website appears in Google search results, but is not fully indexed. I have an SEO plugin or tool already installed, and this is a new website so it has never been successfully indexed by Google in the past. The successful candidate must have experience with Wordpress, bug fixing, SEO, and Google indexing. A and a is up and running but something is preventing Google from crawling my website.

Need a software developer to create custom web crawling script to get specific data.

...website's visibility and relevance to search engines. - Off-page optimization: This involves building high-quality backlinks, creating social media profiles, and submitting the website to relevant directories to improve its authority and online presence. - Technical SEO: This includes optimizing website speed, fixing broken links, improving mobile-friendliness, and ensuring proper indexing and crawling by search engines. I have a specific target region/demographic for my SEO campaign [Note: this information was not provided by the client]. I understand that SEO is a long-term process and I am willing to invest time and effort to see results. However, I would like to see noticeable improvements within a timeframe of 6 months. The ideal candidate should have: - Proven expe...

...how after the plasterer leaves the kitchen company, tilers, electricians, plumbers and carpenters turn up and pretty much destroy the house. I want it to be really ridiculous, some examples are the tiler shooting tiles onto the wall with a tile bazooka, a truck delivering the kitchen but they reversed through the house to deliver the kitchen, the Electrition has fallen through the roof while crawling in the ceiling so his legs are just dangling from the ceiling, the plumber is inside the house on a excavator because the pipes are in the wrong place and he’s smashed down walls to get the excavator in, the carpenter is smashing up the place because he’s trying to make a door jamb fit in a hole that’s to small for the door, there’s a burst water pipe squirti...