Clojure etl jobs

...in data migration across CRP, SIT, UAT, and GoLive phases in Oracle Fusion ERP implementation, utilizing ETL tools. * Experience in SQL Server, T-SQL, and SSIS Job Responsibilities: Data Extraction: * Extract data from various source systems, including flat files, spreadsheets, legacy databases, and external APIs. * Develop and optimize SQL queries and scripts for efficient data retrieval. Data Transformation: * Perform data cleansing, validation, and transformation to ensure data quality. * Convert data from source formats to Oracle-compatible formats. * Apply business rules and data mapping for aligning source data with target data structures. ETL Development: * Develop and maintain ETL (Extract, Transform, Load) processes using standard tools. * Handle data integ...

Job Title: Senior Master Data Management (MDM) Developer Monday to Friday Remote via Zoom 2-3 hours/day IST Night (7pm to 12amIST) Salary: 30,000 INR per mont...Monday to Friday Remote via Zoom 2-3 hours/day IST Night (7pm to 12amIST) Salary: 30,000 INR per month We are seeking a Senior Master Data Management (MDM) Developer with a focus on healthcare. Strong SQL skills for data manipulation and querying. Experience with ETL development and integration. Familiarity with Azure Data Factory and cloud/on-prem migrations is advantageous. One needs to be available in the mentioned timings and willing to work by screen share and remote control via Zoom. Skills Required: Master Data Management (MDM) ETL development and integration Strong SQL skills SSIS Azure Data Factory Cloud/o...

As a client, I am in search of a skilled Python developer with experience in XML parsing and handling database operations. I require a Python utility program to parse Talend jobs and determine source and target column details. Key Requirements: * Parse XML files - The Talend mapping file will be supplied in XML format. We're in need of someone fluent with Python's X...or potentially other formats. It should be flexibly designed as we will use this data to load into a database. * Database expertise - An understanding of database interactions and structure are vital. This project will require both a keen eye for detail and the ability to create efficient, effective code. The utility must be well-documented and easy to maintain. Experience with Talend or other ETL tools wo...

I'm constructing a robust real estate scoring model, and need a skillful database engineer proficient in Google Big Query. The project involves: 1. ETL Development: Your main task is to develop an ETL process. You should have experience managing CSV files and integrating APIs. 2. Data Management: The data to handle includes property addresses, sale price history, property characteristics, and people details related to properties. The expert should know how to organize and manage this huge amount of data effectively. 3. Real-Time Updates: Due to the volatile nature of the real estate market, the ability to integrate real-time updates is an asset. An expert in Google Big Query, effective in handling CSV files, and capable of seamless API integration would make the best f...

I'm in search of a Azure Data Factory expert who is well-versed in Delta tables, Parquet, and Dedicated SQL pool. As per the requirement, I have all the data and specifications ready. The successful freelancer will need to be familiar with advanced transform...in search of a Azure Data Factory expert who is well-versed in Delta tables, Parquet, and Dedicated SQL pool. As per the requirement, I have all the data and specifications ready. The successful freelancer will need to be familiar with advanced transformations as the ETL complexity level is high. It's a plus if you have prior and proven experience in handling such projects. Key Skills Required: - Expertise in Azure Data Factory - Deep knowledge of Delta tables, Parquet and Dedicated SQL pool - Familiarity with adv...

Job Description: Develop & support Power BI artefacts and participate in the analytics product development Participate in the analysis of requirements to understand the business and system needs regarding user interfaces, reporting enhancements, new functionality, and back-end ETL operations, should be able to provide estimates against requirements. Code and modify reports using DAX & MDX programming languages. Document all code using standard forms and procedures, adhering to all naming conventions Participate in unit, system, and user acceptance testing Adopt new and emerging technologies to provide solutions to client needs. Additional responsibilities will be team performance management, timesheet reviews, ensuring project delivery on time. Required Primary Skills: ...

"Airflow pipeline I require a professional with experience in Airflow and dbt for my project. You will be handling an ETL/ELT integration specifically for data migration. This migration task involves moving specific tables from one MySQL database to another one. I'm trying to migrate data from mysql to mysql. We need to transfer incremental data on an hourly basis. We are looking to use open source ETL or ELT. In addition to airflow, you can use other open source ETL tools.".

...to be connected to Azure; this is contract management system. 2. One more company recently which was acquired has financial data on-prem on different tenant we need to bring in this data to the Azure cloud. Note : Business central is for the financial data and GL entry ,Dynamics 365 is for the sales and invoicing ,Hubspot for the sales lead, Service Now is the ticketing system , Workato is the ETL tool which connects all the system,Kimble one is the project management platform. There is one billing system acquire going on we might need to bring in the data from this source also. Data Handling: Perform necessary data normalization or transformation before storing the data in Azure SQL Database. The ideal candidate for this role will have past experience with Azure SQL Database...

...with experience in Airflow and dbt for my project. You will be handling an ETL/ELT integration specifically for data migration. This migration task involves moving specific tables from one MySQL database to another one. Your key responsibilities for this project will include: - Identifying the specific tables needed for migration, as indicated by me. - Developing and implementing an efficient migration process using Airflow and dbt. - Ensuring successful and accurate data migration. Ideal skills: - Incremental data must be transferred every hour. - Transfer speed is important. - Extensive experience in Airflow & dbt. - Proficient in MySQL databases. - Experience in data migration projects. - Thorough knowledge of ETL/ELT processes. - Attention to detail as the task ...

...Timings : 10:00 am - 07:00 pm IST Remote (India) • Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines. -Connect with stakeholders daily to discuss project progress and updates. -Work within an Agile process to deliver projects in a timely and efficient manner. -Have worked extensively on Redshift and understands performance tuning techniques and management of Redshift data workloads. -Design and develop Airflow DAGs to schedule and manage ETL workflows. -Implement best practices for data engineering, including data modeling, data warehousing, and data pipeline architecture. -Monitor and troubleshoot ETL pipelines to ensure smooth operation. -Custom Python functions to handle data quality and validation. ...

I need an ETL developer to streamline and automate an existing manual process. This includes: - Transferring two encrypted *. files from a source server to a local machine. - Decrypting these files into *.csv format. - Running SQL on one file and a PowerShell command on the other file to create a staging table. - Setting up a Microsoft SQL Server database into read-write mode from read-only. - Reverting the database back to read-only upon task completion. - Sending a report to the user upon task completion. Ideal skills for this job include advanced SQL, PowerShell, and ETL development. Familiarity with read/write permissions on Microsoft SQL Server and experience with data transfer and encryption/decryption processes are also highly beneficial.

I am seeking an IICS (Informatica Intelligent Cloud Services) specialist to effectively set up and integrate AWS S3/EC2 services with this platform. Key Requirements: - Set up project infrastructure per given specifications - Ensure successful integration of AWS S3 and EC2 with IICS - Establish robust connections and develop ETL processes - Configure parameter files as part of the setup While not specified, an intermediate or advanced level of IICS proficiency would be ideal for this task. Previous experience with AWS services is a must. I have not stated a timeframe for completion, indicating flexibility; however, I would appreciate if interested freelancers could provide their estimated timelines of delivery in their responses.

I am looking for an experienced Senior Data Engineer for in...looking for an experienced Senior Data Engineer for interview training. Your primary responsibility would be data cleaning and preprocessing, design and optimize database and implement ETL processes. Key responsibilities include: - Clean and preprocess data to ensure its quality and efficiency. - Design and optimize databases, aiming for both flexibility and speed. - Implement ETL (Extract, Transform, Load) processes to facilitate the effective and secure moving of data. Skills and Experience: - Proficient in Python, SQL, and Hadoop. - Expertise in handling medium-sized databases (1GB-1TB). - Proven track record in ETL processes handling. Your expertise in these areas will be crucial to the successful com...

Objective: Upgrade an existing ETL process to enhance data validation, error handling, and user feedback for structured Excel files uploaded to SharePoint 2019, for subsequent transfer to a database. The solution could be also implemented via Azure Functions and Sharepoint online if deemed possible. Existing Setup: A basic ETL process developed with the Luigi library, lacking advanced validation and user feedback mechanisms. Solution uses parsing code (Pandas) for Excel files data parsing that can be utilized for the upgrade also. There are 4 different excel files, so 4 different parsers will be provided. The system should manage four unique Excel file types, each in its own SharePoint folder, requiring separate parsers for their distinct structures. An initial selection...

Looking for an experienced MSBI and powerbi developer. You would be working with me to troubleshoot and optimize an existing MSBI solution and powerbi report. Your tasks include but are not limited to: - Data Modeling - ETL Development - Report Development For this project, your primary tools would be: - SQL Server Integration Services (SSIS) - SQL Server Analysis Services (SSAS) - power bi Your experience and skills in these areas will be critical to the success of our project. Only serious and high-quality professionals need apply. This is a challenging role and previous experience in the given areas is a must. Let's enhance our MSBI solution together and make it the best it can be.

...manage and organize data within it. - Ability to ensure data security and integrity throughout the extraction and integration process. **Ideal Skills and Experience:** - Proven track record of working with POS systems, especially Clover, for sales data extraction. - Strong background in cloud-computing platforms, specifically Azure. - Comfortable with setting up and managing data pipelines and ETL processes. - Experience in setting up automated workflows between different technological platforms. - Knowledge of data visualization and interpretation, to guide the final format of sales data within Snowflake. This project requires a freelancer who can swiftly understand our data infrastructure and implement a solution that minimizes manual intervention and maximizes data utility f...

...guidance especially in the area of ETL processes. I need to focus on: - This is an interview support role, You are supposed to help in live interviews. • Extraction techniques – The primary data sources of my interest are platforms like Spark, AWS, Azure, GCP, and Hive. I want to understand effective methods for data extraction from these particular sources. Ideal Skills and Experience: - Expertise in ETL tools for data extraction - Hands-on experience with Spark, AWS, Azure, GCP, Hive - Profound knowledge in data engineering - Experience in career coaching or mentoring will be a bonus - SQL -Python This assistance will give me a competitive edge in my upcoming interviews by providing me with practical skills, knowledge, and confidence in addressing ETL...

I'm seeking an experienced Data Engineer with proficiency in SQL and PySpark. Key Responsibilities: - Develop and optimize our ETL processes. - Enhance our data pipeline for smoother operations. The ideal candidate should deliver efficient extraction, transformation, and loading of data, which is critical to our project's success. Skills and Experience: - Proficient in SQL and PySpark - Proven experience in ETL process development - Previous experience in data pipeline optimization Your expertise will significantly improve our data management systems, and your ability to deliver effectively and promptly will be highly appreciated.

I am urgently requiring the development of an incremental ETL pipeline. To do this job successfully, you will need to provide two main deliverable: -Architecture diagram: Demonstrates your plan for the ETL pipeline structure. -ERD diagram: A depiction of entities, their relationships, and the data flow. Ideal Skills and Experiences: -Broad experience in ETL, particularly incremental ETL pipelines. -Proficiency in creating architecture and ERD diagrams, showcasing complex data flows. - Familiarity with a variety of technologies for implementing ETL process. While we skipped the preference, a broad knowledge base will be advantageous. I am in a hurry for this project, therefore your ability to work efficiently and effectively within a tight schedule ...

Looking for a seasoned AWS expert to assist in optimizing EC2 instances, S3 storage, and Lambda functions for improved performance. The ideal candidate will have: Mainly working on Aws glue jobs , S3,lambda, Crawler, step functions, dynamo DB ,Clouldwatchs, Appflow, parameter store , API Gateways, ETL Jobs,splunk logs CloudFormation - important big bucket,Spunk, Jenkins pipelines* CI/CD * , YAML for creating templates and manifest files * Sql mandatory for date base * ⁠using Postman for API calls * ⁠Python and spark Musk know

...integrity throughout the data lifecycle. - Implement monitoring and alerting systems to proactively identify and address issues. - Stay up-to-date with emerging technologies and best practices in data engineering. *Requirements:* - Bachelor's degree in Computer Science, Engineering, or a related field. - Proven experience as a data engineer or similar role, with a strong understanding of data modeling, ETL processes, and database technologies. - Proficiency in at least one programming language commonly used in data engineering (e.g., Python, Java, Scala). - Hands-on experience with cloud platforms (e.g., AWS, Google Cloud, Azure) and related services (e.g., S3, GCP BigQuery, Azure Data Lake). - Familiarity with containerization and orchestration tools such as Docker and Kub...

I'...of data from spreadsheets. The end goal is to improve the data accuracy. Your Key Responsibilities: - Understand the current data structure in the spreadsheet - Merge information from various sections accurately - Ensure no data is lost or inaccurately merged during the integration process Required Skills and Experience: - Extensive experience with data extraction, transformation, and loading (ETL) tools - Proficient in data integration methodologies - Expert in handling spreadsheets and large sets of data - Proven track record of improving data accuracy When submitting your application, please include your experience in handling similar projects. Your ability to demonstrate a clear understanding of this project and its requirements will be a key factor in my decision-...

...and maintaining Oracle databases to support our organization's applications and business processes. This role requires proficiency in SQL, PL/SQL, database functions, triggers, ETL processes, and database performance tuning. Key Responsibilities: • Design, develop, and implement Oracle databases to meet business requirements. • Perform database administration tasks, including installation, configuration, backup and recovery, and monitoring. • Write efficient SQL and PL/SQL code to support application development, including the creation of functions, stored procedures, triggers, and packages. • Develop and maintain ETL processes to extract, transform, and load data from various sources into Oracle databases. • Optimize database performance t...

...Malaysian freelancer) to play a pivotal role in our dynamic team. In addition to the responsibilities mentioned earlier, the successful candidate will be entrusted with deploying Talend solutions in both testing and production environments. If you possess hands-on experience in deploying ETL solutions and ensuring their seamless operation in varied environments, we encourage you to apply. Responsibilities:- Talend Job Design: Develop efficient and scalable Talend job designs for various data integration and ETL processes. Implement best practices for job design to ensure maintainability, reusability, and performance. Version Control: Implement version control for Talend jobs to manage changes, updates, and rollbacks effectively. Ensure proper documentation of changes ma...

...looking for a skilled Python ETL expert to assist us with a data transformation project. Our goal is to extract, transform, and load data from various sources into a structured format for analysis. We need someone with experience in automating ETL pipelines, handling data quality issues, and ensuring seamless integration. Key Responsibilities: Retrieve data from all sorts of places – databases, APIs, and flat files. Work on raw data, making it clean, consistent, and accurate. Handle missing values, normalize data, and some feature engineering. Load the transformed data where it belongs, ensuring it gets along with our storage solution. Test and validate, ensuring our data stays true to its transformed self. Proven experience in Python development with a focus on ...

I need an expert well-versed in Python or Access macro to...Your solution should be capable of aggregating the data where necessary. - Category-wise CSV Export: Export the resulting data into .csv files, with one file per category of data. The ideal freelancer for this project should have the following skills: - An excellent understanding of Python or Macro Access programming. - Proven experience working with data extraction, transformation and loading (ETL). - Familiarity with data aggregation and analytical operations. - A good understanding of CSV files and how to manipulate them in Python or Access. - Past work with data-based projects is a bonus. The effectiveness of your solution in addressing these tasks will be a key determinant of its success. Looking forward to working...

...of the source data, we process the data, create analyses and credit ratings Where we are now: - We have Crawlers that create datasets from 2020. (Several terabytes of data for 4.5 million profiles of Polish companies) - we have a product design (report structure) What we need: - that in order to proceed with IT development, we need to develop a Data Ddocumentation that would describe (Crawlers, etl and rating model) My proposal for the first task is: - source data analysis - to design a documentation structure that would enable IT development, testing and maintenance of the databases, something else ? P.S. A demonstration of initiative for the proposed activities is welcome. If you have experience in documenting 100% of the data flow then you can suggest additional steps/tas...

...need professionals with an extensive understanding of data extraction, transformation, and integration in the banking domain. Your overarching role will encompass integrating our banking system with a new data warehouse. NOTE: The team or company must have relevant experience in the BANKING DOMAIN. Note: Looking for a full-time available team. Key Responsibilities: - Apply your expertise in ETL tools to extract, transform, and organize multiple data streams. - You will integrate the cleaned data with other various parts of our banking systems. Analytical Requirement: - Your role will also demand you to leverage your solid background in developing insightful reports in the banking context. While not mandatory, the ability to pull in data from customer’s transaction...

...processing of data, utilizing web scraping technology, APIs, and python libraries like beautiful soup for processing. For this project, the main tasks include: - Designing and implementing effective scraping systems to collect and process data - Integrating to our current system Skills and Experience Ideal candidates for this project should have: - Proven experience in data engineering, API, ETL, and scraping projects. - Proficiency in Python Libraries such as BeautifulSoup or Scrapy. - Demonstrable past work related to data science or data engineering Here are some additional details to help you understand the specific project requirements: Background on What My Clients Need | Information to Help Salespeople: I help sales teams perform better by providing them informatio...

We are looking for ETL ODI Developer for long term contract role who can support us 8 hours a day from Mon- Friday Experience 7+ years Long Term Work Hours 10.30 am – 07.30 pm IST Remote Key Skills : • 7 + years’ experience in building function & store procedure using ODI too and 3+ Years in Project Management • Strong in Agile, JIRA • Design Data warehouse table and applicable documentation. • Develop table creation scripts and execute the scripts for the Database Administrator (DBA). • Analyze data sources and determine data relationships to pair with the business requirements documentation (BRD). • Familiarity with data quality issues and Exact, Transform, Load (ETL) processes • Design and develop Extract Load Transform ...

Share Resumes. • Solutioning/Architecting/Developing Informatica Cloud Data Integration (CDI) and Cloud Application Integration (CAI) processes • Building/Unit Testing/Deploying the Mappings/Taskflows in CDI • Implement transformations within CDI mappings based on Business logic • Building CAI assets (Service Connectors, App Connections, Process Objects and Processes) to integrate applications in real time • Creating Functional Design Documents (FDD) for solutioning and sharing with Solution Architects and other team members to explain the development methods or procedures and follow the patterns to complete development • Bug fixing, Unit testing & UAT? Go-live & Production support • Training & Knowledge Transfer Essenti...

I'm looking for an expert in Azure ETL pipelines who can help me maintain and optimize my existing structures. My primary focus is on data transformation of structured data. The role also involves carrying out any necessary data cleaning or enrichment processes during the transformation. Key tasks: - Data transformation optimization - Data cleaning - Data enrichment Ideal skills: - Extensive experience with Azure ETL pipelines - Proven ability with data transformation - Proficiency with structured data - Data cleaning and enrichment skills.

I'm in need of a distinguished SQL developer with an adept background in ETL Testing within Snowflake. The successful candidate will be responsible for: - Carrying out efficient ETL Testing in Snowflake. This will involve knowledge in specific areas such as data extraction, data transformation, and data loading. - Execution and validation of SQL Test Cases. This specifically includes conducting data integrity tests and data retrieval tests. The ideal candidate should have proven experience in SQL development, ETL testing and knowledge in the Snowflake environment. An understanding of database design, query optimization, and database administration will be highly beneficial. The candidate should be detail-oriented and should have a keen eye for identifying and a...

I'm seeking a highly skilled Data Engineer / Data Architect for my tech startup. Key Responsibilities: * Data modeling * Database design * Execution of ETL (Extract, Transform, Load) processes * Building and managing a Data Warehouse * Creating and managing API The core function of the Data Warehouse will be to integrate data from various sources. Ideal Skills & Experience: * Proficient in AWS * Strong foundation in Data modeling and Database design * Experience in building ETL processes * Knowledge in Data Warehouse construction, management, and data integration * API design and development Only apply if you are comfortable with AWS tech stack.

I need a software development company to build a custom application to be used on a Windows platform. This app...platform. This application is aimed at generating reports from tally using data analytics . It will be an tax audit software ( am an CA ). I will retail the software. I have already prepared working model in power query. Ideal skills for this job include: - Advanced knowledge in software development - Familiarity with tally tdl - I want excel pivot table like UI - Expert in web scrapping and ETL function - knowledge of sql server ( I will provide raw data in sql ) - Good understanding of data management - Ability to convert this into single exe file - give me control panel . as I want to sell it furhur - on few areas AI may be used - integration with other software...

I'm searching for a skilled data analyst who can help me perform an Extract, Load, Transform (ELT) task on my internal database. The chosen freelancer should: - Be able to handle a medium to high data volume (1 GB to 10 GB). - Have experience in extracting data from multiple sources. - Be familiar with loading data into Snowflake. - Understand how to transform data using HiTouch (reverse ETL). - Be efficient with data analysis using Mixpanel In-depth knowledge of Google Analytics is also beneficial. I expect you to have demonstrable experience in ELT tasks and be capable of delivering quality results accurately and timely. Your understanding of these processes will be crucial to the success of this project.

I'm looking for an expert in ETL processes and Tableau. For my project, the right freelancer should be able to extract, transform, and load financial data. The information that should be included in the Tableau dashboard is revenue data, expense data and profit margin data. Key Skills and Experience: - Knowledge of ETL processes - Experience with Tableau - Background in Financial data analysis I require monthly updates on the dashboard. It's essential that you're detail-oriented, accurate, and timely. This job requires clear communication and a high level of precision. If you have the necessary skills and experience, please bid.

...guidance especially in the area of ETL processes. I need to focus on: - This is an interview support role, You are supposed to help in live interviews. • Extraction techniques – The primary data sources of my interest are platforms like Spark, AWS, Azure, GCP, and Hive. I want to understand effective methods for data extraction from these particular sources. Ideal Skills and Experience: - Expertise in ETL tools for data extraction - Hands-on experience with Spark, AWS, Azure, GCP, Hive - Profound knowledge in data engineering - Experience in career coaching or mentoring will be a bonus - SQL -Python This assistance will give me a competitive edge in my upcoming interviews by providing me with practical skills, knowledge, and confidence in addressing ETL...

We are looking for Full Time employee who can work on our payroll. Responsibilities: ETL Development: ● Design, develop, and implement robust ETL processes using AWS Glue, AWS Data Pipeline, or custom scripts as needed. ● Ensure the efficient extraction, transformation, and loading of data from diverse sources into our data warehouse. Data Warehousing: ● Design and maintain data warehouse solutions on AWS, with a focus on scalability, performance, and reliability. ● Implement and optimize data models for efficient storage and retrieval in AWS Redshift. AWS Service Utilization: ● Leverage AWS services such as S3, Lambda, Glue, Redshift, and others to build end-to-end data solutions. ● Stay abreast of AWS developments and recommend the adoption of new services to enhance ou...

I'm in need of a skilled developer with experience constructing ETL pipelines that extract data from CSV files. Main project tasks: - Extract data from CSV file source - Perform necessary transformations including data cleaning and data enrichment. Required experience and skills: - Proven track record in developing ETL pipelines - Proficiency in handling and processing CSV files - Expertise in performing data cleaning and data enrichment actions. This task requires fine attention to detail and a pragmatic approach to data management. I'm expecting an efficient and error-free pipeline construction. Your responsibility will be to ensure high-quality, cleaned, and enriched data output from the provided CSV files.

Require ETL script to be written in Python /Django Task :Pull data from source and transform it into desired response API endpoint would be final delivery

...management, Triggers etc. ü Proven expert in Modeling enterprise-grade databases and architecture. Very strong at Entity Relationship. ü 5+ years of experience modeling loosely coupled relational databases that can store tera or petabytes of data. ü 3+ years of proven expert in working with large Data Warehouse. ü Must have ability to conceptualize ideas and data streams to schemas. ü Expert at ETL transformations using SSIS. ü Good knowledge of analytics tool like AWS Quick Sight. ü 4+ Years of proven experience working with high volume multi-client SAAS solutions. ü Must be expert at communicating technical issues to non-business users. ü Must have business-oriented thought process. ü Proven expert at setting up standards and co...

...train others. If you're skilled in data analysis, visualization, SQL, programming (Python, R), and other BI essentials, join us for global projects. What We're Looking For: Business Intelligence Experts with Training Skills: Data analysis, visualization, and SQL Programming (Python, R) Business acumen and problem-solving Effective communication and domain expertise Data warehousing and modeling ETL processes and OLAP Statistical analysis and machine learning Big data technologies (Hadoop, Spark) Agile methodologies and data-driven decision-making Cloud technologies (AWS, Azure) and data security NoSQL databases and web scraping Natural Language Processing (NLP) and sentiment analysis API integration and data architecture Why Work With Us: Global Opportunities: Collab...

We're looking for an amazing python dev to take over work on an a Python Flask container - mainly responsible for ETL and data analysis. You: - A strong developer with experience in data pipeline and data analysis work. - Expert in libraries like Pandas, NumPy, and other python data manipulation libs. - A familiarity with networking protocols such as Websockets, HTTP, and FTP. - Familiarity with Data Warehouses like BigQuery - Familiarity with Looker - Ability to support US West Coast hours Do you do your own coding, or do you coordinate development through an agency? Can you describe a recent project where you utilized Pandas and NumPy for data analysis or manipulation? What challenges did you face and how did you overcome them? Can you explain a scenario where you had to ...

...manage and organize data within it. - Ability to ensure data security and integrity throughout the extraction and integration process. **Ideal Skills and Experience:** - Proven track record of working with POS systems, especially Clover, for sales data extraction. - Strong background in cloud-computing platforms, specifically Azure. - Comfortable with setting up and managing data pipelines and ETL processes. - Experience in setting up automated workflows between different technological platforms. - Knowledge of data visualization and interpretation, to guide the final format of sales data within Snowflake. This project requires a freelancer who can swiftly understand our data infrastructure and implement a solution that minimizes manual intervention and maximizes data utility f...

...Intelligence (MS BI) technologies for over 8 years, we're in search of a consultant who has a strong background in Power BI and particularly in SQL Server Integration Services (SSIS,SSRS,SSAS). While expertise in other MS BI areas is appreciated, SSIS should be your main strength. Tasks will revolve around data modeling and visualization, report creation and customization, as well as data integration and ETL processes depending upon the demand of the projects on hand. Please note, however, that all the mentioned tasks are equally important in this role. Ideal candidate should: - Have proven experience with Power BI and SSIS,SSRS - Be comfortable tackling both the big picture and diving into the details - Provide examples of past work showcasing their expertise in these ar...

I require an ETL Informatica IDQ specialist for my project. The Ideal Candidate: - Should have extensive experience in ETL Informatica IDQ specifically. 5+ Years IDQ experience recent 10.4 5+ years Data warehousing 5+ years informatica Share RESUME.

I've got a 2GB CSV file with a JSON column that I need processed using the Azure synapse ETL pipeline. The JSON structure would need to be exploded into all their respective columns. - The desired output format should be in Parquet. Familiarity with creating Parquet files is therefore essential. - Must be capable of extracting all columns from the JSON structure. - Experience with Azure Synapse and ETL pipeline is important. - Strong JSON understanding and manipulation skill is necessary. - This project needs to be completed ASAP, so a prompt response and swift action will be appreciated.

I am looking for a skilled Informatica professional to set up Informatica 12.x on my enterprise-level. Informatica admin needed to: Platform Setup: Des...efficient data processing. Security and Access Control: Implement robust security measures and access controls to safeguard sensitive data and ensure compliance with industry regulations. Performance Tuning: Monitor system performance, identify bottlenecks, and implement optimization strategies to ensure the platform operates at peak efficiency. Workflow and Job Scheduling: Create and manage ETL workflows and job schedules, troubleshoot issues, and ensure data processing jobs are executed successfully. Backup and Recovery: Develop and maintain backup and disaster recovery plans to safeguard critical data and ensure business c...

Build a glue etl using pyspark to transfer data from mysql to postgres. facing challenges in column mappings between the 2 sources, the target database has datatypes enums and text arrays. should solve the erros in column mappings Should have prior experience ingesting data into postgres enum datatype