Hello,
As a Cloud Data Engineer with 6 years of experts and hands-on experiences, I have experiences in the following areas, tools and technologies for your proejct;
► BIG DATA & DATA ENGINEERING
Apache Spark, Apache Airflow, Hadoop, ClickHouse, MapReduce, YARN, Pig, Hive, HBase, Kafka, Druid, Flink, Presto (incl. Athena)
► CLOUD
AWS: EMR, S3, Athena, Glue, EC2, RDS, Redshift, Lambda, VPC, DynamoDB, Kinesis
GCP: Dataflow, Composer (Apache Airflow), BigQuery, Pub/Sub, Dataproc, Cloud Data Fusion
Message Queue, Cloud Functions, Container Optimized Instance, Kubernetes
Azure: Data Factory, HDInsight
► OTHER SKILLS & TOOLS
Docker, Terraform, Kubernetes, Pentaho, NoSQL databases
Python, Scala, Java
Some of my major projects included
— AWS-based serverless Lakehouse (using S3, EMR, Glue, AWS Batch, Athena); domain: marketing.
— GCP-based ML-oriented ETL infrastructure (using Airflow, Dataflow); domain: food tech.
— ClickHouse-based real-time events tracking system; domain: media.
— Adopted Druid+Superset BI toolset; domain: ad tech.
Skills:
— Once again: Python/Java expert and experienced Data Engineer �
— Extensive experience with MapReduce and BI tools.
— Effective communicator, responsible, team-oriented.
— Major remote experience: I build an effective work process for a distributed team.
— Interested in high load back-end development, ML, and analytical researches.
— Airflow contributor and plugin creator.
Best Regards,
Oleksandr