2+ Apache Spark Jobs in Coimbatore | Apache Spark Job openings in Coimbatore
Apply to 2+ Apache Spark Jobs in Coimbatore on CutShort.io. Explore the latest Apache Spark Job opportunities across top companies like Google, Amazon & Adobe.
Job Summary:
Seeking an experienced Senior Data Engineer to lead data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. The role involves working on scalable pipelines, large-scale distributed systems, and data lake management.
Core Responsibilities:
· Build and manage high-volume data pipelines using Spark/Databricks.
· Implement ELT frameworks using Azure Data Factory/Synapse Pipelines.
· Optimize large-scale datasets in Delta/Iceberg formats.
· Implement robust data quality, monitoring, and governance layers.
· Collaborate with Data Scientists, Analysts, and Business stakeholders.
Technical Stack:
· Big Data: Apache Spark, Kafka, Hive, Airflow, Hudi/Iceberg
· Cloud: Azure (Synapse, ADF, ADLS Gen2), Databricks, AWS (Glue/S3)
· Languages: Python, Scala, SQL
· Storage Formats: Delta Lake, Iceberg, Parquet, ORC
· CI/CD: Azure DevOps, Terraform (infra as code), Git
Senior Data Engineer (Apache Stack + Databricks/Synapse)
Share cv to
Thirega@ vysystems dot com - WhatsApp - 91Five0033Five2Three
Amazon Web Services (AWS)- Must have the experience of leading teams and drive customer interactions
- Must have multiple successful deployments user stories
- Extensive hands on experience in Apache Spark along with HiveQL
- Sound knowledge in Amazon Web Services or any other Cloud environment.
- Experienced in data flow orchestration using Apache Airflow
- JSON, XML, CSV, Parquet file formats with snappy compression.
- File movements between HDFS and AWS S3
- Experience in shell scripting and scripting to automate report generation and migration of reports to AWS S3
- Worked in building a data pipeline using Pandas and Flask FrameworkGood Familiarity with Anaconda and Jupyternotebook
Follow Cutshort