Data Engineer

Cloud infrastructure solutions and support company. (SE1)

Data Engineer

Data Engineer at Cloud infrastructure solutions and support company. (SE1) · Pune · 2 - 6 years · ₹12L - ₹16L / yr · Posted 12 Oct 2021

Data Engineer

at Cloud infrastructure solutions and support company. (SE1)

Agency job

via Multi Recruit

2 - 6 yrs

₹12L - ₹16L / yr

Pune

Skills

SQL

ETL

Data engineering

Big Data

Java

Python

PySpark

Design, create, test, and maintain data pipeline architecture in collaboration with the Data Architect.
Build the infrastructure required for extraction, transformation, and loading of data from a wide variety of data sources using Java, SQL, and Big Data technologies.
Support the translation of data needs into technical system requirements. Support in building complex queries required by the product teams.
Build data pipelines that clean, transform, and aggregate data from disparate sources
Develop, maintain and optimize ETLs to increase data accuracy, data stability, data availability, and pipeline performance.
Engage with Product Management and Business to deploy and monitor products/services on cloud platforms.
Stay up-to-date with advances in data persistence and big data technologies and run pilots to design the data architecture to scale with the increased data sets of consumer experience.
Handle data integration, consolidation, and reconciliation activities for digital consumer / medical products.

Job Qualifications:

Bachelor’s or master's degree in Computer Science, Information management, Statistics or related field
5+ years of experience in the Consumer or Healthcare industry in an analytical role with a focus on building on data pipelines, querying data, analyzing, and clearly presenting analyses to members of the data science team.
Technical expertise with data models, data mining.
Hands-on Knowledge of programming languages in Java, Python, R, and Scala.
Strong knowledge in Big data tools like the snowflake, AWS Redshift, Hadoop, map-reduce, etc.
Having knowledge in tools like AWS Glue, S3, AWS EMR, Streaming data pipelines, Kafka/Kinesis is desirable.
Hands-on knowledge in SQL and No-SQL database design.
Having knowledge in CI/CD for the building and hosting of the solutions.
Having AWS certification is an added advantage.
Having Strong knowledge in visualization tools like Tableau, QlikView is an added advantage
A team player capable of working and integrating across cross-functional teams for implementing project requirements. Experience in technical requirements gathering and documentation.
Ability to work effectively and independently in a fast-paced agile environment with tight deadlines
A flexible, pragmatic, and collaborative team player with the innate ability to engage with data architects, analysts, and scientists

Design, create, test, and maintain data pipeline architecture in collaboration with the Data Architect.
Build the infrastructure required for extraction, transformation, and loading of data from a wide variety of data sources using Java, SQL, and Big Data technologies.
Support the translation of data needs into technical system requirements. Support in building complex queries required by the product teams.
Build data pipelines that clean, transform, and aggregate data from disparate sources
Develop, maintain and optimize ETLs to increase data accuracy, data stability, data availability, and pipeline performance.
Engage with Product Management and Business to deploy and monitor products/services on cloud platforms.
Stay up-to-date with advances in data persistence and big data technologies and run pilots to design the data architecture to scale with the increased data sets of consumer experience.
Handle data integration, consolidation, and reconciliation activities for digital consumer / medical products.

Job Qualifications:

Bachelor’s or master's degree in Computer Science, Information management, Statistics or related field
5+ years of experience in the Consumer or Healthcare industry in an analytical role with a focus on building on data pipelines, querying data, analyzing, and clearly presenting analyses to members of the data science team.
Technical expertise with data models, data mining.
Hands-on Knowledge of programming languages in Java, Python, R, and Scala.
Strong knowledge in Big data tools like the snowflake, AWS Redshift, Hadoop, map-reduce, etc.
Having knowledge in tools like AWS Glue, S3, AWS EMR, Streaming data pipelines, Kafka/Kinesis is desirable.
Hands-on knowledge in SQL and No-SQL database design.
Having knowledge in CI/CD for the building and hosting of the solutions.
Having AWS certification is an added advantage.
Having Strong knowledge in visualization tools like Tableau, QlikView is an added advantage
A team player capable of working and integrating across cross-functional teams for implementing project requirements. Experience in technical requirements gathering and documentation.
Ability to work effectively and independently in a fast-paced agile environment with tight deadlines
A flexible, pragmatic, and collaborative team player with the innate ability to engage with data architects, analysts, and scientists

Users love Cutshort

Read about what our users have to say about finding their next opportunity on Cutshort.

Shubham Vishwakarma

Full Stack Developer - Averlon

I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.

Companies hiring on Cutshort

Similar jobs (10)

Data Engineer – AWS, PySpark & Airflow

at NeoGenCode Technologies Pvt Ltd

2 candid answers

Posted by Akshay Patil

Gurugram

5 - 10 yrs

₹6L - ₹16L / yr

Amazon Web Services (AWS)

Data engineering

Data Lake Architecture

Amazon S3

AWS Glue

+15 more

🚀 Job Title : Data Engineer – AWS, PySpark & Airflow

Experience : 5+ Years

Industry : Health Insurance / BFSI

Location : Sector 43, Gurugram, Haryana

Work Mode : Work From Office (5 Days a Week)

Employment Type : Full-Time

Role Overview :

We are looking for an experienced Data Engineer with strong hands-on expertise in AWS Data Engineering, Data Lake Architecture, Python, SQL, PySpark, and Apache Airflow.

The candidate will be responsible for designing, developing, and maintaining scalable, reliable, and high-performance data pipelines using AWS cloud technologies. The role involves building end-to-end ETL / ELT workflows, implementing data transformation and processing solutions, optimizing pipeline performance, and ensuring the reliability of production data systems.

Mandatory Skills :

Key Responsibilities :

Design, develop, and maintain end-to-end data pipelines for data ingestion, staging, transformation, curation, and consumption.
Build and manage AWS-based data lake solutions using Amazon S3, AWS Glue, Amazon EMR, Athena, and Redshift.
Develop, maintain, and optimize ETL / ELT pipelines using Python, SQL, Apache Spark, and PySpark.
Implement and manage workflow orchestration using Apache Airflow, including scheduling, dependency management, retries, monitoring, and failure handling.
Integrate data from multiple structured and semi-structured sources.
Design and maintain data models, dimensional models, and analytical data marts.
Work with relational databases, including PostgreSQL and Amazon RDS.
Optimize Spark jobs, SQL queries, data transformations, and cloud resource utilization.
Implement data quality checks, validation rules, and error-handling mechanisms.
Monitor production data pipelines, troubleshoot failures, and perform root cause analysis (RCA).
Ensure pipeline reliability, scalability, performance, and compliance with service-level agreements (SLAs).
Follow data security, access management, governance, and sensitive-data protection best practices.
Collaborate with business, analytics, and application teams to deliver effective data solutions.
Maintain technical documentation and follow established data engineering standards.

Required Technical Skills :

AWS Data Engineering : Strong hands-on experience with AWS-based data engineering and data lake architecture.
AWS Services : Amazon S3, AWS Glue, Glue Data Catalog, Amazon EMR, Athena, Redshift, AWS Lambda, and IAM.
Programming : Strong proficiency in Python and SQL.
Big Data Processing : Hands-on experience with Apache Spark and PySpark.
Workflow Orchestration : Experience developing and managing workflows using Apache Airflow.
ETL/ELT Development : Experience building scalable data ingestion, transformation, and processing pipelines.
Database Technologies : Good understanding of relational databases and PostgreSQL concepts.
Data Modelling : Experience with data modelling, data transformation, and analytical data structures.
Production Support : Ability to monitor pipelines, troubleshoot failures, perform RCA, and resolve production issues.
Performance Optimization : Experience optimizing SQL queries, Spark jobs, and data processing workflows.

Good-to-Have Skills :

Experience with AWS Database Migration Service (DMS) and Amazon RDS.
Familiarity with dbt for data transformation and modelling.
Knowledge of Delta Lake, Apache Iceberg, or Apache Hudi.
Exposure to Apache Kafka and real-time data streaming.
Familiarity with GraphQL, Hasura, or similar data-serving technologies.
Understanding of data governance and data quality frameworks.
Experience in the BFSI, healthcare, or health insurance domain.

Candidate Requirements :

5+ years of relevant experience in Data Engineering.
Proven hands-on experience building and maintaining production-grade data pipelines on AWS.
Strong understanding of data lake architecture and large-scale data processing.
Practical experience with Apache Airflow and Spark / PySpark-based transformations.
Strong programming, analytical, debugging, and problem-solving skills.
Experience handling production incidents, pipeline failures, and performance bottlenecks.
Ability to take ownership of deliverables and work effectively with cross-functional teams.
Good communication, collaboration, and technical documentation skills.
Willingness to work from the Gurugram office five days a week.

Ideal Candidate :

The ideal candidate is a hands-on Data Engineer who can independently build, optimize, and maintain AWS-based data pipelines, work with large datasets using PySpark, orchestrate workflows through Airflow, and resolve production issues in a business-critical environment.

🚀 Job Title : Data Engineer – AWS, PySpark & Airflow

Experience : 5+ Years

Industry : Health Insurance / BFSI

Location : Sector 43, Gurugram, Haryana

Work Mode : Work From Office (5 Days a Week)

Employment Type : Full-Time

Role Overview :

We are looking for an experienced Data Engineer with strong hands-on expertise in AWS Data Engineering, Data Lake Architecture, Python, SQL, PySpark, and Apache Airflow.

Mandatory Skills :

Key Responsibilities :

Design, develop, and maintain end-to-end data pipelines for data ingestion, staging, transformation, curation, and consumption.
Build and manage AWS-based data lake solutions using Amazon S3, AWS Glue, Amazon EMR, Athena, and Redshift.
Develop, maintain, and optimize ETL / ELT pipelines using Python, SQL, Apache Spark, and PySpark.
Implement and manage workflow orchestration using Apache Airflow, including scheduling, dependency management, retries, monitoring, and failure handling.
Integrate data from multiple structured and semi-structured sources.
Design and maintain data models, dimensional models, and analytical data marts.
Work with relational databases, including PostgreSQL and Amazon RDS.
Optimize Spark jobs, SQL queries, data transformations, and cloud resource utilization.
Implement data quality checks, validation rules, and error-handling mechanisms.
Monitor production data pipelines, troubleshoot failures, and perform root cause analysis (RCA).
Ensure pipeline reliability, scalability, performance, and compliance with service-level agreements (SLAs).
Follow data security, access management, governance, and sensitive-data protection best practices.
Collaborate with business, analytics, and application teams to deliver effective data solutions.
Maintain technical documentation and follow established data engineering standards.

Required Technical Skills :

AWS Data Engineering : Strong hands-on experience with AWS-based data engineering and data lake architecture.
AWS Services : Amazon S3, AWS Glue, Glue Data Catalog, Amazon EMR, Athena, Redshift, AWS Lambda, and IAM.
Programming : Strong proficiency in Python and SQL.
Big Data Processing : Hands-on experience with Apache Spark and PySpark.
Workflow Orchestration : Experience developing and managing workflows using Apache Airflow.
ETL/ELT Development : Experience building scalable data ingestion, transformation, and processing pipelines.
Database Technologies : Good understanding of relational databases and PostgreSQL concepts.
Data Modelling : Experience with data modelling, data transformation, and analytical data structures.
Production Support : Ability to monitor pipelines, troubleshoot failures, perform RCA, and resolve production issues.
Performance Optimization : Experience optimizing SQL queries, Spark jobs, and data processing workflows.

Good-to-Have Skills :

Experience with AWS Database Migration Service (DMS) and Amazon RDS.
Familiarity with dbt for data transformation and modelling.
Knowledge of Delta Lake, Apache Iceberg, or Apache Hudi.
Exposure to Apache Kafka and real-time data streaming.
Familiarity with GraphQL, Hasura, or similar data-serving technologies.
Understanding of data governance and data quality frameworks.
Experience in the BFSI, healthcare, or health insurance domain.

Candidate Requirements :

5+ years of relevant experience in Data Engineering.
Proven hands-on experience building and maintaining production-grade data pipelines on AWS.
Strong understanding of data lake architecture and large-scale data processing.
Practical experience with Apache Airflow and Spark / PySpark-based transformations.
Strong programming, analytical, debugging, and problem-solving skills.
Experience handling production incidents, pipeline failures, and performance bottlenecks.
Ability to take ownership of deliverables and work effectively with cross-functional teams.
Good communication, collaboration, and technical documentation skills.
Willingness to work from the Gurugram office five days a week.

Ideal Candidate :

Data Platform Engineer (SDE 2 / SDE 3)

at NeoGenCode Technologies Pvt Ltd

2 candid answers

Posted by Akshay Patil

Remote, Pune

3 - 8 yrs

₹14L - ₹30L / yr

Data Platform Engineer

Java

Python

SQL

Data engineering

+13 more

Job Title : Data Platform Engineer (SDE 2 / SDE 3)

Experience : 3 to 8 Years (SDE2 : 3 to 5 Years | SDE3 : 5.5 to 8 Years)

Location : Remote (Contract) → Pune (Post Conversion)

Employment Type : Contract-to-Hire (3 Months)

Mandatory Skills :

Java / Python / Go, SQL, Data Engineering, ETL / ELT, Apache Airflow, Apache Spark / Flink, Kafka, AWS/GCP/Azure, Distributed Systems, API Development, Docker, Kubernetes, CI/CD, System Design

Role Overview :

We are looking for a Data Platform Engineer with strong backend engineering expertise to build scalable, high-performance data platforms and distributed systems. The ideal candidate should have hands-on experience in developing production-grade backend applications along with designing and maintaining modern data pipelines.

Key Responsibilities :

Build and maintain scalable ETL/ELT and data processing pipelines.
Develop backend services and APIs using Java (preferred), Python, or Go.
Design batch and real-time data pipelines using Spark, Flink, Kafka, and Airflow.
Optimize SQL queries, data models, and distributed data systems.
Work with cloud platforms (AWS, GCP, or Azure) and container technologies (Docker, Kubernetes).
Implement CI/CD, monitoring, logging, and performance optimization.
Collaborate with product and engineering teams on scalable system design and architecture.

Required Qualifications :

SDE 2 : 3 to 5 years of Backend + Data Engineering experience.
SDE 3 : 5.5 to 8 years of Backend + Data Engineering experience.
Strong coding skills in Java (preferred), Python, or Go.
Excellent SQL and data modeling knowledge.
Hands-on experience with Airflow, Apache Spark / Apache Flink, or similar technologies.
Experience building scalable backend services and APIs.
Good understanding of distributed systems, system design. and scalable architecture.
Experience with cloud platforms (AWS, Azure, or GCP).
Hands-on experience with Docker and Kubernetes.
Strong understanding of CI/CD pipelines.
Excellent problem-solving and debugging skills.

Good to Have :

Experience with Snowflake, BigQuery, or Redshift.
Understanding of Kafka, Kinesis, or Pub/Sub
Performance optimization of large-scale distributed systems
FinTech or high-scale distributed systems experience.
Knowledge of data governance, security, and compliance.

Preferred Candidate Profile :

Strong Backend + Data Engineering experience
Experience building scalable production systems
Strong ownership mindset
Good system design knowledge
Experience processing millions of events using Kafka/Spark
Built APIs supporting data infrastructure
Production engineering experience

Interview Process :

Take-home Coding Assignment (48 Hours)
Leadership & Strategy Round (1 Hour)
Technical Depth – Data Engineering & Performance (1 Hour)
Culture & Values Fit (30 Minutes)

Job Title : Data Platform Engineer (SDE 2 / SDE 3)

Experience : 3 to 8 Years (SDE2 : 3 to 5 Years | SDE3 : 5.5 to 8 Years)

Location : Remote (Contract) → Pune (Post Conversion)

Employment Type : Contract-to-Hire (3 Months)

Mandatory Skills :

Java / Python / Go, SQL, Data Engineering, ETL / ELT, Apache Airflow, Apache Spark / Flink, Kafka, AWS/GCP/Azure, Distributed Systems, API Development, Docker, Kubernetes, CI/CD, System Design

Role Overview :

Key Responsibilities :

Build and maintain scalable ETL/ELT and data processing pipelines.
Develop backend services and APIs using Java (preferred), Python, or Go.
Design batch and real-time data pipelines using Spark, Flink, Kafka, and Airflow.
Optimize SQL queries, data models, and distributed data systems.
Work with cloud platforms (AWS, GCP, or Azure) and container technologies (Docker, Kubernetes).
Implement CI/CD, monitoring, logging, and performance optimization.
Collaborate with product and engineering teams on scalable system design and architecture.

Required Qualifications :

SDE 2 : 3 to 5 years of Backend + Data Engineering experience.
SDE 3 : 5.5 to 8 years of Backend + Data Engineering experience.
Strong coding skills in Java (preferred), Python, or Go.
Excellent SQL and data modeling knowledge.
Hands-on experience with Airflow, Apache Spark / Apache Flink, or similar technologies.
Experience building scalable backend services and APIs.
Good understanding of distributed systems, system design. and scalable architecture.
Experience with cloud platforms (AWS, Azure, or GCP).
Hands-on experience with Docker and Kubernetes.
Strong understanding of CI/CD pipelines.
Excellent problem-solving and debugging skills.

Good to Have :

Experience with Snowflake, BigQuery, or Redshift.
Understanding of Kafka, Kinesis, or Pub/Sub
Performance optimization of large-scale distributed systems
FinTech or high-scale distributed systems experience.
Knowledge of data governance, security, and compliance.

Preferred Candidate Profile :

Strong Backend + Data Engineering experience
Experience building scalable production systems
Strong ownership mindset
Good system design knowledge
Experience processing millions of events using Kafka/Spark
Built APIs supporting data infrastructure
Production engineering experience

Interview Process :

Take-home Coding Assignment (48 Hours)
Leadership & Strategy Round (1 Hour)
Technical Depth – Data Engineering & Performance (1 Hour)
Culture & Values Fit (30 Minutes)

Data Engineer

at VY SYSTEMS PRIVATE LIMITED

Posted by Jancy A

Bengaluru (Bangalore)

5 - 7 yrs

₹4L - ₹20L / yr

Data Engineer,

Python

ETL

DevOps

Job Summary

Role Overview

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, Advanced SQL, CI/CD, DevOps, and Data Analytics. The ideal candidate should have hands-on experience designing and developing scalable data pipelines, transforming large datasets, and supporting data-driven applications.

Experience with Google Cloud Platform (GCP) will be an added advantage.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
Develop complex and optimized SQL queries, stored procedures, and data transformations.
Build and maintain reliable data integration workflows across multiple data sources.
Perform data cleansing, validation, transformation, and quality checks.
Analyze data and provide insights to support business and technical requirements.
Implement and maintain CI/CD pipelines for data engineering applications.
Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
Troubleshoot data pipeline failures, performance issues, and production incidents.
Optimize data processing workflows for performance, scalability, and reliability.
Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
Follow best practices for version control, testing, documentation, and deployment.
Contribute to cloud-based data engineering initiatives, preferably on GCP.

Required Skills

5–7 years of hands-on experience in Data Engineering.
Strong programming skills in Python.
Strong expertise in Advanced SQL and database concepts.
Hands-on experience with ETL/ELT processes and data pipelines.
Good understanding of Data Warehousing and Data Modeling concepts.
Experience with CI/CD practices and tools.
Strong understanding of DevOps principles, automation, and deployment processes.
Strong data analytics and problem-solving skills.
Experience working with large datasets and performance optimization.
Good understanding of Git/version control and software development best practices.

Good to Have

Hands-on experience with Google Cloud Platform (GCP).
Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
Experience with containerization/orchestration technologies such as Docker/Kubernetes.
Experience with workflow orchestration tools such as Airflow.
Knowledge of cloud-based data architecture and distributed data processing.

Preferred Candidate Profile

Strong analytical and problem-solving abilities.
Good communication and stakeholder management skills.
Ability to work independently as well as in a collaborative team environment.
Strong ownership of data pipelines and production systems.
Candidates who can join at short notice are preferred.

Mandatory Skills

Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops

Job Summary

Role Overview

Experience with Google Cloud Platform (GCP) will be an added advantage.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
Develop complex and optimized SQL queries, stored procedures, and data transformations.
Build and maintain reliable data integration workflows across multiple data sources.
Perform data cleansing, validation, transformation, and quality checks.
Analyze data and provide insights to support business and technical requirements.
Implement and maintain CI/CD pipelines for data engineering applications.
Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
Troubleshoot data pipeline failures, performance issues, and production incidents.
Optimize data processing workflows for performance, scalability, and reliability.
Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
Follow best practices for version control, testing, documentation, and deployment.
Contribute to cloud-based data engineering initiatives, preferably on GCP.

Required Skills

5–7 years of hands-on experience in Data Engineering.
Strong programming skills in Python.
Strong expertise in Advanced SQL and database concepts.
Hands-on experience with ETL/ELT processes and data pipelines.
Good understanding of Data Warehousing and Data Modeling concepts.
Experience with CI/CD practices and tools.
Strong understanding of DevOps principles, automation, and deployment processes.
Strong data analytics and problem-solving skills.
Experience working with large datasets and performance optimization.
Good understanding of Git/version control and software development best practices.

Good to Have

Hands-on experience with Google Cloud Platform (GCP).
Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
Experience with containerization/orchestration technologies such as Docker/Kubernetes.
Experience with workflow orchestration tools such as Airflow.
Knowledge of cloud-based data architecture and distributed data processing.

Preferred Candidate Profile

Strong analytical and problem-solving abilities.
Good communication and stakeholder management skills.
Ability to work independently as well as in a collaborative team environment.
Strong ownership of data pipelines and production systems.
Candidates who can join at short notice are preferred.

Mandatory Skills

Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops

ETL Developer

at VY SYSTEMS PRIVATE LIMITED

Posted by Banu S

Bengaluru (Bangalore)

6 - 10 yrs

₹5L - ₹30L / yr

ETL

ETL Development

Python

PySpark

Hiring: ETL Developer – Hadoop & PySpark

Location: Bangalore

Experience: 6–10 Years

Budget: 25–30 LPA

Work Mode: Weekdays

Job Description

We are looking for an experienced ETL Developer with strong expertise in Hadoop, PySpark, Python, and SQL to design, develop, and optimize data processing and transformation solutions. The ideal candidate should have hands-on experience working with big data technologies, building scalable ETL pipelines, and managing data for data warehousing and analytics.

Key Responsibilities

Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
Develop data transformation workflows using PySpark and Python.
Work with Hadoop and other big data technologies to process and manage large datasets.
Write complex SQL queries for data extraction, transformation, validation, and analysis.
Implement data cleansing, data quality checks, and data validation processes.
Develop and optimize data integration workflows for data warehousing solutions.
Troubleshoot ETL failures, performance issues, and data discrepancies.
Optimize PySpark jobs and SQL queries for performance and scalability.
Collaborate with data engineers, architects, analysts, and other stakeholders to deliver reliable data solutions.
Ensure data accuracy, consistency, and adherence to established data governance standards.

Mandatory Skills

Strong hands-on experience in ETL development.
Good experience with Hadoop and big data technologies.
Proficiency in PySpark and Python programming.
Strong SQL skills, including complex queries and data manipulation.
Experience in data processing, data transformation, and data integration.
Understanding of data warehousing concepts and architecture.
Experience in developing, debugging, and optimizing ETL pipelines.

Required Experience

6–10 years of relevant experience in ETL development and data engineering.
Practical experience handling large datasets using Hadoop and PySpark.
Strong analytical, problem-solving, and troubleshooting skills.
Ability to work independently and collaborate effectively within a team.

Interested candidates can share their updated resumes along with current CTC, expected CTC, notice period, and total relevant experience.

Hiring: ETL Developer – Hadoop & PySpark

Location: Bangalore

Experience: 6–10 Years

Budget: 25–30 LPA

Work Mode: Weekdays

Job Description

Key Responsibilities

Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
Develop data transformation workflows using PySpark and Python.
Work with Hadoop and other big data technologies to process and manage large datasets.
Write complex SQL queries for data extraction, transformation, validation, and analysis.
Implement data cleansing, data quality checks, and data validation processes.
Develop and optimize data integration workflows for data warehousing solutions.
Troubleshoot ETL failures, performance issues, and data discrepancies.
Optimize PySpark jobs and SQL queries for performance and scalability.
Collaborate with data engineers, architects, analysts, and other stakeholders to deliver reliable data solutions.
Ensure data accuracy, consistency, and adherence to established data governance standards.

Mandatory Skills

Strong hands-on experience in ETL development.
Good experience with Hadoop and big data technologies.
Proficiency in PySpark and Python programming.
Strong SQL skills, including complex queries and data manipulation.
Experience in data processing, data transformation, and data integration.
Understanding of data warehousing concepts and architecture.
Experience in developing, debugging, and optimizing ETL pipelines.

Required Experience

6–10 years of relevant experience in ETL development and data engineering.
Practical experience handling large datasets using Hadoop and PySpark.
Strong analytical, problem-solving, and troubleshooting skills.
Ability to work independently and collaborate effectively within a team.

Interested candidates can share their updated resumes along with current CTC, expected CTC, notice period, and total relevant experience.

Senior Data Engineer (AWS, Python/Java) immediate joiner only

at Staffnixcom

Posted by Mayank Choudhary

Bengaluru (Bangalore)

6 - 9 yrs

₹37L - ₹42L / yr

Amazon Web Services (AWS)

Python

Strong Data Engineer (AWS) Profile

Mandatory (Experience 1) – Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.

Mandatory (Experience 2) – Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.

Mandatory (Experience 3) – Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.

Mandatory (Experience 4) – Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.

Mandatory (Experience 5) – Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.

Mandatory (Experience 6) – Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.

Mandatory (Education) – B.Tech / B.E./ M.Tech

Strong Data Engineer (AWS) Profile

Mandatory (Experience 1) – Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.

Mandatory (Experience 2) – Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.

Mandatory (Experience 3) – Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.

Mandatory (Experience 4) – Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.

Mandatory (Experience 5) – Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.

Mandatory (Experience 6) – Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.

Mandatory (Education) – B.Tech / B.E./ M.Tech

Senior Data Engineer (AWS, Python/Java) immediate joiner only

at Staffnixcom

Posted by Mayank Choudhary

Bengaluru (Bangalore)

6 - 9 yrs

₹37L - ₹42L / yr

Data engineering

Python

Strong Data Engineer (AWS) Profile

Mandatory (Experience 1) – Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.

Mandatory (Experience 2) – Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.

Mandatory (Experience 3) – Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.

Mandatory (Experience 4) – Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.

Mandatory (Experience 5) – Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.

Mandatory (Experience 6) – Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.

Mandatory (Education) – B.Tech / B.E./ M.Tech

Strong Data Engineer (AWS) Profile

Mandatory (Experience 1) – Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.

Mandatory (Experience 2) – Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.

Mandatory (Experience 3) – Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.

Mandatory (Experience 4) – Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.

Mandatory (Experience 5) – Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.

Mandatory (Experience 6) – Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.

Mandatory (Education) – B.Tech / B.E./ M.Tech

Data Engineer

at Wissen Technology

4 recruiters

Posted by Anisha Jindal

Bengaluru (Bangalore), Mumbai

5 - 14 yrs

Best in industry

Data engineering

Python

PySpark

DAX

PowerBI

Job Summary

We are looking for a skilled and experienced Data Engineer to join our growing data team. The ideal candidate will have strong expertise in Python, PySpark, Data Modeling, and Power BI, with hands-on experience in designing, developing, and optimizing scalable data solutions. The role requires working closely with business stakeholders, data architects, and analytics teams to build robust data pipelines and semantic models that enable data-driven decision-making.

Technical Skills

Strong hands-on experience in Python and PySpark development.
Expertise in building and optimizing Data Engineering solutions and ETL pipelines.
Strong understanding of Data Modeling concepts (Star Schema, Snowflake Schema, Dimensional Modeling).
Experience with Power BI Data Modeling and Semantic Layer development.
Proficiency in DAX (Data Analysis Expressions).
Experience designing and managing Semantic Models in Power BI.
Strong SQL skills and experience working with large datasets.
Knowledge of data warehousing concepts and best practices.

Preferred Skills

Experience with cloud platforms such as Azure, AWS, or GCP.
Exposure to modern data platforms like Databricks.
Understanding of data governance and data quality frameworks.

Job Summary

Technical Skills

Strong hands-on experience in Python and PySpark development.
Expertise in building and optimizing Data Engineering solutions and ETL pipelines.
Strong understanding of Data Modeling concepts (Star Schema, Snowflake Schema, Dimensional Modeling).
Experience with Power BI Data Modeling and Semantic Layer development.
Proficiency in DAX (Data Analysis Expressions).
Experience designing and managing Semantic Models in Power BI.
Strong SQL skills and experience working with large datasets.
Knowledge of data warehousing concepts and best practices.

Preferred Skills

Experience with cloud platforms such as Azure, AWS, or GCP.
Exposure to modern data platforms like Databricks.
Understanding of data governance and data quality frameworks.

Data Integration Developer

at Neuvamacro Technology Pvt Ltd

Posted by Karthik Palani

Remote, Bengaluru (Bangalore), Mumbai

8 - 10 yrs

₹13L - ₹16L / yr

SQL

Python

ETL

Data warehouse architecture

Data modeling

+4 more

Shift: Second Shift (2:00 PM – 10:00 PM)

Joining: Immediate

About the Role

We are looking for an experienced Data Integration Developer with strong hands-on expertise in SQL, Python, ETL development, Snowflake, and AWS data services.

The ideal candidate should have extensive experience in designing, developing, and maintaining data integration solutions, building data warehouse architectures, and implementing scalable ETL pipelines. Strong SQL development and performance optimization skills are essential for this role.

This is a remote position; however, candidates must be available to visit our Mumbai or Bangalore location for three days during the onboarding process.

Key Responsibilities

SQL Development & Performance Optimization

Develop complex SQL queries, stored procedures, and data transformation logic.
Optimize SQL queries for performance, scalability, and efficiency.
Perform advanced data analysis, validation, and reconciliation.
Troubleshoot complex SQL and data-related issues.

ETL Development & Data Integration

Design, develop, and maintain ETL pipelines and data integration workflows.
Work with SQL and Python for data extraction, transformation, and loading.
Ensure data accuracy, consistency, and integrity across systems.
Identify and resolve data pipeline failures and integration issues.

Data Warehousing & Data Modeling

Design and maintain data warehouse architectures.
Develop data warehouse schemas based on business requirements.
Work on dimensional data modeling, including Facts and Dimensions.
Implement Slowly Changing Dimensions (SCD).
Support data analysis and business information reconciliation.

Snowflake & AWS

Develop and manage data solutions using Snowflake.
Work with AWS services, particularly Amazon S3.
Manage AWS connections and data access configurations.
Build and maintain cloud-based data integration solutions.

Python & Apache Airflow

Develop data processing and automation solutions using Python.
Build, schedule, and monitor workflows using Apache Airflow.
Troubleshoot workflow failures and improve pipeline reliability.

Agile & Development Lifecycle

Participate in Agile development processes, sprint planning, and team discussions.
Contribute throughout all stages of the software development lifecycle.
Collaborate with cross-functional teams to understand business and technical requirements.
Perform root cause analysis and implement appropriate corrective actions.

Required Skills

Advanced SQL – Strong hands-on expertise is mandatory.
Python Development
ETL Development and Data Integration
Snowflake
Apache Airflow
AWS Services, particularly Amazon S3
AWS Connection Management
Data Warehouse Architecture
Data Modeling (Facts & Dimensions)
Slowly Changing Dimensions (SCD)
SQL Query Optimization and Performance Tuning
Data Analysis and Reconciliation
Troubleshooting and Root Cause Analysis
Agile Development Methodologies
Strong understanding of the Software Development Life Cycle (SDLC)

Experience & Qualifications

8–10 years of overall experience in Data Engineering / Data Integration / ETL Development.
Strong professional experience in advanced SQL and Python.
Hands-on experience with Snowflake, Apache Airflow, and AWS.
Proven experience designing and developing data warehouse solutions.
Experience working across the complete product development lifecycle.
Ability to independently troubleshoot complex technical challenges.

Additional Requirements

Immediate joiners preferred.
Must be comfortable working in the second shift (2:00 PM – 10:00 PM).
Position is remote, with a mandatory three-day onboarding visit to Mumbai or Bangalore.
Strong communication, analytical, and problem-solving skills.
Ability to work independently and collaboratively in an Agile environment.

Shift: Second Shift (2:00 PM – 10:00 PM)

Joining: Immediate

About the Role

We are looking for an experienced Data Integration Developer with strong hands-on expertise in SQL, Python, ETL development, Snowflake, and AWS data services.

This is a remote position; however, candidates must be available to visit our Mumbai or Bangalore location for three days during the onboarding process.

Key Responsibilities

SQL Development & Performance Optimization

Develop complex SQL queries, stored procedures, and data transformation logic.
Optimize SQL queries for performance, scalability, and efficiency.
Perform advanced data analysis, validation, and reconciliation.
Troubleshoot complex SQL and data-related issues.

ETL Development & Data Integration

Design, develop, and maintain ETL pipelines and data integration workflows.
Work with SQL and Python for data extraction, transformation, and loading.
Ensure data accuracy, consistency, and integrity across systems.
Identify and resolve data pipeline failures and integration issues.

Data Warehousing & Data Modeling

Design and maintain data warehouse architectures.
Develop data warehouse schemas based on business requirements.
Work on dimensional data modeling, including Facts and Dimensions.
Implement Slowly Changing Dimensions (SCD).
Support data analysis and business information reconciliation.

Snowflake & AWS

Develop and manage data solutions using Snowflake.
Work with AWS services, particularly Amazon S3.
Manage AWS connections and data access configurations.
Build and maintain cloud-based data integration solutions.

Python & Apache Airflow

Develop data processing and automation solutions using Python.
Build, schedule, and monitor workflows using Apache Airflow.
Troubleshoot workflow failures and improve pipeline reliability.

Agile & Development Lifecycle

Participate in Agile development processes, sprint planning, and team discussions.
Contribute throughout all stages of the software development lifecycle.
Collaborate with cross-functional teams to understand business and technical requirements.
Perform root cause analysis and implement appropriate corrective actions.

Required Skills

Advanced SQL – Strong hands-on expertise is mandatory.
Python Development
ETL Development and Data Integration
Snowflake
Apache Airflow
AWS Services, particularly Amazon S3
AWS Connection Management
Data Warehouse Architecture
Data Modeling (Facts & Dimensions)
Slowly Changing Dimensions (SCD)
SQL Query Optimization and Performance Tuning
Data Analysis and Reconciliation
Troubleshooting and Root Cause Analysis
Agile Development Methodologies
Strong understanding of the Software Development Life Cycle (SDLC)

Experience & Qualifications

8–10 years of overall experience in Data Engineering / Data Integration / ETL Development.
Strong professional experience in advanced SQL and Python.
Hands-on experience with Snowflake, Apache Airflow, and AWS.
Proven experience designing and developing data warehouse solutions.
Experience working across the complete product development lifecycle.
Ability to independently troubleshoot complex technical challenges.

Additional Requirements

Immediate joiners preferred.
Must be comfortable working in the second shift (2:00 PM – 10:00 PM).
Position is remote, with a mandatory three-day onboarding visit to Mumbai or Bangalore.
Strong communication, analytical, and problem-solving skills.
Ability to work independently and collaboratively in an Agile environment.

Data Engineer

at VY SYSTEMS PRIVATE LIMITED

Posted by Pavithra E

Hyderabad, Pune

5 - 9 yrs

₹18L - ₹20L / yr

PySpark

SQL

Python

Data Engineer Short Hiring Post

🚨 Hiring: Data Engineer

🔹 Experience: 5–9 Years

🔹 Location: Bangalore / Hyderabad

🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling

🔹 Process: L1 Virtual → L2 F2F Karat Test

🔹 F2F: Bangalore / Hyderabad Location

🔹 Positions: Immediate requirement

⚠️ Note: Candidates must be available for F2F Karat immediately after L1.

#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners

Data Engineer Short Hiring Post

🚨 Hiring: Data Engineer

🔹 Experience: 5–9 Years

🔹 Location: Bangalore / Hyderabad

🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling

🔹 Process: L1 Virtual → L2 F2F Karat Test

🔹 F2F: Bangalore / Hyderabad Location

🔹 Positions: Immediate requirement

⚠️ Note: Candidates must be available for F2F Karat immediately after L1.

#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners

ETL Developer – Hadoop & PySpark

at VY SYSTEMS PRIVATE LIMITED

Posted by Vaishnavi E

Bengaluru (Bangalore)

6 - 10 yrs

₹2L - ₹22L / yr

ETL Development

Hadoop

PySpark

Python

SQL

+3 more

Job Description

We are looking for an experienced ETL Developer with strong expertise in ETL development, Hadoop, PySpark, Python, and SQL. The candidate should have hands-on experience in developing data pipelines, processing and transforming large datasets, and working with data warehousing and big data technologies.