Senior Data engineer

at Pluginlive

Senior Data engineer

Pluginlive

Company

Home

Senior Data engineer at Pluginlive · Chennai · 5 - 20 years · ₹10L - ₹30L / yr · Bootstrapped · Posted 16 Sep 2024

Senior Data engineer

at Pluginlive

Posted by Harsha Saggi

5 - 20 yrs

₹10L - ₹30L / yr

Chennai

Skills

Python

SQL

Java

Django

Flask

Company: PluginLive

About the company:

PluginLive Technology Pvt Ltd is a leading provider of innovative HR solutions. Our mission is to transform the hiring process through technology and make it easier for organizations to find, attract, and hire top talent. We are looking for a passionate and experienced Data Engineering Lead to guide the data strategy and engineering efforts for our Campus Hiring Digital Recruitment SaaS Platform.

Role Overview:

The Data Engineering Lead will be responsible for leading the data engineering team and driving the development of data infrastructure, pipelines, and analytics capabilities for our Campus Hiring Digital Recruitment SaaS Platform. This role requires a deep understanding of data engineering, big data technologies, and team leadership. The ideal candidate will have a strong technical background, excellent leadership skills, and a proven track record of building robust data systems.

Job Description

Position: Data Engineering Lead - Campus Hiring Digital Recruitment SaaS Platform

Location: Chennai

Minimum Qualification: Bachelor’s degree in computer science, Engineering, Data Science, or a related field. Master’s degree or equivalent is a plus.

Experience: 7+ years of experience in data engineering, with at least 3 years in a leadership role.

CTC: 20-30 LPA

Employment Type: Full Time

Key Responsibilities:

Data Strategy and Vision:

Develop and communicate a clear data strategy and vision for the Campus Hiring Digital Recruitment SaaS Platform.
Conduct market research and competitive analysis to identify trends, opportunities, and data needs.
Define and prioritize the data roadmap, aligning it with business goals and customer requirements.

Data Infrastructure Development:

Design, build, and maintain scalable data infrastructure and pipelines to support data collection, storage, processing, and analysis.
Ensure the reliability, scalability, and performance of the data infrastructure.
Implement best practices in data management, including data governance, data quality, and data security.

Data Pipeline Management:

Oversee the development and maintenance of ETL (Extract, Transform, Load) processes.
Ensure data is accurately and efficiently processed and available for analytics and reporting.
Monitor and optimize data pipelines for performance and cost efficiency.

Data Analytics and Reporting:

Collaborate with data analysts and data scientists to build and deploy advanced analytics and machine learning models.
Develop and maintain data models, dashboards, and reports to provide insights and support decision-making.
Ensure data is easily accessible and usable by stakeholders across the organization.

Team Leadership:

Lead, mentor, and guide a team of data engineers, fostering a culture of collaboration, continuous improvement, and innovation.
Conduct code reviews, provide constructive feedback, and ensure adherence to development standards.
Collaborate with cross-functional teams including product, engineering, and marketing to ensure alignment and delivery of data goals.

Stakeholder Collaboration:

Work closely with stakeholders to understand business requirements and translate them into technical specifications.
Communicate effectively with non-technical stakeholders to explain data concepts and progress.
Participate in strategic planning and decision-making processes.

Skills Required:

Proven experience in designing and building scalable data infrastructures and pipelines.
Strong proficiency in programming languages such as Python, R, Data visualization tools like Power BI, Tableau, Qlik, Google Analytics
Expertise in big data technologies such as Apache Airflow, Hadoop, Spark, Kafka, and cloud data platforms like AWS, Oracle Cloud.
Solid understanding of database technologies, both SQL and NoSQL.
Experience with data modeling, data warehousing, and ETL processes.
Strong analytical and problem-solving abilities.
Excellent communication, collaboration, and leadership skills.

Preferred Qualifications:

Experience in HR technology or recruitment platforms.
Familiarity with machine learning and AI technologies.
Knowledge of data governance and data security best practices.
Contributions to open-source projects or active participation in the tech community.

Company: PluginLive

About the company:

Role Overview:

Job Description

Position: Data Engineering Lead - Campus Hiring Digital Recruitment SaaS Platform

Location: Chennai

Minimum Qualification: Bachelor’s degree in computer science, Engineering, Data Science, or a related field. Master’s degree or equivalent is a plus.

Experience: 7+ years of experience in data engineering, with at least 3 years in a leadership role.

CTC: 20-30 LPA

Employment Type: Full Time

Key Responsibilities:

Data Strategy and Vision:

Develop and communicate a clear data strategy and vision for the Campus Hiring Digital Recruitment SaaS Platform.
Conduct market research and competitive analysis to identify trends, opportunities, and data needs.
Define and prioritize the data roadmap, aligning it with business goals and customer requirements.

Data Infrastructure Development:

Design, build, and maintain scalable data infrastructure and pipelines to support data collection, storage, processing, and analysis.
Ensure the reliability, scalability, and performance of the data infrastructure.
Implement best practices in data management, including data governance, data quality, and data security.

Data Pipeline Management:

Oversee the development and maintenance of ETL (Extract, Transform, Load) processes.
Ensure data is accurately and efficiently processed and available for analytics and reporting.
Monitor and optimize data pipelines for performance and cost efficiency.

Data Analytics and Reporting:

Collaborate with data analysts and data scientists to build and deploy advanced analytics and machine learning models.
Develop and maintain data models, dashboards, and reports to provide insights and support decision-making.
Ensure data is easily accessible and usable by stakeholders across the organization.

Team Leadership:

Lead, mentor, and guide a team of data engineers, fostering a culture of collaboration, continuous improvement, and innovation.
Conduct code reviews, provide constructive feedback, and ensure adherence to development standards.
Collaborate with cross-functional teams including product, engineering, and marketing to ensure alignment and delivery of data goals.

Stakeholder Collaboration:

Work closely with stakeholders to understand business requirements and translate them into technical specifications.
Communicate effectively with non-technical stakeholders to explain data concepts and progress.
Participate in strategic planning and decision-making processes.

Skills Required:

Proven experience in designing and building scalable data infrastructures and pipelines.
Strong proficiency in programming languages such as Python, R, Data visualization tools like Power BI, Tableau, Qlik, Google Analytics
Expertise in big data technologies such as Apache Airflow, Hadoop, Spark, Kafka, and cloud data platforms like AWS, Oracle Cloud.
Solid understanding of database technologies, both SQL and NoSQL.
Experience with data modeling, data warehousing, and ETL processes.
Strong analytical and problem-solving abilities.
Excellent communication, collaboration, and leadership skills.

Preferred Qualifications:

Experience in HR technology or recruitment platforms.
Familiarity with machine learning and AI technologies.
Knowledge of data governance and data security best practices.
Contributions to open-source projects or active participation in the tech community.

Users love Cutshort

Read about what our users have to say about finding their next opportunity on Cutshort.

Shubham Vishwakarma

Full Stack Developer - Averlon

I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.

Companies hiring on Cutshort

About Pluginlive

Founded :

2022

Type :

Services

Size

Stage :

Bootstrapped

About

Shifting gears… Go to the Next Level!

Harsha Saggi

Connect

Company social profiles

N/A

Similar jobs (10)

Lead - Data Engineering

at Tatvic

2 recruiters

Posted by TatvicDigital AnalyticsPrivateLimited

Ahmedabad

6 - 9 yrs

₹15L - ₹25L / yr

Google Cloud Platform (GCP)

ETL architecture

1. Company Description:

Tatvic, is a marketing analytics company focusing on generating insights from data using long association with Google & its infrastructure. We breed,recognize and reward performance. As a company we are growing very fast & we are under transformation. To enable this transformation we need future leaders with eyesight which balances execution & strategic understanding.

Website: www.tatvic.com

2. Role Details

Role: Lead Data Engineer
Band: L4
Department: Data Engineering
Reporting to: L5 / Head of Delivery
Location: Ahmedabad
Job Type: Full Time
Experience: 6-8 years of relevant experience
Education: Bachelor’s or Master’s degree in Computer Science, Data Science, or a related technical field.

3. Mission & Role Overview

Mission: To create high-level organizational impact by leading a team to design and deliver scalable, high-performance cloud-native data solutions. The Lead Data Engineer (L4) is responsible for ensuring technical excellence—specifically in BigQuery optimization and secure GCP/AWS architectures—while driving the transition to Agentic AI workflows and creating value through the development of L1-L3 engineers.
Role Overview: This is a pivotal leadership role on the Management Track, responsible for end-to-end data outcomes for the function. The Lead Engineer ensures the quality of engineering execution, manages team utilization and resource allocation, and serves as the primary technical representative in cross-functional forums with Marketing, Strategy, and Product teams.

4. Key Responsibilities

Lead the design, development, and deployment of scalable and efficient cloud-based data pipelines and architectures.
Build reusable data solutions and software components that serve multiple customers and use cases.
Architect cloud environments using GCP, ensuring best practices in IAM, Security, and Networking.
Conduct Exploratory Data Analysis (EDA) on clickstream and behavioral datasets to derive insights.
Manage end-to-end ETL processes, including data extraction, transformation, and loading using Python, SQL, and Airflow.
Design and build production-grade software (APIs, services, internal tools) with sound engineering practices — version control, testing, CI/CD, and code review.
Extend data platforms with AI capabilities — integrating LLMs, RAG pipelines, and agentic workflows where they add clear business value.
Perform Root Cause Analysis (RCA) and resolve data quality, pipeline, or application issues.
Collaborate cross-functionally with Marketing, Insights, Product, and Engineering teams to align data initiatives with business goals.
Document architectures, design decisions, and operations clearly and consistently.
Coach and mentor team members, ensuring technical excellence and continuous learning.
Participate in recruitment and knowledge-sharing through interviews, blogs, and webinars.

5. Must-Have & Good-to-Have Skills

Must-Have Skills:

Strong understanding of clickstream data and data pipeline architecture
Hands-on experience with cloud platforms (GCP)
Expertise in SQL, Python, and Airflow
Solid software engineering foundation — API design, testing, version control, and deployment practices
Proven ability to perform exploratory data analysis and data validation
Knowledge of cloud security, IAM, and networking best practices
Working knowledge of building or integrating AI/LLM-based components into existing systems
Excellent problem-solving, stakeholder management, and communication skills

Good-to-Have Skills:

Certifications such as: Google Cloud Professional Data Engineer / Cloud Architect
AWS Certified Data Analytics or Azure Data Engineer Associate
Google Analytics 4 (GAIQ)
Experience in marketing attribution, commerce data, or analytics ecosystems
Exposure to cost optimization and cloud resource management
Experience with agent frameworks (LangChain, ADK, MCP) or LLM APIs
Experience building automation frameworks or internal data tools

6. Not-a-Fit Criteria

Individual Contributor Focus: Operates as the team's best individual contributor instead of building team capability or empowering L3s.
Information Hoarding: Keeps architectural patterns or tribal knowledge trapped in their head rather than creating reusable playbooks and documentation.
Avoidance of Conflict: Avoids difficult performance conversations with underperforming team members, allowing capability gaps to persist.
Activity Over Outcomes: Measures success by "tasks completed" rather than pipeline stability, data accuracy, or business impact.
Functional Siloing: Fails to build partnerships with Marketing or Product teams, leading to misaligned data delivery.

Our Culture & Perks - Where Passion Meets Purpose:

Competitive CTC: Based on experience and skillset
Medical Insurance for you and your family
Return-to-Work Program for New Moms — because career breaks shouldn’t break momentum
1 Work From Home Day/Month — flexibility when life calls
2 Birthday Leaves/Year — one for you, one for someone you love
Monthly Engagement Activities — from game nights to learning sessions
Inclusive Culture & Equal Opportunity Employer — diversity drives our innovation
We Think Like Engineers & Athletes: we build, we iterate, we scale, we win — together
5-day workweek — because balance drives brilliance

1. Company Description:

Website: www.tatvic.com

2. Role Details

Role: Lead Data Engineer
Band: L4
Department: Data Engineering
Reporting to: L5 / Head of Delivery
Location: Ahmedabad
Job Type: Full Time
Experience: 6-8 years of relevant experience
Education: Bachelor’s or Master’s degree in Computer Science, Data Science, or a related technical field.

3. Mission & Role Overview

Mission: To create high-level organizational impact by leading a team to design and deliver scalable, high-performance cloud-native data solutions. The Lead Data Engineer (L4) is responsible for ensuring technical excellence—specifically in BigQuery optimization and secure GCP/AWS architectures—while driving the transition to Agentic AI workflows and creating value through the development of L1-L3 engineers.
Role Overview: This is a pivotal leadership role on the Management Track, responsible for end-to-end data outcomes for the function. The Lead Engineer ensures the quality of engineering execution, manages team utilization and resource allocation, and serves as the primary technical representative in cross-functional forums with Marketing, Strategy, and Product teams.

4. Key Responsibilities

Lead the design, development, and deployment of scalable and efficient cloud-based data pipelines and architectures.
Build reusable data solutions and software components that serve multiple customers and use cases.
Architect cloud environments using GCP, ensuring best practices in IAM, Security, and Networking.
Conduct Exploratory Data Analysis (EDA) on clickstream and behavioral datasets to derive insights.
Manage end-to-end ETL processes, including data extraction, transformation, and loading using Python, SQL, and Airflow.
Design and build production-grade software (APIs, services, internal tools) with sound engineering practices — version control, testing, CI/CD, and code review.
Extend data platforms with AI capabilities — integrating LLMs, RAG pipelines, and agentic workflows where they add clear business value.
Perform Root Cause Analysis (RCA) and resolve data quality, pipeline, or application issues.
Collaborate cross-functionally with Marketing, Insights, Product, and Engineering teams to align data initiatives with business goals.
Document architectures, design decisions, and operations clearly and consistently.
Coach and mentor team members, ensuring technical excellence and continuous learning.
Participate in recruitment and knowledge-sharing through interviews, blogs, and webinars.

5. Must-Have & Good-to-Have Skills

Must-Have Skills:

Strong understanding of clickstream data and data pipeline architecture
Hands-on experience with cloud platforms (GCP)
Expertise in SQL, Python, and Airflow
Solid software engineering foundation — API design, testing, version control, and deployment practices
Proven ability to perform exploratory data analysis and data validation
Knowledge of cloud security, IAM, and networking best practices
Working knowledge of building or integrating AI/LLM-based components into existing systems
Excellent problem-solving, stakeholder management, and communication skills

Good-to-Have Skills:

Certifications such as: Google Cloud Professional Data Engineer / Cloud Architect
AWS Certified Data Analytics or Azure Data Engineer Associate
Google Analytics 4 (GAIQ)
Experience in marketing attribution, commerce data, or analytics ecosystems
Exposure to cost optimization and cloud resource management
Experience with agent frameworks (LangChain, ADK, MCP) or LLM APIs
Experience building automation frameworks or internal data tools

6. Not-a-Fit Criteria

Individual Contributor Focus: Operates as the team's best individual contributor instead of building team capability or empowering L3s.
Information Hoarding: Keeps architectural patterns or tribal knowledge trapped in their head rather than creating reusable playbooks and documentation.
Avoidance of Conflict: Avoids difficult performance conversations with underperforming team members, allowing capability gaps to persist.
Activity Over Outcomes: Measures success by "tasks completed" rather than pipeline stability, data accuracy, or business impact.
Functional Siloing: Fails to build partnerships with Marketing or Product teams, leading to misaligned data delivery.

Our Culture & Perks - Where Passion Meets Purpose:

Competitive CTC: Based on experience and skillset
Medical Insurance for you and your family
Return-to-Work Program for New Moms — because career breaks shouldn’t break momentum
1 Work From Home Day/Month — flexibility when life calls
2 Birthday Leaves/Year — one for you, one for someone you love
Monthly Engagement Activities — from game nights to learning sessions
Inclusive Culture & Equal Opportunity Employer — diversity drives our innovation
We Think Like Engineers & Athletes: we build, we iterate, we scale, we win — together
5-day workweek — because balance drives brilliance

Data Engineer

at VY SYSTEMS PRIVATE LIMITED

Posted by Jancy A

Bengaluru (Bangalore)

5 - 7 yrs

₹4L - ₹20L / yr

Data Engineer,

Python

ETL

DevOps

Job Summary

Role Overview

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, Advanced SQL, CI/CD, DevOps, and Data Analytics. The ideal candidate should have hands-on experience designing and developing scalable data pipelines, transforming large datasets, and supporting data-driven applications.

Experience with Google Cloud Platform (GCP) will be an added advantage.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
Develop complex and optimized SQL queries, stored procedures, and data transformations.
Build and maintain reliable data integration workflows across multiple data sources.
Perform data cleansing, validation, transformation, and quality checks.
Analyze data and provide insights to support business and technical requirements.
Implement and maintain CI/CD pipelines for data engineering applications.
Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
Troubleshoot data pipeline failures, performance issues, and production incidents.
Optimize data processing workflows for performance, scalability, and reliability.
Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
Follow best practices for version control, testing, documentation, and deployment.
Contribute to cloud-based data engineering initiatives, preferably on GCP.

Required Skills

5–7 years of hands-on experience in Data Engineering.
Strong programming skills in Python.
Strong expertise in Advanced SQL and database concepts.
Hands-on experience with ETL/ELT processes and data pipelines.
Good understanding of Data Warehousing and Data Modeling concepts.
Experience with CI/CD practices and tools.
Strong understanding of DevOps principles, automation, and deployment processes.
Strong data analytics and problem-solving skills.
Experience working with large datasets and performance optimization.
Good understanding of Git/version control and software development best practices.

Good to Have

Hands-on experience with Google Cloud Platform (GCP).
Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
Experience with containerization/orchestration technologies such as Docker/Kubernetes.
Experience with workflow orchestration tools such as Airflow.
Knowledge of cloud-based data architecture and distributed data processing.

Preferred Candidate Profile

Strong analytical and problem-solving abilities.
Good communication and stakeholder management skills.
Ability to work independently as well as in a collaborative team environment.
Strong ownership of data pipelines and production systems.
Candidates who can join at short notice are preferred.

Mandatory Skills

Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops

Job Summary

Role Overview

Experience with Google Cloud Platform (GCP) will be an added advantage.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
Develop complex and optimized SQL queries, stored procedures, and data transformations.
Build and maintain reliable data integration workflows across multiple data sources.
Perform data cleansing, validation, transformation, and quality checks.
Analyze data and provide insights to support business and technical requirements.
Implement and maintain CI/CD pipelines for data engineering applications.
Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
Troubleshoot data pipeline failures, performance issues, and production incidents.
Optimize data processing workflows for performance, scalability, and reliability.
Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
Follow best practices for version control, testing, documentation, and deployment.
Contribute to cloud-based data engineering initiatives, preferably on GCP.

Required Skills

5–7 years of hands-on experience in Data Engineering.
Strong programming skills in Python.
Strong expertise in Advanced SQL and database concepts.
Hands-on experience with ETL/ELT processes and data pipelines.
Good understanding of Data Warehousing and Data Modeling concepts.
Experience with CI/CD practices and tools.
Strong understanding of DevOps principles, automation, and deployment processes.
Strong data analytics and problem-solving skills.
Experience working with large datasets and performance optimization.
Good understanding of Git/version control and software development best practices.

Good to Have

Hands-on experience with Google Cloud Platform (GCP).
Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
Experience with containerization/orchestration technologies such as Docker/Kubernetes.
Experience with workflow orchestration tools such as Airflow.
Knowledge of cloud-based data architecture and distributed data processing.

Preferred Candidate Profile

Strong analytical and problem-solving abilities.
Good communication and stakeholder management skills.
Ability to work independently as well as in a collaborative team environment.
Strong ownership of data pipelines and production systems.
Candidates who can join at short notice are preferred.

Mandatory Skills

Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops

GCP Data Engineer

at VY SYSTEMS PRIVATE LIMITED

Posted by Jancy A

Bengaluru (Bangalore), Hyderabad

9 - 15 yrs

₹10L - ₹28L / yr

Google Cloud Platform (GCP)

Python

PySpark

Oracle

Leadership

GCP Data Engineering Lead

Experience: 9+ Years

Lead Experience: 2+ Years

Location: Bangalore / Hyderabad

Key Skills:

Strong experience in GCP Data Engineering and BigQuery
Hands-on experience with Oracle Exadata / PL-SQL
Strong knowledge of PySpark / Scala and Python
Experience with GoldenGate, Kafka and CDC
Hands-on experience with Apache Airflow
Good experience in CI/CD and DevOps practices
Experience with Terraform / Infrastructure as Code
Exposure to AI/LLM technologies and GenAI solutions
Strong understanding of data architecture, ETL/ELT and data pipelines

Roles & Responsibilities:

Lead the design and development of scalable GCP data engineering solutions.
Design and implement batch and real-time data pipelines using BigQuery, PySpark, Kafka/CDC and Airflow.
Work with Oracle Exadata/PL-SQL and GoldenGate for data integration and migration.
Implement CI/CD pipelines and infrastructure automation using Terraform.
Explore and integrate AI/LLM capabilities into data engineering solutions.
Lead technical discussions, code reviews, solution design and mentor team members.
Collaborate with business and technical teams to deliver high-quality data solutions.

GCP Data Engineering Lead

Experience: 9+ Years

Lead Experience: 2+ Years

Location: Bangalore / Hyderabad

Key Skills:

Strong experience in GCP Data Engineering and BigQuery
Hands-on experience with Oracle Exadata / PL-SQL
Strong knowledge of PySpark / Scala and Python
Experience with GoldenGate, Kafka and CDC
Hands-on experience with Apache Airflow
Good experience in CI/CD and DevOps practices
Experience with Terraform / Infrastructure as Code
Exposure to AI/LLM technologies and GenAI solutions
Strong understanding of data architecture, ETL/ELT and data pipelines

Roles & Responsibilities:

Lead the design and development of scalable GCP data engineering solutions.
Design and implement batch and real-time data pipelines using BigQuery, PySpark, Kafka/CDC and Airflow.
Work with Oracle Exadata/PL-SQL and GoldenGate for data integration and migration.
Implement CI/CD pipelines and infrastructure automation using Terraform.
Explore and integrate AI/LLM capabilities into data engineering solutions.
Lead technical discussions, code reviews, solution design and mentor team members.
Collaborate with business and technical teams to deliver high-quality data solutions.

Technical Lead

at Chiacon Consulting

Posted by Jagriti Verma

Gurugram

5 - 10 yrs

₹8L - ₹10L / yr

Microsoft Windows Azure

Datawarehousing

Automation

Data Structures

SQL Azure

+1 more

Technical Lead

Job Description

Role Overview

We are seeking a highly skilled Data Engineering Lead with minimum of 5 years of hands-on experience in Azure data engineering, data warehousing, and automation-driven integration. The ideal candidate should be a proven technical lead, capable of driving end-to-end

project delivery and working closely with customer teams. This is a Work from Office / Customer Site role.

Roles and Responsibilities

• Lead the design, development, and delivery of data engineering and automation projects.

• Architect and implement ETL/ELT pipelines using Azure Data Factory.

• Design and manage enterprise data warehouses including dimensional modeling and

schema optimization.

• Manage Azure components including Storage Accounts, Data Lakes, Azure SQL, and Synapse.

• Drive automation initiatives across data ingestion and transformation workflows.

• Collaborate with business and technical stakeholders to translate requirements into scalable solutions.

• Mentor team members and enforce engineering best practices.

• Serve as the technical anchor responsible for ensuring high-quality, on-time delivery.

Skills and Qualification

• Strong hands-on experience with Azure Data Factory, Azure Storage, Azure SQL/Synapse.

• Deep understanding of data warehousing concepts: star/snowflake schemas, fact/dimension modeling.

• Experience with automation-led data engineering solutions.

• Strong troubleshooting, optimization, and analytical skills.

• Excellent communication and stakeholder management abilities.

• Proven experience as a Technical Lead leading teams and delivery.

Must Have

• Min of 5 years of relevant data engineering experience.

• Strong Azure Data Engineering and Data Warehousing expertise.

• Proven Technical Lead experience.

• Ability to work from office and customer site.

• Strong ownership mindset with a focus on quality and delivery excellence.

Technical Lead

Job Description

Role Overview

project delivery and working closely with customer teams. This is a Work from Office / Customer Site role.

Roles and Responsibilities

• Lead the design, development, and delivery of data engineering and automation projects.

• Architect and implement ETL/ELT pipelines using Azure Data Factory.

• Design and manage enterprise data warehouses including dimensional modeling and

schema optimization.

• Manage Azure components including Storage Accounts, Data Lakes, Azure SQL, and Synapse.

• Drive automation initiatives across data ingestion and transformation workflows.

• Collaborate with business and technical stakeholders to translate requirements into scalable solutions.

• Mentor team members and enforce engineering best practices.

• Serve as the technical anchor responsible for ensuring high-quality, on-time delivery.

Skills and Qualification

• Strong hands-on experience with Azure Data Factory, Azure Storage, Azure SQL/Synapse.

• Deep understanding of data warehousing concepts: star/snowflake schemas, fact/dimension modeling.

• Experience with automation-led data engineering solutions.

• Strong troubleshooting, optimization, and analytical skills.

• Excellent communication and stakeholder management abilities.

• Proven experience as a Technical Lead leading teams and delivery.

Must Have

• Min of 5 years of relevant data engineering experience.

• Strong Azure Data Engineering and Data Warehousing expertise.

• Proven Technical Lead experience.

• Ability to work from office and customer site.

• Strong ownership mindset with a focus on quality and delivery excellence.

Tech Lead – Data Lake Platform

at NeoGenCode Technologies Pvt Ltd

2 candid answers

Posted by Akshay Patil

Gurugram

7 - 12 yrs

₹12L - ₹18L / yr

Amazon Web Services (AWS)

Amazon S3

Amazon EMR

athena

Amazon Redshift

+17 more

Job Title : Tech Lead – Data Lake Platform

Number of Positions : 2

Experience : 7+ Years

Role Type : Technical Lead / Data Platform Lead

Domain : Data Engineering / Data Platform / AWS

About the Role :

We are looking for an experienced Tech Lead – Data Lake Platform to lead the design, development, and operations of an enterprise-scale AWS-based Data Lake Platform.

The platform will ingest data from multiple business systems, process it through structured data layers, and serve data for analytics, reporting, APIs, and operational applications.

As a Tech Lead, you will be responsible for setting the technical direction, leading data engineering teams, driving platform reliability and performance, and owning the platform's delivery, governance, and production support end to end.

Core Tech Stack :

AWS | S3 | EMR | Glue | Athena | Redshift | DMS | Lambda | RDS | IAM | Airflow | Spark/PySpark | SQL | Data Modeling | Hasura | GraphQL | DBT | PostgreSQL/Aurora | Kafka

Key Responsibilities :

Own the overall Data Lake Platform architecture, covering data ingestion, staging, curated layers, consumption, analytics, and API/data serving.
Lead the design and development of production-grade data pipelines using AWS Glue, EMR/Spark, Airflow, DBT, Athena, and Redshift.
Design scalable batch and analytics pipelines with a focus on reliability, performance, data quality, and maintainability.
Own the API/data-serving architecture from consumption data → RDS/PostgreSQL → Hasura GraphQL → Lambda/API Gateway.
Drive improvements in platform stability, including orchestration failures, cluster sizing, pipeline SLAs, query performance, and production reliability.
Design and implement appropriate AWS security, access control, IAM, PII handling, and data governance practices.
Lead technical discussions, architecture decisions, code reviews, and engineering best practices.
Mentor and guide data engineers while ensuring high-quality and scalable engineering delivery.
Own production support, incident management, troubleshooting, and root-cause analysis (RCA) for critical data platform issues.
Develop and maintain runbooks, operational procedures, monitoring, and incident response practices.
Collaborate with business, product, application, and analytics teams to onboard new datasets and support reporting, API, and data consumption requirements.
Ensure the platform meets defined availability, performance, security, data quality, and compliance requirements.

Must-Have Skills :

7+ years of experience in Data Engineering, Data Platform Engineering, or related areas, including experience in technical leadership or architecture.
Strong hands-on experience with AWS Data Services, including:
Amazon S3
AWS EMR
AWS Glue
Amazon Athena
Amazon Redshift
AWS DMS
AWS Lambda
Amazon RDS
AWS IAM
Strong production experience with Apache Airflow.
Strong hands-on experience with Apache Spark / PySpark.
Strong SQL skills and experience with data modeling, including layered data architecture, data marts, and enterprise data models.
Experience with Hasura or a similar GraphQL layer for PostgreSQL-based data/API serving.
Strong understanding of data lake architecture and enterprise data platforms.
Proven experience leading engineers and driving technical decisions.
Hands-on experience with production support, troubleshooting, incident management, and RCA.
Strong understanding of data platform performance, scalability, reliability, and SLA management.

Good-to-Have Skills :

Experience with DBT and modern data transformation practices.
Experience with lakehouse table formats on Amazon S3, such as Apache Iceberg or similar technologies.
Strong knowledge of Amazon Redshift workload optimization, including :
Distribution keys
Sort keys
Spectrum
External tables
Experience with Kafka or other streaming/data ingestion technologies.
Experience with PostgreSQL / Amazon Aurora as a data-serving layer.
Experience with Lambda and API Gateway for API-based data serving.
Experience with enterprise data governance, data quality, security, and compliance.
Experience in BFSI / Banking / Financial Services / Insurance domain.

Ideal Candidate :

The ideal candidate is a hands-on Data Platform / Data Engineering Lead who can operate at both the architecture and implementation level. You should be comfortable designing an AWS Data Lake from end to end, leading engineers, troubleshooting production issues, and working closely with business and application teams to deliver reliable data products.

Job Title : Tech Lead – Data Lake Platform

Number of Positions : 2

Experience : 7+ Years

Role Type : Technical Lead / Data Platform Lead

Domain : Data Engineering / Data Platform / AWS

About the Role :

We are looking for an experienced Tech Lead – Data Lake Platform to lead the design, development, and operations of an enterprise-scale AWS-based Data Lake Platform.

The platform will ingest data from multiple business systems, process it through structured data layers, and serve data for analytics, reporting, APIs, and operational applications.

Core Tech Stack :

AWS | S3 | EMR | Glue | Athena | Redshift | DMS | Lambda | RDS | IAM | Airflow | Spark/PySpark | SQL | Data Modeling | Hasura | GraphQL | DBT | PostgreSQL/Aurora | Kafka

Key Responsibilities :

Own the overall Data Lake Platform architecture, covering data ingestion, staging, curated layers, consumption, analytics, and API/data serving.
Lead the design and development of production-grade data pipelines using AWS Glue, EMR/Spark, Airflow, DBT, Athena, and Redshift.
Design scalable batch and analytics pipelines with a focus on reliability, performance, data quality, and maintainability.
Own the API/data-serving architecture from consumption data → RDS/PostgreSQL → Hasura GraphQL → Lambda/API Gateway.
Drive improvements in platform stability, including orchestration failures, cluster sizing, pipeline SLAs, query performance, and production reliability.
Design and implement appropriate AWS security, access control, IAM, PII handling, and data governance practices.
Lead technical discussions, architecture decisions, code reviews, and engineering best practices.
Mentor and guide data engineers while ensuring high-quality and scalable engineering delivery.
Own production support, incident management, troubleshooting, and root-cause analysis (RCA) for critical data platform issues.
Develop and maintain runbooks, operational procedures, monitoring, and incident response practices.
Collaborate with business, product, application, and analytics teams to onboard new datasets and support reporting, API, and data consumption requirements.
Ensure the platform meets defined availability, performance, security, data quality, and compliance requirements.

Must-Have Skills :

7+ years of experience in Data Engineering, Data Platform Engineering, or related areas, including experience in technical leadership or architecture.
Strong hands-on experience with AWS Data Services, including:
Amazon S3
AWS EMR
AWS Glue
Amazon Athena
Amazon Redshift
AWS DMS
AWS Lambda
Amazon RDS
AWS IAM
Strong production experience with Apache Airflow.
Strong hands-on experience with Apache Spark / PySpark.
Strong SQL skills and experience with data modeling, including layered data architecture, data marts, and enterprise data models.
Experience with Hasura or a similar GraphQL layer for PostgreSQL-based data/API serving.
Strong understanding of data lake architecture and enterprise data platforms.
Proven experience leading engineers and driving technical decisions.
Hands-on experience with production support, troubleshooting, incident management, and RCA.
Strong understanding of data platform performance, scalability, reliability, and SLA management.

Good-to-Have Skills :

Experience with DBT and modern data transformation practices.
Experience with lakehouse table formats on Amazon S3, such as Apache Iceberg or similar technologies.
Strong knowledge of Amazon Redshift workload optimization, including :
Distribution keys
Sort keys
Spectrum
External tables
Experience with Kafka or other streaming/data ingestion technologies.
Experience with PostgreSQL / Amazon Aurora as a data-serving layer.
Experience with Lambda and API Gateway for API-based data serving.
Experience with enterprise data governance, data quality, security, and compliance.
Experience in BFSI / Banking / Financial Services / Insurance domain.

Ideal Candidate :

Data engineer Microsoft fabric

at Jade Business Services JBS

Posted by shruthi k

The recruiter has not been active on this job recently. You may apply but please expect a delayed response.

Remote, Pune

7 - 18 yrs

₹1L - ₹35L / yr (ESOP available)

microsoft fabric,dataengineering,one lake

Data Engineer – Microsoft Fabric

Location: Pune, India

Work Mode: Hybrid

Experience: 6+ Years

Employment Type: Full-time contactor

Compensation: As per market standards, commensurate with experience and expertise

Shift Timings: 2:00 PM – 11:00 PM IST

Notice Period: 0 – 15 days

About the Role

Jade Business Services (JBS) is seeking a Data Engineer – Microsoft Fabric to join our Pune team and work on enterprise-scale data transformation and analytics initiatives.

We are looking for a hands-on Data Engineer with strong experience in Microsoft Fabric, SQL, Python/PySpark and modern data engineering practices. The candidate will be responsible for building scalable data pipelines, implementing Lakehouse and Warehouse solutions, developing data models and supporting governed, reliable and AI-ready data platforms.

The ideal candidate should be comfortable working with architects, engineering teams and client stakeholders to translate business requirements into scalable and production-ready data solutions.

Roles and Responsibilities

Design and develop data solutions using Microsoft Fabric, including OneLake, Lakehouse, Warehouse and Data Factory pipelines.
Build and maintain scalable ETL/ELT pipelines for batch and incremental data processing.
Develop data ingestion and transformation pipelines using Fabric Data Factory, SQL, Python and/or PySpark.
Implement Medallion Architecture using Bronze, Silver and Gold layers.
Work with Lakehouse and Fabric Warehouse for enterprise data processing and analytics.
Develop and maintain data models, tables, views and optimized SQL queries.
Build and support semantic models for Power BI and analytical workloads.
Implement data quality, validation, monitoring and error-handling mechanisms.
Work with metadata, lineage and governance requirements using Microsoft Purview.
Implement data security, access controls and role-based permissions across data platforms.
Support Data Product and domain-oriented data architecture principles.
Follow DataOps practices including CI/CD, deployment, monitoring and production support.
Troubleshoot pipeline failures, performance issues and data quality problems.
Optimize data pipelines, queries and storage for performance and cost efficiency.
Work closely with Data Architects and business stakeholders to understand requirements and implement technical solutions.
Participate in technical design discussions, code reviews and architecture reviews.
Maintain technical documentation, data flow diagrams and pipeline documentation.
Support production deployments, incident resolution and SLA-driven data platform operations.
Identify opportunities for automation and AI-assisted improvements across data engineering processes.

Qualifications and Skills

6+ years of experience in Data Engineering, Data Integration or Data Platform development.
Strong hands-on experience with Microsoft Fabric.
Experience with:
Microsoft Fabric Lakehouse
Fabric Warehouse
OneLake
Fabric Data Factory / Pipelines
Semantic Models
Strong understanding of Lakehouse and Medallion Architecture.
Strong SQL development and query optimization skills.
Hands-on experience with Python and/or PySpark.
Experience developing enterprise ETL/ELT and data integration pipelines.
Experience with batch and incremental data processing.
Understanding of data modelling concepts including dimensional modelling.
Knowledge of data quality, metadata, lineage and data governance.
Working knowledge of Microsoft Purview.
Understanding of Data Mesh and Data Product concepts.
Experience with CI/CD, version control, monitoring and DataOps practices.
Understanding of cloud security, access controls and data privacy.
Good troubleshooting and problem-solving skills.
Strong communication skills and ability to work with distributed and client-facing teams.

Preferred Skills

Microsoft Fabric or Azure Data certifications.
Experience migrating workloads from Azure Synapse, SQL Server, Databricks or other data platforms to Microsoft Fabric.
Experience implementing Medallion Architecture on Microsoft Fabric.
Experience with Power BI and semantic modelling.
Exposure to AI/ML, Generative AI or Agentic AI use cases on enterprise data platforms.
Experience working with Data Products or domain-oriented data solutions.
Experience in Energy & Utilities, Healthcare, Financial Services or Insurance.
Experience working with US or international enterprise clients.

What We Expect

The ideal candidate should be hands-on first and capable of independently building, troubleshooting and optimizing Fabric data solutions. You should be able to explain the technical decisions behind your implementation and work effectively with architects and engineering teams to deliver production-ready solutions.

Data Engineer – Microsoft Fabric

Location: Pune, India

Work Mode: Hybrid

Experience: 6+ Years

Employment Type: Full-time contactor

Compensation: As per market standards, commensurate with experience and expertise

Shift Timings: 2:00 PM – 11:00 PM IST

Notice Period: 0 – 15 days

About the Role

Jade Business Services (JBS) is seeking a Data Engineer – Microsoft Fabric to join our Pune team and work on enterprise-scale data transformation and analytics initiatives.

The ideal candidate should be comfortable working with architects, engineering teams and client stakeholders to translate business requirements into scalable and production-ready data solutions.

Roles and Responsibilities

Design and develop data solutions using Microsoft Fabric, including OneLake, Lakehouse, Warehouse and Data Factory pipelines.
Build and maintain scalable ETL/ELT pipelines for batch and incremental data processing.
Develop data ingestion and transformation pipelines using Fabric Data Factory, SQL, Python and/or PySpark.
Implement Medallion Architecture using Bronze, Silver and Gold layers.
Work with Lakehouse and Fabric Warehouse for enterprise data processing and analytics.
Develop and maintain data models, tables, views and optimized SQL queries.
Build and support semantic models for Power BI and analytical workloads.
Implement data quality, validation, monitoring and error-handling mechanisms.
Work with metadata, lineage and governance requirements using Microsoft Purview.
Implement data security, access controls and role-based permissions across data platforms.
Support Data Product and domain-oriented data architecture principles.
Follow DataOps practices including CI/CD, deployment, monitoring and production support.
Troubleshoot pipeline failures, performance issues and data quality problems.
Optimize data pipelines, queries and storage for performance and cost efficiency.
Work closely with Data Architects and business stakeholders to understand requirements and implement technical solutions.
Participate in technical design discussions, code reviews and architecture reviews.
Maintain technical documentation, data flow diagrams and pipeline documentation.
Support production deployments, incident resolution and SLA-driven data platform operations.
Identify opportunities for automation and AI-assisted improvements across data engineering processes.

Qualifications and Skills

6+ years of experience in Data Engineering, Data Integration or Data Platform development.
Strong hands-on experience with Microsoft Fabric.
Experience with:
Microsoft Fabric Lakehouse
Fabric Warehouse
OneLake
Fabric Data Factory / Pipelines
Semantic Models
Strong understanding of Lakehouse and Medallion Architecture.
Strong SQL development and query optimization skills.
Hands-on experience with Python and/or PySpark.
Experience developing enterprise ETL/ELT and data integration pipelines.
Experience with batch and incremental data processing.
Understanding of data modelling concepts including dimensional modelling.
Knowledge of data quality, metadata, lineage and data governance.
Working knowledge of Microsoft Purview.
Understanding of Data Mesh and Data Product concepts.
Experience with CI/CD, version control, monitoring and DataOps practices.
Understanding of cloud security, access controls and data privacy.
Good troubleshooting and problem-solving skills.
Strong communication skills and ability to work with distributed and client-facing teams.

Preferred Skills

Microsoft Fabric or Azure Data certifications.
Experience migrating workloads from Azure Synapse, SQL Server, Databricks or other data platforms to Microsoft Fabric.
Experience implementing Medallion Architecture on Microsoft Fabric.
Experience with Power BI and semantic modelling.
Exposure to AI/ML, Generative AI or Agentic AI use cases on enterprise data platforms.
Experience working with Data Products or domain-oriented data solutions.
Experience in Energy & Utilities, Healthcare, Financial Services or Insurance.
Experience working with US or international enterprise clients.

What We Expect

Data / AI / GenAI – Engineering & Analytics

at NeoGenCode Technologies Pvt Ltd

2 candid answers

Posted by Akshay Patil

Gurugram, Noida, Ahmedabad

4 - 8 yrs

₹6L - ₹12L / yr

PowerBI

Python

ETL

Data engineering

Azure Data Factory

+8 more

Job Title : Data / AI / GenAI – Engineering & Analytics

Experience : 4+ Years

Location : Noida / Gurugram / Ahmedabad

Employment Type : Full-time | On-site at client location

Working Days : 5 days per week

Working Hours : Standard business hours

Position Overview : We are seeking professionals to deliver modern data engineering, business intelligence, cloud analytics and AI-powered applications, according to their specialization.

Mandatory Skills :

Python, SQL, ETL/ELT, Data Engineering, Power BI, Azure Data Factory, Databricks, PySpark, Data Modeling, GenAI/LLMs, RAG, Vector Databases, AI Agents (as per role specialization).

Key Roles / Specializations :

Power BI Developers : Dashboards, DAX, Power Query, reporting and data modeling
Azure Data Engineers : Azure Data Factory, Data Lake, Azure SQL and cloud pipelines
Databricks Engineers : Apache Spark, PySpark, Delta Lake and scalable processing
Data Engineers : ETL/ELT, SQL, Python, data warehousing and quality
GenAI / RAG Engineers : LLMs, prompts, embeddings, vector databases and retrieval pipelines
Agentic AI Engineers : AI agents, tool calling, orchestration and multi-agent workflows

Key Responsibilities :

Design, develop and maintain data pipelines, analytics solutions or AI applications.
Build and optimize data integration, transformation and processing workflows.
Develop dashboards and analytical models for business decision-making.
Use cloud data platforms and modern engineering frameworks.
Develop and evaluate LLM applications, RAG solutions or AI agents as applicable.
Ensure data quality, security, performance and reliability.
Collaborate with engineering teams and stakeholders on testing, deployment and documentation.

Required Skills & Qualifications :

Bachelor’s degree in Computer Science, IT, Data Science, Engineering or related discipline.
4–5 years of relevant experience in data, analytics, AI or software engineering.
Hands-on expertise in at least one listed specialization; not all technologies are required.
Relevant Python, SQL, DAX or other technical skills depending on the position.
Understanding of cloud platforms, APIs, integration and software practices.

Preferred Qualifications :

Azure, Databricks or Microsoft Fabric experience.
For GenAI roles : Azure OpenAI, OpenAI APIs, LangChain, LlamaIndex or LangGraph.
For relevant roles : vector databases, CI/CD, Docker, MLOps or LLMOps.

Note : Candidates are expected to have expertise in their relevant specialization, not in every listed technology or role.

Job Title : Data / AI / GenAI – Engineering & Analytics

Experience : 4+ Years

Location : Noida / Gurugram / Ahmedabad

Employment Type : Full-time | On-site at client location

Working Days : 5 days per week

Working Hours : Standard business hours

Position Overview : We are seeking professionals to deliver modern data engineering, business intelligence, cloud analytics and AI-powered applications, according to their specialization.

Mandatory Skills :

Python, SQL, ETL/ELT, Data Engineering, Power BI, Azure Data Factory, Databricks, PySpark, Data Modeling, GenAI/LLMs, RAG, Vector Databases, AI Agents (as per role specialization).

Key Roles / Specializations :

Power BI Developers : Dashboards, DAX, Power Query, reporting and data modeling
Azure Data Engineers : Azure Data Factory, Data Lake, Azure SQL and cloud pipelines
Databricks Engineers : Apache Spark, PySpark, Delta Lake and scalable processing
Data Engineers : ETL/ELT, SQL, Python, data warehousing and quality
GenAI / RAG Engineers : LLMs, prompts, embeddings, vector databases and retrieval pipelines
Agentic AI Engineers : AI agents, tool calling, orchestration and multi-agent workflows

Key Responsibilities :

Design, develop and maintain data pipelines, analytics solutions or AI applications.
Build and optimize data integration, transformation and processing workflows.
Develop dashboards and analytical models for business decision-making.
Use cloud data platforms and modern engineering frameworks.
Develop and evaluate LLM applications, RAG solutions or AI agents as applicable.
Ensure data quality, security, performance and reliability.
Collaborate with engineering teams and stakeholders on testing, deployment and documentation.

Required Skills & Qualifications :

Bachelor’s degree in Computer Science, IT, Data Science, Engineering or related discipline.
4–5 years of relevant experience in data, analytics, AI or software engineering.
Hands-on expertise in at least one listed specialization; not all technologies are required.
Relevant Python, SQL, DAX or other technical skills depending on the position.
Understanding of cloud platforms, APIs, integration and software practices.

Preferred Qualifications :

Azure, Databricks or Microsoft Fabric experience.
For GenAI roles : Azure OpenAI, OpenAI APIs, LangChain, LlamaIndex or LangGraph.
For relevant roles : vector databases, CI/CD, Docker, MLOps or LLMOps.

Note : Candidates are expected to have expertise in their relevant specialization, not in every listed technology or role.

Tech Lead – Data Lake Platform

at NeoGenCode Technologies Pvt Ltd

2 candid answers

Posted by Divya Sharma

Gurugram

5 - 12 yrs

₹8L - ₹16L / yr

Amazon Web Services (AWS)

Data engineering

PySpark

Apache Airflow

SQL

+4 more

Tech Lead – Data Lake Platform

Recruitment & Payroll Company: Neogencode Technologies

Job Location: Sector 43, Gurugram, Haryana

Work Mode: On-site (Work from Office)

Working Days: 5 days a week

Experience Required: 5+ years

Employment Type: Full-time

About the Opportunity

We are hiring a Tech Lead – Data Lake Platform. This opportunity involves working on an enterprise-grade AWS data lake platform supporting data engineering, analytics, reporting, and business applications.

The candidate will be employed on the payroll of Neogencode Technologies and will work at the client location in Sector 43, Gurugram.

Key Responsibilities

Own and design end-to-end data lake architecture, including data ingestion, staging, curated layers, and data consumption.
Lead data engineering pipelines using AWS services, Apache Airflow, Spark/PySpark, and dbt.
Manage AWS data services such as S3, EMR, Glue, Athena, Redshift, Lambda, RDS, IAM, and DMS.
Design and support API/data-serving solutions using PostgreSQL, Hasura GraphQL, or similar technologies.
Improve pipeline performance, platform reliability, monitoring, and service-level agreements (SLAs).
Ensure data security, access control, personally identifiable information (PII) protection, and governance.
Mentor data engineers, conduct code reviews, and handle production incidents and root-cause analysis.
Collaborate with business and application teams to deliver data and reporting solutions.

Required Skills

5+ years of experience in data engineering or platform engineering, including technical leadership.
Strong hands-on experience with AWS data services and enterprise data lake architecture.
Production experience with Apache Airflow, Spark/PySpark, and SQL.
Strong understanding of data modeling and layered data architecture.
Experience with PostgreSQL and Hasura GraphQL or a similar API/data-serving layer.
Proven experience in technical leadership, production support, and troubleshooting.

Good to Have

Experience with dbt and lakehouse table formats.
Knowledge of Kafka or streaming data ingestion.
Experience with Redshift optimization and workload design.

Tech Lead – Data Lake Platform

Recruitment & Payroll Company: Neogencode Technologies

Job Location: Sector 43, Gurugram, Haryana

Work Mode: On-site (Work from Office)

Working Days: 5 days a week

Experience Required: 5+ years

Employment Type: Full-time

About the Opportunity

The candidate will be employed on the payroll of Neogencode Technologies and will work at the client location in Sector 43, Gurugram.

Key Responsibilities

Own and design end-to-end data lake architecture, including data ingestion, staging, curated layers, and data consumption.
Lead data engineering pipelines using AWS services, Apache Airflow, Spark/PySpark, and dbt.
Manage AWS data services such as S3, EMR, Glue, Athena, Redshift, Lambda, RDS, IAM, and DMS.
Design and support API/data-serving solutions using PostgreSQL, Hasura GraphQL, or similar technologies.
Improve pipeline performance, platform reliability, monitoring, and service-level agreements (SLAs).
Ensure data security, access control, personally identifiable information (PII) protection, and governance.
Mentor data engineers, conduct code reviews, and handle production incidents and root-cause analysis.
Collaborate with business and application teams to deliver data and reporting solutions.

Required Skills

5+ years of experience in data engineering or platform engineering, including technical leadership.
Strong hands-on experience with AWS data services and enterprise data lake architecture.
Production experience with Apache Airflow, Spark/PySpark, and SQL.
Strong understanding of data modeling and layered data architecture.
Experience with PostgreSQL and Hasura GraphQL or a similar API/data-serving layer.
Proven experience in technical leadership, production support, and troubleshooting.

Good to Have

Experience with dbt and lakehouse table formats.
Knowledge of Kafka or streaming data ingestion.
Experience with Redshift optimization and workload design.

Data Engineer

at VY SYSTEMS PRIVATE LIMITED

1 recruiter

Posted by Dharani S

Bengaluru (Bangalore)

5 - 9 yrs

₹3L - ₹20L / yr

Python

DevOps

PySpark

Job Description

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines.
Develop data processing solutions using Python.
Write complex and optimized SQL queries, stored procedures, and data transformations.
Build and maintain data ingestion and integration workflows.
Implement data quality, validation, monitoring, and error-handling processes.
Develop and maintain CI/CD pipelines for data engineering applications.
Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
Collaborate with data analysts, data scientists, software engineers, and business teams.
Optimize data pipelines for performance, reliability, and scalability.
Troubleshoot production data issues and ensure timely resolution.
Follow best practices for version control, code quality, testing, and deployment.

Mandatory Skills

Python
ETL
SQL
CI/CD
DevOps
Git / Version Control
Strong problem-solving and debugging skills

Job Description

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines.
Develop data processing solutions using Python.
Write complex and optimized SQL queries, stored procedures, and data transformations.
Build and maintain data ingestion and integration workflows.
Implement data quality, validation, monitoring, and error-handling processes.
Develop and maintain CI/CD pipelines for data engineering applications.
Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
Collaborate with data analysts, data scientists, software engineers, and business teams.
Optimize data pipelines for performance, reliability, and scalability.
Troubleshoot production data issues and ensure timely resolution.
Follow best practices for version control, code quality, testing, and deployment.

Mandatory Skills

Python
ETL
SQL
CI/CD
DevOps
Git / Version Control
Strong problem-solving and debugging skills

Senior Data Engineer

at QuaXigma IT solutions Private Limited

Posted by SaiSruthi Nuthanpati

Tirupati, Chennai

5 - 10 yrs

Best in industry

SQL

Python

Stored Procedures

Amazon Web Services (AWS)

Microsoft Windows Azure

+10 more

About Us:

The QX Impact was launched with a mission to make A.I accessible and affordable and deliver AI Products/Solutions at scale for the enterprises by bringing the power of Data, AI, and Engineering to drive digital transformation. We believe without insights; businesses will continue to face challenges to better understand their customers and even lose them. Secondly, without insights businesses won't’ be able to deliver differentiated products/services; and finally, without insights, businesses can’t achieve a new level of “Operational Excellence” is crucial to remain competitive, meeting rising customer expectations, expanding markets, and digitalization.

Job Summary:

We are looking for a Senior Data Engineer who is creative, collaborative, and adaptable to join our agile team of data scientists, engineers, and UX developers. The role focuses on building and maintaining robust data pipelines to support advanced analytics, data science, and BI solutions.

As a Senior Data Engineer, you will work with internal and external data, collaborate with data scientists, and contribute to the design, development, and deployment of innovative solutions.

Key Responsibilities:

Design, develop, test, and maintain optimal data pipeline and ETL architectures.
Map out data systems and define/design required integrations, ETL, BI, and AI systems/processes.
Prepare and optimize data for predictive and prescriptive modeling.
Collaborate with teams to integrate ERP data into the enterprise data lake, ensuring seamless flow and quality.
Enhance cloud data infrastructure on AWS or Azure for scalability and performance.
Utilize big data tools and frameworks to optimize data acquisition and preparation.
Build architectures to move data to/from data lakes and data warehouses for advanced analytics.
Develop and curate data models for analytics, dashboards, and reports.
Conduct code reviews, maintain production-level code, and implement testing approaches.
Monitor, troubleshoot, and resolve data ingestion workflows to maintain reliability and uptime.
Drive innovation and implement efficient new approaches to data engineering tasks.

Must-Have Skills:

Bachelor’s degree in Computer Science, Mathematics, Engineering, or a related field.
5+ years of experience working with enterprise data platforms, including building and managing data lakes.
3–5 years of experience designing and implementing data warehouse solutions.
Expertise in SQL, including developing stored procedures (SP) and applying advanced data design concepts.
Proficiency in Spark (Python/Scala) and Spark Streaming for real-time data pipelines.
Experience with AWS or Azure services (e.g., AWS Glue, Azure Data Factory, Redshift, Snowflake).
Familiarity with big data tools such as Apache Kafka, Apache Spark, or Flink.
Hands-on experience with orchestration tools (e.g., Apache Airflow, Prefect).
Knowledge of CI/CD processes, version control (e.g., Git, Jenkins), and deployment automation.
Strong problem-solving, communication, and collaboration skills.

Good-to-Have Skills:

Experience in integrating ERP data into data lakes.
Experience with traditional ETL tools (e.g., Talend, Pentaho).

Competencies:

Tech Savvy - Anticipating and adopting innovations in business-building digital and technology applications.
Self-Development - Actively seeking new ways to grow and be challenged using both formal and informal development channels.
Action Oriented - Taking on new opportunities and tough challenges with a sense of urgency, high energy, and enthusiasm.
Customer Focus - Building strong customer relationships and delivering customer-centric solutions.
Optimize Work Processes - Knowing the most effective and efficient processes to get things done, with a focus on continuous improvement.

Why Join Us?

Be part of a collaborative and agile team driving cutting-edge AI and data engineering solutions.
Work on impactful projects that make a difference across industries.
Opportunities for professional growth and continuous learning.
Competitive salary and benefits package.

Application Details

Ready to make an impact? Apply today and become part of the QX Impact team!

About Us:

Job Summary:

As a Senior Data Engineer, you will work with internal and external data, collaborate with data scientists, and contribute to the design, development, and deployment of innovative solutions.

Key Responsibilities:

Design, develop, test, and maintain optimal data pipeline and ETL architectures.
Map out data systems and define/design required integrations, ETL, BI, and AI systems/processes.
Prepare and optimize data for predictive and prescriptive modeling.
Collaborate with teams to integrate ERP data into the enterprise data lake, ensuring seamless flow and quality.
Enhance cloud data infrastructure on AWS or Azure for scalability and performance.
Utilize big data tools and frameworks to optimize data acquisition and preparation.
Build architectures to move data to/from data lakes and data warehouses for advanced analytics.
Develop and curate data models for analytics, dashboards, and reports.
Conduct code reviews, maintain production-level code, and implement testing approaches.
Monitor, troubleshoot, and resolve data ingestion workflows to maintain reliability and uptime.
Drive innovation and implement efficient new approaches to data engineering tasks.

Must-Have Skills:

Bachelor’s degree in Computer Science, Mathematics, Engineering, or a related field.
5+ years of experience working with enterprise data platforms, including building and managing data lakes.
3–5 years of experience designing and implementing data warehouse solutions.
Expertise in SQL, including developing stored procedures (SP) and applying advanced data design concepts.
Proficiency in Spark (Python/Scala) and Spark Streaming for real-time data pipelines.
Experience with AWS or Azure services (e.g., AWS Glue, Azure Data Factory, Redshift, Snowflake).
Familiarity with big data tools such as Apache Kafka, Apache Spark, or Flink.
Hands-on experience with orchestration tools (e.g., Apache Airflow, Prefect).
Knowledge of CI/CD processes, version control (e.g., Git, Jenkins), and deployment automation.
Strong problem-solving, communication, and collaboration skills.

Good-to-Have Skills:

Experience in integrating ERP data into data lakes.
Experience with traditional ETL tools (e.g., Talend, Pentaho).

Competencies:

Tech Savvy - Anticipating and adopting innovations in business-building digital and technology applications.
Self-Development - Actively seeking new ways to grow and be challenged using both formal and informal development channels.
Action Oriented - Taking on new opportunities and tough challenges with a sense of urgency, high energy, and enthusiasm.
Customer Focus - Building strong customer relationships and delivering customer-centric solutions.
Optimize Work Processes - Knowing the most effective and efficient processes to get things done, with a focus on continuous improvement.

Why Join Us?

Be part of a collaborative and agile team driving cutting-edge AI and data engineering solutions.
Work on impactful projects that make a difference across industries.
Opportunities for professional growth and continuous learning.
Competitive salary and benefits package.

Application Details

Ready to make an impact? Apply today and become part of the QX Impact team!

Why apply to jobs via Cutshort