Data Engineer – GCP + Spark + DBT at NeoGenCode Technologies Pvt Ltd · Bengaluru (Bangalore) · 8 - 12 years · ₹15L - ₹22L / yr · Raised funding · Posted 28 Jul 2025
GitJob Title : Data Engineer – GCP + Spark + DBT
Location : Bengaluru (On-site at Client Location | 3 Days WFO)
Experience : 8 to 12 Years
Level : Associate Architect
Type : Full-time
Job Overview :
We are looking for a seasoned Data Engineer to join the Data Platform Engineering team supporting a Unified Data Platform (UDP). This role requires hands-on expertise in DBT, GCP, BigQuery, and PySpark, with a solid foundation in CI/CD, data pipeline optimization, and agile delivery.
Mandatory Skills : GCP, DBT, Google Dataform, BigQuery, PySpark/Spark SQL, Advanced SQL, CI/CD, Git, Agile Methodologies.
Key Responsibilities :
- Design, build, and optimize scalable data pipelines using BigQuery, DBT, and PySpark.
- Leverage GCP-native services like Cloud Storage, Pub/Sub, Dataproc, Cloud Functions, and Composer for ETL/ELT workflows.
- Implement and maintain CI/CD for data engineering projects with Git-based version control.
- Collaborate with cross-functional teams including Infra, Security, and DataOps for reliable, secure, and high-quality data delivery.
- Lead code reviews, mentor junior engineers, and enforce best practices in data engineering.
- Participate in Agile sprints, backlog grooming, and Jira-based project tracking.
Must-Have Skills :
- Strong experience with DBT, Google Dataform, and BigQuery
- Hands-on expertise with PySpark/Spark SQL
- Proficient in GCP for data engineering workflows
- Solid knowledge of SQL optimization, Git, and CI/CD pipelines
- Agile team experience and strong problem-solving abilities
Nice-to-Have Skills :
- Familiarity with Databricks, Delta Lake, or Kafka
- Exposure to data observability and quality frameworks (e.g., Great Expectations, Soda)
- Knowledge of MDM patterns, Terraform, or IaC is a plus
Shubham Vishwakarma
Full Stack Developer - Averlon
About NeoGenCode Technologies Pvt Ltd
About
Welcome to Neogencode Technologies, an IT services and consulting firm that provides innovative solutions to help businesses achieve their goals. Our team of experienced professionals is committed to providing tailored services to meet the specific needs of each client. Our comprehensive range of services includes software development, web design and development, mobile app development, cloud computing, cybersecurity, digital marketing, and skilled resource acquisition. We specialize in helping our clients find the right skilled resources to meet their unique business needs. At Neogencode Technologies, we prioritize communication and collaboration with our clients, striving to understand their unique challenges and provide customized solutions that exceed their expectations. We value long-term partnerships with our clients and are committed to delivering exceptional service at every stage of the engagement. Whether you are a small business looking to improve your processes or a large enterprise seeking to stay ahead of the competition, Neogencode Technologies has the expertise and experience to help you succeed. Contact us today to learn more about how we can support your business growth and provide skilled resources to meet your business needs.
Candid answers by the company
IT & Engineering Talent Staffing
- Provides full-time and contract-based hiring, delivering handpicked, pre‑screened developers across tech stacks—ranging from web, mobile, AI/ML, Web3/blockchain.
- Maintains a bench o vetted candidates, offering fast delivery of interview-ready profiles—often within 24 hours.
- Offers payroll management, handling compliance, tax, attendance, and documentation for both contractors and full-time employees.
2. End-to-End Project Delivery
- Delivers full-stack development solutions: web, mobile, cloud, AI/ML, Blockchain/Web3.
- Manages entire project lifecycle—requirements gathering, design (UI/UX), development, deployment, and ongoing support .
3. Additional Offerings
- Expands into cybersecurity consulting, digital marketing, and cloud platform services (like AWS, GCP, Azure) .
- Provides strategic IT consulting to align technology solutions with business objectives
Company social profiles
Similar jobs (10)
PythonJob Summary
Role Overview
We are looking for an experienced Data Engineer with strong expertise in Python, ETL, Advanced SQL, CI/CD, DevOps, and Data Analytics. The ideal candidate should have hands-on experience designing and developing scalable data pipelines, transforming large datasets, and supporting data-driven applications.
Experience with Google Cloud Platform (GCP) will be an added advantage.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
- Develop complex and optimized SQL queries, stored procedures, and data transformations.
- Build and maintain reliable data integration workflows across multiple data sources.
- Perform data cleansing, validation, transformation, and quality checks.
- Analyze data and provide insights to support business and technical requirements.
- Implement and maintain CI/CD pipelines for data engineering applications.
- Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
- Troubleshoot data pipeline failures, performance issues, and production incidents.
- Optimize data processing workflows for performance, scalability, and reliability.
- Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
- Follow best practices for version control, testing, documentation, and deployment.
- Contribute to cloud-based data engineering initiatives, preferably on GCP.
Required Skills
- 5–7 years of hands-on experience in Data Engineering.
- Strong programming skills in Python.
- Strong expertise in Advanced SQL and database concepts.
- Hands-on experience with ETL/ELT processes and data pipelines.
- Good understanding of Data Warehousing and Data Modeling concepts.
- Experience with CI/CD practices and tools.
- Strong understanding of DevOps principles, automation, and deployment processes.
- Strong data analytics and problem-solving skills.
- Experience working with large datasets and performance optimization.
- Good understanding of Git/version control and software development best practices.
Good to Have
- Hands-on experience with Google Cloud Platform (GCP).
- Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
- Experience with containerization/orchestration technologies such as Docker/Kubernetes.
- Experience with workflow orchestration tools such as Airflow.
- Knowledge of cloud-based data architecture and distributed data processing.
Preferred Candidate Profile
- Strong analytical and problem-solving abilities.
- Good communication and stakeholder management skills.
- Ability to work independently as well as in a collaborative team environment.
- Strong ownership of data pipelines and production systems.
- Candidates who can join at short notice are preferred.
Mandatory Skills
Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops
PythonRole Overview
We are looking for a GCP Data Engineer with 10+ years of experience to design, develop, and optimize scalable cloud-based data solutions. The ideal candidate will have strong hands-on expertise in GCP, BigQuery, and advanced SQL, with experience building data pipelines and working with large-scale datasets.
Key Responsibilities
- Design and develop scalable data pipelines and ETL/ELT processes on GCP.
- Build, optimize, and maintain data solutions using Google BigQuery.
- Develop complex SQL queries for data transformation, aggregation, and analysis.
- Design efficient data models and optimize pipelines for performance, scalability, and cost.
- Integrate data from multiple sources and ensure data quality, reliability, and availability.
- Troubleshoot pipeline and data issues and drive continuous improvement.
- Collaborate with data architects, analysts, application teams, and business stakeholders.
- Follow best practices for cloud security, data governance, testing, and documentation.
Required Skills
- 8+ years of Data Engineering experience
- Strong hands-on experience with GCP, Django, and MongoDB
- Extensive experience with BigQuery
- Advanced SQL skills
- Strong understanding of ETL/ELT and data pipeline development
- Data modeling and data warehousing experience
- Experience handling large-scale datasets and performance optimization
- Strong problem-solving and communication skills
Good to Have
- GCP services such as Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, or Cloud Functions
- Python or other data engineering languages
- Experience with data governance and security
- Agile development experience
The recruiter has not been active on this job recently. You may apply but please expect a delayed response.
Job Summary
We are seeking a highly skilled GCP Data Engineer with strong expertise in Google Cloud Platform (GCP), Python, ETL, and modern data engineering technologies. The ideal candidate should have hands-on experience designing and building scalable data pipelines using BigQuery, Dataflow, Pub/Sub, Airflow, and modern data lake technologies such as Apache Iceberg or Delta Lake.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines on Google Cloud Platform.
- Build and optimize data processing workflows using Python and Google Cloud Dataflow (Apache Beam).
- Develop and manage large-scale analytical data models in BigQuery.
- Implement event-driven data ingestion using Google Cloud Pub/Sub.
- Create, schedule, and monitor workflows using Apache Airflow and Autosys.
- Design and implement modern data lake architectures using Apache Iceberg or Delta Lake.
- Optimize query performance, storage, and compute costs in GCP.
- Ensure data quality, governance, security, and compliance across data platforms.
- Collaborate with Data Scientists, Analysts, and Application teams to deliver scalable data solutions.
- Troubleshoot production issues and continuously improve pipeline reliability and performance.
Mandatory Skills
- Strong hands-on experience with Google Cloud Platform (GCP).
- Proficiency in Python programming.
- Experience in designing and implementing ETL/ELT pipelines.
- Strong knowledge of BigQuery.
- Experience with Google Cloud Dataflow (Apache Beam).
- Experience with Google Cloud Pub/Sub.
- Hands-on experience with Apache Airflow.
- Experience in job scheduling using Autosys.
- Experience with modern table formats such as Apache Iceberg or Delta Lake.
- Strong SQL and data modeling skills.
Preferred Skills
- Experience with Cloud Storage, Dataproc, Cloud Composer, and Cloud Functions.
- Knowledge of CI/CD pipelines and DevOps practices.
- Experience with Docker and Kubernetes.
- Familiarity with Git and Agile/Scrum methodologies.
- Knowledge of data warehousing and dimensional modeling.
- Exposure to streaming and real-time data processing.
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- 4–8+ years of experience in Data Engineering with hands-on expertise in GCP technologies.
Required Experience
- Strong experience in developing enterprise-grade data pipelines using Python and GCP.
- Hands-on experience with BigQuery, Dataflow, Pub/Sub, and Airflow.
- Experience scheduling and monitoring batch workflows using Autosys.
- Experience implementing modern data lake architectures using Apache Iceberg or Delta Lake.
- Strong understanding of ETL best practices, performance tuning, and data optimization.
- Excellent analytical, troubleshooting, and problem-solving skills.
Mandatory Skills
- Google Cloud Platform (GCP)
- Python
- ETL
- BigQuery
- Autosys
- Apache Airflow
- Google Cloud Pub/Sub
- Google Cloud Dataflow (Apache Beam)
- Apache Iceberg / Delta Lake
- SQL & Data Modeling
Job Title : Senior Data Engineer – GCP & Apache Spark
Job Details :
- Number of Positions : 3
- Employment Type : Contract-to-Hire (C2H)
- Experience Required : 5 to 8 Years
- Location : Manyata Tech Park, Bengaluru
- Work Mode : Hybrid – 3 Days in Office per Week
- Preferred Notice Period : Immediate to 15–20 Days
- Project Duration : Initially 6–8 months, with a strong possibility of extension for long-term projects
- Relocation : Candidates from outside Bengaluru can apply if they are willing to relocate.
Role Overview :
We are looking for experienced Senior Data Engineers with strong expertise in Google Cloud Platform (GCP), Apache Spark, BigQuery, and SQL to design, develop, and maintain scalable data pipelines and distributed data processing solutions.
The ideal candidate should have hands-on experience with cloud-based data engineering, end-to-end pipeline architecture, system design, performance optimization, debugging, and CI/CD practices.
Mandatory skills :
Apache Spark (PySpark / Scala), SQL, GCP, BigQuery, GCP Dataproc, Data Pipelines, System Design, CI/CD, Debugging & Problem-Solving.
Key Responsibilities :
- Design, develop, and maintain scalable data pipelines using Apache Spark, PySpark, and/or Scala.
- Build and optimize batch and distributed data processing workflows on GCP.
- Work extensively with BigQuery for data transformation, validation, analytics, and query optimization.
- Develop and maintain data workflows using GCP services, including Dataproc, BigQuery, and Cloud Storage.
- Design end-to-end data pipeline architectures with a focus on scalability, reliability, and performance.
- Write and optimize complex SQL queries for large-scale datasets.
- Troubleshoot and resolve data processing issues, pipeline failures, and performance bottlenecks.
- Implement CI/CD practices for data engineering workflows, testing, and deployments.
- Monitor pipeline performance and identify opportunities for continuous improvement.
- Collaborate with cross-functional teams to understand requirements and deliver reliable data solutions.
- Participate in code reviews, testing, production deployments, and production support.
Mandatory Skills :
- Strong hands-on experience with Apache Spark, including PySpark and/or Scala.
- Strong SQL skills and experience working with large-scale datasets.
- Hands-on experience with Google Cloud Platform (GCP).
- Practical experience with BigQuery and data warehousing concepts.
- Working knowledge of GCP Dataproc.
- Strong understanding of data pipelines, end-to-end workflows, and system design.
- Practical experience with CI/CD and deployment processes.
- Excellent debugging, troubleshooting, and problem-solving skills.
Preferred Skills :
- Experience with Apache Airflow, Google Cloud Composer, or similar orchestration tools.
- Knowledge of Kafka or other streaming and messaging technologies.
- Proficiency in Python for data engineering and automation.
- Experience with Git-based development workflows.
- Knowledge of Linux/Unix environments and shell scripting.
- Familiarity with monitoring, logging, data quality, and governance practices.
Required Qualifications :
- 5 to 8 years of experience in Data Engineering, Big Data Engineering, or a related field.
- Proven experience developing and maintaining scalable data pipelines.
- Strong understanding of distributed data processing and cloud-based data architecture.
- Ability to troubleshoot complex technical issues and optimize pipeline performance.
- Strong communication, collaboration, and ownership skills.
Interview Process :
Round 1 : AI-Based Interview
- Candidates must complete the AI interview as soon as possible after receiving the invitation.
Round 2 : Virtual Interview Drive
- Interviews are planned for the upcoming drive; exact dates will be communicated separately.
Round 3 : Online Client Interview
- Shortlisted candidates will attend the online interview.
- Candidates must be available throughout the day on the scheduled interview date.
Data Engineer – Contract Opportunity
We are looking for an experienced Data Engineer with 5+ years of experience to work on subscriber activation, churn, FTE, and future reporting requirements.
Key Responsibilities:
- Work on data requirements related to Subscriber Activation, Churn, FTE, and future reporting.
- Work with BigQuery as the centralized data warehouse.
- Develop and maintain data ingestion and integration pipelines across multiple source systems.
- Design and implement ETL/ELT processes.
- Develop data models for reporting and analytics.
- Integrate BigQuery with Power BI or similar reporting tools.
Required Skills:
- Strong hands-on experience with GCP & BigQuery
- Data warehouse architecture, design, and implementation
- Data ingestion/integration across multiple source systems
- ETL/ELT and data pipeline development
- Data modelling for reporting and analytics
- Experience integrating BigQuery with Power BI or similar reporting tools
Contract: 1 month initially, with potential extension
Compensation: ₹6–7 LPA
Work Mode: Remote
Important
Since this is only a 1-month contract, mention “Potential extension” rather than saying it will definitely be extended.
PythonExperience: 6+ years overall Data Engineering experience.
Must-have — candidates should have hands-on experience in ALL of these:
- GCP (Google Cloud Platform) – strong hands-on experience
- Python – data engineering/ETL development
- SQL – advanced SQL, query optimization, data transformation
- BigQuery – strong hands-on experience with development, optimization and data warehousing
- Data Engineering / ETL – building and maintaining data pipelines
- GCP data services – preferably Cloud Storage, Dataflow, Pub/Sub, Composer/Airflow, etc.
- Data warehousing / dimensional modeling
PythonJob Description – Senior GCP Data Engineer
Position: Senior GCP Data Engineer
Experience: 8+ Years
Budget: Up to 30 LPA
Location: Bangalore / Hyderabad
Relocation: Candidates from Chennai / Pune willing to relocate are also welcome.
Notice Period: Immediate to 15 Days
Interview Process:
- Round 1: Virtual Interview
- Round 2 / DM Round: Face-to-Face Interview
Required Skills
- Strong hands-on experience in GCP Data Engineering.
- Good programming experience in Python.
- Strong experience with PySpark / Apache Spark.
- Proficiency in SQL and database concepts.
- Experience in developing and maintaining data pipelines.
- Good understanding of data processing, transformation, and integration.
- Experience in handling large-scale data processing.
- Strong problem-solving and debugging skills.
- Good understanding of cloud-based data engineering concepts.
- Experience in optimizing data pipelines for performance and scalability.
Preferred Skills
- Previous experience in a Lead Data Engineer role.
- Experience in technical leadership, mentoring, and guiding team members.
- Ability to collaborate with cross-functional teams and deliver data engineering solutions.
Roles & Responsibilities
- Design, develop, and maintain scalable data pipelines using GCP.
- Develop data processing and transformation workflows using Python and PySpark.
- Write and optimize SQL queries for efficient data processing.
- Build and maintain reliable data engineering solutions.
- Troubleshoot data pipeline issues and optimize performance.
- Ensure data quality, reliability, and scalability.
- Collaborate with stakeholders and team members to deliver project requirements.
- Lead and guide team members where applicable.
- Important: Candidates must have strong Senior Data Engineer experience. Lead experience is preferred but not mandatory.
Python🚀 Hiring: Senior GCP Data Engineer
We are looking for experienced Senior Data Engineers with strong hands-on expertise in Google Cloud Platform (GCP), Python, PySpark, and SQL. Candidates with team lead experience will be preferred.
📌 Job Details:
- Role: Senior GCP Data Engineer
- Experience: 8+ Years
- Budget: Up to ₹30 LPA
- Location: Bangalore / Hyderabad
- Relocation: Candidates from Chennai and Pune willing to relocate are welcome.
- Notice Period: Immediate to 15 days
- Interview Process: L1 – Virtual | L2 / DM – Face-to-Face (F2F)
💻 Required Technical Skills:
- Strong hands-on experience with GCP data engineering.
- Excellent proficiency in Python, PySpark, and SQL.
- Experience in designing, developing, and optimizing data pipelines.
- Good understanding of ETL/ELT processes and large-scale data processing.
- Experience working with cloud-based data platforms and data warehousing.
- Strong analytical, problem-solving, and communication skills.
⭐ Preferred Skills:
- Previous experience as a Technical Lead or leading a data engineering team.
- Knowledge of GCP services such as BigQuery, Dataproc, Dataflow, and Cloud Storage.
PythonExperience: 5+ Years
Employment Type: Full-Time
Role Overview
We are looking for an experienced GCP Data Engineer with 5+ years of experience in data engineering and strong hands-on expertise in Google BigQuery, Google Cloud Storage (GCS), Airflow/Cloud Composer, Python, and Vertex AI. The candidate should be capable of designing, developing, and maintaining scalable data pipelines and cloud-based data solutions on Google Cloud Platform.
Key Skills – Mandatory
- BigQuery – Strong hands-on experience in data warehousing, SQL, optimization, and performance tuning.
- Google Cloud Storage (GCS) – Experience with data storage, file management, and integration with data pipelines.
- Airflow / Cloud Composer – Experience in developing, scheduling, monitoring, and managing data workflows.
- Python – Strong programming skills for data engineering, ETL/ELT development, automation, and pipeline implementation.
- Vertex AI – Experience working with ML/AI workflows, model integration, or data pipelines supporting AI/ML solutions.
Good to Have / Added Advantage
- Dataproc – Experience with distributed data processing and Spark-based workloads.
- Cloud Data Fusion – Experience in building and managing data integration pipelines.
- Cloud Run – Understanding of deploying and running containerized applications/services on GCP.
- Experience with ETL/ELT processes and data pipeline development.
- Knowledge of GCP data architecture and cloud-native services.
- Experience in data quality, validation, monitoring, and troubleshooting.
Responsibilities
- Design, develop, and maintain scalable GCP-based data pipelines.
- Build and optimize data solutions using BigQuery and Cloud Storage.
- Develop and manage workflows using Airflow / Cloud Composer.
- Write efficient and reusable Python code for data processing and automation.
- Support Vertex AI integrations and AI/ML data workflows.
- Monitor pipeline performance and troubleshoot data processing issues.
- Work with cross-functional teams to understand data requirements and deliver reliable solutions.
- Implement best practices for data security, quality, scalability, and performance.
You must have :
- 5+ years of overall experience in Data Engineering.
- Strong hands-on experience with BigQuery, GCS, Airflow/Cloud Composer, Python, and Vertex AI.
- Strong understanding of data engineering concepts, ETL/ELT, data pipelines, and cloud technologies.
- Dataproc, Data Fusion, and Cloud Run experience will be an added advantage.
PythonGCP Data engineer (L1 virtual, L2 or DM F2F )
GCP + Python + Pyspark + sql + Lead
8+yrs
Bangalore/hyderabad location ( relocation profiles also from Chennai pune location)
immediate to 15days
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon