GCP Data Engineer at VY SYSTEMS PRIVATE LIMITED · Bengaluru (Bangalore), Hyderabad · 5 - 18 years · ₹4L - ₹35L / yr · Profitable · Posted 16 Sep 2026

Senior Data Engineer – Ab Initio | GCP | Spark | Agentic AI
Location: Bangalore
Experience: 5+ Years
Role: Senior Data Engineer
Work Mode: Bangalore
Job Summary
We are looking for an experienced Senior Data Engineer with strong expertise in Ab Initio, GCP, Apache Spark, and Agentic AI. The ideal candidate will have hands-on experience designing and developing scalable data engineering solutions, building data pipelines, and working with modern cloud and AI technologies.
The candidate should be comfortable working across traditional enterprise data platforms and emerging Generative AI / Agentic AI solutions.
Key Responsibilities
- Design, develop, and maintain scalable and high-performance data pipelines using Ab Initio, Spark, and GCP services.
- Develop and optimize complex ETL/ELT workflows using Ab Initio.
- Build and maintain data processing solutions using Apache Spark / PySpark.
- Develop cloud-based data solutions on Google Cloud Platform (GCP).
- Work with GCP data services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, or equivalent services.
- Perform data integration, transformation, cleansing, and validation.
- Optimize data pipelines for performance, scalability, reliability, and cost.
- Collaborate with data architects,

About VY SYSTEMS PRIVATE LIMITED
About
Vy Systems is a Global Technology consulting, Solutions, and Managed Technology Services company. We service our customers with ‘RESPONSIVENESS’ as a key factor and we believe that timely response to any transaction increases the operational efficiency and accelerates the revenue and profitability to our customers.
The Company is founded and managed by a team of professionals having more than two+ decades of global experience in the business of Technology Consulting and Services.
Tech stack
Similar jobs (10)
GCP Data engineer
Exp 9+yrs
Bangalore/Hyderabad
Lead exp+ Python + pyspark + sql or plsql
immediate to 15days joiners
Call or WhatsApp: 6 3 8 5 0 7 6 1 4 2
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
Job Description: Python + AI
Company: Wissen Technology
Location: Bangalore, India
Experience: 5+Years
Employment Type: Full-Time
Role: Python + AI / Data Engineer
About the Role
Wissen Technology is looking for experienced Python + AI / Data Engineering professionals to join our technology team in Bangalore. The ideal candidate will have strong hands-on experience in Python, Artificial Intelligence, Generative AI, PySpark, Snowflake, and data pipeline development.
The candidate should be capable of designing and developing scalable data and AI solutions, building robust ETL/ELT pipelines, working with large datasets, and integrating AI/ML capabilities into enterprise applications.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Python and PySpark.
- Develop robust ETL/ELT pipelines for processing large volumes of structured and unstructured data.
- Build and optimize data processing solutions using Apache Spark / PySpark.
- Develop data ingestion and transformation pipelines into Snowflake.
- Design and implement scalable Snowflake data models, tables, views, and SQL transformations.
- Work with batch and, where applicable, real-time data processing pipelines.
- Build and integrate AI and Generative AI solutions using Python.
- Develop LLM-based applications, RAG solutions, AI agents, and AI-powered services.
- Integrate AI models with enterprise data platforms and data pipelines.
- Develop REST APIs and microservices using FastAPI, Flask, or Django.
- Perform data cleansing, transformation, validation, and quality checks.
- Optimize PySpark jobs, SQL queries, Snowflake workloads, and data pipelines for performance and scalability.
- Implement data pipeline monitoring, logging, error handling, and alerting.
- Work with cloud platforms such as AWS, Azure, or GCP.
- Collaborate with Data Scientists, Data Engineers, Software Engineers, Architects, and business stakeholders.
- Participate in technical design, architecture, code reviews, and production support.
- Mentor junior engineers and contribute to engineering best practices.
Preferred Qualifications
- Bachelor's or master's degree in computer science, Engineering, Data Science, Artificial Intelligence, or a related field.
- Experience working on enterprise-scale AI and data engineering projects.
- Experience combining Python + PySpark + Snowflake + AI/GenAI in production environments.
- Experience with Databricks is an advantage.
- Experience with AI Agents / Agentic AI and tool/function calling.
- Knowledge of distributed systems and cloud-native architecture.
- Experience leading technical initiatives or mentoring engineering teams.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
🚀 Hiring: Senior GCP Data Engineer
We are looking for experienced Senior Data Engineers with strong hands-on expertise in Google Cloud Platform (GCP), Python, PySpark, and SQL. Candidates with team lead experience will be preferred.
📌 Job Details:
- Role: Senior GCP Data Engineer
- Experience: 8+ Years
- Budget: Up to ₹30 LPA
- Location: Bangalore / Hyderabad
- Relocation: Candidates from Chennai and Pune willing to relocate are welcome.
- Notice Period: Immediate to 15 days
- Interview Process: L1 – Virtual | L2 / DM – Face-to-Face (F2F)
💻 Required Technical Skills:
- Strong hands-on experience with GCP data engineering.
- Excellent proficiency in Python, PySpark, and SQL.
- Experience in designing, developing, and optimizing data pipelines.
- Good understanding of ETL/ELT processes and large-scale data processing.
- Experience working with cloud-based data platforms and data warehousing.
- Strong analytical, problem-solving, and communication skills.
⭐ Preferred Skills:
- Previous experience as a Technical Lead or leading a data engineering team.
- Knowledge of GCP services such as BigQuery, Dataproc, Dataflow, and Cloud Storage.
Experience: 6+ years overall Data Engineering experience.
Must-have — candidates should have hands-on experience in ALL of these:
- GCP (Google Cloud Platform) – strong hands-on experience
- Python – data engineering/ETL development
- SQL – advanced SQL, query optimization, data transformation
- BigQuery – strong hands-on experience with development, optimization and data warehousing
- Data Engineering / ETL – building and maintaining data pipelines
- GCP data services – preferably Cloud Storage, Dataflow, Pub/Sub, Composer/Airflow, etc.
- Data warehousing / dimensional modeling
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
Job Title : Senior Data Engineer – GCP & Apache Spark
Job Details :
- Number of Positions : 3
- Employment Type : Contract-to-Hire (C2H)
- Experience Required : 5 to 8 Years
- Location : Manyata Tech Park, Bengaluru
- Work Mode : Hybrid – 3 Days in Office per Week
- Preferred Notice Period : Immediate to 15–20 Days
- Project Duration : Initially 6–8 months, with a strong possibility of extension for long-term projects
- Relocation : Candidates from outside Bengaluru can apply if they are willing to relocate.
Role Overview :
We are looking for experienced Senior Data Engineers with strong expertise in Google Cloud Platform (GCP), Apache Spark, BigQuery, and SQL to design, develop, and maintain scalable data pipelines and distributed data processing solutions.
The ideal candidate should have hands-on experience with cloud-based data engineering, end-to-end pipeline architecture, system design, performance optimization, debugging, and CI/CD practices.
Mandatory skills :
Apache Spark (PySpark / Scala), SQL, GCP, BigQuery, GCP Dataproc, Data Pipelines, System Design, CI/CD, Debugging & Problem-Solving.
Key Responsibilities :
- Design, develop, and maintain scalable data pipelines using Apache Spark, PySpark, and/or Scala.
- Build and optimize batch and distributed data processing workflows on GCP.
- Work extensively with BigQuery for data transformation, validation, analytics, and query optimization.
- Develop and maintain data workflows using GCP services, including Dataproc, BigQuery, and Cloud Storage.
- Design end-to-end data pipeline architectures with a focus on scalability, reliability, and performance.
- Write and optimize complex SQL queries for large-scale datasets.
- Troubleshoot and resolve data processing issues, pipeline failures, and performance bottlenecks.
- Implement CI/CD practices for data engineering workflows, testing, and deployments.
- Monitor pipeline performance and identify opportunities for continuous improvement.
- Collaborate with cross-functional teams to understand requirements and deliver reliable data solutions.
- Participate in code reviews, testing, production deployments, and production support.
Mandatory Skills :
- Strong hands-on experience with Apache Spark, including PySpark and/or Scala.
- Strong SQL skills and experience working with large-scale datasets.
- Hands-on experience with Google Cloud Platform (GCP).
- Practical experience with BigQuery and data warehousing concepts.
- Working knowledge of GCP Dataproc.
- Strong understanding of data pipelines, end-to-end workflows, and system design.
- Practical experience with CI/CD and deployment processes.
- Excellent debugging, troubleshooting, and problem-solving skills.
Preferred Skills :
- Experience with Apache Airflow, Google Cloud Composer, or similar orchestration tools.
- Knowledge of Kafka or other streaming and messaging technologies.
- Proficiency in Python for data engineering and automation.
- Experience with Git-based development workflows.
- Knowledge of Linux/Unix environments and shell scripting.
- Familiarity with monitoring, logging, data quality, and governance practices.
Required Qualifications :
- 5 to 8 years of experience in Data Engineering, Big Data Engineering, or a related field.
- Proven experience developing and maintaining scalable data pipelines.
- Strong understanding of distributed data processing and cloud-based data architecture.
- Ability to troubleshoot complex technical issues and optimize pipeline performance.
- Strong communication, collaboration, and ownership skills.
Interview Process :
Round 1 : AI-Based Interview
- Candidates must complete the AI interview as soon as possible after receiving the invitation.
Round 2 : Virtual Interview Drive
- Interviews are planned for the upcoming drive; exact dates will be communicated separately.
Round 3 : Online Client Interview
- Shortlisted candidates will attend the online interview.
- Candidates must be available throughout the day on the scheduled interview date.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
role: data engineer
Python pyspark, SQL, data engineer
5+yrs
Bangalore/Hyderabad
immediate to 15days.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
About Us:
The QX Impact was launched with a mission to make A.I accessible and affordable and deliver AI Products/Solutions at scale for the enterprises by bringing the power of Data, AI, and Engineering to drive digital transformation. We believe without insights; businesses will continue to face challenges to better understand their customers and even lose them. Secondly, without insights businesses won't’ be able to deliver differentiated products/services; and finally, without insights, businesses can’t achieve a new level of “Operational Excellence” is crucial to remain competitive, meeting rising customer expectations, expanding markets, and digitalization.
Job Summary:
We are looking for a Senior Data Engineer who is creative, collaborative, and adaptable to join our agile team of data scientists, engineers, and UX developers. The role focuses on building and maintaining robust data pipelines to support advanced analytics, data science, and BI solutions.
As a Senior Data Engineer, you will work with internal and external data, collaborate with data scientists, and contribute to the design, development, and deployment of innovative solutions.
Key Responsibilities:
- Design, develop, test, and maintain optimal data pipeline and ETL architectures.
- Map out data systems and define/design required integrations, ETL, BI, and AI systems/processes.
- Prepare and optimize data for predictive and prescriptive modeling.
- Collaborate with teams to integrate ERP data into the enterprise data lake, ensuring seamless flow and quality.
- Enhance cloud data infrastructure on AWS or Azure for scalability and performance.
- Utilize big data tools and frameworks to optimize data acquisition and preparation.
- Build architectures to move data to/from data lakes and data warehouses for advanced analytics.
- Develop and curate data models for analytics, dashboards, and reports.
- Conduct code reviews, maintain production-level code, and implement testing approaches.
- Monitor, troubleshoot, and resolve data ingestion workflows to maintain reliability and uptime.
- Drive innovation and implement efficient new approaches to data engineering tasks.
Must-Have Skills:
- Bachelor’s degree in Computer Science, Mathematics, Engineering, or a related field.
- 5+ years of experience working with enterprise data platforms, including building and managing data lakes.
- 3–5 years of experience designing and implementing data warehouse solutions.
- Expertise in SQL, including developing stored procedures (SP) and applying advanced data design concepts.
- Proficiency in Spark (Python/Scala) and Spark Streaming for real-time data pipelines.
- Experience with AWS or Azure services (e.g., AWS Glue, Azure Data Factory, Redshift, Snowflake).
- Familiarity with big data tools such as Apache Kafka, Apache Spark, or Flink.
- Hands-on experience with orchestration tools (e.g., Apache Airflow, Prefect).
- Knowledge of CI/CD processes, version control (e.g., Git, Jenkins), and deployment automation.
- Strong problem-solving, communication, and collaboration skills.
Good-to-Have Skills:
- Experience in integrating ERP data into data lakes.
- Experience with traditional ETL tools (e.g., Talend, Pentaho).
Competencies:
- Tech Savvy - Anticipating and adopting innovations in business-building digital and technology applications.
- Self-Development - Actively seeking new ways to grow and be challenged using both formal and informal development channels.
- Action Oriented - Taking on new opportunities and tough challenges with a sense of urgency, high energy, and enthusiasm.
- Customer Focus - Building strong customer relationships and delivering customer-centric solutions.
- Optimize Work Processes - Knowing the most effective and efficient processes to get things done, with a focus on continuous improvement.
Why Join Us?
- Be part of a collaborative and agile team driving cutting-edge AI and data engineering solutions.
- Work on impactful projects that make a difference across industries.
- Opportunities for professional growth and continuous learning.
- Competitive salary and benefits package.
Application Details
Ready to make an impact? Apply today and become part of the QX Impact team!
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
9+yrs
Bangalore/Hyderabad
immediate to 15days.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
Job Title: GCP Data Engineer
Experience: 6–8 years
Location : Bangalore
What are they looking for?
They need a Google Cloud Platform (GCP) Data Engineer who can build and maintain data pipelines, process large volumes of data, and manage data storage using Google Cloud services.
The candidate should have hands-on experience with batch and real-time data processing, ETL workflows, and cloud data warehousing.
Mandatory / Core Skills
- Google Cloud Platform (GCP)
- Google Cloud Dataflow
- Pub/Sub
- Apache Beam
- BigQuery
- Cloud Storage / Bigtable
- Cloud Composer (Apache Airflow)
- ETL / data pipeline development
- Apache Spark / Hadoop using Dataproc
Supporting Skills
- Cloud Data Fusion
- NoSQL databases
- Data quality and governance
- Data security, access controls, and encryption
- Batch and real-time streaming
- Preparing datasets for BI, analytics, and machine learning
GCP Data engineer (L1 virtual, L2 or DM F2F )
GCP + Python + Pyspark + sql + Lead
8+yrs
Bangalore/hyderabad location ( relocation profiles also from Chennai pune location)
immediate to 15days











