GCP Data Engineer

Experience: 5+ Years

Employment Type: Full-Time

Role Overview

We are looking for an experienced GCP Data Engineer with 5+ years of experience in data engineering and strong hands-on expertise in Google BigQuery, Google Cloud Storage (GCS), Airflow/Cloud Composer, Python, and Vertex AI. The candidate should be capable of designing, developing, and maintaining scalable data pipelines and cloud-based data solutions on Google Cloud Platform.

Key Skills – Mandatory

• BigQuery – Strong hands-on experience in data warehousing, SQL, optimization, and performance tuning.
• Google Cloud Storage (GCS) – Experience with data storage, file management, and integration with data pipelines.
• Airflow / Cloud Composer – Experience in developing, scheduling, monitoring, and managing data workflows.
• Python – Strong programming skills for data engineering, ETL/ELT development, automation, and pipeline implementation.
• Vertex AI – Experience working with ML/AI workflows, model integration, or data pipelines supporting AI/ML solutions.

Good to Have / Added Advantage

• Dataproc – Experience with distributed data processing and Spark-based workloads.
• Cloud Data Fusion – Experience in building and managing data integration pipelines.
• Cloud Run – Understanding of deploying and running containerized applications/services on GCP.
• Experience with ETL/ELT processes and data pipeline development.
• Knowledge of GCP data architecture and cloud-native services.
• Experience in data quality, validation, monitoring, and troubleshooting.

Responsibilities

• Design, develop, and maintain scalable GCP-based data pipelines.
• Build and optimize data solutions using BigQuery and Cloud Storage.
• Develop and manage workflows using Airflow / Cloud Composer.
• Write efficient and reusable Python code for data processing and automation.
• Support Vertex AI integrations and AI/ML data workflows.
• Monitor pipeline performance and troubleshoot data processing issues.
• Work with cross-functional teams to understand data requirements and deliver reliable solutions.
• Implement best practices for data security, quality, scalability, and performance.

You must have :

• 5+ years of overall experience in Data Engineering.
• Strong hands-on experience with BigQuery, GCS, Airflow/Cloud Composer, Python, and Vertex AI.
• Strong understanding of data engineering concepts, ETL/ELT, data pipelines, and cloud technologies.
• Dataproc, Data Fusion, and Cloud Run experience will be an added advantage.

Role Overview

We are looking for a GCP Data Engineer with 10+ years of experience to design, develop, and optimize scalable cloud-based data solutions. The ideal candidate will have strong hands-on expertise in GCP, BigQuery, and advanced SQL, with experience building data pipelines and working with large-scale datasets.

Key Responsibilities

• Design and develop scalable data pipelines and ETL/ELT processes on GCP.
• Build, optimize, and maintain data solutions using Google BigQuery.
• Develop complex SQL queries for data transformation, aggregation, and analysis.
• Design efficient data models and optimize pipelines for performance, scalability, and cost.
• Integrate data from multiple sources and ensure data quality, reliability, and availability.
• Troubleshoot pipeline and data issues and drive continuous improvement.
• Collaborate with data architects, analysts, application teams, and business stakeholders.
• Follow best practices for cloud security, data governance, testing, and documentation.

Required Skills

• 8+ years of Data Engineering experience
• Strong hands-on experience with GCP, Django, and MongoDB
• Extensive experience with BigQuery
• Advanced SQL skills
• Strong understanding of ETL/ELT and data pipeline development
• Data modeling and data warehousing experience
• Experience handling large-scale datasets and performance optimization
• Strong problem-solving and communication skills

Good to Have

• GCP services such as Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, or Cloud Functions
• Python or other data engineering languages
• Experience with data governance and security
• Agile development experience

Experience: 6+ years overall Data Engineering experience.

Must-have — candidates should have hands-on experience in ALL of these:

1. GCP (Google Cloud Platform) – strong hands-on experience
2. Python – data engineering/ETL development
3. SQL – advanced SQL, query optimization, data transformation
4. BigQuery – strong hands-on experience with development, optimization and data warehousing
5. Data Engineering / ETL – building and maintaining data pipelines
6. GCP data services – preferably Cloud Storage, Dataflow, Pub/Sub, Composer/Airflow, etc.
7. Data warehousing / dimensional modeling

Job Summary

We are seeking a highly skilled GCP Data Engineer with strong expertise in Google Cloud Platform (GCP), Python, ETL, and modern data engineering technologies. The ideal candidate should have hands-on experience designing and building scalable data pipelines using BigQuery, Dataflow, Pub/Sub, Airflow, and modern data lake technologies such as Apache Iceberg or Delta Lake.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines on Google Cloud Platform.
• Build and optimize data processing workflows using Python and Google Cloud Dataflow (Apache Beam).
• Develop and manage large-scale analytical data models in BigQuery.
• Implement event-driven data ingestion using Google Cloud Pub/Sub.
• Create, schedule, and monitor workflows using Apache Airflow and Autosys.
• Design and implement modern data lake architectures using Apache Iceberg or Delta Lake.
• Optimize query performance, storage, and compute costs in GCP.
• Ensure data quality, governance, security, and compliance across data platforms.
• Collaborate with Data Scientists, Analysts, and Application teams to deliver scalable data solutions.
• Troubleshoot production issues and continuously improve pipeline reliability and performance.

Mandatory Skills

• Strong hands-on experience with Google Cloud Platform (GCP).
• Proficiency in Python programming.
• Experience in designing and implementing ETL/ELT pipelines.
• Strong knowledge of BigQuery.
• Experience with Google Cloud Dataflow (Apache Beam).
• Experience with Google Cloud Pub/Sub.
• Hands-on experience with Apache Airflow.
• Experience in job scheduling using Autosys.
• Experience with modern table formats such as Apache Iceberg or Delta Lake.
• Strong SQL and data modeling skills.

Preferred Skills

• Experience with Cloud Storage, Dataproc, Cloud Composer, and Cloud Functions.
• Knowledge of CI/CD pipelines and DevOps practices.
• Experience with Docker and Kubernetes.
• Familiarity with Git and Agile/Scrum methodologies.
• Knowledge of data warehousing and dimensional modeling.
• Exposure to streaming and real-time data processing.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 4–8+ years of experience in Data Engineering with hands-on expertise in GCP technologies.

Required Experience

• Strong experience in developing enterprise-grade data pipelines using Python and GCP.
• Hands-on experience with BigQuery, Dataflow, Pub/Sub, and Airflow.
• Experience scheduling and monitoring batch workflows using Autosys.
• Experience implementing modern data lake architectures using Apache Iceberg or Delta Lake.
• Strong understanding of ETL best practices, performance tuning, and data optimization.
• Excellent analytical, troubleshooting, and problem-solving skills.

Mandatory Skills

• Google Cloud Platform (GCP)
• Python
• ETL
• BigQuery
• Autosys
• Apache Airflow
• Google Cloud Pub/Sub
• Google Cloud Dataflow (Apache Beam)
• Apache Iceberg / Delta Lake
• SQL & Data Modeling

• Design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data ingestion and transformation.
• Develop and optimize data lake and data warehouse architectures (e.g., Snowflake, BigQuery, Redshift).
• Work with cloud platforms GCP, Azure to manage data infrastructure.
• GCP as mandatory skills
• Collaborate with analytics and product teams to understand data needs and deliver solutions.
• Ensure data quality, reliability, security, and compliance across all data systems.
• Mentor junior data engineers and contribute to best practices and code reviews.
• Monitor and troubleshoot data pipeline performance and resolve data-related issues.
• Automate data validation, monitoring, and alerting processes.
• 8+ years of experience in data engineering or software engineering with a data focus.
• Proficient in SQL and at least one programming language (e.g., Python, Scala, Java).
• Experience with modern data warehousing tools (e.g., Snowflake, Redshift, BigQuery).
• Strong understanding of data modeling, data lakes, and ETL/ELT design.
• Hands-on experience with orchestration tools like Airflow, dbt, or similar.
• Solid experience with cloud data platforms (AWS/GCP/Azure).
• Familiarity with CI/CD pipelines, containerization (Docker/Kubernetes), and version control (Git).
• Experience working in a DevOps or DataOps environment.
• Knowledge of data governance, lineage, and cataloging tools (e.g., Collibra, Alation).
• Familiarity with streaming technologies (Kafka, Spark Streaming, Flink).
• Experience supporting machine learning workflows and data science initiatives.

Senior Data Engineer – Ab Initio | GCP | Spark | Agentic AI

Location: Bangalore

Experience: 5+ Years

Role: Senior Data Engineer

Work Mode: Bangalore

Job Summary

We are looking for an experienced Senior Data Engineer with strong expertise in Ab Initio, GCP, Apache Spark, and Agentic AI. The ideal candidate will have hands-on experience designing and developing scalable data engineering solutions, building data pipelines, and working with modern cloud and AI technologies.

The candidate should be comfortable working across traditional enterprise data platforms and emerging Generative AI / Agentic AI solutions.

Key Responsibilities

• Design, develop, and maintain scalable and high-performance data pipelines using Ab Initio, Spark, and GCP services.
• Develop and optimize complex ETL/ELT workflows using Ab Initio.
• Build and maintain data processing solutions using Apache Spark / PySpark.
• Develop cloud-based data solutions on Google Cloud Platform (GCP).
• Work with GCP data services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, or equivalent services.
• Perform data integration, transformation, cleansing, and validation.
• Optimize data pipelines for performance, scalability, reliability, and cost.
• Collaborate with data architects,
• 
  

What you'll need

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical field.
• 5+ years of professional data engineering experience.
• Experience designing and building cloud-native data solutions.
• Strong expertise with Google Cloud Platform, including BigQuery. Experience developing transformation frameworks using dbt.
• Strong SQL and Python programming skills.
• Experience with PostgreSQL or other relational databases.
• Experience orchestrating workflows using Apache Airflow or Cloud Composer.
• Experience implementing Infrastructure as Code using Terraform. Experience building CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, or similar platforms.
• Experience developing scalable batch and streaming data pipelines. Strong problem-solving skills with the ability to balance scalability, reliability, and cloud cost optimization.

Preferred Qualifications

• Experience with Pub/Sub, Datastream, Dataflow, Cloud Storage, Cloud Functions, or Cloud Run.
• Experience building multi-tenant SaaS platforms.
• Experience implementing metadata-driven governance, lineage, and data quality frameworks.
• Experience supporting AI, machine learning, or customer-facing analytics platforms.
• Experience with Kubernetes and Docker.

Click "Apply Now" in https://gosuperedtech.com/career/ai-system-engineer to apply

Role Overview

We are looking for a Junior AI System Engineer who is eager to build a career in AI systems, automation, backend workflows, cloud infrastructure, and intelligent product operations.

This role offers an opportunity to work closely with experienced engineers, product teams, and AI specialists on real AI-powered systems that support learners, educators, schools, and internal business operations.

At GoSuper EdTech, our cloud infrastructure is built on Google Cloud Platform — GCP. You will get hands-on exposure to GCP-based systems, backend services, AI integrations, deployment workflows, monitoring, cloud storage, databases, and automation pipelines.

You will help design, integrate, test, monitor, and maintain AI-enabled systems using modern tools such as AI APIs, LLMs, automation workflows, backend services, databases, GCP services, cloud deployment tools, and monitoring systems.

This role is ideal if you are curious about AI, comfortable with technical problem-solving, and interested in building reliable systems that connect software, data, cloud infrastructure, automation, and intelligent workflows.

What You’ll Do

• Support the development and maintenance of AI-powered systems, tools, and workflows.
• Assist in integrating AI APIs, LLM platforms, automation tools, and backend services into GoSuper products.
• Work with OpenAI, Gemini, Claude, or similar AI platforms under the guidance of senior engineers.
• Support AI and backend workflows deployed on Google Cloud Platform — GCP.
• Assist with GCP-based services such as Cloud Run, Compute Engine, Cloud Functions, Cloud Storage, Firebase, Firestore, Cloud SQL, BigQuery, Pub/Sub, Cloud Logging, and Cloud Monitoring, based on project needs.
• Help build AI workflows for content generation, chatbot systems, smart recommendations, internal automation, and productivity tools.
• Support backend integrations using Node.js, Python, REST APIs, webhooks, and third-party services.
• Assist in designing and maintaining system workflows that connect databases, applications, AI models, cloud services, and business tools.
• Work with databases such as PostgreSQL, MongoDB, Firebase, Firestore, Supabase, or similar platforms.
• Help test AI outputs, validate workflows, debug issues, and improve system reliability.
• Monitor system performance, API usage, errors, logs, workflow failures, and cloud service health.
• Support deployment, configuration, and maintenance of AI-enabled product features on GCP.
• Collaborate with product managers, developers, designers, QA teams, and business teams to understand requirements and deliver working solutions.
• Participate in daily standups, sprint planning, technical discussions, and team meetings.
• Document AI workflows, system logic, API integrations, prompts, GCP configurations, deployment steps, and troubleshooting processes.
• Continuously learn and apply best practices in AI systems, backend engineering, automation, GCP cloud infrastructure, and production support.

What We’re Looking For

• 6 months to 1 year of experience in AI systems, backend development, software engineering, automation, DevOps support, cloud support, system integration, or relevant internship/project experience.
• Basic understanding of AI tools, LLMs, APIs, automation workflows, and software systems.
• Working knowledge of JavaScript, TypeScript, or Python.
• Basic backend development experience with Node.js, Express, NestJS, FastAPI, or similar frameworks.
• Basic understanding of Google Cloud Platform — GCP or willingness to learn GCP-based deployment and monitoring workflows.
• Understanding of REST APIs, webhooks, third-party integrations, and data flow between systems.
• Interest in AI APIs, prompt workflows, chatbot systems, automation tools, and intelligent product features.
• Basic understanding of databases such as PostgreSQL, MongoDB, Firebase, Firestore, Supabase, or Redis.
• Ability to debug technical issues across APIs, workflows, logs, backend services, and cloud deployments.
• Good analytical thinking and problem-solving ability.
• Ability to write clear documentation for workflows, integrations, cloud configurations, and technical processes.
• Eagerness to learn new tools, AI platforms, system design concepts, GCP services, and cloud technologies.
• Good communication skills to work with technical and non-technical teams.
• Ownership mindset and willingness to take responsibility for assigned tasks.
• Comfortable working in a fast-paced startup environment.

Nice to Have

• Familiarity with AI APIs such as OpenAI, Gemini, Claude, or similar platforms.
• Basic understanding of prompt engineering and LLM-based workflows.
• Exposure to LangChain, LlamaIndex, embeddings, vector databases, or retrieval-augmented generation.
• Basic experience with GCP services such as Cloud Run, Cloud Functions, Firebase, Firestore, Cloud Storage, Cloud SQL, BigQuery, Pub/Sub, Cloud Logging, or Cloud Monitoring.
• Exposure to Google AI tools, Vertex AI, Gemini API, or AI-related services on GCP.
• Experience with automation tools, workflow builders, webhooks, or integration platforms.
• Exposure to Docker, CI/CD pipelines, GitHub Actions, deployment workflows, or cloud-based release processes.
• Experience working with logs, monitoring tools, API testing tools, or debugging platforms.
• Familiarity with Postman, Git, GitHub, Notion, Zoho, Slack, or similar productivity tools.
• Experience building chatbots, AI assistants, internal tools, or automated workflows.
• Personal, academic, internship, or open-source projects related to AI, automation, backend systems, GCP, or cloud tools.
• Interest in SaaS, EdTech, AI-powered products, and startup environments.

What You’ll Gain

• Hands-on experience building AI-powered systems in a real startup environment.
• Practical exposure to AI APIs, LLM workflows, automation systems, backend services, and GCP cloud infrastructure.
• Mentorship from senior engineers and product leaders.
• Experience working across AI, backend engineering, databases, APIs, integrations, deployment, system monitoring, and cloud operations.
• Opportunity to contribute to real product features used by learners, educators, schools, and institutions.
• Exposure to SaaS product development, EdTech workflows, AI-driven business solutions, and GCP-based production systems.
• Learning culture that encourages experimentation, feedback, and continuous improvement.
• Opportunity to understand how AI systems are designed, deployed, monitored, scaled, and improved in production.
• Access to Cult Elite and Cult Play Pass, offering wellness and lifestyle benefits to keep you energized and inspired.

Compensation

• Competitive salary with performance-based bonuses.
• Equity ownership through ESOPs — own a piece of the company you help build.
• Flexible remote work options with occasional Bengaluru office meetups.
• Health and wellness perks, including Cult Elite membership and Cult Play Pass for employees.
• Learning and development support to help you grow in AI systems, backend engineering, automation, SaaS, and GCP cloud technologies.
• Team retreats, virtual hangouts, and a collaborative work culture.

We are looking for a AI System Engineer who is eager to build a career in AI systems, automation, backend workflows, cloud infrastructure, and intelligent product operations. Apply in https://gosuperedtech.com/career/ai-system-engineer

Job Title: Data Engineer – PySpark | Oracle | GCP

Experience: 5–7 Years

Location: Hyderabad

Notice Period: Immediate Joiners Preferred

Job Summary

We are seeking an experienced Data Engineer with strong expertise in PySpark, Oracle, and Google Cloud Platform (GCP) to design, develop, and optimize scalable data pipelines. The ideal candidate should have hands-on experience in ETL development, data integration, and cloud-based data engineering solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/data pipelines using PySpark.
• Extract, transform, and load data from Oracle databases into GCP environments.
• Build and optimize batch data processing workflows for high performance and reliability.
• Develop data engineering solutions using GCP services.
• Ensure data quality through validation, monitoring, and troubleshooting.
• Optimize SQL queries and ETL jobs for performance and scalability.

Required Skills

• 5–7 years of experience as a Data Engineer.
• Strong hands-on experience with PySpark.
• Solid experience with Oracle Database and advanced SQL.
• Hands-on experience with Google Cloud Platform (GCP).
• Strong understanding of ETL processes and data warehousing concepts.

Work Location: Hyderabad

Notice Period: Immediate Joiners Preferred