GCP Developer

Role Overview

We are looking for a GCP Data Engineer with 10+ years of experience to design, develop, and optimize scalable cloud-based data solutions. The ideal candidate will have strong hands-on expertise in GCP, BigQuery, and advanced SQL, with experience building data pipelines and working with large-scale datasets.

Key Responsibilities

• Design and develop scalable data pipelines and ETL/ELT processes on GCP.
• Build, optimize, and maintain data solutions using Google BigQuery.
• Develop complex SQL queries for data transformation, aggregation, and analysis.
• Design efficient data models and optimize pipelines for performance, scalability, and cost.
• Integrate data from multiple sources and ensure data quality, reliability, and availability.
• Troubleshoot pipeline and data issues and drive continuous improvement.
• Collaborate with data architects, analysts, application teams, and business stakeholders.
• Follow best practices for cloud security, data governance, testing, and documentation.

Required Skills

• 8+ years of Data Engineering experience
• Strong hands-on experience with GCP, Django, and MongoDB
• Extensive experience with BigQuery
• Advanced SQL skills
• Strong understanding of ETL/ELT and data pipeline development
• Data modeling and data warehousing experience
• Experience handling large-scale datasets and performance optimization
• Strong problem-solving and communication skills

Good to Have

• GCP services such as Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, or Cloud Functions
• Python or other data engineering languages
• Experience with data governance and security
• Agile development experience

Experience: 5+ Years

Employment Type: Full-Time

Role Overview

We are looking for an experienced GCP Data Engineer with 5+ years of experience in data engineering and strong hands-on expertise in Google BigQuery, Google Cloud Storage (GCS), Airflow/Cloud Composer, Python, and Vertex AI. The candidate should be capable of designing, developing, and maintaining scalable data pipelines and cloud-based data solutions on Google Cloud Platform.

Key Skills – Mandatory

• BigQuery – Strong hands-on experience in data warehousing, SQL, optimization, and performance tuning.
• Google Cloud Storage (GCS) – Experience with data storage, file management, and integration with data pipelines.
• Airflow / Cloud Composer – Experience in developing, scheduling, monitoring, and managing data workflows.
• Python – Strong programming skills for data engineering, ETL/ELT development, automation, and pipeline implementation.
• Vertex AI – Experience working with ML/AI workflows, model integration, or data pipelines supporting AI/ML solutions.

Good to Have / Added Advantage

• Dataproc – Experience with distributed data processing and Spark-based workloads.
• Cloud Data Fusion – Experience in building and managing data integration pipelines.
• Cloud Run – Understanding of deploying and running containerized applications/services on GCP.
• Experience with ETL/ELT processes and data pipeline development.
• Knowledge of GCP data architecture and cloud-native services.
• Experience in data quality, validation, monitoring, and troubleshooting.

Responsibilities

• Design, develop, and maintain scalable GCP-based data pipelines.
• Build and optimize data solutions using BigQuery and Cloud Storage.
• Develop and manage workflows using Airflow / Cloud Composer.
• Write efficient and reusable Python code for data processing and automation.
• Support Vertex AI integrations and AI/ML data workflows.
• Monitor pipeline performance and troubleshoot data processing issues.
• Work with cross-functional teams to understand data requirements and deliver reliable solutions.
• Implement best practices for data security, quality, scalability, and performance.

You must have :

• 5+ years of overall experience in Data Engineering.
• Strong hands-on experience with BigQuery, GCS, Airflow/Cloud Composer, Python, and Vertex AI.
• Strong understanding of data engineering concepts, ETL/ELT, data pipelines, and cloud technologies.
• Dataproc, Data Fusion, and Cloud Run experience will be an added advantage.

Experience: 6+ years overall Data Engineering experience.

Must-have — candidates should have hands-on experience in ALL of these:

1. GCP (Google Cloud Platform) – strong hands-on experience
2. Python – data engineering/ETL development
3. SQL – advanced SQL, query optimization, data transformation
4. BigQuery – strong hands-on experience with development, optimization and data warehousing
5. Data Engineering / ETL – building and maintaining data pipelines
6. GCP data services – preferably Cloud Storage, Dataflow, Pub/Sub, Composer/Airflow, etc.
7. Data warehousing / dimensional modeling

GCP Data Engineering Lead

Experience: 9+ Years

Lead Experience: 2+ Years

Location: Bangalore / Hyderabad

Key Skills:

• Strong experience in GCP Data Engineering and BigQuery
• Hands-on experience with Oracle Exadata / PL-SQL
• Strong knowledge of PySpark / Scala and Python
• Experience with GoldenGate, Kafka and CDC
• Hands-on experience with Apache Airflow
• Good experience in CI/CD and DevOps practices
• Experience with Terraform / Infrastructure as Code
• Exposure to AI/LLM technologies and GenAI solutions
• Strong understanding of data architecture, ETL/ELT and data pipelines

Roles & Responsibilities:

• Lead the design and development of scalable GCP data engineering solutions.
• Design and implement batch and real-time data pipelines using BigQuery, PySpark, Kafka/CDC and Airflow.
• Work with Oracle Exadata/PL-SQL and GoldenGate for data integration and migration.
• Implement CI/CD pipelines and infrastructure automation using Terraform.
• Explore and integrate AI/LLM capabilities into data engineering solutions.
• Lead technical discussions, code reviews, solution design and mentor team members.
• Collaborate with business and technical teams to deliver high-quality data solutions.

• Design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data ingestion and transformation.
• Develop and optimize data lake and data warehouse architectures (e.g., Snowflake, BigQuery, Redshift).
• Work with cloud platforms GCP, Azure to manage data infrastructure.
• GCP as mandatory skills
• Collaborate with analytics and product teams to understand data needs and deliver solutions.
• Ensure data quality, reliability, security, and compliance across all data systems.
• Mentor junior data engineers and contribute to best practices and code reviews.
• Monitor and troubleshoot data pipeline performance and resolve data-related issues.
• Automate data validation, monitoring, and alerting processes.
• 8+ years of experience in data engineering or software engineering with a data focus.
• Proficient in SQL and at least one programming language (e.g., Python, Scala, Java).
• Experience with modern data warehousing tools (e.g., Snowflake, Redshift, BigQuery).
• Strong understanding of data modeling, data lakes, and ETL/ELT design.
• Hands-on experience with orchestration tools like Airflow, dbt, or similar.
• Solid experience with cloud data platforms (AWS/GCP/Azure).
• Familiarity with CI/CD pipelines, containerization (Docker/Kubernetes), and version control (Git).
• Experience working in a DevOps or DataOps environment.
• Knowledge of data governance, lineage, and cataloging tools (e.g., Collibra, Alation).
• Familiarity with streaming technologies (Kafka, Spark Streaming, Flink).
• Experience supporting machine learning workflows and data science initiatives.

Job Summary

We are seeking a highly skilled GCP Data Engineer with strong expertise in Google Cloud Platform (GCP), Python, ETL, and modern data engineering technologies. The ideal candidate should have hands-on experience designing and building scalable data pipelines using BigQuery, Dataflow, Pub/Sub, Airflow, and modern data lake technologies such as Apache Iceberg or Delta Lake.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines on Google Cloud Platform.
• Build and optimize data processing workflows using Python and Google Cloud Dataflow (Apache Beam).
• Develop and manage large-scale analytical data models in BigQuery.
• Implement event-driven data ingestion using Google Cloud Pub/Sub.
• Create, schedule, and monitor workflows using Apache Airflow and Autosys.
• Design and implement modern data lake architectures using Apache Iceberg or Delta Lake.
• Optimize query performance, storage, and compute costs in GCP.
• Ensure data quality, governance, security, and compliance across data platforms.
• Collaborate with Data Scientists, Analysts, and Application teams to deliver scalable data solutions.
• Troubleshoot production issues and continuously improve pipeline reliability and performance.

Mandatory Skills

• Strong hands-on experience with Google Cloud Platform (GCP).
• Proficiency in Python programming.
• Experience in designing and implementing ETL/ELT pipelines.
• Strong knowledge of BigQuery.
• Experience with Google Cloud Dataflow (Apache Beam).
• Experience with Google Cloud Pub/Sub.
• Hands-on experience with Apache Airflow.
• Experience in job scheduling using Autosys.
• Experience with modern table formats such as Apache Iceberg or Delta Lake.
• Strong SQL and data modeling skills.

Preferred Skills

• Experience with Cloud Storage, Dataproc, Cloud Composer, and Cloud Functions.
• Knowledge of CI/CD pipelines and DevOps practices.
• Experience with Docker and Kubernetes.
• Familiarity with Git and Agile/Scrum methodologies.
• Knowledge of data warehousing and dimensional modeling.
• Exposure to streaming and real-time data processing.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 4–8+ years of experience in Data Engineering with hands-on expertise in GCP technologies.

Required Experience

• Strong experience in developing enterprise-grade data pipelines using Python and GCP.
• Hands-on experience with BigQuery, Dataflow, Pub/Sub, and Airflow.
• Experience scheduling and monitoring batch workflows using Autosys.
• Experience implementing modern data lake architectures using Apache Iceberg or Delta Lake.
• Strong understanding of ETL best practices, performance tuning, and data optimization.
• Excellent analytical, troubleshooting, and problem-solving skills.

Mandatory Skills

• Google Cloud Platform (GCP)
• Python
• ETL
• BigQuery
• Autosys
• Apache Airflow
• Google Cloud Pub/Sub
• Google Cloud Dataflow (Apache Beam)
• Apache Iceberg / Delta Lake
• SQL & Data Modeling

What you'll need

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical field.
• 5+ years of professional data engineering experience.
• Experience designing and building cloud-native data solutions.
• Strong expertise with Google Cloud Platform, including BigQuery. Experience developing transformation frameworks using dbt.
• Strong SQL and Python programming skills.
• Experience with PostgreSQL or other relational databases.
• Experience orchestrating workflows using Apache Airflow or Cloud Composer.
• Experience implementing Infrastructure as Code using Terraform. Experience building CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, or similar platforms.
• Experience developing scalable batch and streaming data pipelines. Strong problem-solving skills with the ability to balance scalability, reliability, and cloud cost optimization.

Preferred Qualifications

• Experience with Pub/Sub, Datastream, Dataflow, Cloud Storage, Cloud Functions, or Cloud Run.
• Experience building multi-tenant SaaS platforms.
• Experience implementing metadata-driven governance, lineage, and data quality frameworks.
• Experience supporting AI, machine learning, or customer-facing analytics platforms.
• Experience with Kubernetes and Docker.

🚨 Hiring: GCP Data Engineer

We are looking for experienced GCP Data Engineers to join our team!

🔹 Experience: 9+ Years

🔹 Relevant Experience: 4+ Years in GCP Data Engineering

🔹 Required Skills: GCP, Oracle PL/SQL, Python, PySpark

🔹 Location: Bangalore / Hyderabad

🔹 Notice Period: Immediate to 10 Days Preferred

Key Skills:

🔹 Strong hands-on experience in GCP Data Engineering

🔹 Good experience with PySpark & Python

🔹 Strong knowledge of Oracle PL/SQL

🔹 Experience in data processing, ETL, and data pipelines

🔹 Good understanding of cloud-based data engineering

📩 Interested candidates can share their updated CV via DM.

#Hiring #GCPDataEngineer #GCP #DataEngineering #PySpark #Python #OraclePLSQL #DataEngineer #BangaloreJobs #HyderabadJobs #ImmediateJoiner #TechJobs #ITJobs #HiringNow

Description

We are looking for Senior Data Engineers to join our AdTech team and build scalable, high-performance data platforms that power advertising insights and analytics. The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Spark and Scala.

You will work on designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
• Build and optimize distributed data applications using Spark and Scala.
• Develop reliable, high-performance data pipelines for batch and streaming workloads.
• Work with large datasets to ensure data quality, consistency, and performance.
• Collaborate with engineering, product, and analytics teams to deliver robust data solutions.
• Optimize data workflows for scalability, reliability, and cost efficiency.
• Deploy and manage data workloads in cloud and containerized environments.
• Troubleshoot production issues and continuously improve platform performance.

Requirements

Candidates who demonstrate:

• 5+ years of experience in Data Engineering or Big Data Engineering.
• Strong hands-on experience with Apache Spark and Scala.
• Experience building and maintaining ETL pipelines.
• Familiarity with Google Cloud Storage (GCS).
• Experience with Kubernetes (K8s).
• Strong SQL skills and understanding of distributed data processing.
• Excellent debugging, problem-solving, and performance optimization skills.
• Strong communication and collaboration skills.

Good to Have

• Experience with AWS and cloud-native data services.
• Familiarity with streaming technologies such as Kafka.
• Experience working on large-scale data platforms or AdTech systems.
• Exposure to orchestration tools such as Airflow.

Benefits

• Best-in-class salary: We hire strong talent and compensate accordingly.
• Proximity Talks: Meet and learn from designers, engineers, product leaders, and AI practitioners.
• Continuous learning: Work with a world-class team and stay close to the latest in AI, engineering, and product development.
• High-impact work: Build AI-first systems and products used at scale by global clients.

About Us

Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.

Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.

Location: Hyderabad / Chennai

Experience: 5+ years

Employment type: Full-time, permanent

Work Hours: General Shift

website: www.amazech.com

Qualifications: 

• B.E./B.Tech/M.E./M.Tech in Computer Science, Information Technology, Data Science, or related disciplines.
• Strong academic background with relevant industry experience in Data Engineering and Data Warehousing.

Key Responsibilities:

·       Design, develop, and maintain scalable data warehouse solutions using Snowflake.

·       Write, optimize, troubleshoot, and enhance Snowflake SQL queries with a focus on performance and scalability.

·       Develop and support ETL processes using Talend to ensure reliable and efficient data movement.

·       Collaborate with business, analytics, and application teams to enable reporting, dashboards, metrics, and data exploration capabilities.

·       Perform data analysis and resolve issues across data ingestion, transformation, and reporting pipelines.

·       Debug and troubleshoot Python-based data processing scripts and automation workflows.

·       Implement best practices for data quality, testing, deployment, and code reviews.

·       Work across UI, API, and Data Warehouse layers to support end-to-end data integration and business requirements.

·       Monitor, optimize, and maintain data warehouse performance and operational stability.

·       Create and maintain technical documentation, data models, and process workflows.

Required Skills and Experience: 

·       Strong hands-on expertise in Snowflake Data Warehouse.

·       Advanced SQL skills with experience handling large-scale datasets.

·       Strong understanding of Data Warehousing concepts, dimensional modelling, and data architecture.

·       Hands-on experience with Analytical SQL functions, query tuning, and performance optimization.

·       Experience developing and maintaining ETL solutions using Talend.

·       Proficiency in Python for scripting, debugging, automation, and data processing.

·       Experience integrating UI, API, and Data Warehouse workflows.

·       Strong problem-solving and analytical skills.

·       Experience with testing, code reviews, and deployment best practices.

·       Excellent communication and stakeholder management skills.