GCP Data Engineer at Egen Solutions · Hyderabad · 4 - 8 years · ₹12L - ₹25L / yr · Profitable · Posted 11 Aug 2025
PythonWe are looking for a skilled and motivated Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for designing, developing, and maintaining robust and scalable ETL (Extract, Transform, Load) data pipelines. The role involves working with various GCP services, implementing data ingestion and transformation logic, and ensuring data quality and consistency across systems.
Key Responsibilities:
- Design, develop, test, and maintain scalable ETL data pipelines using Python.
- Work extensively on Google Cloud Platform (GCP) services such as:
- Dataflow for real-time and batch data processing
- Cloud Functions for lightweight serverless compute
- BigQuery for data warehousing and analytics
- Cloud Composer for orchestration of data workflows (based on Apache Airflow)
- Google Cloud Storage (GCS) for managing data at scale
- IAM for access control and security
- Cloud Run for containerized applications
- Perform data ingestion from various sources and apply transformation and cleansing logic to ensure high-quality data delivery.
- Implement and enforce data quality checks, validation rules, and monitoring.
- Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver efficient data solutions.
- Manage version control using GitHub and participate in CI/CD pipeline deployments for data projects.
- Write complex SQL queries for data extraction and validation from relational databases such as SQL Server, Oracle, or PostgreSQL.
- Document pipeline designs, data flow diagrams, and operational support procedures.
Required Skills:
- 4–8 years of hands-on experience in Python for backend or data engineering projects.
- Strong understanding and working experience with GCP cloud services (especially Dataflow, BigQuery, Cloud Functions, Cloud Composer, etc.).
- Solid understanding of data pipeline architecture, data integration, and transformation techniques.
- Experience in working with version control systems like GitHub and knowledge of CI/CD practices.
- Strong experience in SQL with at least one enterprise database (SQL Server, Oracle, PostgreSQL, etc.).
Shubham Vishwakarma
Full Stack Developer - Averlon
About Egen Solutions
About
Company social profiles
Similar jobs (10)
Job Summary
We are seeking a highly skilled GCP Data Engineer with strong expertise in Google Cloud Platform (GCP), Python, ETL, and modern data engineering technologies. The ideal candidate should have hands-on experience designing and building scalable data pipelines using BigQuery, Dataflow, Pub/Sub, Airflow, and modern data lake technologies such as Apache Iceberg or Delta Lake.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines on Google Cloud Platform.
- Build and optimize data processing workflows using Python and Google Cloud Dataflow (Apache Beam).
- Develop and manage large-scale analytical data models in BigQuery.
- Implement event-driven data ingestion using Google Cloud Pub/Sub.
- Create, schedule, and monitor workflows using Apache Airflow and Autosys.
- Design and implement modern data lake architectures using Apache Iceberg or Delta Lake.
- Optimize query performance, storage, and compute costs in GCP.
- Ensure data quality, governance, security, and compliance across data platforms.
- Collaborate with Data Scientists, Analysts, and Application teams to deliver scalable data solutions.
- Troubleshoot production issues and continuously improve pipeline reliability and performance.
Mandatory Skills
- Strong hands-on experience with Google Cloud Platform (GCP).
- Proficiency in Python programming.
- Experience in designing and implementing ETL/ELT pipelines.
- Strong knowledge of BigQuery.
- Experience with Google Cloud Dataflow (Apache Beam).
- Experience with Google Cloud Pub/Sub.
- Hands-on experience with Apache Airflow.
- Experience in job scheduling using Autosys.
- Experience with modern table formats such as Apache Iceberg or Delta Lake.
- Strong SQL and data modeling skills.
Preferred Skills
- Experience with Cloud Storage, Dataproc, Cloud Composer, and Cloud Functions.
- Knowledge of CI/CD pipelines and DevOps practices.
- Experience with Docker and Kubernetes.
- Familiarity with Git and Agile/Scrum methodologies.
- Knowledge of data warehousing and dimensional modeling.
- Exposure to streaming and real-time data processing.
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- 4–8+ years of experience in Data Engineering with hands-on expertise in GCP technologies.
Required Experience
- Strong experience in developing enterprise-grade data pipelines using Python and GCP.
- Hands-on experience with BigQuery, Dataflow, Pub/Sub, and Airflow.
- Experience scheduling and monitoring batch workflows using Autosys.
- Experience implementing modern data lake architectures using Apache Iceberg or Delta Lake.
- Strong understanding of ETL best practices, performance tuning, and data optimization.
- Excellent analytical, troubleshooting, and problem-solving skills.
Mandatory Skills
- Google Cloud Platform (GCP)
- Python
- ETL
- BigQuery
- Autosys
- Apache Airflow
- Google Cloud Pub/Sub
- Google Cloud Dataflow (Apache Beam)
- Apache Iceberg / Delta Lake
- SQL & Data Modeling
We are hiring a GCP Data Engineer to build scalable data pipelines on Google Cloud.
Responsibilities
- Build batch and streaming pipelines with Dataflow and Pub/Sub
- Model and optimise data warehouses in BigQuery
- Run large-scale processing on Dataproc
- Orchestrate workflows with Cloud Composer
Requirements
- 2+ years of data engineering on Google Cloud
- Hands-on with BigQuery and Dataflow
- Strong data modelling and SQL optimisation skills
PythonJob Summary
Role Overview
We are looking for an experienced Data Engineer with strong expertise in Python, ETL, Advanced SQL, CI/CD, DevOps, and Data Analytics. The ideal candidate should have hands-on experience designing and developing scalable data pipelines, transforming large datasets, and supporting data-driven applications.
Experience with Google Cloud Platform (GCP) will be an added advantage.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
- Develop complex and optimized SQL queries, stored procedures, and data transformations.
- Build and maintain reliable data integration workflows across multiple data sources.
- Perform data cleansing, validation, transformation, and quality checks.
- Analyze data and provide insights to support business and technical requirements.
- Implement and maintain CI/CD pipelines for data engineering applications.
- Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
- Troubleshoot data pipeline failures, performance issues, and production incidents.
- Optimize data processing workflows for performance, scalability, and reliability.
- Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
- Follow best practices for version control, testing, documentation, and deployment.
- Contribute to cloud-based data engineering initiatives, preferably on GCP.
Required Skills
- 5–7 years of hands-on experience in Data Engineering.
- Strong programming skills in Python.
- Strong expertise in Advanced SQL and database concepts.
- Hands-on experience with ETL/ELT processes and data pipelines.
- Good understanding of Data Warehousing and Data Modeling concepts.
- Experience with CI/CD practices and tools.
- Strong understanding of DevOps principles, automation, and deployment processes.
- Strong data analytics and problem-solving skills.
- Experience working with large datasets and performance optimization.
- Good understanding of Git/version control and software development best practices.
Good to Have
- Hands-on experience with Google Cloud Platform (GCP).
- Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
- Experience with containerization/orchestration technologies such as Docker/Kubernetes.
- Experience with workflow orchestration tools such as Airflow.
- Knowledge of cloud-based data architecture and distributed data processing.
Preferred Candidate Profile
- Strong analytical and problem-solving abilities.
- Good communication and stakeholder management skills.
- Ability to work independently as well as in a collaborative team environment.
- Strong ownership of data pipelines and production systems.
- Candidates who can join at short notice are preferred.
Mandatory Skills
Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops
PythonRole Overview
We are looking for a GCP Data Engineer with 10+ years of experience to design, develop, and optimize scalable cloud-based data solutions. The ideal candidate will have strong hands-on expertise in GCP, BigQuery, and advanced SQL, with experience building data pipelines and working with large-scale datasets.
Key Responsibilities
- Design and develop scalable data pipelines and ETL/ELT processes on GCP.
- Build, optimize, and maintain data solutions using Google BigQuery.
- Develop complex SQL queries for data transformation, aggregation, and analysis.
- Design efficient data models and optimize pipelines for performance, scalability, and cost.
- Integrate data from multiple sources and ensure data quality, reliability, and availability.
- Troubleshoot pipeline and data issues and drive continuous improvement.
- Collaborate with data architects, analysts, application teams, and business stakeholders.
- Follow best practices for cloud security, data governance, testing, and documentation.
Required Skills
- 8+ years of Data Engineering experience
- Strong hands-on experience with GCP, Django, and MongoDB
- Extensive experience with BigQuery
- Advanced SQL skills
- Strong understanding of ETL/ELT and data pipeline development
- Data modeling and data warehousing experience
- Experience handling large-scale datasets and performance optimization
- Strong problem-solving and communication skills
Good to Have
- GCP services such as Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, or Cloud Functions
- Python or other data engineering languages
- Experience with data governance and security
- Agile development experience
PythonExperience: 6+ years overall Data Engineering experience.
Must-have — candidates should have hands-on experience in ALL of these:
- GCP (Google Cloud Platform) – strong hands-on experience
- Python – data engineering/ETL development
- SQL – advanced SQL, query optimization, data transformation
- BigQuery – strong hands-on experience with development, optimization and data warehousing
- Data Engineering / ETL – building and maintaining data pipelines
- GCP data services – preferably Cloud Storage, Dataflow, Pub/Sub, Composer/Airflow, etc.
- Data warehousing / dimensional modeling
PythonExperience: 5+ Years
Employment Type: Full-Time
Role Overview
We are looking for an experienced GCP Data Engineer with 5+ years of experience in data engineering and strong hands-on expertise in Google BigQuery, Google Cloud Storage (GCS), Airflow/Cloud Composer, Python, and Vertex AI. The candidate should be capable of designing, developing, and maintaining scalable data pipelines and cloud-based data solutions on Google Cloud Platform.
Key Skills – Mandatory
- BigQuery – Strong hands-on experience in data warehousing, SQL, optimization, and performance tuning.
- Google Cloud Storage (GCS) – Experience with data storage, file management, and integration with data pipelines.
- Airflow / Cloud Composer – Experience in developing, scheduling, monitoring, and managing data workflows.
- Python – Strong programming skills for data engineering, ETL/ELT development, automation, and pipeline implementation.
- Vertex AI – Experience working with ML/AI workflows, model integration, or data pipelines supporting AI/ML solutions.
Good to Have / Added Advantage
- Dataproc – Experience with distributed data processing and Spark-based workloads.
- Cloud Data Fusion – Experience in building and managing data integration pipelines.
- Cloud Run – Understanding of deploying and running containerized applications/services on GCP.
- Experience with ETL/ELT processes and data pipeline development.
- Knowledge of GCP data architecture and cloud-native services.
- Experience in data quality, validation, monitoring, and troubleshooting.
Responsibilities
- Design, develop, and maintain scalable GCP-based data pipelines.
- Build and optimize data solutions using BigQuery and Cloud Storage.
- Develop and manage workflows using Airflow / Cloud Composer.
- Write efficient and reusable Python code for data processing and automation.
- Support Vertex AI integrations and AI/ML data workflows.
- Monitor pipeline performance and troubleshoot data processing issues.
- Work with cross-functional teams to understand data requirements and deliver reliable solutions.
- Implement best practices for data security, quality, scalability, and performance.
You must have :
- 5+ years of overall experience in Data Engineering.
- Strong hands-on experience with BigQuery, GCS, Airflow/Cloud Composer, Python, and Vertex AI.
- Strong understanding of data engineering concepts, ETL/ELT, data pipelines, and cloud technologies.
- Dataproc, Data Fusion, and Cloud Run experience will be an added advantage.
The recruiter has not been active on this job recently. You may apply but please expect a delayed response.
Python
Amazon Web Services (AWS)Skills Referential (Required knowledge, skills and abilities)
Technical Skills:
Python
Pyspark
SQL
ETL Aws, Azure, gcp
Python🚨 Hiring: GCP Data Engineer
We are looking for experienced GCP Data Engineers to join our team!
🔹 Experience: 9+ Years
🔹 Relevant Experience: 4+ Years in GCP Data Engineering
🔹 Required Skills: GCP, Oracle PL/SQL, Python, PySpark
🔹 Location: Bangalore / Hyderabad
🔹 Notice Period: Immediate to 10 Days Preferred
Key Skills:
🔹 Strong hands-on experience in GCP Data Engineering
🔹 Good experience with PySpark & Python
🔹 Strong knowledge of Oracle PL/SQL
🔹 Experience in data processing, ETL, and data pipelines
🔹 Good understanding of cloud-based data engineering
📩 Interested candidates can share their updated CV via DM.
#Hiring #GCPDataEngineer #GCP #DataEngineering #PySpark #Python #OraclePLSQL #DataEngineer #BangaloreJobs #HyderabadJobs #ImmediateJoiner #TechJobs #ITJobs #HiringNow
Data Engineer Short Hiring Post
🚨 Hiring: Data Engineer
🔹 Experience: 5–9 Years
🔹 Location: Bangalore / Hyderabad
🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling
🔹 Process: L1 Virtual → L2 F2F Karat Test
🔹 F2F: Bangalore / Hyderabad Location
🔹 Positions: Immediate requirement
⚠️ Note: Candidates must be available for F2F Karat immediately after L1.
#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners
Job Description
We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines.
- Develop data processing solutions using Python.
- Write complex and optimized SQL queries, stored procedures, and data transformations.
- Build and maintain data ingestion and integration workflows.
- Implement data quality, validation, monitoring, and error-handling processes.
- Develop and maintain CI/CD pipelines for data engineering applications.
- Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
- Collaborate with data analysts, data scientists, software engineers, and business teams.
- Optimize data pipelines for performance, reliability, and scalability.
- Troubleshoot production data issues and ensure timely resolution.
- Follow best practices for version control, code quality, testing, and deployment.
Mandatory Skills
- Python
- ETL
- SQL
- CI/CD
- DevOps
- Git / Version Control
- Strong problem-solving and debugging skills
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon