Senior Data Engineer

We are hiring for a Data Engineer & Java scala for a full-time opportunity in Bangalore.

Experience: 4–10 Years

Must-Have Skills:

Java / Scala / Python, Data Engineering, Databricks / Snowflake, and Cloud (AWS / Azure / GCP).

We are looking for strong software engineers with hands-on experience in data platform development, scalable backend programming, data integration, and cloud technologies.

Experience in Banking, Capital Markets, Market Risk, or Regulatory Reporting will be an added advantage.

Job Description – Data Engineer

Role: Data Engineer

Experience: 9+ Years

Location: Bangalore / Hyderabad

Notice Period: Immediate to 15 Days

Interview Process: 1st Round – Virtual | 2nd Round – F2F

Mandatory Skills

• Python
• PySpark
• SQL
• Azure Data Engineering

Job Description

We are looking for an experienced Azure Data Engineer with 9+ years of experience and strong hands-on expertise in Python, PySpark, SQL, and Azure Data Engineering.

Key Responsibilities

• Develop and maintain scalable data engineering solutions using Azure.
• Build and optimize data processing pipelines using PySpark and Python.
• Write complex SQL queries for data extraction and transformation.
• Work with Azure data services and cloud-based data platforms.
• Perform data processing, transformation, and integration.
• Troubleshoot data pipeline and production issues.
• Collaborate with technical and business teams to deliver data solutions.

Preferred: Immediate to 15 Days joiners.

Job Title : Senior Data Engineer – GCP & Apache Spark

Job Details :

• Number of Positions : 3
• Employment Type : Contract-to-Hire (C2H)
• Experience Required : 5 to 8 Years
• Location : Manyata Tech Park, Bengaluru
• Work Mode : Hybrid – 3 Days in Office per Week
• Preferred Notice Period : Immediate to 15–20 Days
• Project Duration : Initially 6–8 months, with a strong possibility of extension for long-term projects
• Relocation : Candidates from outside Bengaluru can apply if they are willing to relocate.

Role Overview :

We are looking for experienced Senior Data Engineers with strong expertise in Google Cloud Platform (GCP), Apache Spark, BigQuery, and SQL to design, develop, and maintain scalable data pipelines and distributed data processing solutions.

The ideal candidate should have hands-on experience with cloud-based data engineering, end-to-end pipeline architecture, system design, performance optimization, debugging, and CI/CD practices.

Mandatory skills :

Apache Spark (PySpark / Scala), SQL, GCP, BigQuery, GCP Dataproc, Data Pipelines, System Design, CI/CD, Debugging & Problem-Solving.

Key Responsibilities :

• Design, develop, and maintain scalable data pipelines using Apache Spark, PySpark, and/or Scala.
• Build and optimize batch and distributed data processing workflows on GCP.
• Work extensively with BigQuery for data transformation, validation, analytics, and query optimization.
• Develop and maintain data workflows using GCP services, including Dataproc, BigQuery, and Cloud Storage.
• Design end-to-end data pipeline architectures with a focus on scalability, reliability, and performance.
• Write and optimize complex SQL queries for large-scale datasets.
• Troubleshoot and resolve data processing issues, pipeline failures, and performance bottlenecks.
• Implement CI/CD practices for data engineering workflows, testing, and deployments.
• Monitor pipeline performance and identify opportunities for continuous improvement.
• Collaborate with cross-functional teams to understand requirements and deliver reliable data solutions.
• Participate in code reviews, testing, production deployments, and production support.

Mandatory Skills :

• Strong hands-on experience with Apache Spark, including PySpark and/or Scala.
• Strong SQL skills and experience working with large-scale datasets.
• Hands-on experience with Google Cloud Platform (GCP).
• Practical experience with BigQuery and data warehousing concepts.
• Working knowledge of GCP Dataproc.
• Strong understanding of data pipelines, end-to-end workflows, and system design.
• Practical experience with CI/CD and deployment processes.
• Excellent debugging, troubleshooting, and problem-solving skills.

Preferred Skills :

• Experience with Apache Airflow, Google Cloud Composer, or similar orchestration tools.
• Knowledge of Kafka or other streaming and messaging technologies.
• Proficiency in Python for data engineering and automation.
• Experience with Git-based development workflows.
• Knowledge of Linux/Unix environments and shell scripting.
• Familiarity with monitoring, logging, data quality, and governance practices.

Required Qualifications :

• 5 to 8 years of experience in Data Engineering, Big Data Engineering, or a related field.
• Proven experience developing and maintaining scalable data pipelines.
• Strong understanding of distributed data processing and cloud-based data architecture.
• Ability to troubleshoot complex technical issues and optimize pipeline performance.
• Strong communication, collaboration, and ownership skills.

Interview Process :

Round 1 : AI-Based Interview

• Candidates must complete the AI interview as soon as possible after receiving the invitation.

Round 2 : Virtual Interview Drive

• Interviews are planned for the upcoming drive; exact dates will be communicated separately.

Round 3 : Online Client Interview

• Shortlisted candidates will attend the online interview.
• Candidates must be available throughout the day on the scheduled interview date.

Job Summary

Role Overview

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, Advanced SQL, CI/CD, DevOps, and Data Analytics. The ideal candidate should have hands-on experience designing and developing scalable data pipelines, transforming large datasets, and supporting data-driven applications.

Experience with Google Cloud Platform (GCP) will be an added advantage.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
• Develop complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain reliable data integration workflows across multiple data sources.
• Perform data cleansing, validation, transformation, and quality checks.
• Analyze data and provide insights to support business and technical requirements.
• Implement and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
• Troubleshoot data pipeline failures, performance issues, and production incidents.
• Optimize data processing workflows for performance, scalability, and reliability.
• Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
• Follow best practices for version control, testing, documentation, and deployment.
• Contribute to cloud-based data engineering initiatives, preferably on GCP.

Required Skills

• 5–7 years of hands-on experience in Data Engineering.
• Strong programming skills in Python.
• Strong expertise in Advanced SQL and database concepts.
• Hands-on experience with ETL/ELT processes and data pipelines.
• Good understanding of Data Warehousing and Data Modeling concepts.
• Experience with CI/CD practices and tools.
• Strong understanding of DevOps principles, automation, and deployment processes.
• Strong data analytics and problem-solving skills.
• Experience working with large datasets and performance optimization.
• Good understanding of Git/version control and software development best practices.

Good to Have

• Hands-on experience with Google Cloud Platform (GCP).
• Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
• Experience with containerization/orchestration technologies such as Docker/Kubernetes.
• Experience with workflow orchestration tools such as Airflow.
• Knowledge of cloud-based data architecture and distributed data processing.

Preferred Candidate Profile

• Strong analytical and problem-solving abilities.
• Good communication and stakeholder management skills.
• Ability to work independently as well as in a collaborative team environment.
• Strong ownership of data pipelines and production systems.
• Candidates who can join at short notice are preferred.

Mandatory Skills

 Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

9+yrs

Bangalore/Hyderabad

immediate to 15days.

Description

We are looking for Senior Data Engineers to join our Data Platform team and build scalable, high-performance data platforms that power data processing, analytics, and downstream applications.

The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Apache Spark and Python Scala.

You will be responsible for designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL and data processing pipelines for large-scale datasets.
• Build and optimize distributed data applications using Apache Spark and Python Scala.
• Develop reliable, high-performance data pipelines for batch and streaming workloads.
• Design and manage data workflows using Apache Airflow.
• Build and operate data workloads on AWS, with strong usage of Amazon S3 for large-scale data storage.
• Work with large datasets to ensure data quality, consistency, reliability, and performance.
• Collaborate with engineering, product, analytics, and other platform teams to deliver robust data solutions.
• Optimize data workflows for scalability, reliability, performance, and cost efficiency.
• Troubleshoot production issues, identify bottlenecks, and continuously improve platform performance.

Requirements

Candidates who demonstrate:

• 5+ years of experience in Data Engineering, Big Data Engineering, or a similar role.
• Strong hands-on experience with Apache Spark and Scala.
• Experience designing, building, and maintaining large-scale ETL pipelines.
• Strong hands-on experience with AWS, particularly Amazon S3.
• Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
• Strong SQL skills and a solid understanding of distributed data processing concepts.
• Experience working with batch and/or streaming data pipelines.
• Excellent debugging, problem-solving, and performance optimization skills.
• Strong communication and collaboration skills.

Good to Have

• Experience with Databricks and the broader Databricks data platform.
• Familiarity with streaming technologies such as Apache Kafka.
• Experience working on large-scale data platforms handling high-volume data workloads.
• Exposure to additional AWS data services and cloud-native data architectures.

Description

We are looking for Senior Data Engineers to join our AdTech team and build scalable, high-performance data platforms that power advertising insights and analytics. The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Spark and Scala.

You will work on designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
• Build and optimize distributed data applications using Spark and Scala.
• Develop reliable, high-performance data pipelines for batch and streaming workloads.
• Work with large datasets to ensure data quality, consistency, and performance.
• Collaborate with engineering, product, and analytics teams to deliver robust data solutions.
• Optimize data workflows for scalability, reliability, and cost efficiency.
• Deploy and manage data workloads in cloud and containerized environments.
• Troubleshoot production issues and continuously improve platform performance.

Requirements

Candidates who demonstrate:

• 5+ years of experience in Data Engineering or Big Data Engineering.
• Strong hands-on experience with Apache Spark and Scala.
• Experience building and maintaining ETL pipelines.
• Familiarity with Google Cloud Storage (GCS).
• Experience with Kubernetes (K8s).
• Strong SQL skills and understanding of distributed data processing.
• Excellent debugging, problem-solving, and performance optimization skills.
• Strong communication and collaboration skills.

Good to Have

• Experience with AWS and cloud-native data services.
• Familiarity with streaming technologies such as Kafka.
• Experience working on large-scale data platforms or AdTech systems.
• Exposure to orchestration tools such as Airflow.

Benefits

• Best-in-class salary: We hire strong talent and compensate accordingly.
• Proximity Talks: Meet and learn from designers, engineers, product leaders, and AI practitioners.
• Continuous learning: Work with a world-class team and stay close to the latest in AI, engineering, and product development.
• High-impact work: Build AI-first systems and products used at scale by global clients.

About Us

Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.

Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.

Job Summary

We are looking for a skilled and experienced Data Engineer to join our growing data team. The ideal candidate will have strong expertise in Python, PySpark, Data Modeling, and Power BI, with hands-on experience in designing, developing, and optimizing scalable data solutions. The role requires working closely with business stakeholders, data architects, and analytics teams to build robust data pipelines and semantic models that enable data-driven decision-making.

Technical Skills

• Strong hands-on experience in Python and PySpark development.
• Expertise in building and optimizing Data Engineering solutions and ETL pipelines.
• Strong understanding of Data Modeling concepts (Star Schema, Snowflake Schema, Dimensional Modeling).
• Experience with Power BI Data Modeling and Semantic Layer development.
• Proficiency in DAX (Data Analysis Expressions).
• Experience designing and managing Semantic Models in Power BI.
• Strong SQL skills and experience working with large datasets.
• Knowledge of data warehousing concepts and best practices.

Preferred Skills

• Experience with cloud platforms such as Azure, AWS, or GCP.
• Exposure to modern data platforms like Databricks.
• Understanding of data governance and data quality frameworks.

Job Description – Senior GCP Data Engineer

Position: Senior GCP Data Engineer

Experience: 8+ Years

Budget: Up to 30 LPA

Location: Bangalore / Hyderabad

Relocation: Candidates from Chennai / Pune willing to relocate are also welcome.

Notice Period: Immediate to 15 Days

Interview Process:

• Round 1: Virtual Interview
• Round 2 / DM Round: Face-to-Face Interview

Required Skills

• Strong hands-on experience in GCP Data Engineering.
• Good programming experience in Python.
• Strong experience with PySpark / Apache Spark.
• Proficiency in SQL and database concepts.
• Experience in developing and maintaining data pipelines.
• Good understanding of data processing, transformation, and integration.
• Experience in handling large-scale data processing.
• Strong problem-solving and debugging skills.
• Good understanding of cloud-based data engineering concepts.
• Experience in optimizing data pipelines for performance and scalability.

Preferred Skills

• Previous experience in a Lead Data Engineer role.
• Experience in technical leadership, mentoring, and guiding team members.
• Ability to collaborate with cross-functional teams and deliver data engineering solutions.

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines using GCP.
• Develop data processing and transformation workflows using Python and PySpark.
• Write and optimize SQL queries for efficient data processing.
• Build and maintain reliable data engineering solutions.
• Troubleshoot data pipeline issues and optimize performance.
• Ensure data quality, reliability, and scalability.
• Collaborate with stakeholders and team members to deliver project requirements.
• Lead and guide team members where applicable.
• Important: Candidates must have strong Senior Data Engineer experience. Lead experience is preferred but not mandatory.

Responsibilities and JD

Job Description: We are looking for a Senior Developer with strong expertise in PySpark, Databricks, and Snowflake to build scalable data engineering solutions and enterprise data platforms.

Key Responsibilities:

1. Design, develop, and maintain ETL/ELT pipelines using PySpark, Databricks, and Snowflake.
2. Develop batch and real-time data processing solutions for structured and semi-structured data.
3. Build and optimize Databricks notebooks, workflows, and Delta Lake solutions.
4. Design and implement Snowflake databases, schemas, views, stored procedures, tasks, and streams.
5. Develop scalable data models, data marts, and data warehouse solutions.
6. Optimize PySpark jobs, Databricks workloads, and Snowflake queries for performance and cost efficiency.
7. Implement data quality, validation, governance, and security controls.
8. Collaborate with business stakeholders, architects, and cross-functional teams to deliver data solutions.
9. Manage source control and CI/CD deployments using Git and Azure DevOps.
10. Troubleshoot production issues, perform root cause analysis, and ensure pipeline reliability.
11. Mentor junior team members and participate in code reviews and technical design discussions.

Required Skills: PySpark, Databricks, Snowflake, Python, SQL.

Experience: 5+ years of Data Engineering experience with strong hands-on expertise in PySpark, Databricks, and Snowflake.