Cassandra DBA at Cornertree · Bengaluru (Bangalore), Pune, Hyderabad, Gurugram, Noida · 5 - 10 years · ₹15L - ₹30L / yr · Raised funding · Posted 25 Nov 2024

Skills:
Experience with Cassandra, including installing configuring and monitoring a Cassandra cluster.
Experience with Cassandra data modeling and CQL scripting. Experience with DataStax Enterprise Graph
Experience with both Windows and Linux Operating Systems. Knowledge of Microsoft .NET Framework (C#, NETCore).
Ability to perform effectively in a team-oriented environment

About Cornertree
About
Connect with the team
Similar jobs (9)
1st virtual , 2nd round F2F -
Python pyspark, SQL, data engineer
5+yrs
Bang/hyderabad
immediate to 15days.
1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
9+yrs
Bangalore/Hyderabad
immediate to 15days.
role: data engineer
Python pyspark, SQL, data engineer
5+yrs
Bangalore/Hyderabad
immediate to 15days.
1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
Bang/hyderabad
immediate to 15days.
The recruiter has not been active on this job recently. You may apply but please expect a delayed response.
Responsibilities and JD
Job Description: We are looking for a Senior Developer with strong expertise in PySpark, Databricks, and Snowflake to build scalable data engineering solutions and enterprise data platforms.
Key Responsibilities:
- Design, develop, and maintain ETL/ELT pipelines using PySpark, Databricks, and Snowflake.
- Develop batch and real-time data processing solutions for structured and semi-structured data.
- Build and optimize Databricks notebooks, workflows, and Delta Lake solutions.
- Design and implement Snowflake databases, schemas, views, stored procedures, tasks, and streams.
- Develop scalable data models, data marts, and data warehouse solutions.
- Optimize PySpark jobs, Databricks workloads, and Snowflake queries for performance and cost efficiency.
- Implement data quality, validation, governance, and security controls.
- Collaborate with business stakeholders, architects, and cross-functional teams to deliver data solutions.
- Manage source control and CI/CD deployments using Git and Azure DevOps.
- Troubleshoot production issues, perform root cause analysis, and ensure pipeline reliability.
- Mentor junior team members and participate in code reviews and technical design discussions.
Required Skills: PySpark, Databricks, Snowflake, Python, SQL.
Experience: 5+ years of Data Engineering experience with strong hands-on expertise in PySpark, Databricks, and Snowflake.
Job Title : Senior Data Engineer – Databricks
Experience : 14 to 20 Years
Location : HSR Layout, Bangalore
Work Mode : Hybrid – 3 Days WFO
Shift : 11:30 AM – 07:30 PM IST
Positions : 2
Notice Period : Immediate Joiners Only
Interview : 1 Technical Round + 2 Client Rounds
Role Overview :
We are looking for a Senior Data Engineer to build and lead enterprise-scale data platforms for a Switzerland-based commodity client.
The role requires a strong hands-on Data Engineering professional with expertise in Databricks, PySpark, Python, SQL, and AWS, along with technical leadership and stakeholder management experience.
Must-Have Skills :
- 14 to 20 years of Data Engineering experience
- Databricks & Apache Spark / PySpark
- Python & SQL
- AWS Cloud
- Lakehouse Architecture
- ETL / ELT & Distributed Data Processing
- Batch & Streaming Pipelines
- Data Pipeline Optimization & Data Modeling
- CDC & Incremental Processing
- Git, CI/CD & Testing
- Data Quality, Monitoring & Observability
- Technical Leadership & Stakeholder Management
Key Responsibilities :
- Design and build scalable data pipelines using Databricks, PySpark, Python, SQL, and AWS.
- Own data products from design through production.
- Develop batch / streaming pipelines and reusable ETL / ELT frameworks.
- Optimize pipelines for performance, scalability, reliability, and cost.
- Design scalable data architectures and data models.
- Implement data quality, monitoring, lineage, and CI/CD practices.
- Lead technical discussions and mentor engineering teams.
- Collaborate with business stakeholders, architects, product owners, and engineering teams.
- Remain hands-on while providing technical leadership.
Ideal Candidate :
A 14 to 20 years experienced, hands-on Data Engineering leader with strong Databricks + PySpark + AWS expertise, excellent communication, stakeholder management, and experience delivering enterprise-scale data platforms.
🔴 Super Urgent : Only Bangalore-based immediate joiners.
About the Role
We are looking for a Senior Data Engineer with strong hands-on expertise in Databricks, Python, PySpark, and SQL to build scalable, high-performance data engineering solutions. You’ll architect and develop large scale, high-performance data pipelines capable of handling massive real-time and batch data volumes across multiple business systems. Databricks is the core enterprise data and processing platform for this role. You will also use Apache Airflow for workflow orchestration and dbt for ELT transformations, and will contribute to designing reliable, secure, and governed data platforms that enable analytics, reporting, and AI-driven use cases.
Key Responsibilities
- Design and implement large-scale data pipelines using Python/PySpark, Databricks, and Microsoft Fabric.
- Develop and optimize data processing workloads in Databricks using PySpark and Spark SQL, with a strong focus on scalability, reliability, performance, and maintainability.
- Develop and maintain dbt models including layered architecture, incremental models, snapshots, macros, testing, and documentation.
- Design, develop, and maintain Apache Airflow DAGs for orchestrating reliable, scalable, and observable data pipelines.
- Design and implement data quality, observability, and governance frameworks, including automated testing, monitoring, lineage, access control, and data privacy standards.
- Partner with analytics, product, and business stakeholders to turn requirements into trustworthy datasets, and raise the engineering bar through design discussions, code reviews, and mentoring junior engineers.
Required Skills
- Strong expertise in Python for developing scalable, modular, and production-ready data engineering applications.
- Strong expertise in PySpark, including DataFrame API, Spark SQL, Structured Streaming, partitioning strategies, joins, caching, handling data skew, and Spark performance optimization.
- Strong hands-on experience with Databricks for data ingestion, transformation, processing, and optimization, including Delta Lake, Unity Catalog, Databricks Workflows, notebooks, jobs, and Databricks-native data engineering capabilities.
- Strong experience in Databricks/Spark performance tuning, including query and job optimization, partitioning, file sizing, caching, join optimization, handling data skew, and efficient use of compute resources.
- Hands-on experience with Delta Lake, including transactional data processing, schema management, incremental data processing, and reliable batch and streaming data pipelines.
- Hands-on experience in developing dbt projects using layered architecture, incremental models, snapshots, macros/Jinja, testing, documentation, and deployment best practices.
- Expertise in advanced SQL and data modelling — dimensional modeling, slowly changing dimensions, schema evolution, and query optimization.
- Hands-on experience in developing and managing Apache Airflow DAGs, scheduling workflows, dependency management, retries, backfills, and operational monitoring.
- Hands-on experience with at least one major cloud platform (AWS, Azure or GCP).
- Strong problem-solving skills and the ability to work independently with business and analytics stakeholders.
Nice to Have
- Hands-on exposure to Microsoft Fabric for data integration and analytics.
- Experience using AI coding assistants (e.g. Claude Code, GitHub Copilot) as part of a development workflow.
- Familiarity with modern DevOps practices, including CI/CD pipelines, Infrastructure as Code (IaC), and containerization (Docker/Kubernetes).
- Domain expertise in financial services.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Python and PySpark.
- Work extensively with Databricks and Delta Lake, including Delta tables, MERGE operations, schema evolution, Time Travel, and data optimization.
- Develop, schedule, and monitor Apache Airflow DAGs for workflow orchestration and dependency management.
- Work with Kubernetes for containerized application deployment, orchestration, monitoring, and troubleshooting.
- Develop reusable and optimized Python code following software engineering best practices.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Implement data quality checks, error handling, logging, and monitoring mechanisms.
- Use Git for version control, branching, pull requests, code reviews, and collaborative development.
- Troubleshoot pipeline failures, investigate data issues, and implement reliable solutions.
- Collaborate with cross-functional teams to understand business requirements and deliver efficient data solutions.
Required Technical Skills
- Python: Strong programming skills, including data manipulation, exception handling, modular programming, and debugging.
- PySpark: Hands-on experience with Spark DataFrames, Spark SQL, joins, transformations, partitioning, and performance tuning.
- Databricks: Strong understanding of Databricks architecture, notebooks, workflows, compute management, and job execution.
- Delta Lake: Experience with Delta tables, ACID transactions, MERGE/UPSERT, schema enforcement, schema evolution, VACUUM, and Time Travel.
- Apache Airflow: Experience developing DAGs, managing task dependencies, configuring retries, scheduling workflows, and troubleshooting failures.
- Kubernetes: Practical knowledge of pods, deployments, services, ConfigMaps, Secrets, resource management, and troubleshooting containerized workloads.
- Git: Proficiency in branching strategies, pull requests, merging, and resolving conflicts.
- SQL: Strong SQL skills, including complex joins, CTEs, window functions, and query optimization.
Good to Have
- Experience with Docker and containerization best practices.
- Experience with Databricks Unity Catalog and data governance.
- Knowledge of CI/CD pipelines and automated deployments.
- Experience deploying and managing Airflow workloads on Kubernetes.
- Familiarity with Kubernetes monitoring, autoscaling, and resource optimization.
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Understanding of data lakehouse architecture and data modeling.
- Experience with batch and incremental data processing.
- Familiarity with data quality frameworks and automated testing.
Preferred Candidate Profile
- Strong analytical and problem-solving skills.
- Ability to independently design, develop, deploy, and troubleshoot data pipelines.
- Good understanding of distributed data processing and container orchestration.
- Experience working in an Agile development environment.
- Strong communication and collaboration skills.
GCP Data engineer
Exp 9+yrs
Bangalore/Hyderabad
Lead exp+ Python + pyspark + sql or plsql
immediate to 15days joiners
Call or WhatsApp: 6 3 8 5 0 7 6 1 4 2









