python backend developer at Wissen Technology · Pune · 4 - 8 years · ₹1L - ₹20L / yr · Profitable · Posted 16 Jul 2026

Company Name – Wissen Technology
Group of companies in India – Wissen Technology & Wissen Infotech
Work Location – Pune
While you may already know about Wissen and the company history, here is a quick rundown for you.
About Wissen Technology:
· The Wissen Group was founded in the year 2000. Wissen Technology, a part of Wissen Group, was established in the year 2015.
· Wissen Technology is a specialized technology company that delivers high-end consulting for organizations in the Banking & Finance, Telecom, and Healthcare domains. We help clients build world class products.
· Our workforce has highly skilled professionals, with leadership and senior management executives who have graduated from Ivy League Universities like Wharton, MIT, IITs, IIMs, and NITs and with rich work experience in some of the biggest companies in the world.
· Wissen Technology has grown its revenues by 400% in these five years without any external funding or investments.
· Globally present with offices US, India, UK, Australia, Mexico, and Canada.
· We offer an array of services including Application Development, Artificial Intelligence & Machine Learning, Big Data & Analytics, Visualization & Business Intelligence, Robotic Process Automation, Cloud, Mobility, Agile & DevOps, Quality Assurance & Test Automation.
· Wissen Technology has been certified as a Great Place to Work®.
· Wissen Technology has been voted as the Top 20 AI/ML vendor by CIO Insider in 2020.
· Over the years, Wissen Group has successfully delivered $650 million worth of projects for more than 20 of the Fortune 500 companies.
· We have served client across sectors like Banking, Telecom, Healthcare, Manufacturing, and Energy. They include likes of Morgan Stanley, Goldman Sachs, MSCI, StateStreet, Flipkart, Swiggy, Trafigura, GE to name a few.
Job Description:
Experience Required: 3–5years
Job Title: Application Development Engineer (Python – Backtesting & Index Platforms)
Role Overview
We are seeking a strong Python Application Engineer to help build a next-generation Index Backtesting and Rebalance Platform.
In this role, you will design and develop deterministic, scalable calculation engines that convert financial methodologies into production-grade software.
You will work on portfolio construction, rebalancing logic, and historical simulations while consuming reference and market data from Snowflake using python as your core processing tool.
Key Responsibilities
Engine Development: Design and implement modular, reusable Python components for index construction, rebalancing, and backtesting.
Large-Scale Simulation: Use Pandas, NumPy, and PySpark to run historical calculations across long time horizons and multiple index variants.
Workflow Integration: Integrate engines with orchestrators such as Airflow or Temporal using parameterized, config-driven execution.
Reference Data Consumption: Query and utilize pricing, security master, and corporate action data from Snowflake.
Quality & Reconciliation: Build automated test harnesses to validate outputs, compare against benchmarks, and guarantee reproducibility.
Performance Optimization: Improve runtime efficiency through vectorization, caching, and distributed computing patterns.
Cross-Team Collaboration: Partner with Business, Index Ops, and Platform teams to accelerate research-to-production onboarding.
Required Technical Capabilities
Python Expertise: Strong proficiency in Python application development with emphasis on clean architecture and maintainable design.
Data & Numerical Libraries: Deep experience with Pandas and NumPy; working knowledge of PySpark for distributed workloads.
Financial Computation: Ability to implement portfolio mathematics, weighting algorithms, and time-series transformations.
Config-Driven Systems: Experience building rule-based or metadata-driven processing frameworks.
Database Skills: Strong SQL and experience consuming structured data from Snowflake.
Testing Discipline: Expertise in unit testing, regression testing, and deterministic replay of calculations.
Orchestration Integration: Familiarity with Airflow, Temporal, or similar workflow engines.
Cloud Infrastructure: Solid understanding of AWS ecosystem services (S3, Lambda, IAM).
Technical Skills:
- Technical Skills:
- Python, PySpark, Pandas, NumPy, AWS (S3, Lambda), Apache Airflow, SQL, Financial Services Domain Knowledge, Strong Communication Skills.

About Wissen Technology
About
The Wissen Group was founded in the year 2000. Wissen Technology, a part of Wissen Group, was established in the year 2015. Wissen Technology is a specialized technology company that delivers high-end consulting for organizations in the Banking & Finance, Telecom, and Healthcare domains.
With offices in US, India, UK, Australia, Mexico, and Canada, we offer an array of services including Application Development, Artificial Intelligence & Machine Learning, Big Data & Analytics, Visualization & Business Intelligence, Robotic Process Automation, Cloud, Mobility, Agile & DevOps, Quality Assurance & Test Automation.
Leveraging our multi-site operations in the USA and India and availability of world-class infrastructure, we offer a combination of on-site, off-site and offshore service models. Our technical competencies, proactive management approach, proven methodologies, committed support and the ability to quickly react to urgent needs make us a valued partner for any kind of Digital Enablement Services, Managed Services, or Business Services.
We believe that the technology and thought leadership that we command in the industry is the direct result of the kind of people we have been able to attract, to form this organization (you are one of them!).
Our workforce consists of 1000+ highly skilled professionals, with leadership and senior management executives who have graduated from Ivy League Universities like MIT, Wharton, IITs, IIMs, and BITS and with rich work experience in some of the biggest companies in the world.
Wissen Technology has been certified as a Great Place to Work®. The technology and thought leadership that the company commands in the industry is the direct result of the kind of people Wissen has been able to attract. Wissen is committed to providing them the best possible opportunities and careers, which extends to providing the best possible experience and value to our clients.
Connect with the team
Similar jobs (10)
We are hiring for a Python Developer at Wissen Technology!
📍 Location: Pune (Hybrid)
💼 Experience: 3–6 Years
⏱️ Notice Period: Immediate / 15 days preferred
🔧 Key Skills:
• Strong experience in Python
• Hands-on with Pandas & NumPy
• Experience with AWS (S3, Lambda preferred)
• Good understanding of data processing & APIs
• SQL knowledge
🏢 About Wissen Technology:
Wissen Technology, part of the Wissen Group (est. 2000), is a fast-growing technology company specializing in high-end consulting across Banking, Finance, Telecom, and Healthcare domains.
✔️ Global presence – US, India, UK, Australia, Mexico & Canada
✔️ Certified Great Place to Work®
✔️ Trusted by Fortune 500 clients like Morgan Stanley, Goldman Sachs, and more
✔️ Strong growth with 400% revenue increase in recent years
🌐 Website: www.wissen.com
🔗 LinkedIn: https://www.linkedin.com/company/wissen-technology/
If you’re interested or have relevant candidates, please share your resume at [your email].
#Hiring #PythonDeveloper #PuneJobs #AWS #ImmediateJoiner
While you may already know about Wissen and the company history, here is a quick rundown for you.
About Wissen Technology:
· The Wissen Group was founded in the year 2000. Wissen Technology, a part of Wissen Group, was established in the year 2015.
· Wissen Technology is a specialized technology company that delivers high-end consulting for organizations in the Banking & Finance, Telecom, and Healthcare domains. We help clients build world class products.
· Our workforce has highly skilled professionals, with leadership and senior management executives who have graduated from Ivy League Universities like Wharton, MIT, IITs, IIMs, and NITs and with rich work experience in some of the biggest companies in the world.
· Wissen Technology has grown its revenues by 400% in these five years without any external funding or investments.
· Globally present with offices US, India, UK, Australia, Mexico, and Canada.
· We offer an array of services including Application Development, Artificial Intelligence & Machine Learning, Big Data & Analytics, Visualization & Business Intelligence, Robotic Process Automation, Cloud, Mobility, Agile & DevOps, Quality Assurance & Test Automation.
· Wissen Technology has been certified as a Great Place to Work®.
· Wissen Technology has been voted as the Top 20 AI/ML vendor by CIO Insider in 2020.
· Over the years, Wissen Group has successfully delivered $650 million worth of projects for more than 20 of the Fortune 500 companies.
· We have served client across sectors like Banking, Telecom, Healthcare, Manufacturing, and Energy. They include likes of Morgan Stanley, Goldman Sachs, MSCI, StateStreet, Flipkart, Swiggy, Trafigura, GE to name a few.
De
Job Title: Application Development Engineer (Python – Backtesting & Index Platforms)
Role Overview
Key Responsibilities
Engine Development: Design and implement modular, reusable Python components for index construction, rebalancing, and backtesting.
Large-Scale Simulation: Use Pandas, NumPy, and PySpark to run historical calculations across long time horizons and multiple index variants.
Workflow Integration: Integrate engines with orchestrators such as Airflow or Temporal using parameterized, config-driven execution.
Reference Data Consumption: Query and utilize pricing, security master, and corporate action data from Snowflake.
Quality & Reconciliation: Build automated test harnesses to validate outputs, compare against benchmarks, and guarantee reproducibility.
Performance Optimization: Improve runtime efficiency through vectorization, caching, and distributed computing patterns.
Cross-Team Collaboration: Partner with Business, Index Ops, and Platform teams to accelerate research-to-production onboarding.
Required Technical Capabilities
Python Expertise: Strong proficiency in Python application development with emphasis on clean architecture and maintainable design.
Data & Numerical Libraries: Deep experience with Pandas and NumPy; working knowledge of PySpark for distributed workloads.
Financial Computation: Ability to implement portfolio mathematics, weighting algorithms, and time-series transformations.
Config-Driven Systems: Experience building rule-based or metadata-driven processing frameworks.
Database Skills: Strong SQL and experience consuming structured data from Snowflake.
Testing Discipline: Expertise in unit testing, regression testing, and deterministic replay of calculations.
Orchestration Integration: Familiarity with Airflow, Temporal, or similar workflow engines.
Cloud Infrastructure: Solid understanding of AWS ecosystem services (S3, Lambda, IAM) and how they integrate with the Snowflake Data Cloud.

About the company
The client is a quantitative investment firm focused on Indian financial markets. They operate a multi-strategy, multi-manager platform designed to generate consistent, risk- adjusted returns.
Their approach combines systematic investment methods, rigorous quantitative research and institutional-grade manager evaluation. We bring together research, technology and data to build scalable investment solutions.
Role Overview
We are seeking a Quantitative Developer with strong C++ and Python expertise to convert mathematical models and research prototypes into reliable, high-performance analytical engines.
You will work closely with Quantitative Research, Data Engineering, AI and Product Engineering teams throughout the full model lifecycle—from research handover and production implementation to validation, deployment and ongoing support.
This role is ideal for someone who enjoys working at the intersection of quantitative finance, numerical computing and production software engineering.
Key Responsibilities
Research Production
- Translate mathematical models and Python research prototypes into robust, production-quality C++.
- Develop reusable components for risk analytics, forecasting, portfolio analysis and simulation.
- Build efficient Python interfaces for C++ components using pybind11 or similar technologies.
- Ensure production implementations remain mathematically and numerically consistent with the underlying research.
- Establish clear and reproducible processes for transitioning models from research to production.
Engine Development and Validation
- Design and develop analytical engines capable of processing historical, batch and streaming data.
- Integrate calculation components with data pipelines, APIs, databases and downstream applications.
- Validate production implementations against research prototypes, benchmark datasets and expected results.
- Develop automated numerical, unit, integration, regression and performance tests.
- Identify and resolve numerical stability, precision and edge-case issues.
- Optimize calculation speed, memory usage, concurrency and scalability.
- Profile and benchmark critical components to meet defined performance requirements.
Deployment and Delivery
- Package analytical engines as libraries, services, APIs or containers.
- Support deployment across internal infrastructure and client-controlled environments.
- Configure engines for different datasets, workflows and institutional requirements.
- Assist with integration testing, production upgrades, issue diagnosis and technical troubleshooting.
- Implement appropriate logging, monitoring and error-handling capabilities.
- Document interfaces, assumptions, configurations, dependencies and deployment requirements.
Collaboration and Ownership
- Work closely with Quantitative Research, Data Engineering, AI and Product Engineering teams.
- Participate in technical design discussions, code reviews and quantitative model reviews.
- Communicate implementation trade-offs, constraints and risks clearly to technical and quantitative stakeholders.
- Take end-to-end ownership of assigned components, from research handover through production deployment and support.
- Contribute to engineering standards, reusable libraries and development best practices.
Required Qualifications
- Bachelor’s or master’s degree in Computer Science, Engineering, Mathematics, Statistics, Physics, Quantitative Finance or a related discipline.
- Strong professional programming experience in modern C++, including object- oriented and generic programming.
- Proficiency in Python and scientific-computing libraries such as NumPy, pandas or SciPy.
- Experience translating mathematical or analytical prototypes into production software.
- Strong understanding of algorithms, data structures, software architecture and design principles.
- Experience building automated unit, integration and performance tests.
- Familiarity with numerical methods, floating-point behaviour and numerical validation.
- Experience profiling and optimizing compute-intensive or data-intensive applications.
- Proficiency with Git and modern software-development practices.
- Strong analytical, debugging and problem-solving skills.
- •Ability to work effectively with both researchers and software engineers.
Preferred Qualifications
- Experience with pybind11, Boost.Python, Cython or similar interoperability technologies.
- Knowledge of quantitative finance, portfolio analytics, risk modelling, forecasting or simulation.
- Familiarity with time-series data and financial-market datasets.
- Experience developing applications that process batch or real-time streaming data.
- Exposure to concurrent, parallel or distributed computing.
- Experience with containerization and deployment technologies such as Docker.
- Familiarity with Linux environments, CI/CD pipelines and cloud or on-premises infrastructure.
- Experience building analytical libraries, calculation services or APIs for institutional users.
- Knowledge of Indian financial markets is advantageous
Description
We are looking for Senior Data Engineers to join our Data Platform team and build scalable, high-performance data platforms that power data processing, analytics, and downstream applications.
The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Apache Spark and Python Scala.
You will be responsible for designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL and data processing pipelines for large-scale datasets.
- Build and optimize distributed data applications using Apache Spark and Python Scala.
- Develop reliable, high-performance data pipelines for batch and streaming workloads.
- Design and manage data workflows using Apache Airflow.
- Build and operate data workloads on AWS, with strong usage of Amazon S3 for large-scale data storage.
- Work with large datasets to ensure data quality, consistency, reliability, and performance.
- Collaborate with engineering, product, analytics, and other platform teams to deliver robust data solutions.
- Optimize data workflows for scalability, reliability, performance, and cost efficiency.
- Troubleshoot production issues, identify bottlenecks, and continuously improve platform performance.
Requirements
Candidates who demonstrate:
- 5+ years of experience in Data Engineering, Big Data Engineering, or a similar role.
- Strong hands-on experience with Apache Spark and Scala.
- Experience designing, building, and maintaining large-scale ETL pipelines.
- Strong hands-on experience with AWS, particularly Amazon S3.
- Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
- Strong SQL skills and a solid understanding of distributed data processing concepts.
- Experience working with batch and/or streaming data pipelines.
- Excellent debugging, problem-solving, and performance optimization skills.
- Strong communication and collaboration skills.
Good to Have
- Experience with Databricks and the broader Databricks data platform.
- Familiarity with streaming technologies such as Apache Kafka.
- Experience working on large-scale data platforms handling high-volume data workloads.
- Exposure to additional AWS data services and cloud-native data architectures.
Senior Data Engineer – PySpark & Oracle
Experience: 7+ Years
Location: Bangalore
Notice Period: Immediate to 10 Days
Key Skills:
- Strong expertise in Data Modeling, Data Design & Modernization
- Primary skills: PySpark, Oracle SQL/PLSQL
- Secondary skills: Python, ETL & Data Pipelines
- Experience with Kafka and Hadoop
- Exposure to AWS / Azure / GCP
- Good knowledge of Git and JIRA
Roles & Responsibilities:
- Design, develop, and modernize scalable data models and data architecture.
- Develop and optimize data processing solutions using PySpark and Oracle SQL/PLSQL.
- Build and maintain robust ETL workflows and data pipelines.
- Work with Kafka, Hadoop, and cloud platforms for data processing and integration.
- Perform data transformation, optimization, and performance tuning.
- Collaborate with technical teams on data design, development, testing, and deployment.
Data Engineer Short Hiring Post
🚨 Hiring: Data Engineer
🔹 Experience: 5–9 Years
🔹 Location: Bangalore / Hyderabad
🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling
🔹 Process: L1 Virtual → L2 F2F Karat Test
🔹 F2F: Bangalore / Hyderabad Location
🔹 Positions: Immediate requirement
⚠️ Note: Candidates must be available for F2F Karat immediately after L1.
#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners
Job Summary
We are seeking a skilled Python Developer with experience in building scalable applications using Python frameworks such as Flask and Django. The ideal candidate should have hands-on experience with PySpark for big data processing and strong SQL skills for data analysis and database management.
Key Responsibilities
- Design, develop, test, and maintain Python-based applications.
- Develop RESTful APIs using Flask and/or Django.
- Build scalable data processing pipelines using PySpark.
- Write efficient SQL queries, stored procedures, and optimize database performance.
- Integrate applications with databases and third-party services.
- Collaborate with cross-functional teams to gather and implement business requirements.
- Troubleshoot, debug, and enhance application performance.
- Follow coding standards, perform code reviews, and maintain technical documentation.
Required Skills
- Strong proficiency in Python.
- Hands-on experience with Flask and/or Django.
- Experience with PySpark for large-scale data processing.
- Strong knowledge of SQL (writing complex queries, joins, indexing, optimization).
- Understanding of REST APIs and microservices architecture.
- Experience with Git/version control.
- Knowledge of relational databases such as PostgreSQL, MySQL, SQL Server, or Oracle.
- Good problem-solving and analytical skills.
Preferred Skills
- Experience with cloud platforms (AWS, Azure, or GCP).
- Familiarity with Docker and Kubernetes.
- Knowledge of CI/CD pipelines.
- Experience with Airflow or other workflow orchestration tools.
- Understanding of Agile/Scrum methodologies.
We are looking for a skilled Python & PySpark Developer with strong expertise in Big Data technologies, Spark, SQL/PL-SQL, and REST API development using Flask or Django. The ideal candidate should have experience building scalable data pipelines, processing large datasets, developing APIs, and working with distributed computing frameworks.
Key Responsibilities
- Develop, optimize, and maintain scalable data pipelines using PySpark and Apache Spark.
- Design, develop, and optimize complex SQL and PL/SQL queries, stored procedures, functions, and database objects.
- Build and maintain RESTful APIs using Flask or Django.
- Develop robust Python applications for data engineering and backend services.
- Process and analyze large-scale datasets using Big Data technologies.
- Optimize Spark jobs for performance, scalability, and reliability.
- Integrate APIs with internal and external systems.
- Collaborate with cross-functional teams including Data Engineers, Data Scientists, and Application Developers.
- Troubleshoot production issues and implement performance improvements.
- Follow coding standards, version control, and CI/CD best practices.
Mandatory Skills
- Strong proficiency in Python programming.
- Hands-on experience with PySpark and Apache Spark.
- Strong SQL coding skills.
- Experience with PL/SQL development.
- Experience in Big Data ecosystem.
- REST API development using Flask or Django.
- Experience in developing and consuming Python APIs.
- Knowledge of data processing, ETL, and distributed computing.
- Experience with Git/version control.
Preferred Skills
- Experience with Hadoop ecosystem (Hive, HDFS, YARN).
- Exposure to cloud platforms such as AWS, Azure, or GCP.
- Knowledge of Airflow or other workflow orchestration tools.
- Experience with Docker and Kubernetes.
- Familiarity with Kafka or other streaming technologies.
- Understanding of CI/CD pipelines.
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
- 4–8+ years of experience in Python and Big Data development (can be adjusted based on the role).
Required Experience
- Strong hands-on experience in Python, PySpark, and Apache Spark.
- Extensive experience writing optimized SQL and PL/SQL code.
- Experience developing REST APIs using Flask or Django.
- Experience working with large-scale data processing and ETL pipelines.
- Strong analytical, debugging, and problem-solving skills.
Mandatory Skills: Python, PySpark, SQL Coding, Apache Spark, Big Data, Flask/Django (REST API), PL/SQL, Python APIs.
Role Summary:
We are looking for a Forward Deployed Engineer with strong hands-on experience in Databricks and Generative AI/Claude to work closely with clients, business stakeholders, and internal engineering teams. The ideal candidate will combine strong Data Engineering, Software Engineering, Databricks, and Generative AI skills with the ability to understand business problems and rapidly build, deploy, and optimize production-ready solutions. This is a client-facing, hands-on engineering role where you will work from problem discovery and solution design through POC development, production deployment, and ongoing optimization.
Key Responsibilities:
Forward Deployed Engineering
- Work directly with clients and stakeholders to understand business and technical requirements.
- Translate business problems into scalable data, AI, and software solutions.
- Design and develop POCs and rapidly validate technical solutions.
- Convert successful POCs into reliable, production-ready applications.
- Work closely with client engineering and data teams during implementation and deployment.
- Troubleshoot production issues and continuously optimize deployed solutions.
- Act as a technical bridge between clients, delivery teams, data engineers, AI engineers, and architects.
Databricks & Data Engineering
- Design and develop scalable data solutions using Databricks, PySpark, Python, and SQL.
- Build and optimize data ingestion, transformation, and ETL/ELT pipelines.
- Work with Databricks Lakehouse, Delta Lake, and Unity Catalog.
- Develop Databricks Workflows and production data pipelines.
- Implement data processing solutions for structured and semi-structured datasets.
- Optimize Databricks workloads for performance, scalability, reliability, and cost.
- Integrate Databricks with databases, APIs, cloud platforms, and enterprise applications.
Generative AI & Claude
- Build enterprise AI solutions using Claude and other Large Language Models (LLMs).
- Integrate Claude APIs into applications and business workflows.
- Develop RAG (Retrieval-Augmented Generation) solutions using enterprise data.
- Work with embeddings, vector search, semantic search, and knowledge retrieval.
- Develop AI-powered applications for summarization, classification, information extraction, question answering, and document processing.
- Implement prompt engineering, structured outputs, tool/function calling, and context management.
- Develop and integrate AI agents and multi-step AI workflows where applicable.
- Evaluate LLM responses for accuracy, relevance, groundedness, latency, and cost.
- Implement appropriate AI guardrails, security, and data privacy controls.
Production & Deployment
- Deploy AI and data solutions into production environments.
- Work with APIs, microservices, Git, CI/CD, containers, and cloud platforms.
- Monitor application and pipeline performance and troubleshoot issues.
- Collaborate with Data Scientists and ML Engineers to productionize AI/ML models.
- Ensure solutions meet security, scalability, reliability, and maintainability requirements.
Required Skills & Experience
- 4+ years of experience in Data Engineering, Software Engineering, AI/ML Engineering, or a related field.
- Strong hands-on experience with Databricks.
- Strong proficiency in Python, PySpark, and SQL.
- Experience with Delta Lake and Lakehouse Architecture.
- Experience working with Generative AI / LLMs.
- Hands-on experience with Claude / Anthropic APIs is preferred.
- Experience with RAG, embeddings, vector databases, and semantic search.
- Strong understanding of REST APIs and enterprise integrations.
- Experience developing production-grade applications and data pipelines.
- Strong problem-solving and troubleshooting capabilities.
- Excellent communication and client-facing skills.
Preferred Skills
- Experience with Claude Code / Anthropic ecosystem.
- Experience with OpenAI, Azure OpenAI, AWS Bedrock, or other LLM platforms.
- Experience with LangChain, LangGraph, LlamaIndex, or equivalent frameworks.
- Experience with Databricks Unity Catalog, Workflows, and MLflow.
- Experience with AWS, Azure, or GCP.
- Experience with Docker, Kubernetes, and CI/CD.
- Exposure to AI agents and agentic workflows.
- Knowledge of AI evaluation, guardrails, security, and responsible AI.
- Experience working in consulting, client delivery, or customer-facing engineering environments.
Key Competencies
- Strong customer-facing and stakeholder management skills.
- Ability to understand ambiguous business problems and translate them into technical solutions.
- Strong ownership and execution mindset.
- Ability to rapidly prototype, iterate, and productionize solutions.
- Strong analytical and troubleshooting skills.
- Comfortable working in fast-paced and dynamic client environments.
- Excellent written and verbal communication.
- Ability to work independently as well as collaboratively with distributed teams.
The recruiter has not been active on this job recently. You may apply but please expect a delayed response.
Skills Referential (Required knowledge, skills and abilities)
Technical Skills:
Python
Pyspark
SQL
ETL Aws, Azure, gcp
About the Role
We are looking for a Senior Data Engineer with strong hands-on expertise in Databricks, Python, PySpark, and SQL to build scalable, high-performance data engineering solutions. You’ll architect and develop large scale, high-performance data pipelines capable of handling massive real-time and batch data volumes across multiple business systems. Databricks is the core enterprise data and processing platform for this role. You will also use Apache Airflow for workflow orchestration and dbt for ELT transformations, and will contribute to designing reliable, secure, and governed data platforms that enable analytics, reporting, and AI-driven use cases.
Key Responsibilities
- Design and implement large-scale data pipelines using Python/PySpark, Databricks, and Microsoft Fabric.
- Develop and optimize data processing workloads in Databricks using PySpark and Spark SQL, with a strong focus on scalability, reliability, performance, and maintainability.
- Develop and maintain dbt models including layered architecture, incremental models, snapshots, macros, testing, and documentation.
- Design, develop, and maintain Apache Airflow DAGs for orchestrating reliable, scalable, and observable data pipelines.
- Design and implement data quality, observability, and governance frameworks, including automated testing, monitoring, lineage, access control, and data privacy standards.
- Partner with analytics, product, and business stakeholders to turn requirements into trustworthy datasets, and raise the engineering bar through design discussions, code reviews, and mentoring junior engineers.
Required Skills
- Strong expertise in Python for developing scalable, modular, and production-ready data engineering applications.
- Strong expertise in PySpark, including DataFrame API, Spark SQL, Structured Streaming, partitioning strategies, joins, caching, handling data skew, and Spark performance optimization.
- Strong hands-on experience with Databricks for data ingestion, transformation, processing, and optimization, including Delta Lake, Unity Catalog, Databricks Workflows, notebooks, jobs, and Databricks-native data engineering capabilities.
- Strong experience in Databricks/Spark performance tuning, including query and job optimization, partitioning, file sizing, caching, join optimization, handling data skew, and efficient use of compute resources.
- Hands-on experience with Delta Lake, including transactional data processing, schema management, incremental data processing, and reliable batch and streaming data pipelines.
- Hands-on experience in developing dbt projects using layered architecture, incremental models, snapshots, macros/Jinja, testing, documentation, and deployment best practices.
- Expertise in advanced SQL and data modelling — dimensional modeling, slowly changing dimensions, schema evolution, and query optimization.
- Hands-on experience in developing and managing Apache Airflow DAGs, scheduling workflows, dependency management, retries, backfills, and operational monitoring.
- Hands-on experience with at least one major cloud platform (AWS, Azure or GCP).
- Strong problem-solving skills and the ability to work independently with business and analytics stakeholders.
Nice to Have
- Hands-on exposure to Microsoft Fabric for data integration and analytics.
- Experience using AI coding assistants (e.g. Claude Code, GitHub Copilot) as part of a development workflow.
- Familiarity with modern DevOps practices, including CI/CD pipelines, Infrastructure as Code (IaC), and containerization (Docker/Kubernetes).
- Domain expertise in financial services.





