Data Engineer

🚀 Hiring: Data Engineer ( Azure ) at Deqode

⭐ Experience: 5+ Years

📍 Location: Pune, Bhopal, Jaipur, Gurgaon, Delhi, Banglore,

⭐ Work Mode:- Hybrid

⏱️ Notice Period: Immediate Joiners

(Only immediate joiners & candidates serving notice period)

⭐ Hiring: Databricks Data Engineer – Lakeflow | Streaming | DBSQL | Data Intelligence

We are looking for a Databricks Data Engineer ( Azure ) to build reliable, scalable, and governed data pipelines powering analytics, operational reporting, and the Data Intelligence Layer.

🔹 Key Responsibilities

✅ Build optimized batch pipelines using Delta Lake (partitioning, OPTIMIZE, Z-ORDER, VACUUM)

✅ Implement incremental ingestion using Databricks Autoloader with schema evolution & checkpointing

✅ Develop Structured Streaming pipelines with watermarking, late data handling & restart safety

✅ Implement declarative pipelines using Lakeflow

✅ Design idempotent, replayable pipelines with safe backfills

✅ Optimize Spark workloads (AQE, skew handling, shuffle & join tuning)

✅ Build curated datasets for Databricks SQL (DBSQL), dashboards & downstream applications

✅ Package and deploy using Databricks Repos & Asset Bundles (CI/CD)

Ensure governance using Unity Catalog and embedded data quality checks

✅ Mandatory Skills (Must Have)

👉 Databricks & Delta Lake (Advanced Optimization & Performance Tuning)

👉 Structured Streaming & Autoloader Implementation

👉 Databricks SQL (DBSQL) & Data Modeling for Analytics

🚀 Hiring: Data Engineer | GCP + Spark + Python + .NET |

| 6–10 Yrs | Gurugram (Hybrid)

We’re looking for a skilled Data Engineer with strong hands-on experience in GCP, Spark-Scala, Python, and .NET.

📍 Location: Suncity, Sector 54, Gurugram (Hybrid – 3 days onsite)

💼 Experience: 6–10 Years

⏱️ Notice Period :- Immediate Joiner

Required Skills:

• 5+ years of experience in distributed computing (Spark) and software development.
• 3+ years of experience in Spark-Scala
• 5+ years of experience in Data Engineering.
• 5+ years of experience in Python.
• Fluency in working with databases (preferably Postgres).
• Have a sound understanding of object-oriented programming and development principles.
• Experience working in an Agile Scrum or Kanban development environment.
• Experience working with version control software (preferably Git).
• Experience with CI/CD pipelines.
• Experience with automated testing, including integration/delta, Load, and Performance

* Python (3 to 6 years): Strong expertise in data workflows and automation

* Spark (PySpark): Hands-on experience with large-scale data processing

* Pandas: For detailed data analysis and validation

* Delta Lake: Managing structured and semi-structured datasets at scale

* SQL: Querying and performing operations on Delta tables

* Azure Cloud: Compute and storage services

* Orchestrator: Good experience with either ADF or Airflow

Roles & Responsibilities

• Data Engineering Excellence: Design and implement data pipelines using formats like JSON, Parquet, CSV, and ORC, utilizing batch and streaming ingestion.
• Cloud Data Migration Leadership: Lead cloud migration projects, developing scalable Spark pipelines.
• Medallion Architecture: Implement Bronze, Silver, and gold tables for scalable data systems.
• Spark Code Optimization: Optimize Spark code to ensure efficient cloud migration.
• Data Modeling: Develop and maintain data models with strong governance practices.
• Data Cataloging & Quality: Implement cataloging strategies with Unity Catalog to maintain high-quality data.
• Delta Live Table Leadership: Lead the design and implementation of Delta Live Tables (DLT) pipelines for secure, tamper-resistant data management.
• Customer Collaboration: Collaborate with clients to optimize cloud migrations and ensure best practices in design and governance.

Educational Qualifications

• Experience: Minimum 5 years of hands-on experience in data engineering, with a proven track record in complex pipeline development and cloud-based data migration projects.
• Education: Bachelor’s or higher degree in Computer Science, Data Engineering, or a related field.
• Skills
• Must-have: Proficiency in Spark, SQL, Python, and other relevant data processing technologies. Strong knowledge of Databricks and its components, including Delta Live Table (DLT) pipeline implementations. Expertise in on-premises to cloud Spark code optimization and Medallion Architecture.

Good to Have

• Familiarity with AWS services (experience with additional cloud platforms like GCP or Azure is a plus).

Soft Skills

• Excellent communication and collaboration skills, with the ability to work effectively with clients and internal teams.
• Certifications
• AWS/GCP/Azure Data Engineer Certification.

About Corridor Platforms

Corridor Platforms is a leader in next-generation risk decisioning and responsible AI governance, empowering banks and lenders to build transparent, compliant, and data-driven solutions. Our platforms combine advanced analytics, real-time data integration, and GenAI to support complex financial decision workflows for regulated industries.

Role Overview

As a Backend Engineer at Corridor Platforms, you will:

• Architect, develop, and maintain backend components for our Risk Decisioning Platform.
• Build and orchestrate scalable backend services that automate, optimize, and monitor high-value credit and risk decisions in real time.
• Integrate with ORM layers – such as SQLAlchemy – and multi RDBMS solutions (Postgres, MySQL, Oracle, MSSQL, etc) to ensure data integrity, scalability, and compliance.
• Collaborate closely with Product Team, Data Scientists, QA Teams to create extensible APIs, workflow automation, and AI governance features.
• Architect workflows for privacy, auditability, versioned traceability, and role-based access control, ensuring adherence to regulatory frameworks.
• Take ownership from requirements to deployment, seeing your code deliver real impact in the lives of customers and end users.

Technical Skills

• Languages: Python 3.9+, SQL, JavaScript/TypeScript, Angular
• Frameworks: Flask, SQLAlchemy, Celery, Marshmallow, Apache Spark
• Databases: PostgreSQL, Oracle, SQL Server, Redis
• Tools: pytest, Docker, Git, Nx
• Cloud: Experience with AWS, Azure, or GCP preferred
• Monitoring: Familiarity with OpenTelemetry and logging frameworks

Why Join Us?

• Cutting-Edge Tech: Work hands-on with the latest AI, cloud-native workflows, and big data tools—all within a single compliant platform.
• End-to-End Impact: Contribute to mission-critical backend systems, from core data models to live production decision services.
• Innovation at Scale: Engineer solutions that process vast data volumes, helping financial institutions innovate safely and effectively.
• Mission-Driven: Join a passionate team advancing fair, transparent, and compliant risk decisioning at the forefront of fintech and AI governance.

What We’re Looking For

• Proficiency in Python, SQLAlchemy (or similar ORM), and SQL databases.
• Experience developing and maintaining scalable backend services, including API, data orchestration, ML workflows,  and workflow automation.
• Solid understanding of data modeling, distributed systems, and backend architecture for regulated environments.
• Curiosity and drive to work at the intersection of AI/ML, fintech, and regulatory technology.
• Experience mentoring and guiding junior developers.

Ready to build backends that shape the future of decision intelligence and responsible AI?

Apply now and become part of the innovation at Corridor Platforms!

Job title - Python developer

Exp – 4 to 6 years

Location – Pune/Mum/B’lore

PFB JD

Requirements:

• Proven experience as a Python Developer
• Strong knowledge of core Python and Pyspark concepts
• Experience with web frameworks such as Django or Flask
• Good exposure to any cloud platform (GCP Preferred)
• CI/CD exposure required
• Solid understanding of RESTful APIs and how to build them
• Experience working with databases like Oracle DB and MySQL
• Ability to write efficient SQL queries and optimize database performance
• Strong problem-solving skills and attention to detail
• Strong SQL programing (stored procedure, functions)
• Excellent communication and interpersonal skill

Roles and Responsibilities

• Design, develop, and maintain data pipelines and ETL processes using pyspark
• Work closely with data scientists and analysts to provide them with clean, structured data.
• Optimize data storage and retrieval for performance and scalability.
• Collaborate with cross-functional teams to gather data requirements.
• Ensure data quality and integrity through data validation and cleansing processes.
• Monitor and troubleshoot data-related issues to ensure data pipeline reliability.
• Stay up to date with industry best practices and emerging technologies in data engineering.

Profile: AWS Data Engineer

Mode- Hybrid

Experience- 5+7 years

Locations - Bengaluru, Pune, Chennai, Mumbai, Gurugram

Roles and Responsibilities

• Design and maintain ETL pipelines using AWS Glue and Python/PySpark
• Optimize SQL queries for Redshift and Athena
• Develop Lambda functions for serverless data processing
• Configure AWS DMS for database migration and replication
• Implement infrastructure as code with CloudFormation
• Build optimized data models for performance
• Manage RDS databases and AWS service integrations
• Troubleshoot and improve data processing efficiency
• Gather requirements from business stakeholders
• Implement data quality checks and validation
• Document data pipelines and architecture
• Monitor workflows and implement alerting
• Keep current with AWS services and best practices

Required Technical Expertise:

• Python/PySpark for data processing
• AWS Glue for ETL operations
• Redshift and Athena for data querying
• AWS Lambda and serverless architecture
• AWS DMS and RDS management
• CloudFormation for infrastructure
• SQL optimization and performance tuning

Azure DE

Primary Responsibilities -

• Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
• Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure Create data models for analytics purposes
• Utilizing Azure Data Factory or comparable technologies, create and maintain ETL (Extract, Transform, Load) operations
• Use Azure Data Factory and Databricks to assemble large, complex data sets
• Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.
• Ensure data security and compliance
• Collaborate with data engineers, and other stakeholders to understand requirements and translate them into scalable and reliable data platform architectures

Required skills:

• Blend of technical expertise, analytical problem-solving, and collaboration with cross-functional teams
• Azure DevOps
• Apache Spark, Python
• SQL proficiency
• Azure Databricks knowledge
• Big data technologies

The DEs should be well versed in coding, spark core and data ingestion using Azure. Moreover, they need to be decent in terms of communication skills. They should also have core Azure DE skills and coding skills (pyspark, python and SQL).

Out of the 7 open demands, 5 of The Azure Data Engineers should have minimum 5 years of relevant Data Engineering experience.

EXP-Developer-4 to 12 years

         Must have low-level design and development skills.  Should able to design a solution for given use cases. 

• Agile delivery-  Person must able to show design and code on a daily basis
• Must be an experienced PySpark developer and  Scala coding.   Primary skill is PySpark
• Must have experience in designing job orchestration, sequence, metadata design, Audit trail, dynamic parameter passing and error/exception handling
• Good experience with unit, integration and UAT support
• Able to design and code reusable components and functions
• Should able to review design, code & provide review comments with justification
• Zeal to learn new tool/technologies and adoption
• Good to have experience with Devops and CICD