Data Engineer

🚀 Hiring: Data Engineer | GCP + Spark + Python + .NET |

| 6–10 Yrs | Gurugram (Hybrid)

We’re looking for a skilled Data Engineer with strong hands-on experience in GCP, Spark-Scala, Python, and .NET.

📍 Location: Suncity, Sector 54, Gurugram (Hybrid – 3 days onsite)

💼 Experience: 6–10 Years

⏱️ Notice Period :- Immediate Joiner

Required Skills:

• 5+ years of experience in distributed computing (Spark) and software development.
• 3+ years of experience in Spark-Scala
• 5+ years of experience in Data Engineering.
• 5+ years of experience in Python.
• Fluency in working with databases (preferably Postgres).
• Have a sound understanding of object-oriented programming and development principles.
• Experience working in an Agile Scrum or Kanban development environment.
• Experience working with version control software (preferably Git).
• Experience with CI/CD pipelines.
• Experience with automated testing, including integration/delta, Load, and Performance

* Python (3 to 6 years): Strong expertise in data workflows and automation

* Spark (PySpark): Hands-on experience with large-scale data processing

* Pandas: For detailed data analysis and validation

* Delta Lake: Managing structured and semi-structured datasets at scale

* SQL: Querying and performing operations on Delta tables

* Azure Cloud: Compute and storage services

* Orchestrator: Good experience with either ADF or Airflow

About Corridor Platforms

Corridor Platforms is a leader in next-generation risk decisioning and responsible AI governance, empowering banks and lenders to build transparent, compliant, and data-driven solutions. Our platforms combine advanced analytics, real-time data integration, and GenAI to support complex financial decision workflows for regulated industries.

Role Overview

As a Backend Engineer at Corridor Platforms, you will:

• Architect, develop, and maintain backend components for our Risk Decisioning Platform.
• Build and orchestrate scalable backend services that automate, optimize, and monitor high-value credit and risk decisions in real time.
• Integrate with ORM layers – such as SQLAlchemy – and multi RDBMS solutions (Postgres, MySQL, Oracle, MSSQL, etc) to ensure data integrity, scalability, and compliance.
• Collaborate closely with Product Team, Data Scientists, QA Teams to create extensible APIs, workflow automation, and AI governance features.
• Architect workflows for privacy, auditability, versioned traceability, and role-based access control, ensuring adherence to regulatory frameworks.
• Take ownership from requirements to deployment, seeing your code deliver real impact in the lives of customers and end users.

Technical Skills

• Languages: Python 3.9+, SQL, JavaScript/TypeScript, Angular
• Frameworks: Flask, SQLAlchemy, Celery, Marshmallow, Apache Spark
• Databases: PostgreSQL, Oracle, SQL Server, Redis
• Tools: pytest, Docker, Git, Nx
• Cloud: Experience with AWS, Azure, or GCP preferred
• Monitoring: Familiarity with OpenTelemetry and logging frameworks

Why Join Us?

• Cutting-Edge Tech: Work hands-on with the latest AI, cloud-native workflows, and big data tools—all within a single compliant platform.
• End-to-End Impact: Contribute to mission-critical backend systems, from core data models to live production decision services.
• Innovation at Scale: Engineer solutions that process vast data volumes, helping financial institutions innovate safely and effectively.
• Mission-Driven: Join a passionate team advancing fair, transparent, and compliant risk decisioning at the forefront of fintech and AI governance.

What We’re Looking For

• Proficiency in Python, SQLAlchemy (or similar ORM), and SQL databases.
• Experience developing and maintaining scalable backend services, including API, data orchestration, ML workflows,  and workflow automation.
• Solid understanding of data modeling, distributed systems, and backend architecture for regulated environments.
• Curiosity and drive to work at the intersection of AI/ML, fintech, and regulatory technology.
• Experience mentoring and guiding junior developers.

Ready to build backends that shape the future of decision intelligence and responsible AI?

Apply now and become part of the innovation at Corridor Platforms!

Roles & Responsibilities

• Data Engineering Excellence: Design and implement data pipelines using formats like JSON, Parquet, CSV, and ORC, utilizing batch and streaming ingestion.
• Cloud Data Migration Leadership: Lead cloud migration projects, developing scalable Spark pipelines.
• Medallion Architecture: Implement Bronze, Silver, and gold tables for scalable data systems.
• Spark Code Optimization: Optimize Spark code to ensure efficient cloud migration.
• Data Modeling: Develop and maintain data models with strong governance practices.
• Data Cataloging & Quality: Implement cataloging strategies with Unity Catalog to maintain high-quality data.
• Delta Live Table Leadership: Lead the design and implementation of Delta Live Tables (DLT) pipelines for secure, tamper-resistant data management.
• Customer Collaboration: Collaborate with clients to optimize cloud migrations and ensure best practices in design and governance.

Educational Qualifications

• Experience: Minimum 5 years of hands-on experience in data engineering, with a proven track record in complex pipeline development and cloud-based data migration projects.
• Education: Bachelor’s or higher degree in Computer Science, Data Engineering, or a related field.
• Skills
• Must-have: Proficiency in Spark, SQL, Python, and other relevant data processing technologies. Strong knowledge of Databricks and its components, including Delta Live Table (DLT) pipeline implementations. Expertise in on-premises to cloud Spark code optimization and Medallion Architecture.

Good to Have

• Familiarity with AWS services (experience with additional cloud platforms like GCP or Azure is a plus).

Soft Skills

• Excellent communication and collaboration skills, with the ability to work effectively with clients and internal teams.
• Certifications
• AWS/GCP/Azure Data Engineer Certification.

Profile: AWS Data Engineer

Mode- Hybrid

Experience- 5+7 years

Locations - Bengaluru, Pune, Chennai, Mumbai, Gurugram

Roles and Responsibilities

• Design and maintain ETL pipelines using AWS Glue and Python/PySpark
• Optimize SQL queries for Redshift and Athena
• Develop Lambda functions for serverless data processing
• Configure AWS DMS for database migration and replication
• Implement infrastructure as code with CloudFormation
• Build optimized data models for performance
• Manage RDS databases and AWS service integrations
• Troubleshoot and improve data processing efficiency
• Gather requirements from business stakeholders
• Implement data quality checks and validation
• Document data pipelines and architecture
• Monitor workflows and implement alerting
• Keep current with AWS services and best practices

Required Technical Expertise:

• Python/PySpark for data processing
• AWS Glue for ETL operations
• Redshift and Athena for data querying
• AWS Lambda and serverless architecture
• AWS DMS and RDS management
• CloudFormation for infrastructure
• SQL optimization and performance tuning

Key Responsibilities

• Develop and maintain Python-based applications.
• Design and optimize SQL queries and databases.
• Collaborate with cross-functional teams to define, design, and ship new features.
• Write clean, maintainable, and efficient code.
• Troubleshoot and debug applications.
• Participate in code reviews and contribute to team knowledge sharing.

Qualifications and Required Skills

• Strong proficiency in Python programming.
• Experience with SQL and database management.
• Experience with web frameworks such as Django or Flask.
• Knowledge of front-end technologies like HTML, CSS, and JavaScript.
• Familiarity with version control systems like Git.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork skills.

Good to Have Skills

• Experience with cloud platforms like AWS or Azure.
• Knowledge of containerization technologies like Docker.
• Familiarity with continuous integration and continuous deployment (CI/CD) pipelines

Azure DE

Primary Responsibilities -

• Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
• Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure Create data models for analytics purposes
• Utilizing Azure Data Factory or comparable technologies, create and maintain ETL (Extract, Transform, Load) operations
• Use Azure Data Factory and Databricks to assemble large, complex data sets
• Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.
• Ensure data security and compliance
• Collaborate with data engineers, and other stakeholders to understand requirements and translate them into scalable and reliable data platform architectures

Required skills:

• Blend of technical expertise, analytical problem-solving, and collaboration with cross-functional teams
• Azure DevOps
• Apache Spark, Python
• SQL proficiency
• Azure Databricks knowledge
• Big data technologies

The DEs should be well versed in coding, spark core and data ingestion using Azure. Moreover, they need to be decent in terms of communication skills. They should also have core Azure DE skills and coding skills (pyspark, python and SQL).

Out of the 7 open demands, 5 of The Azure Data Engineers should have minimum 5 years of relevant Data Engineering experience.

·      The Objective:

You will play a crucial role in designing, implementing, and maintaining our data infrastructure, run tests and update the systems

·      Job function and requirements

o  Expert in Python, Pandas and Numpy with knowledge of Python web Framework such as Django and Flask.

o  Able to integrate multiple data sources and databases into one system.

o  Basic understanding of frontend technologies like HTML, CSS, JavaScript.

o  Able to build data pipelines.

o  Strong unit test and debugging skills.

o  Understanding of fundamental design principles behind a scalable application

o  Good understanding of RDBMS databases among Mysql or Postgresql.

o  Able to analyze and transform raw data.

·      About us

Mitibase helps companies find warm prospects every month that are most relevant, and then helps their team to act on those with automation. We do so by automatically tracking key accounts and contacts for job changes and relationships triggers and surfaces them as warm leads in your sales pipeline.

EXP-Developer-4 to 12 years

         Must have low-level design and development skills.  Should able to design a solution for given use cases. 

• Agile delivery-  Person must able to show design and code on a daily basis
• Must be an experienced PySpark developer and  Scala coding.   Primary skill is PySpark
• Must have experience in designing job orchestration, sequence, metadata design, Audit trail, dynamic parameter passing and error/exception handling
• Good experience with unit, integration and UAT support
• Able to design and code reusable components and functions
• Should able to review design, code & provide review comments with justification
• Zeal to learn new tool/technologies and adoption
• Good to have experience with Devops and CICD