11+ Hadoop Jobs in Kochi (Cochin) | Hadoop Job openings in Kochi (Cochin)

SENIOR DATA ENGINEER:

ROLE SUMMARY:

Own the design and delivery of petabyte-scale data platforms and pipelines across AWS and modern Lakehouse stacks. You’ll architect, code, test, optimize, and operate ingestion, transformation, storage, and serving layers. This role requires autonomy, strong engineering judgment, and partnership with project managers, infrastructure teams, testers, and customer architects to land secure, cost-efficient, and high-performing solutions.

RESPONSIBILITIES:

• Architecture and design: Create HLD/LLD/SAD, source–target mappings, data contracts, and optimal designs aligned to requirements.
• Pipeline development: Build and test robust ETL/ELT for batch, micro-batch, and streaming across RDBMS, flat files, APIs, and event sources.
• Performance and cost tuning: Profile and optimize jobs, right-size infrastructure, and model license/compute/storage costs.
• Data modeling and storage: Design schemas and SCD strategies; manage relational, NoSQL, data lakes, Delta Lakes, and Lakehouse tables.
• DevOps and release: Establish coding standards, templates, CI/CD, configuration management, and monitored release processes.
• Quality and reliability: Define DQ rules and lineage; implement SLA tracking, failure detection, RCA, and proactive defect mitigation.
• Security and governance: Enforce IAM best practices, retention, audit/compliance; implement PII detection and masking.
• Orchestration: Schedule and govern pipelines with Airflow and serverless event-driven patterns.
• Stakeholder collaboration: Clarify requirements, present design options, conduct demos, and finalize architectures with customer teams.
• Leadership: Mentor engineers, set FAST goals, drive upskilling and certifications, and support module delivery and sprint planning.

REQUIRED QUALIFICATIONS:

• Experience: 15+ years designing distributed systems at petabyte scale; 10+ years building data lakes and multi-source ingestion.
•  Cloud (AWS): IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail.
• Programming: Python (preferred), PySpark, SQL for analytics, window functions, and performance tuning.
• ETL tools: AWS Glue, Informatica, Databricks, GCP DataProc; orchestration with Airflow.
• Lakehouse/warehousing: Snowflake, BigQuery, Delta Lake/Lakehouse; schema design, partitioning, clustering, performance optimization.
• DevOps/IaC: Terraform with 15+ years of practice; CI/CD (GitHub Actions, Jenkins) with 10+ years; config governance and release management.
• Serverless and events: Design event-driven distributed systems on AWS.
• NoSQL: 2–3 years with DocumentDB including data modeling and performance considerations.
• AI services: AWS Entity Resolution, AWS Comprehend; run custom LLMs on Amazon SageMaker; use LLMs for PII classification.

NICE-TO-HAVE QUALIFICATIONS:

• Data governance automation: 10+ years defining audit, compliance, retention standards and automating governance workflows.
• Table and file formats: Apache Parquet; Apache Iceberg as analytical table format.
• Advanced LLM workflows: RAG and agentic patterns over proprietary data; re-ranking with index/vector store results.
• Multi-cloud exposure: Azure ADF/ADLS, GCP Dataflow/DataProc; FinOps practices for cross-cloud cost control.

OUTCOMES AND MEASURES:

• Engineering excellence: Adherence to processes, standards, and SLAs; reduced defects and non-compliance; fewer recurring issues.
• Efficiency: Faster run times and lower resource consumption with documented cost models and performance baselines.
• Operational reliability: Faster detection, response, and resolution of failures; quick turnaround on production bugs; strong release success.
• Data quality and security: High DQ pass rates, robust lineage, minimal security incidents, and audit readiness.
• Team and customer impact: On-time milestones, clear communication, effective demos, improved satisfaction, and completed certifications/training.

LOCATION AND SCHEDULE:

●      Location: Outside US (OUS).

●      Schedule: Minimum 6 hours of overlap with US time zones.

Key Responsibilities:

● Client Engagement: Serve as the primary point of contact for assigned clients, understanding their unique operation processes and requirements. Build and maintain strong relationships to facilitate successful implementations.

● Project Management: Lead the end-to-end implementation , ensuring projects are delivered on time, within scope, and within budget. Coordinate with cross-functional teams to align resources and objectives.

● Process Analysis and Improvement: Evaluate clients' existing operation workflows, identify inefficiencies, and recommend optimized processes leveraging platform. Utilize process mapping and data analysis to drive continuous improvement.

● Data Analysis: Analyze substantial datasets to ensure accurate configuration and integration. Employ statistical tools and SQL-based queries to interpret data and provide actionable insights.

● Problem Solving: Break down complex problems into manageable components, developing effective solutions in collaboration with clients and internal teams.

● Process Excellence: Advocate for and implement best practices in process management, utilizing methodologies such as Lean Six Sigma to enhance operational efficiency.

● Customer Excellence: Ensure a superior customer experience by proactively addressing client needs, providing training and support, and promptly resolving any issues that arise.

Qualifications:

● Minimum of 3 years of experience in project management, preferably in financial services, software implementation, consulting or analytics.

● Strong analytical skills with experience in data analysis, SQL querying, and handling large datasets.

● Excellent communication and interpersonal skills, with the ability to manage client relationships effectively.

● Demonstrated ability to lead cross-functional teams and manage multiple projects concurrently.

● Proven expertise in financial operation processes and related software solutions is a plus

● Proficiency in developing business intelligence solutions or with low-code tools is a plus

Why Join ?

● Opportunity to work with a cutting-edge financial technology company.

● Collaborative and innovative work environment.

● Competitive compensation and benefits package.

• ● Professional development and growth opportunities.
• 
  
• Must have 3+ years of project/program management experience in Financial Services/Banking/NBFC/Fintech companies only.
• Hands-on proficiency in data analysis and SQL querying, with ability to work on large datasets
• Ability to lead end-to-end implementation projects and manage cross-functional teams effectively.
• Experience in process analysis, optimization, and mapping for operational efficiency.
• Strong client-facing communication and stakeholder management capabilities.
• Good expertise in financial operations processes and workflows with proven implementation experience.

Job Summary:

Seeking an experienced Senior Data Engineer to lead data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. The role involves working on scalable pipelines, large-scale distributed systems, and data lake management.

Core Responsibilities:

· Build and manage high-volume data pipelines using Spark/Databricks.

· Implement ELT frameworks using Azure Data Factory/Synapse Pipelines.

· Optimize large-scale datasets in Delta/Iceberg formats.

· Implement robust data quality, monitoring, and governance layers.

· Collaborate with Data Scientists, Analysts, and Business stakeholders.

Technical Stack:

· Big Data: Apache Spark, Kafka, Hive, Airflow, Hudi/Iceberg

· Cloud: Azure (Synapse, ADF, ADLS Gen2), Databricks, AWS (Glue/S3)

· Languages: Python, Scala, SQL

· Storage Formats: Delta Lake, Iceberg, Parquet, ORC

· CI/CD: Azure DevOps, Terraform (infra as code), Git

Senior Data Engineer (Apache Stack + Databricks/Synapse)

Share cv to

Thirega@ vysystems dot com - WhatsApp - 91Five0033Five2Three

• Knowledge on CAMPUS LAN Architecture and implementation

• Having exposure to Cisco’s latest technologies like SDA, DNAC

• Hand-on experience on designing and implementing Cisco SDA

• Strong Routing & Switching, Cisco SDA, DNAC, Cisco ISE skillset is a must.

• Hands On expertise on Cisco Prime, ISE, Cisco Meraki

• Hands on and troubleshooting SDA

• Experience on L 3

KEY RESPONSIBILITIES

• Design, build, and maintain high-performance, reusable, and reliable code. 
• Can work solely or as a team player for existing and upcoming projects. 
• Ability to work independently and within a team  
• Understanding of the Software Development Life cycle. 
• Good understanding of object-oriented design principles 

SKILLS AND EXPERIENCE

• At least 3 yrs. of experience in .NET Core, Angular & MVC
• At least 3 yrs. of experience in MS SQL Server  
• Expert in Angular, Web APIs
• Knowledge of object-oriented programming, SOLID principles, and design patterns
• Excellent problem-solving skills  
• Must communicate in English effectively  
• Analyze the existing codebase
• Design and implement solutions

We prefer candidates from Kerala.

Coordinate internal resources and third parties/vendors for the flawless execution of projects
Ensure that all projects are delivered on-time, within scope and within budget
Developing project scopes and objectives, involving all relevant stakeholders and ensuring technical feasibility
Ensure resource availability and allocation
Develop a detailed project plan to track progress
Use appropriate verification techniques to manage changes in project scope, schedule and costs
Measure project performance using appropriate systems, tools and techniques
Report and escalate to management as needed
Manage the relationship with the client and all stakeholders
Perform risk management to minimize project risks
Establish and maintain relationships with third parties/vendors
Create and maintain comprehensive project document

Requirements and skills
Great educational background, preferably in the fields of computer science or engineering for technical project managers
Proven working experience as a project administrator in the information technology sector
Solid technical background, with understanding or hands-on experience in software development and web technologies
Excellent client-facing and internal communication skills
Excellent written and verbal communication skills
Solid organizational skills including attention to detail and multi-tasking skills

• Determine the appropriate approach for new software assignments
• Provide solutions to overcome complex problems using the latest technologies
• Developing high-quality and detailed designs, Identify, prioritize and execute tasks in the software development life cycle
• Proactively communicate with the client to get things done within the deadline.
• Develop tools and applications by producing clean, efficient code.
• Automate tasks through appropriate tools and scripting
• Provide high-level maintenance and update of an existing applications to improve functionality and process flow
• Oversee the maintenance of technical documentation of applications as architecture, code, and procedures

• 3+ Experience with RPA software (UiPath) (ONLY/MANDATORY)
• Experience in programming .NET (C# or VB), Java + Node.JS-based programming
• UiPath RPA Implementations and Working Knowledge: Studio, BOTs, Orchestrators, RE Framework and Three E2E Implementations as a Core RPA Developer.
• Skill with scripting languages such as VBA/VBS, Python, NodeJS and JavaScript
• Proficiency with Microsoft Power Platform, SQL and NoSQL database experiences
• Orchestrator configuration and component usage knowledge
• hands on experience of orchestrator queue and load distribution
• Proficiency with JIRA, Jenkins, Github, Artifactory and CI/CD development
• Excellent analytical and problem-solving skills, including business process expertise
• Experience developing design documents for automation processes

• BE/ B.Tech/ MCA/ BCA
• Strong organizational and project management skills.
• Proficiency with fundamental front-end languages such as HTML, CSS, and JavaScript.
• Familiarity with JavaScript frameworks such as Angular JS, React
• Familiarity with database technology such as MySQL, Oracle, and MongoDB.
• Excellent verbal communication skills.
• Good problem-solving skills.
• Attention to detail.