11+ Apache Spark Jobs in Kochi (Cochin) | Apache Spark Job openings in Kochi (Cochin)

Job Summary:

Seeking an experienced Senior Data Engineer to lead data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. The role involves working on scalable pipelines, large-scale distributed systems, and data lake management.

Core Responsibilities:

· Build and manage high-volume data pipelines using Spark/Databricks.

· Implement ELT frameworks using Azure Data Factory/Synapse Pipelines.

· Optimize large-scale datasets in Delta/Iceberg formats.

· Implement robust data quality, monitoring, and governance layers.

· Collaborate with Data Scientists, Analysts, and Business stakeholders.

Technical Stack:

· Big Data: Apache Spark, Kafka, Hive, Airflow, Hudi/Iceberg

· Cloud: Azure (Synapse, ADF, ADLS Gen2), Databricks, AWS (Glue/S3)

· Languages: Python, Scala, SQL

· Storage Formats: Delta Lake, Iceberg, Parquet, ORC

· CI/CD: Azure DevOps, Terraform (infra as code), Git

Senior Data Engineer (Apache Stack + Databricks/Synapse)

Share cv to

Thirega@ vysystems dot com - WhatsApp - 91Five0033Five2Three

SENIOR DATA ENGINEER:

ROLE SUMMARY:

Own the design and delivery of petabyte-scale data platforms and pipelines across AWS and modern Lakehouse stacks. You’ll architect, code, test, optimize, and operate ingestion, transformation, storage, and serving layers. This role requires autonomy, strong engineering judgment, and partnership with project managers, infrastructure teams, testers, and customer architects to land secure, cost-efficient, and high-performing solutions.

RESPONSIBILITIES:

• Architecture and design: Create HLD/LLD/SAD, source–target mappings, data contracts, and optimal designs aligned to requirements.
• Pipeline development: Build and test robust ETL/ELT for batch, micro-batch, and streaming across RDBMS, flat files, APIs, and event sources.
• Performance and cost tuning: Profile and optimize jobs, right-size infrastructure, and model license/compute/storage costs.
• Data modeling and storage: Design schemas and SCD strategies; manage relational, NoSQL, data lakes, Delta Lakes, and Lakehouse tables.
• DevOps and release: Establish coding standards, templates, CI/CD, configuration management, and monitored release processes.
• Quality and reliability: Define DQ rules and lineage; implement SLA tracking, failure detection, RCA, and proactive defect mitigation.
• Security and governance: Enforce IAM best practices, retention, audit/compliance; implement PII detection and masking.
• Orchestration: Schedule and govern pipelines with Airflow and serverless event-driven patterns.
• Stakeholder collaboration: Clarify requirements, present design options, conduct demos, and finalize architectures with customer teams.
• Leadership: Mentor engineers, set FAST goals, drive upskilling and certifications, and support module delivery and sprint planning.

REQUIRED QUALIFICATIONS:

• Experience: 15+ years designing distributed systems at petabyte scale; 10+ years building data lakes and multi-source ingestion.
•  Cloud (AWS): IAM, VPC, EC2, EKS/ECS, S3, RDS, DMS, Lambda, CloudWatch, CloudFormation, CloudTrail.
• Programming: Python (preferred), PySpark, SQL for analytics, window functions, and performance tuning.
• ETL tools: AWS Glue, Informatica, Databricks, GCP DataProc; orchestration with Airflow.
• Lakehouse/warehousing: Snowflake, BigQuery, Delta Lake/Lakehouse; schema design, partitioning, clustering, performance optimization.
• DevOps/IaC: Terraform with 15+ years of practice; CI/CD (GitHub Actions, Jenkins) with 10+ years; config governance and release management.
• Serverless and events: Design event-driven distributed systems on AWS.
• NoSQL: 2–3 years with DocumentDB including data modeling and performance considerations.
• AI services: AWS Entity Resolution, AWS Comprehend; run custom LLMs on Amazon SageMaker; use LLMs for PII classification.

NICE-TO-HAVE QUALIFICATIONS:

• Data governance automation: 10+ years defining audit, compliance, retention standards and automating governance workflows.
• Table and file formats: Apache Parquet; Apache Iceberg as analytical table format.
• Advanced LLM workflows: RAG and agentic patterns over proprietary data; re-ranking with index/vector store results.
• Multi-cloud exposure: Azure ADF/ADLS, GCP Dataflow/DataProc; FinOps practices for cross-cloud cost control.

OUTCOMES AND MEASURES:

• Engineering excellence: Adherence to processes, standards, and SLAs; reduced defects and non-compliance; fewer recurring issues.
• Efficiency: Faster run times and lower resource consumption with documented cost models and performance baselines.
• Operational reliability: Faster detection, response, and resolution of failures; quick turnaround on production bugs; strong release success.
• Data quality and security: High DQ pass rates, robust lineage, minimal security incidents, and audit readiness.
• Team and customer impact: On-time milestones, clear communication, effective demos, improved satisfaction, and completed certifications/training.

LOCATION AND SCHEDULE:

●      Location: Outside US (OUS).

●      Schedule: Minimum 6 hours of overlap with US time zones.

Job Overview

We are looking for a detail-oriented and skilled QA Engineer with expertise in Cypress to join our Quality Assurance team. In this role, you will be responsible for creating and maintaining automated test scripts to ensure the stability and performance of our web applications. You’ll work closely with developers, product managers, and other QA professionals to identify issues early and help deliver a high-quality user experience.

You should have a strong background in test automation, excellent analytical skills, and a passion for improving software quality through efficient testing practices.

Key Responsibilities

• Develop, maintain, and execute automated test cases using Cypress.
• Design robust test strategies and plans based on product requirements and user stories.
• Work with cross-functional teams to identify test requirements and ensure proper coverage.
• Perform regression, integration, smoke, and exploratory testing as needed.
• Report and track defects, and work with developers to resolve issues quickly.
• Collaborate in Agile/Scrum development cycles and contribute to sprint planning and reviews.
• Continuously improve testing tools, processes, and best practices.
• Optimize test scripts for performance, reliability, and maintainability.

Required Skills & Qualifications

• Hands-on experience with Cypress and JavaScript-based test automation.
• Strong understanding of QA methodologies, tools, and processes.
• Experience in testing web applications across multiple browsers and devices.
• Familiarity with REST APIs and tools like Postman or Swagger.
• Experience with version control systems like Git.
• Knowledge of CI/CD pipelines and integrating automated tests (e.g., GitHub Actions, Jenkins).
• Excellent analytical and problem-solving skills.
• Strong written and verbal communication.

Preferred Qualifications

• Experience with other automation tools (e.g., Selenium, Playwright) is a plus.
• Familiarity with performance testing or security testing.
• Background in Agile or Scrum methodologies.
• Basic understanding of DevOps practices.

Required Skills (Technical Competency):

• Solid understanding of Object-Oriented Design and Programming.
• Hands-on experience in Software development in C# [Desktop Applications].
• Strong expertise in OOPS, basic design skills.
• Experience in C#, Winforms is must,
• Knowledge in PRISM, MVVM, LinQ.
• Expertise in multi-threaded programming.
• Strong logical and analytical skills.
• Experience of software development process.
• Expertise in development, testing and debugging skills.
• Knowledge of design patterns and its usage and tools like Visio.
• Experience in UML, SOLID.
• Experience in unit testing frameworks like NUnit.
• Expertise in troubleshooting bugs and finding solutions.
• Strong abstraction, analytical and problem-solving skills.
• Working with multisite teams, Quality conscious and Process Oriented.

Responsibilities:

• Writing and maintaining the modules of Magento 2.X EE websites;
• Write clean, modular, robust code to implement the desired requirements with little or no supervision;
• Work with the QA and Customer Support teams to triage and fix bugs with rapid turnaround;
• Contribute ideas for making the application better and easier to use;
• Create reusable components, which can be configured for different projects;
• Create test plans and perform thorough quality analysis on the code before go live;
• Research on new integration and plug-in capabilities.

• Achieving Stretched Targets In A Result-Focused Environment.
• Making Powerful Sales Presentations Resulting In High Conversion Ratio.
• Handling Objections And Resolving Member/ Customer Conflicts By Constantly Connecting With Them Through Member Meetings.
• Maintaining Strong Client Relationship And High Level Of Customer Service.
• Preparing Pre-Sales Proposals For Prospective Clients.
• Identifying Potential Customers And New Business Opportunities Within And Outside The Venue.
• Keeping Abreast With The Organization's Products And Services.
• Maintaining Consistent Average Productivity.
• Provide Necessary Support In Booking First Time Holiday For New Members And Generate Quality Referrals From Them.
• Qualifications -1 To 5 Years' Quota Carrying Sales Experience
• Experience And Working Knowledge Of CRM Systems
• Demonstrable Track Record Of Over-Achieving Quota
• Strong Written And Verbal Communication Skills

• A minimum of 3 years experience as a Technical Lead in Java., Strong experience in design and implementation of high performing and highly scalable websites/applications
• Strong experience in Spring MVC framework and experience in design patterns, unit testing etc..
• Strong experience in AWS services such as EC2, Lambda, S3, ECS, API Gateway and using tools such as Terraform, Ansible
• Strong experience in Endeca, ATG and BCC, Solr, any Headless CMS
• Considerable experience in databases such as SQL Server/Mongo DB is an added advantage
• Good experience in JavaScript frameworks such as Angular/React/VueJS is an added advantage
• Experience in Adobe AEM is an added advantage
• Creative problem-solving skills and ability to effectively communicate and translate feedback, needs and solutions
• Excellent verbal and written communication skills required

• Provide strategic thinking and leadership pertaining in cloud technologies and building scalable websites
• Provide technical leadership and guidance internally to the team and externally to management
• Communicate effectively across all levels of the organization
• Collaborate within and across teams with strong teamwork orientation
• Help maintain a culture of high quality
• Showcasing excellent work ethic and strong sense of ownership of end result
• Understand the structured software development methodologies including design, development and testing in an Agile environment
• Follow all security practices and identify areas and processes where performance and scalability can be improved
• Make and justify recommendations for improvement., Participate in development of solutions in cases of new technology direction and/or proof of concepts
• Should translate business needs into technical solutions with Strong analytical, conceptual and technical skills
• Support the evaluation, selection, design, development and maintenance of the platforms.

About:

Pangolin is a brand consultancy and creative agency. We help companies create positive change in the world by raising awareness through branding and inspiring through storytelling. We’re growing our team of creatives and strategists. Join us if you’re craving work that makes a real difference in the world, and a work culture that allows you to have a balanced life. 

Role:

Creative Content Writer - B2B/B2C

Description:

Our clients have stories that need telling, we’re looking for storytellers who weave the right narrative for them. Are you comfortable telling stories of social impact, societal change, industrial evolution, and technological innovation? Are you well-versed with corporate communication, writing content for B2B marketing and sales, and enterprise content marketing? Have you worked on a large variety of content types - emails, brochures, flyers, social media content, website copy, sales presentations, webinar decks, articles, and whitepapers? Then you might be a good fit for this job.

Competencies:

• Ability to work on multiple projects simultaneously, effectively juggle diverse writing demands, and deliver on-time
• High retention capacity and the ability to understand client briefs to create effective content strategies.
• An unerring eye for detail and a knack for creativity. 

Skills:

• Ability to interpret data to make recommendations for optimizing online content marketing.
• Unique, engaging, and varied writing styles with good proofreading and research skills.
• Knowledge of keywords, meta tags, SEO, and basic writing terminologies.

Experience:

• Understands marketing KPIs and ROI generated from content marketing programs, including an understanding of how content sources influence audience acquisition pipelines.
• Experienced in writing blogs, product portfolios, reports, web content, brochures, and other marketing collateral for B2B/B2C clients.
• Has a solid foundation in SEO, content strategy, analytics, copywriting, and copy editing.

Location and Type: 

Remote, Full-time

Evaluation:

Portfolios will be checked for expertise in both B2B and B2C domains. Please share only samples of work done by yourself for actual clients. Share only creative work done for an actual company, strictly no SEO blogs or personal blogs. . Shortlisting will be followed by a video interview.