Cutshort logo
Global digital transformation solutions provider.  logo
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
Global digital transformation solutions provider.
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
Global digital transformation solutions provider. 's logo

Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka

at Global digital transformation solutions provider.

Agency job
5 - 8 yrs
₹11L - ₹20L / yr
Hyderabad
Skills
PySpark
Apache Kafka
Data architecture
Amazon Web Services (AWS)
EMR
Apache Airflow
Amazon S3
DMS
Amazon EC2
Scala
Apache Spark
Spark
ETL
CI/CD
Jenkins
Big Data
SQL
Database modeling
OLAP
OLTP
Data Analytics
Automation
Automated testing
Workflow
Data engineering
Data security
HIPAA
Data Science
Datawarehousing
Java
NOSQL Databases
DynamoDB
Cassandra
MongoDB
Data governance
Splunk
Dynatrace

JOB DETAILS:

* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka

* Industry: Global digital transformation solutions provider

* Salary: Best in Industry

* Experience: 5-8 years

* Location: Hyderabad

 

Job Summary

We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.


Key Responsibilities

ETL Pipeline Development & Optimization

  • Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
  • Optimize data pipelines for performance, scalability, fault tolerance, and reliability.

Big Data Processing

  • Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
  • Ensure fault-tolerant, scalable, and high-performance data processing systems.

Cloud Infrastructure Development

  • Build and manage scalable, cloud-native data infrastructure on AWS.
  • Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.

Real-Time & Batch Data Integration

  • Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
  • Ensure consistency, data quality, and a unified view across multiple data sources and formats.

Data Analysis & Insights

  • Partner with business teams and data scientists to understand data requirements.
  • Perform in-depth data analysis to identify trends, patterns, and anomalies.
  • Deliver high-quality datasets and present actionable insights to stakeholders.

CI/CD & Automation

  • Implement and maintain CI/CD pipelines using Jenkins or similar tools.
  • Automate testing, deployment, and monitoring to ensure smooth production releases.

Data Security & Compliance

  • Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
  • Implement data governance practices ensuring data integrity, security, and traceability.

Troubleshooting & Performance Tuning

  • Identify and resolve performance bottlenecks in data pipelines.
  • Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.

Collaboration & Cross-Functional Work

  • Work closely with engineers, data scientists, product managers, and business stakeholders.
  • Participate in agile ceremonies, sprint planning, and architectural discussions.


Skills & Qualifications

Mandatory (Must-Have) Skills

  1. AWS Expertise
  • Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
  • Strong understanding of cloud-native data architectures.
  1. Big Data Technologies
  • Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
  • Experience with Apache Spark and Apache Kafka in production environments.
  1. Data Frameworks
  • Strong knowledge of Spark DataFrames and Datasets.
  1. ETL Pipeline Development
  • Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.
  1. Database Modeling & Data Warehousing
  • Expertise in designing scalable data models for OLAP and OLTP systems.
  1. Data Analysis & Insights
  • Ability to perform complex data analysis and extract actionable business insights.
  • Strong analytical and problem-solving skills with a data-driven mindset.
  1. CI/CD & Automation
  • Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
  • Familiarity with automated testing and deployment workflows.

 

Good-to-Have (Preferred) Skills

  • Knowledge of Java for data processing applications.
  • Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
  • Familiarity with data governance frameworks and compliance tooling.
  • Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
  • Exposure to cost optimization strategies for large-scale cloud data platforms.

 

Skills: big data, scala spark, apache spark, ETL pipeline development

 

******

Notice period - 0 to 15 days only

Job stability is mandatory

Location: Hyderabad

Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer

F2F Interview: 14th Feb 2026

3 days in office, Hybrid model.

 


Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

Deqode
at Deqode
1 recruiter
Shraddha Katare
Posted by Shraddha Katare
Mumbai
2 - 4 yrs
₹4L - ₹8.5L / yr
MERN Stack
Redis
MongoDB
Docker
React.js
+3 more

Profile: MERN Stack Developer

📍 Location: Andheri East, Mumbai

🏤 Work Mode: 5 Days WFO

⏰ Experience: 2+ Years

(Only immediate joiners & candidates who have completed notice period)


What We're Looking For:

✅ 2+ years of MERN stack development experience

✅ MongoDB - database design and complex queries

✅ Express.js - server-side application development

✅ React.js - component-based UI development

✅ Node.js - backend JavaScript runtime

✅ Kafka - event streaming and messaging

✅ Docker - containerization and deployment

✅ Redis - caching and session management

✅ RESTful API design and integration

Read more
Variyas Labs Pvt. Ltd.
at Variyas Labs Pvt. Ltd.
2 candid answers
Sales Team
Posted by Sales Team
Delhi, Gurugram, Noida
4 - 6 yrs
₹15L - ₹24L / yr
OpenStack
Linux/Unix
openshift
Kubernetes

Job Overview:

We are looking for a seasoned OpenStack Administrator with strong expertise in managing large-scale production environments. The ideal candidate should have hands-on experience with Linux, Kubernetes, and OpenShift, and be capable of performing routine maintenance, upgrades, and troubleshooting in complex cloud infrastructures.


The candidate must also be comfortable working with Red Hat support, managing escalations, and communicating effectively with both internal teams and external clients.


Key Skills & Qualifications:

  • Proven experience managing OpenStack infrastructure in production.
  • Strong proficiency in Linux system administration (RHEL/CentOS preferred).
  • Hands-on experience with Kubernetes and OpenShift.
  • Experience with system monitoring, log management, and troubleshooting tools.
  • Familiarity with RH support portal, managing cases, and following up on resolutions.
  • Excellent problem-solving skills and ability to work under pressure.
  • Strong client communication skills and ability to articulate technical issues clearly.
  • Proven ability to work in and manage large-scale production environments.


Candidates with OpenStack certification will be preferred.

Read more
An Indian energy and power company.
An Indian energy and power company.
Agency job
via Jobdost by Sathish Kumar
Ahmedabad
3 - 10 yrs
₹5L - ₹15L / yr
Network Security
Cyber Security
IT security
Web application security
Torrent Power is an Indian energy and power company, having interests in power generation, transmission, distribution and manufacturing and supply of power cables.

Security (AM/Executive)

• To design the security infrastructure / policies for the organisation, implement & monitor the same
• To ensure security compliance with respect to recommendations received from government agencies like CEA, NCIIPC
• Design, review, implement & monitor IT security related controls as part of Internal
• Controls, IFC, ERM
• ISMS certification (ISO 27001) for IT systems; this will include preparation and periodic review of policies and SOPs, regular trainings and maintaining records in prescribed formats
• Conducting internal security audit and generating reports by deploying VA tools
• Periodic security/VAPT audits and implementation of the findings
• IT security related new initiatives like - Security Operations Centre (SOC), Security Information and Event Management (SIEM), cloud security, EMM-enterprise mobility management
• Creating IT Security awareness within the organisation
Read more
CEDRETO MARKETING PRIVATE LIMITED
Ankit Agarwal
Posted by Ankit Agarwal
Bengaluru (Bangalore)
3 - 5 yrs
₹8L - ₹10L / yr
iOS App Development
Objective C
Swift
Xcode
Model-View-View-Model (MVVM)
+1 more

About us:

Do Your Thng (DYT) is a creator community platform with an overarching vision to democratise influence and build a thriving creator community. We offer two solutions. An app for creators that helps them make a living out of content creation. And a self-serve web desk module for marketers to execute influencer campaigns in a quick, frictionless, targeted, and economically attractive manner.

 

  • 2-4years of experience as an iOS developer. 
  • Must have Working Experience in a product based organization at least 18 months.
  • Good Communication skills must. 

 

  • Understanding the project requirement intuitively and providing proper time estimation.-Always read for troubleshooting in all important projects. ● Experience in accessing data as JSON/XML using REST API. 
  • Candidate should have good knowledge of iOS, Swift, xCode and Git. ● Ability to build complex iOS standalone client applications & user interfaces, design & implementation. 
  • Experience with iOS Frameworks such as UIKit, Auto-layout using storyboards, etc. ● Knowledge of Apple's design principals, application interface guidelines, and experience with 
  • AutoLayout. 
  • Good communication skills with the ability to work in a highly-collaborative, agile team 
  • Strong analytical and problem-solving skills 
  • Flexibility to learn, and work with new technologies
Read more
Indiamart Intermesh Limited
NCR (Delhi | Gurgaon | Noida)
2 - 4 yrs
₹5L - ₹6L / yr
Sales
Business Development
Acquisition
Customer Acquisition
Branch Sales Manager About IndiaMART:IndiaMART is India's largest online marketplace, connecting buyers with suppliers. The online channel focuses on providing a platform for buyers, who can be SMEs, large enterprises as well as individuals. Buyers typically gain access to a wider marketplace; diverse portfolios of quality products to choose from and tap a one-stop-shop which caters to all their specific requirements, thereby aiding the discerning buyer make well-informed choices.IndiaMART has over 3000+ employees located across 85+ offices in the country, and with customers spanning across 200 nations. Profile :This is a Middle Management level position for the new client acquisition division. This division has around 1000 people and a large network spread across the country. The job encompasses complete P&L responsibility of a Branch comprising of 10 - 15 Direct Sales Resources. He will be responsible to plan, execute and monitor the sales operations with the objective of attaining accelerated growth of new client acquisition in line with the laid down processes. The role will involve Managing direct sales force in the Branch, increasing the productivity level & overall efficiency of the sales team and ensuring target achievement as per the defined targets.Key Responsibility Area :1. Monitoring and analysis of key data of the region allocated including sales figures, sales personnel, productivity ratios, clients work status and receivables. 2. Identifying the opportunity & problem areas, preparing the action plan for achieving the key sales figures and presenting the Branch Sales reports. Actively driving and ensuring sale target achievement, management of sales support function and timely work completion of customers. 3. Responsible for team training, hand holding and ensuring high retention of team.4. Ensure implementation of training program for skill development5. Record performance metrics and monitoring key parameters to increase the overall productivity of the sales personnel. 6. Drive performance incentive structure and sales promotion schemes7. Collecting market feedback and provide periodic updates to the management for formulation of policy & strategy Education: Btech/MBAExperience: 2+ years of experience in a sale driven environment
Read more
IT Service Based Company
IT Service Based Company
Agency job
via Myna Solutions by Venkat B
Hyderabad
3 - 6 yrs
₹3L - ₹15L / yr
Java
J2EE
Spring Boot
Hibernate (Java)

JD for Java Developer Position (3-5 years)

Notice Period: 10-20 days or within 1 month

1. Excellent Java skills
2. Hands-on working experience with Spring, Hibernate, GIT, Jenkins
3. Strictly adhere to coding standards & unit testing
4. Excellent analytical skills and communication skills are a must
5. Knowledge of MySQL or any other DBMS

Added Advantage
1. ElasticSearch
2. AWS services like lambda, redis, graphql etc
3. Docker

Read more
They are a new lender- the first API delivers Real-time. FA1
They are a new lender- the first API delivers Real-time. FA1
Agency job
via Multi Recruit by Ragul Ragul
Mohali
4 - 8 yrs
₹25L - ₹30L / yr
NodeJS (Node.js)
Node Js Developer
Backend Developer
  • You’ve been building web applications professionally for 4+ years
  • You’re proficient in NodeJS, TypeScript, MySQL, and MongoDB
  • You possess strong OOP and design pattern knowledge
  • Familiar with modern engineering practices: Code Reviews, Continuous Deployment, Automated Testing, etc
  • You write understandable, testable code with an eye towards maintainability and experience with TDD (Test Driven Development)
  • You have the ability to build RESTful APIs
  • Explaining complex technical concepts to designers, support team, and fellow engineers is not a problem for you
  • Well versed with the fundamentals of computer science and distributed systems
  • Frontend experience; (Good to have) Angular
  • Start-up experience, preferably a tech start-up
  • Open-Source contributor
  • You have experience with other programming languages -- e.g., Python, Ruby, PHP, Go, C, etc.
  • Passionate about/experienced with open source and developer tools
  • You have a bachelor's degree in Computer Science, Engineering, or related field, or equivalent training, fellowship, or work experience


Qualifications

  • BE/ B. Tech/ MSc/MTech
  • Candidate should possess the degree from a recognized University/ Institution only

 

 

Read more
Locofast
NCR (Delhi | Gurgaon | Noida)
2 - 5 yrs
₹10L - ₹15L / yr
NodeJS (Node.js)
React.js
Javascript
Fullstack Developer
We are a VC-funded, fast-growing supply-chain startup. In the last year or so, we have grown rapidly both in our scale and our team. We are a group of highly passionate people who are always willing to go the extra mile, and are not afraid of taking risks.

We are now looking to onboard an exceptional Fullstack Developer on our team. They will be an integral part of our technology function and work on challenging, cutting-edge problems, and contribute to the solution design and development for various software systems and subsystems. They will develop new and interesting products for our B2B users and will work with the best minds in the industry.

Please apply if you have:

  • 2-5 years of work experience, preferably in a startup
  • An engineering degree preferably, from a top-tier school (or equivalent experience)
  • Hands-on experience in developing end-to-end applications using Node.Js, ReactJs,
  • Knowledge of HTML/CSS, web standards, best practices
  • Experience of working with databases - MySQL, MongoDB
  • Good understanding of asynchronous request handling, partial page updates, and AJAX
  • An experience in writing clean, semantic code with cross-browser compatibility in mind
  • Proficient understanding of code versioning tools, such as Git/SVN
  • Excellent coding and debugging skills
  • A passion and the ability to make beautiful, smooth, delightful web experiences
  • A Will-Do attitude

We have:

  • A team of aces, from top-tier schools and startups, to work with
  • A fast-paced, data-driven, and fun work environment
  • A huge appetite for new, disruptive, creative and implementable ideas
  • An open work culture

We offer:

  • Competitive salary
  • Great working environment, opportunity to work directly with the founders on very exciting real-life problems.
  • Opportunity to grow really fast.
  • Opportunity to hire a team & mentor them in the future.
  • ESOPs

Hiring process:

  • Shortlist
  • Intro call of 15min
  • 1-2 Tech rounds
  • HR round
  • Offer rollout

If this is something that interests you, let's talk.
Read more
Bengaluru (Bangalore)
3 - 8 yrs
₹5L - ₹14L / yr
NodeJS (Node.js)
Javascript
MongoDB
Express
MEAN stack
Your Mission As a Senior Node.js developer, you will be working directly with our team towards the development of our product, in Node.js, Angularjs, MySQL and more. You are expert in Core JavaScript development and have a sharp knowledge on how to use the object model of JavaScript at best to make your code tidy and reusable. You will use MEAN Stack (MongoDB, Express.js, Angular.js, Node.js) along with Redis for queue management and MySQL for fast transactions. You have a strong product vision, to such an extent that you will be able to turn one line of requirements into a fully fledged feature. We deal with million of billions of records. Think “big data”. You will ensure that your code and queries are properly optimized so as to keep the app quick and responsive – and the servers up! We are deeply versed in reusability (so that we never have to reinvent the wheel). You know about the best Node.js plugins that you can find through npm to fulfill your needs, and you will build reusable components that you will leverage across the codebase. We love to take a step back from our day-to-day activities to improve our product. You will be part of this ongoing process. Your Profile You are first and foremost passionate about product development and digital technologies. You must have completed successful projects with a strong Node.js code Required Skills: core JavaScript experience; Node.js, APIs and modules; experience in handling databases and unit testing is also important. Competencies: excellent bug hunter; good judgement and decision making skills; responsible and motivated; passionate about learning new things. Experience in localisation will be appreciated. Motivated and ready to working in a truly international setting with people from all around the world.
Read more
zeiko technoloiges
Jio Petrol
Posted by Jio Petrol
Udaipur
1 - 1 yrs
₹6L - ₹12L / yr
Search Engine Optimization (SEO)
Digital Marketing
Search Engine Marketing (SEM)
zeiko technologies is largest platfom of digital Marketing service in udaipur , seo experts in udaipur , contentn Marketing service in udaipur , Email-Marketing in udaipur
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos