Cutshort logo
Global digital transformation solutions provider.  logo
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
Global digital transformation solutions provider.
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
Peak Hire Solutions's logo

Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka

at Global digital transformation solutions provider.

Agency job
5 - 8 yrs
₹11L - ₹20L / yr
Hyderabad
Skills
PySpark
Apache Kafka
Data architecture
skill iconAmazon Web Services (AWS)
EMR
Apache Airflow
Amazon S3
DMS
Amazon EC2
skill iconScala
Apache Spark
Spark
ETL
CI/CD
skill iconJenkins
Big Data
SQL
Database modeling
OLAP
OLTP
skill iconData Analytics
Automation
Automated testing
Workflow
Data engineering
Data security
HIPAA
skill iconData Science
Datawarehousing
skill iconJava
NOSQL Databases
DynamoDB
Cassandra
skill iconMongoDB
Data governance
Splunk
Dynatrace

JOB DETAILS:

* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka

* Industry: Global digital transformation solutions provider

* Salary: Best in Industry

* Experience: 5-8 years

* Location: Hyderabad

 

Job Summary

We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.


Key Responsibilities

ETL Pipeline Development & Optimization

  • Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
  • Optimize data pipelines for performance, scalability, fault tolerance, and reliability.

Big Data Processing

  • Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
  • Ensure fault-tolerant, scalable, and high-performance data processing systems.

Cloud Infrastructure Development

  • Build and manage scalable, cloud-native data infrastructure on AWS.
  • Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.

Real-Time & Batch Data Integration

  • Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
  • Ensure consistency, data quality, and a unified view across multiple data sources and formats.

Data Analysis & Insights

  • Partner with business teams and data scientists to understand data requirements.
  • Perform in-depth data analysis to identify trends, patterns, and anomalies.
  • Deliver high-quality datasets and present actionable insights to stakeholders.

CI/CD & Automation

  • Implement and maintain CI/CD pipelines using Jenkins or similar tools.
  • Automate testing, deployment, and monitoring to ensure smooth production releases.

Data Security & Compliance

  • Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
  • Implement data governance practices ensuring data integrity, security, and traceability.

Troubleshooting & Performance Tuning

  • Identify and resolve performance bottlenecks in data pipelines.
  • Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.

Collaboration & Cross-Functional Work

  • Work closely with engineers, data scientists, product managers, and business stakeholders.
  • Participate in agile ceremonies, sprint planning, and architectural discussions.


Skills & Qualifications

Mandatory (Must-Have) Skills

  1. AWS Expertise
  • Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
  • Strong understanding of cloud-native data architectures.
  1. Big Data Technologies
  • Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
  • Experience with Apache Spark and Apache Kafka in production environments.
  1. Data Frameworks
  • Strong knowledge of Spark DataFrames and Datasets.
  1. ETL Pipeline Development
  • Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.
  1. Database Modeling & Data Warehousing
  • Expertise in designing scalable data models for OLAP and OLTP systems.
  1. Data Analysis & Insights
  • Ability to perform complex data analysis and extract actionable business insights.
  • Strong analytical and problem-solving skills with a data-driven mindset.
  1. CI/CD & Automation
  • Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
  • Familiarity with automated testing and deployment workflows.

 

Good-to-Have (Preferred) Skills

  • Knowledge of Java for data processing applications.
  • Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
  • Familiarity with data governance frameworks and compliance tooling.
  • Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
  • Exposure to cost optimization strategies for large-scale cloud data platforms.

 

Skills: big data, scala spark, apache spark, ETL pipeline development

 

******

Notice period - 0 to 15 days only

Job stability is mandatory

Location: Hyderabad

Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer

F2F Interview: 14th Feb 2026

3 days in office, Hybrid model.

 


Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

Mumbai
1 - 5 yrs
₹9L - ₹13L / yr
Sales

Strong Partner Sales Profile (Cloud / Enterprise Ecosystem)

Mandatory (Total Experience): Must have 1+years of B2B sales experience in Cloud / IT Services OR partner / enterprise solution sales exposure within BFSI, Manufacturing, or Pharmaceutical sectors

Mandatory (Experience 1 – Partner / Channel Sales): Must have experience working with channel partners, system integrators, resellers, or strategic alliances

Mandatory (Experience 2 – Partner-Led Deal Execution): Must have supported or driven partner-led sales cycles, including requirement alignment, proposal coordination, and deal closure

Mandatory (Communication & Stakeholder Management): Must be able to coordinate across partners, internal teams, and end customers effectively

Mandatory (Education Background): Must have a technical or IT-oriented education background —BE / BSc IT / MSc IT is mandatory

Read more
South Asia’s Leading Audio Streaming Service
South Asia’s Leading Audio Streaming Service
Agency job
via Qrata by Prajakta Kulkarni
Navi Mumbai
2 - 6 yrs
₹10L - ₹15L / yr
Content Writing
Telugu Content writers
Music
Music Content
Content Strategy

JOB TITLE: ASSOCIATE MANAGER - TELUGU PROGRAMMING

LOCATION: NAVI MUMBAI, INDIA (Remote)


RESPONSIBILITIES:

 Assist in managing the daily Telugu programming and curation strategy. Working knowledge of

Kannada + Malayalam is a big plus

 Basic data analysis and number crunching is an important part of this role

 Work with the Programming Teams as well as other cross-functional teams to achieve various

organizational objectives

 Act as a liaison between various departments to ensure the on time and orderly flow of work

 Curate and maintain a substantial number of varied playlists based on different Moods, Genres

and Activities.

 Think of creative ideas & concepts for Festivals, events & Trending Topics

 Stay on top of new music releases

 Help find ways to organize, facilitate, and improve general programming processes

 Create and update departmental calendar and project status

 Keep entire team informed on the current status of all jobs in progress via status meetings

 Keep abreast of all work in the Programming Department and follow up on due dates that were

established for copy, artwork, playlists and other deliverables


REQUIREMENTS:

 Bachelor’s/ Master’s degree

 Proficient in Telugu – reading and understanding

 3-5 years’ experience in the music industry, ideally programming for music services, working

with a broad range of music content

 Basic data analysis and number crunching

 Passionate about music and have deep knowledge of music culture (Telugu).

 Highly organized, creative self-starter with experience working on multiple projects at the same

time

 Care about attention to detail and organization, have a team spirit, strong work ethic, and a

sense of humor

 Exceptional time management and prioritization skills

 Out-of-the-box thinking and ideas that can significantly contribute to Company’s success.


BENEFITS AND PERKS:


We blur work and play, and you get all the perks of a global company. You will get to

work with a dynamic group of entrepreneurs, who are delivering results and working zealously to

make a difference in the way the world experiences music.

Read more
Top Management Consulting Company
Top Management Consulting Company
Agency job
via People First Consultants by Naveed Mohd
Gurugram, Bengaluru (Bangalore), Chennai
2 - 9 yrs
₹9L - ₹27L / yr
DevOps
Microsoft Windows Azure
gitlab
skill iconAmazon Web Services (AWS)
Google Cloud Platform (GCP)
+15 more
Greetings!!

We are looking out for a technically driven  "ML OPS Engineer" for one of our premium client

COMPANY DESCRIPTION:
This Company is a global management consulting firm. We are the trusted advisor to the world's leading businesses, governments, and institutions. We work with leading organizations across the private, public and social sectors. Our scale, scope, and knowledge allow us to address


Key Skills
• Excellent hands-on expert knowledge of cloud platform infrastructure and administration
(Azure/AWS/GCP) with strong knowledge of cloud services integration, and cloud security
• Expertise setting up CI/CD processes, building and maintaining secure DevOps pipelines with at
least 2 major DevOps stacks (e.g., Azure DevOps, Gitlab, Argo)
• Experience with modern development methods and tooling: Containers (e.g., docker) and
container orchestration (K8s), CI/CD tools (e.g., Circle CI, Jenkins, GitHub actions, Azure
DevOps), version control (Git, GitHub, GitLab), orchestration/DAGs tools (e.g., Argo, Airflow,
Kubeflow)
• Hands-on coding skills Python 3 (e.g., API including automated testing frameworks and libraries
(e.g., pytest) and Infrastructure as Code (e.g., Terraform) and Kubernetes artifacts (e.g.,
deployments, operators, helm charts)
• Experience setting up at least one contemporary MLOps tooling (e.g., experiment tracking,
model governance, packaging, deployment, feature store)
• Practical knowledge delivering and maintaining production software such as APIs and cloud
infrastructure
• Knowledge of SQL (intermediate level or more preferred) and familiarity working with at least
one common RDBMS (MySQL, Postgres, SQL Server, Oracle)
Read more
I can't disclose client name
I can't disclose client name
Agency job
via Innovalus Technologies by Jebarani Elshi
Chennai
5 - 10 yrs
₹3L - ₹15L / yr
Windows Presentation Foundation(WPF)
skill iconC#
skill icon.NET
Key Responsibility:
 
 • Be part of a SCRUM team, do the necessary development in WPF and support/coach the team members in WPF. • Internally motivated, able to work proficiently both independently and in a team environment. • Identify options for potential solutions and assess them for technical suitability. • Adhere to high-quality development principles while delivering solutions. • Participate in peer-reviews of solution designs and related code. • Strong communication skills with both internal team members and external business stakeholders. 
 
Skills Set / Technical Expertise: - 
 • Proficient in C#/.Net, Microsoft SQL, Web API, Entity Framework & Linq • Should have development experience in C# using .Net or .Net Core • Proficient in Microservices, Web API, Entity Framework, LINQ, SQL • Preferable to have working experience in Azure Cloud, Git, CI/CD • Must be strong in Object Oriented Programming, Design Patterns • Excellent Oral & written communication skills Skills 
 
Set / Technical Expertise: - 
• Preferable to have working experience in WPF With MVVM pattern development and data binding • Must be strong in XAML development, Resources, Styles and control templates • Preferable to have working experience in Azure Cloud, Git, CI/CD.
Read more
Founded in 2018,mission to make crypto accessible to everyon
Founded in 2018,mission to make crypto accessible to everyon
Agency job
via LeanHR by YoshitaArora
Remote only
3 - 14 yrs
₹20L - ₹40L / yr
skill iconRuby
skill iconRuby on Rails (ROR)
skill iconAmazon Web Services (AWS)
#Responsibilities:

-Design and build advanced applications from scratch keeping scalability and performance in mind.

-You’ll be working on our P2P and cryptocurrency exchange platforms to build more features around it.

-Collaborating with various stakeholders to determine software requirements

being a financial product, you’ll be involved in building tools to make our user’s data more secure.

-You will be working on blockchain technologies

-You will help maintain code quality and organization.

-Should be able to work with juniors; help and guide them as needed.

-We expect you to learn about new software technologies, and proactively look for ways to make them better.

#You'll be a good fit if:

-You’ve got 3+ years of development experience and an expert in the Ruby and Rails framework.

-You’re a Hands-on expert who can code, architect/design systems, and understand building highly available and scalable microservices on cloud infrastructure.

-You have a very good understanding of Containers and Container Orchestration Technology such as Docker / Kubernetes and AWS Cloud Platform exposure.

-You’ve hands-on experience in using DevOps tools such as Newrelic, Jenkins, Grafana, ELK, etc

-You’re an expert in writing background tasks, test-driven development.

-You enjoy collaborating with the front-end developers, DevOps, and other team members to establish objectives and design more functional, cohesive codes to enhance the user experience.

-You’re committed to quality, and your code solves more problems than it creates ;)
Read more
Semperfi Solution
at Semperfi Solution
1 recruiter
Ambika Jituri
Posted by Ambika Jituri
Bengaluru (Bangalore)
4 - 8 yrs
₹12L - ₹30L / yr
skill icon.NET
ASP.NET MVC
AJAX
LINQ
Web API
+2 more

NET Profile 

 

DESIRED SKILLS

 

·         Experience in Microsoft technologies (Microsoft Dot Net, C#, http://asp.net/" target="_blank">ASP.NET, MVC, Microsoft SQL, .NET Core, Web API, WCF, Bootstrap, JavaScript, Angular JS, Web API, Web services, Rest API)

 

·         Design a complex architecture landscape involving API integrations.

 

·         Hands-on experience in ASP.Net, C#, MVC, WCF, Restful services, Web API, AngularJS, Entity Framework

 

·         Strong analytical & problem-solving skills

 

·         Must have excellent oral & email communication skills.

 

·         Strong analytical and problem-solving skills

 

·         Must have a positive attitude and a yearning to get things done and demonstrate ownership of deliverables.

 

NICE TO HAVE SKILLS

 

·         Knowledge in Microsoft Azure.

 

·         Good Command on Agile Development & Azure DevOps.

 

·         Knowledge in Angular / React / Node JS

 

·       Mainly they are looking for:

 

C#

http://asp.net/" target="_blank">ASP.NET MVC,

AJAX,

LINQ,

jQuery,

WebApi.

 
 
KINDLY REVERT WITH YOUR UPDATED RESUME
 
THANK YOU 
 
Read more
Propellor.ai
at Propellor.ai
5 candid answers
1 video
Kajal Jain
Posted by Kajal Jain
Remote only
1 - 4 yrs
₹5L - ₹15L / yr
skill iconPython
SQL
Spark
Hadoop
Big Data
+2 more

Big Data Engineer/Data Engineer


What we are solving
Welcome to today’s business data world where:
• Unification of all customer data into one platform is a challenge

• Extraction is expensive
• Business users do not have the time/skill to write queries
• High dependency on tech team for written queries

These facts may look scary but there are solutions with real-time self-serve analytics:
• Fully automated data integration from any kind of a data source into a universal schema
• Analytics database that streamlines data indexing, query and analysis into a single platform.
• Start generating value from Day 1 through deep dives, root cause analysis and micro segmentation

At Propellor.ai, this is what we do.
• We help our clients reduce effort and increase effectiveness quickly
• By clearly defining the scope of Projects
• Using Dependable, scalable, future proof technology solution like Big Data Solutions and Cloud Platforms
• Engaging with Data Scientists and Data Engineers to provide End to End Solutions leading to industrialisation of Data Science Model Development and Deployment

What we have achieved so far
Since we started in 2016,
• We have worked across 9 countries with 25+ global brands and 75+ projects
• We have 50+ clients, 100+ Data Sources and 20TB+ data processed daily

Work culture at Propellor.ai
We are a small, remote team that believes in
• Working with a few, but only with highest quality team members who want to become the very best in their fields.
• With each member's belief and faith in what we are solving, we collectively see the Big Picture
• No hierarchy leads us to believe in reaching the decision maker without any hesitation so that our actions can have fruitful and aligned outcomes.
• Each one is a CEO of their domain.So, the criteria while making a choice is so our employees and clients can succeed together!

To read more about us click here:
https://bit.ly/3idXzs0" target="_blank">https://bit.ly/3idXzs0

About the role
We are building an exceptional team of Data engineers who are passionate developers and wants to push the boundaries to solve complex business problems using the latest tech stack. As a Big Data Engineer, you will work with various Technology and Business teams to deliver our Data Engineering offerings to our clients across the globe.

Role Description

• The role would involve big data pre-processing & reporting workflows including collecting, parsing, managing, analysing, and visualizing large sets of data to turn information into business insights
• Develop the software and systems needed for end-to-end execution on large projects
• Work across all phases of SDLC, and use Software Engineering principles to build scalable solutions
• Build the knowledge base required to deliver increasingly complex technology projects
• The role would also involve testing various machine learning models on Big Data and deploying learned models for ongoing scoring and prediction.

Education & Experience
• B.Tech. or Equivalent degree in CS/CE/IT/ECE/EEE 3+ years of experience designing technological solutions to complex data problems, developing & testing modular, reusable, efficient and scalable code to implement those solutions.

Must have (hands-on) experience
• Python and SQL expertise
• Distributed computing frameworks (Hadoop Ecosystem & Spark components)
• Must be proficient in any Cloud computing platforms (AWS/Azure/GCP)  • Experience in in any cloud platform would be preferred - GCP (Big Query/Bigtable, Pub sub, Data Flow, App engine )/ AWS/ Azure

• Linux environment, SQL and Shell scripting Desirable
• Statistical or machine learning DSL like R
• Distributed and low latency (streaming) application architecture
• Row store distributed DBMSs such as Cassandra, CouchDB, MongoDB, etc
. • Familiarity with API design

Hiring Process:
1. One phone screening round to gauge your interest and knowledge of fundamentals
2. An assignment to test your skills and ability to come up with solutions in a certain time
3. Interview 1 with our Data Engineer lead
4. Final Interview with our Data Engineer Lead and the Business Teams

Preferred Immediate Joiners

Read more
A Pre-series A funded FinTech Company
A Pre-series A funded FinTech Company
Agency job
via GoHyre by Avik Majumder
Remote, Bengaluru (Bangalore)
3 - 7 yrs
₹20L - ₹25L / yr
skill iconNodeJS (Node.js)
skill iconMongoDB
NOSQL Databases

Requirements:

  • 3+ years’ experience under your belt developing performance-critical applications that run in a production environment using Node.Js
  • Expertise in Database modeling, preferably relational DB, spotting DB bottlenecks, and writing optimized DB queries.
  • Expertise in common Cloud technologies, AWS/GCP Knowledge of CI/CD tools and workflows.
  • Knowledge in one or more of the following: NoSQL (especially, but not only Redis), Queueing system experience (Kafka, RabbitMQ, ActiveMQ, MSMQ)
  • Good stakeholder management skills, with an ability to build trust and rapport with internal and external stakeholders
  • It is good to have knowledge of Kubernetes and Docker and related deployment technologies.

Roles & Responsibilities:

  • Write well-designed, testable, efficient, and high-quality code.
  • Design (LLD & HLD) and architect technical solutions for the business problems of a large-scale system.
  • Prepare and produce releases of software components.
  • Support continuous improvement by investigating alternatives and technologies.
  • Contribute to all phases of the development lifecycle
  • We believe leadership comes through ownership and so we love people who take ownership of their work and lead by example
Read more
Tailwebs Technology
at Tailwebs Technology
4 recruiters
Sabhyata Sinha
Posted by Sabhyata Sinha
Bengaluru (Bangalore)
1 - 4 yrs
₹3.6L - ₹6.6L / yr
skill iconAndroid Development
android
skill iconKotlin
skill iconJava
SDK
+1 more

Android Developer JD @Tailwebs

We are looking for an Android Developer who possesses a passion for pushing

mobile technologies to the limits and will work with our team of talented engineers

to design and build the next generation of our mobile applications.

Responsibilities:

  • Design, develop, modify, and implement mobile applications using Android.

  • Implement creative and usable user interfaces.

  • Collaborate with cross-functional teams to define, design, and ship new features

  • Work on bug fixing and improving application performance.

  • Continuously discover, evaluate, and implement new technologies to maximize development

    efficiency.

Required Technical Skills:

  • Object-Oriented Programming Skills

  • Strong in Core Java, and Android SDK implementation & good knowledge on Web Services,

    Web APIs & JSON- Kotlin

  • Strong debugging & troubleshooting skills.

  • Good understanding of Android frameworks like Activity Manager, Resource Manager, Location

    Manager, and Notification Manager

  • 1 to 3 years, fresher’s with proven development skills considered.

Job Type: Full-time
Salary offered: INR 250,000 – INR 600,000

Read more
Topprs Educational sales
Topprs Educational sales
Agency job
via Vfind Solutions Pvt Ltd by Kashif Faizu
Remote, Bengaluru (Bangalore)
3 - 7 yrs
₹5L - ₹6L / yr
Sales
Business Development
Telesales
Sales management
Field Sales
+1 more

Team Manager – Direct Sales, Toppr.com Location: PAN India https://toppr.breezy.hr/

About Us

Toppr is an adaptive after-school learning app for classes 5th to 12th. Our vision is to make learning personalized using technology. We believe each child is unique with distinct learning needs.

We are building Toppr as an adaptive platform that can increase engagement, optimize learning and improve outcomes. We have the widest syllabus coverage in India with over 20 subjects, 20 boards and 50 exams. Over 30 lakh students use the learning app.

Toppr was started in 2013 by IIT Bombay alumni ZishaanHayath and HemanthGoteti and is backed by venture capital firms SAIF, Helion and Eight Roads.

If the future of education interests you, we would love to talk about what you can bring to our team.

About the Team

We are the face of Toppr for all our students and their parents.

We help them understand how to leverage Toppr platform for their learning We strive to take Toppr to every single household in India; and then the world.

Role & Responsibilities

  • Provide the team with a vision and objectives; manage key performance indicators.
  • Set the monthly, weekly, daily targets for the team and ensure that the team targets are achieved.
  • Handling and guiding team of Academic Counselor to achieve monthly sales target.
  • Maintain the sales report of the team.
  • Motivating & mentoring team to achieve & exceed targets
  • Design & develop business models as per the location and market situation.
  • Conducting weekly reviews for performance & training.
  • Identifying the areas of improvements & KPI’s
  • Involve team in calling for negotiation and objection handling as and when required.

 

 

 

 

Team Managers at Toppr

  • Are dynamic professionals who drive their team to exceed business targets
  • Possess sound knowledge and understanding of consumer (students and parents) behavior
  • Thrive in a fast paced ambiguous work environment
  • Have excellent communication skills in English and the regional language

Qualification & Skills

  • MBA or Graduate with at least 5 years B2C sales experience
  • Minimum 1 years’ experience of managing a team of 10 – 20 sales executives
  • Experience in to Data Analysis; proficient with using Excel

 

 

 

 

Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos