Cutshort logo
Global digital transformation solutions provider.  logo
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
Global digital transformation solutions provider.
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
Peak Hire Solutions's logo

Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka

at Global digital transformation solutions provider.

Agency job
5 - 8 yrs
₹11L - ₹20L / yr
Hyderabad
Skills
PySpark
Apache Kafka
Data architecture
skill iconAmazon Web Services (AWS)
EMR
Apache Airflow
Amazon S3
DMS
Amazon EC2
skill iconScala
Apache Spark
Spark
ETL
CI/CD
skill iconJenkins
Big Data
SQL
Database modeling
OLAP
OLTP
skill iconData Analytics
Automation
Automated testing
Workflow
Data engineering
Data security
HIPAA
skill iconData Science
Datawarehousing
skill iconJava
NOSQL Databases
DynamoDB
Cassandra
skill iconMongoDB
Data governance
Splunk
Dynatrace

JOB DETAILS:

* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka

* Industry: Global digital transformation solutions provider

* Salary: Best in Industry

* Experience: 5-8 years

* Location: Hyderabad

 

Job Summary

We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.


Key Responsibilities

ETL Pipeline Development & Optimization

  • Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
  • Optimize data pipelines for performance, scalability, fault tolerance, and reliability.

Big Data Processing

  • Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
  • Ensure fault-tolerant, scalable, and high-performance data processing systems.

Cloud Infrastructure Development

  • Build and manage scalable, cloud-native data infrastructure on AWS.
  • Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.

Real-Time & Batch Data Integration

  • Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
  • Ensure consistency, data quality, and a unified view across multiple data sources and formats.

Data Analysis & Insights

  • Partner with business teams and data scientists to understand data requirements.
  • Perform in-depth data analysis to identify trends, patterns, and anomalies.
  • Deliver high-quality datasets and present actionable insights to stakeholders.

CI/CD & Automation

  • Implement and maintain CI/CD pipelines using Jenkins or similar tools.
  • Automate testing, deployment, and monitoring to ensure smooth production releases.

Data Security & Compliance

  • Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
  • Implement data governance practices ensuring data integrity, security, and traceability.

Troubleshooting & Performance Tuning

  • Identify and resolve performance bottlenecks in data pipelines.
  • Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.

Collaboration & Cross-Functional Work

  • Work closely with engineers, data scientists, product managers, and business stakeholders.
  • Participate in agile ceremonies, sprint planning, and architectural discussions.


Skills & Qualifications

Mandatory (Must-Have) Skills

  1. AWS Expertise
  • Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
  • Strong understanding of cloud-native data architectures.
  1. Big Data Technologies
  • Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
  • Experience with Apache Spark and Apache Kafka in production environments.
  1. Data Frameworks
  • Strong knowledge of Spark DataFrames and Datasets.
  1. ETL Pipeline Development
  • Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.
  1. Database Modeling & Data Warehousing
  • Expertise in designing scalable data models for OLAP and OLTP systems.
  1. Data Analysis & Insights
  • Ability to perform complex data analysis and extract actionable business insights.
  • Strong analytical and problem-solving skills with a data-driven mindset.
  1. CI/CD & Automation
  • Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
  • Familiarity with automated testing and deployment workflows.

 

Good-to-Have (Preferred) Skills

  • Knowledge of Java for data processing applications.
  • Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
  • Familiarity with data governance frameworks and compliance tooling.
  • Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
  • Exposure to cost optimization strategies for large-scale cloud data platforms.

 

Skills: big data, scala spark, apache spark, ETL pipeline development

 

******

Notice period - 0 to 15 days only

Job stability is mandatory

Location: Hyderabad

Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer

F2F Interview: 14th Feb 2026

3 days in office, Hybrid model.

 


Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

Leading Ecommerce Electronics Company
Leading Ecommerce Electronics Company
Agency job
via Merito by Jinita Sumaria
Bengaluru (Bangalore)
6 - 8 yrs
₹20L - ₹25L / yr
Agile/Scrum
Certified Scrum Master (CSM)
Certified Scrum Professional (CSP)
Analytical Skills
Mendix

Our client is a Leading Indian consumer electronics start-up and is one of the fastest-growing start-ups. They have also emerged as the fifth largest wearable brand globally. They are super passionate about the impact that they are making in people’s lives and are looking for a fellow self-starter to join our ambitious bunch.

 

Responsibilities

· Using agile methodology values, principles, and practices to plan, manage, and deliver solutions.

· Training, mentoring and supporting scrum teams to follow agile values, principles, and practices.

· Determining and managing tasks, issues, risks, and action items.

· Scheduling and facilitating scrum events, meetings, and decision-making processes.

· Monitoring progress and performance and helping teams to make improvements.

· Planning and organizing demos and product/system testing.

· Ensuring the proper use of collaborative processes and removing impediments for the scrum team.

· Tracking project processes, deliverables, and financials.

· Preparing and presenting status reports to stakeholders.

 

Requirements

· Bachelor's degree in computer science, information technology, or a relevant field preferred.

· 6+ years of experience as a Scrum Master

· 3-4 implementation of LCNC platforms like Mendix.

· Ability to analyze and think quickly and to resolve conflict

· Ability to work with internal stakeholders and vendors

· Strong communication, interpersonal and mentoring skills

· Ability to adapt to a changing environment

· Self-motivation and ability to stay focused in the middle of distraction

Read more
FinTradify
at FinTradify
2 candid answers
Siddharth Srivastava
Posted by Siddharth Srivastava
Noida
0 - 3 yrs
₹1L - ₹2.5L / yr
Receptionist
Effective communication
Front office
MS-Excel
MS-Office
+2 more

We are looking for a good-looking professional and friendly Receptionist to manage our front desk and perform a variety of administrative and clerical tasks. The ideal candidate will be the first point of contact for our company, ensuring a welcoming environment for visitors and providing support to various departments

Read more
Career Forge
at Career Forge
2 candid answers
Mohammad Faiz
Posted by Mohammad Faiz
Delhi, Gurugram, Noida, Ghaziabad, Faridabad
5 - 7 yrs
₹12L - ₹15L / yr
skill iconPython
Apache Spark
PySpark
Data engineering
ETL
+10 more

🚀 Exciting Opportunity: Data Engineer Position in Gurugram 🌐


Hello 


We are actively seeking a talented and experienced Data Engineer to join our dynamic team at Reality Motivational Venture in Gurugram (Gurgaon). If you're passionate about data, thrive in a collaborative environment, and possess the skills we're looking for, we want to hear from you!


Position: Data Engineer  

Location: Gurugram (Gurgaon)  

Experience: 5+ years 


Key Skills:

- Python

- Spark, Pyspark

- Data Governance

- Cloud (AWS/Azure/GCP)


Main Responsibilities:

- Define and set up analytics environments for "Big Data" applications in collaboration with domain experts.

- Implement ETL processes for telemetry-based and stationary test data.

- Support in defining data governance, including data lifecycle management.

- Develop large-scale data processing engines and real-time search and analytics based on time series data.

- Ensure technical, methodological, and quality aspects.

- Support CI/CD processes.

- Foster know-how development and transfer, continuous improvement of leading technologies within Data Engineering.

- Collaborate with solution architects on the development of complex on-premise, hybrid, and cloud solution architectures.


Qualification Requirements:

- BSc, MSc, MEng, or PhD in Computer Science, Informatics/Telematics, Mathematics/Statistics, or a comparable engineering degree.

- Proficiency in Python and the PyData stack (Pandas/Numpy).

- Experience in high-level programming languages (C#/C++/Java).

- Familiarity with scalable processing environments like Dask (or Spark).

- Proficient in Linux and scripting languages (Bash Scripts).

- Experience in containerization and orchestration of containerized services (Kubernetes).

- Education in database technologies (SQL/OLAP and Non-SQL).

- Interest in Big Data storage technologies (Elastic, ClickHouse).

- Familiarity with Cloud technologies (Azure, AWS, GCP).

- Fluent English communication skills (speaking and writing).

- Ability to work constructively with a global team.

- Willingness to travel for business trips during development projects.


Preferable:

- Working knowledge of vehicle architectures, communication, and components.

- Experience in additional programming languages (C#/C++/Java, R, Scala, MATLAB).

- Experience in time-series processing.


How to Apply:

Interested candidates, please share your updated CV/resume with me.


Thank you for considering this exciting opportunity.

Read more
India Bison
Vanita Deokar
Posted by Vanita Deokar
Remote only
5 - 8 yrs
₹5L - ₹15L / yr
skill iconJava
J2EE
skill iconSpring Boot
Hibernate (Java)

Job description

JD - Java Developer (Backend Developer)


Technical Requirement:

  • At least 5 years of experience in Java, J2EE technologies.
  • Expertise in doing development using Java, Spring Boot, Spring modules (Spring Security, Spring MVC, Spring Data JPA, etc.), Hibernate, Web Services
  • Worked on REST web services.
  • Worked on application server : Tomcat
  • Worked on Eclipse IDE
  • Knowledge of Swagger, Postman
  • Worked on Database like MySQL
  • Worked on Git, Maven, CI/CD pipelines
  • Good to have knowledge of using Docker and Mongo DB


Overall Roles & responsibilities:

  • Consult with client team to understand business requirements and with solution architects / team lead to understand the application architecture.
  • Develop/code applications following specifications using the appropriate tools and adhering to the standard coding standards.
  • Coordinate unit testing, integration testing and bug fixing with the development and quality assurance team.
  • Plan for defect fixes as per priority and severity mentioned by QA team.
  • J Attend daily scrum meeting with entire team and client.
Read more
Amazon India
at Amazon India
1 video
58 recruiters
Shane Mathews
Posted by Shane Mathews
Chennai
4 - 15 yrs
₹15L - ₹25L / yr
skill iconJava
skill iconPython
skill iconC++
Linux/Unix
Technical support
+3 more

Are you interested in joining the team behind Amazon’s newest innovation? Come help us work on world class software for our customers!


The Amazon Kindle Reader and Shopping Support Engineering team provides production engineering support and is also responsible for providing multifaceted services to the Kindle digital product family of development teams and working with production operations teams for software product release coordination and deployment. This job requires you to hit the ground running and your ability to learn quickly and work on disparate and overlapping tasks that will define your success


Job responsibilities

  • Provide support of incoming tickets, including extensive troubleshooting tasks, with responsibilities covering multiple products, features and services
  • Work on operations and maintenance driven coding projects, primarily in Java and C++
  • Software deployment support in staging and production environments
  • Develop tools to aid operations and maintenance
  • System and Support status reporting
  • Ownership of one or more Digital products or components
  • Customer notification and workflow coordination and follow-up to maintain service level agreements
  • Work with support team for handing-off or taking over active support issues and creating a team specific knowledge base and skill set


BASIC QUALIFICATIONS

  • 4+ years of software development, or 4+ years of technical support experience
  • Experience troubleshooting and debugging technical systems
  • Experience in Unix
  • Experience scripting in modern program languages
  • Experience in agile/scrum or related collaborative workflow
  • Experience troubleshooting and documenting findings


PREFERRED QUALIFICATIONS

  • Knowledge of distributed applications/enterprise applications
  • Knowledge of UNIX/Linux operating system
  • Experience analyzing and troubleshooting RESTful web API calls
  • Exposure to iOS (SWIFT) and Android (Native) App support & development
Read more
company logo
Agency job
via Avekshaa Technologies by Geetha MN
Bengaluru (Bangalore)
12 - 15 yrs
Best in industry
Marketing
Revenue growth
B2B Marketing
Digital Marketing
Drive the entire inbound marketing efforts towards setting up demand generation engine for the business
• Drive revenue growth through marketing channels
• Work with Inside sales team to deliver consistent business growth
• Enhance the web properties of the company while ensuring optimum SEO and SEM effort towards discoverability
• Set up a social media strategy for the brand with a disproportionate effort towards developing a compelling LinkedIn presence and outreach
• Would be responsible for ensuring sustained thought leadership for the company in the performance engineering category by creating a strong content management engine
• Plan and deliver successful webinars every quarter
• Drive Video Marketing - Solution / Products showcase, Demo, Testimonial and Case Studies
• Create customer presentations and contribute in building key proposals Build Community/s
• Branding / Messaging: Define and refine messaging for the ICP
• Generate value through Event marketing
• Would be expected to lead all communication effort including internal communication, public relations and analyst relations.
Read more
Tekion
at Tekion
4 recruiters
Divya Rao
Posted by Divya Rao
Bengaluru (Bangalore)
4 - 8 yrs
₹25L - ₹45L / yr
skill iconReact.js
skill iconJavascript
skill iconRedux/Flux

Key Responsibilities

  • Design and Build complicated Workflow
  • Understanding the user persona and deliver slick experience
  • Experience in JavaScript (ReactJS / Redux) React Native, HTML, CSS, JQuery, HTTP
  • Strong understating of XML, JSON, DOM, and w3c standards
  • Providing technical expertise for every phase of the project lifecycle; from concept development to solution design, implementation, optimization and support
  • Be an Interface with Stockholder sand architects to create technology specifications from business requirements for one or more modules within the project
  • Responsible to prepare the technical design of the components within the module (one or more of client/web presentation tier, server tier, data access and data model, integration component, package function customization)
  • Proficient in articulating the pros and cons of using a certain technology stack or component or design pattern versus another to the clients and project team and drives selection of technologies, designs to come up with the optimal architecture
  • Drives the performance tuning, re-design and re-factoring for a module
  • Contributes to designing and implementing the build, release process as well as system configuration and deployment characteristics of the application
  • Implements (hands-on) the module components based on prevailing Framework and Open-source standards & its best practices.
  • Mentors the team (people working on the module and other project team members) on technology concepts and best practices for design, implementation and other processes
Read more
Steady Rabbit
at Steady Rabbit
1 recruiter
Kirti Dhingra
Posted by Kirti Dhingra
Remote only
3 - 4 yrs
₹4L - ₹9L / yr
skill iconReact Native
MVC Framework
Payment gateways
skill iconRedux/Flux

Skills

  • Have published at least one original React Native App.
  • Experience with third-party libraries and APIs 

Role 


Should be able to single-handedly build performant mobile apps on the React Native platform. You will be responsible for architecting and building these applications, as well as coordinating with the teams responsible for other layers of the product infrastructure. Building a product is a highly collaborative effort, and as such, a strong team player with a commitment to perfection is required.


Responsibilities 


  • Build pixel-perfect, buttery smooth UIs across mobile platforms.
  • Leverage native APIs for deep integrations with both platforms.
  • Diagnose and fix bugs and performance bottlenecks for performance that feels native.
  • Reach out to the open-source community to encourage and help implement mission-critical software fixes—React Native moves fast and often breaks things.
Read more
Our client company is into Sports equipment. (D1)
Our client company is into Sports equipment. (D1)
Agency job
via Multi Recruit by Ashwini Miniyar
Bengaluru (Bangalore)
3 - 12 yrs
₹10L - ₹20L / yr
performance marketing
Performance Marketing
  • Manage and Develop the Digital Performance Channels
  • Ownership of Top-line building channels like Google Shopping, Google Remarketing, Third Parties for Remarketing
  • Continuously chase ambitious targets of ROAS while maintaining min Top Line thresholds
  • Build effective journeys of remarketing to reduce the overlaps and shorten the touchpoints to convert
  • Exploration of new channels to drive Performance Marketing
  • Be an actor of your channel ecosystem
  • Owner mindset, willingness to take initiatives, and accountability to drive closures
  • Be the key contact window between Users and the technical team
  • Challenge the needs and demonstrate managerial courage towards users satisfaction
  • Develop, through your posture, a collaborative working methodology, an environment of trust and responsibility among the product team

AN IDEAL CANDIDATE WILL HAVE

  • Passion for sport, you like to enjoy the sport with your teammates
  • You have an entrepreneurial mindset, a strong sense of responsibility, autonomy, and drive for results
  • Recognized capacity to understand business needs, bring structure in a complex environment, and lead action plans
  • Strong data analysis skills, ability to perform data analysis and make data-driven decisions
  • You are a technology enthusiast; you care about how things work, and you can successfully collaborate with technical experts
  • You are well versed with Google Shopping and Remarketing as a subject
  • A prior experience in performance marketing/e-commerce is required.

EXPERIENCE

  • 3-12 years of experience in the digital eco-system
  • Prior experience in performance marketing agencies is a plus
  • A Masters degree in marketing is a plus

 

Read more
Sunera Technologies
at Sunera Technologies
1 recruiter
Siri Gandi
Posted by Siri Gandi
Hyderabad
5 - 10 yrs
₹0L - ₹19L / yr
Apex
Visual force
Salesforce
Sunera Technologies is a product based development company.
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos