Cutshort logo
Global digital transformation solutions provider.  logo
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
Global digital transformation solutions provider.
Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
Global digital transformation solutions provider. 's logo

Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka

at Global digital transformation solutions provider.

Agency job
5 - 8 yrs
₹11L - ₹20L / yr
Hyderabad
Skills
PySpark
Apache Kafka
Data architecture
skill iconAmazon Web Services (AWS)
EMR
Apache Airflow
Amazon S3
DMS
Amazon EC2
skill iconScala
Apache Spark
Spark
ETL
CI/CD
skill iconJenkins
Big Data
SQL
Database modeling
OLAP
OLTP
skill iconData Analytics
Automation
Automated testing
Workflow
Data engineering
Data security
HIPAA
skill iconData Science
Datawarehousing
skill iconJava
NOSQL Databases
DynamoDB
Cassandra
skill iconMongoDB
Data governance
Splunk
Dynatrace

JOB DETAILS:

* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka

* Industry: Global digital transformation solutions provider

* Salary: Best in Industry

* Experience: 5-8 years

* Location: Hyderabad

 

Job Summary

We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.


Key Responsibilities

ETL Pipeline Development & Optimization

  • Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
  • Optimize data pipelines for performance, scalability, fault tolerance, and reliability.

Big Data Processing

  • Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
  • Ensure fault-tolerant, scalable, and high-performance data processing systems.

Cloud Infrastructure Development

  • Build and manage scalable, cloud-native data infrastructure on AWS.
  • Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.

Real-Time & Batch Data Integration

  • Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
  • Ensure consistency, data quality, and a unified view across multiple data sources and formats.

Data Analysis & Insights

  • Partner with business teams and data scientists to understand data requirements.
  • Perform in-depth data analysis to identify trends, patterns, and anomalies.
  • Deliver high-quality datasets and present actionable insights to stakeholders.

CI/CD & Automation

  • Implement and maintain CI/CD pipelines using Jenkins or similar tools.
  • Automate testing, deployment, and monitoring to ensure smooth production releases.

Data Security & Compliance

  • Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
  • Implement data governance practices ensuring data integrity, security, and traceability.

Troubleshooting & Performance Tuning

  • Identify and resolve performance bottlenecks in data pipelines.
  • Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.

Collaboration & Cross-Functional Work

  • Work closely with engineers, data scientists, product managers, and business stakeholders.
  • Participate in agile ceremonies, sprint planning, and architectural discussions.


Skills & Qualifications

Mandatory (Must-Have) Skills

  1. AWS Expertise
  • Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
  • Strong understanding of cloud-native data architectures.
  1. Big Data Technologies
  • Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
  • Experience with Apache Spark and Apache Kafka in production environments.
  1. Data Frameworks
  • Strong knowledge of Spark DataFrames and Datasets.
  1. ETL Pipeline Development
  • Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.
  1. Database Modeling & Data Warehousing
  • Expertise in designing scalable data models for OLAP and OLTP systems.
  1. Data Analysis & Insights
  • Ability to perform complex data analysis and extract actionable business insights.
  • Strong analytical and problem-solving skills with a data-driven mindset.
  1. CI/CD & Automation
  • Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
  • Familiarity with automated testing and deployment workflows.

 

Good-to-Have (Preferred) Skills

  • Knowledge of Java for data processing applications.
  • Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
  • Familiarity with data governance frameworks and compliance tooling.
  • Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
  • Exposure to cost optimization strategies for large-scale cloud data platforms.

 

Skills: big data, scala spark, apache spark, ETL pipeline development

 

******

Notice period - 0 to 15 days only

Job stability is mandatory

Location: Hyderabad

Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer

F2F Interview: 14th Feb 2026

3 days in office, Hybrid model.

 


Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

Technomine Group
Careers -Technomine
Posted by Careers -Technomine
Ahmedabad
0 - 1 yrs
₹2.4L - ₹3.6L / yr
Communication Skills
English Proficiency
Customer Service
Customer Support
Customer Success


Position: Customer Service Executive


Company: Technomine Group (UK Travel Operations)


About Us:

Technomine Group is a global outsourcing and operations partner supporting clients across the UK, US, and Canada. Our UK-based travel division delivers premium travel support and customer experience management. We are now welcoming fresh, enthusiastic minds to join us as Travel Desk Admin Executives for our UK travel process.


Role Overview:

An incredible opportunity for freshers to gain international exposure in travel operations! You’ll handle backend travel coordination, ensure smooth booking processes, assist with travel administration, and collaborate with UK teams for seamless operations.


Key Responsibilities:

  • Manage travel bookings, reservations, and related documentation.
  • Handle customer queries via email, chat, or phone.
  • Maintain accurate travel records, itineraries, and invoices.
  • Coordinate with UK counterparts to ensure process adherence.
  • Work in rotational shifts to support UK operations.


Required Skills & Qualifications:

  • Graduate in any discipline (Freshers welcome!)
  • Good command over English (written & verbal)
  • Basic computer proficiency (MS Office, emails, data entry)
  • Strong attention to detail and willingness to learn
  • Comfortable working in rotational shifts


 What We Offer:

  • 5 days working with rotational shifts
  • Opportunity to work with a UK-based travel company
  • Global exposure & career growth within Technomine Group
  • Food provided, Leave Encashment, Provident Fund, Accommodation


Work Type: Full-time, Permanent


Location: Navrangpura, Ahmedabad, Gujarat (In-person role)

Read more
TalkCharge Technologies Pvt Ltd
Gurugram
0 - 4 yrs
₹1L - ₹3L / yr
Customer Support
Customer Service
Communication Skills
Payments
Fintech
Job Description
Excellent verbal & written communication skills.
Manage Email Process- Payment, Refunds, Cancellation etc.
Responsible to provide timely & effective resolutions to customers through emails, chat, etc.
Knowledge of payments and fintech would be plus

Perks and benefits

Ownership and Responsibility.
Medical Insurance.
Read more
Utthunga Technologies
at Utthunga Technologies
6 recruiters
Roshan RM
Posted by Roshan RM
Bengaluru (Bangalore)
2 - 9 yrs
₹7L - ₹14L / yr
skill iconHTML/CSS
skill iconJavascript
skill iconAngular (2+)
skill iconAngularJS (1.x)
ASP.NET
+6 more
Net FS

- Software Developer with minimum 2+ years of development experience in .net
- Write effective and reusable code using the MVVM framework
- Expertise in .net core, OOPS concept and .net coding guidelines
- Solid understanding of Object-Oriented design patterns
- Good in logical and problem solving skills
- Good analytical and debugging skills
- Ability to understand requirements
- Knowledge of unit testing and code analysis tools will be added advantage
- Knowledge of Azure services will be added advantage
- Knowledge of working with XML will be added advantage


Angular
- Looking for an Angular Developer who has deep knowledge of Angular practices and commonly used modules based on extensive work experience.
- Should be flexible to adapt quickly to new technologies as required.
- Hands on and implementation of Angular applications, directives, controllers, services
- Thorough understanding of the responsibilities of the platform, database, API, caching layer, proxies, and other web services used in the system
- Good experience on OOPS Design skills.
- Experience with consuming RESTful services
- Creating self-contained, reusable, and testable modules and components
- Should have good knowledge of the MVVM frameworks
- Exposure to Microsoft Azure is an added advantage
- Critical thinker and problem-solving skills
- Team player
- Great interpersonal and communication skills
- Should have strong programming knowledge in Angular (Angular v8+) with at least 3+ years of relevant experience
Read more
Ylytic
at Ylytic
2 recruiters
Mradul Sharma
Posted by Mradul Sharma
Remote only
1 - 2 yrs
₹3L - ₹6L / yr
skill iconPython
skill iconFlask
skill iconMongoDB
RESTful APIs

Backend Development - Intern

 

Converts into a full time position based on performance


A Backend Developer, for an early-stage Analytics startup who can help us translate Product specs into a high-performing and scalable product across platforms. Also help in improving functionality of product. End goal is to achieve product-market objectives

The candidate will have rich learning into what goes into developing, scaling a product from scratch. A chance to grow into a Backend Development Owner in medium-long term

Responsibilities

  1. Own Back End Development closely with Product Owner

  2. Design, implement and maintain product modules/sub-systems in your scope

  3. Perform bug verification, release testing and support for assigned products.

  4. Conduct unit testing and integration testing.

  5. Make sure the delivered product matches the business requirements.

Expectations from candidate

  1. Ability: Good at logical thinking, problem solving

  2. Inclination: Curious and strong willingness to learn from problems, write good code

  3. Knowledge: Basics of Back End Development (Some academic degree/certification

    or Prior experience/Projects preferred)

  4. Languages/Tool/Framework: Python, MongoDB, GraphQL, GoLang

  5. Interest in product focused work in a fast-moving start-up environment

  6. Attitude: We are looking for a person with an owner mindset who can partner with

    us in building our product with a lot of commitment. We like people who are curious problem solvers and are interested in learning with us and working hard.

What we offer

  1. Great culture focused on outcomes through learning and experimentation

  2. Mentoring by experienced professionals from pedigreed backgrounds (IIT/IIMs)

  3. Growth, ownership, and flexibility to achieve your goals.

 

Read more
Evora IT Solutions
at Evora IT Solutions
10 recruiters
Divyadharshini SU
Posted by Divyadharshini SU
Bengaluru (Bangalore), Noida
3 - 6 yrs
Best in industry
SAPUI5
UI5
skill iconHTML/CSS
RESTful APIs
Webservices
+5 more
Evora is an international system integrator with a focus on mobile and user experience (UX). We are a trusted partner for innovative digital maintenance and service solutions for global and regional customers in Europe, North America, Asia & the Pacific. As SAP Gold Partner, Service Now Specialist, and Click partner (by Salesforce), we offer consulting services, software development, quality assurance services, and system integration based on standard components and products, as well as support & application management. By implementing best-practice solutions while optimizing their processes we help our customers to be successful in tomorrow's world. Together with our partner Parasus we are expanding into the area of SAP procurement, covering technology such as SAP SRM, SAP SLC, ERP MM, SAP S/4, or SAP Ariba. 

Desired Profile:
  • Excellent communication and problem-solving skills,
  • Strong analytical skills Ability to turn business requirements into technical concepts and code,
  • Minimum 3 - 6 years experience in software development in general with 3 years in complex web applications using SAP UI5,
  • Software architecture knowledge of enterprise applications is a must
  • Experience with Web - Technologies (SAP UI5, HTML5, CSS3, JS, REST - WS) is a must
  • Experience with SAP Mobile Platform as well as SAP Web IDE is a plus
  • Knowledge and experience with user experience and design concepts is a plus
Read more
Zycus
at Zycus
10 recruiters
nisha joshi
Posted by nisha joshi
Mumbai
0 - 5 yrs
₹6L - ₹8L / yr
Business Development
Sales
Inside Sales

ZYCUS OVERVIEW:
Zycus is looking out for Seasoned Sales professionals, with a Software sales experience of about 1-5 years, and hands-on experience in the generation of leads and new business opportunities. Headquartered in Princeton, the U.S. in 1998, Zycus has grown every day to be established as an organization which now is a leading global provider of complete Source-to-Pay suite of procurement performance solutions.

We develop cloud-based (SaaS) Source-to-Pay solutions for large global enterprises, and have successfully deployed about 200 solutions to over 1000 Global clients. Our spirit of innovation and our passion to help procurement create greater business impact are reflected among procurement solution deployments that we have undertaken over the years. We are proud to have as our clients, some of the best-of-breed companies across verticals like Manufacturing, Automotives, Banking and Finance, Oil and Gas, Food Processing, Electronics, Telecommunications, Chemicals, Health and Pharma, Education and more. With a team of 1200+ employees, we are present in India with 3 development centers at Bengaluru, Mumbai & Pune and offices in the U.S. (New Jersey, Atlanta, Chicago), U.K., Australia, Dubai, Netherland, and Singapore.

Know more about the LEADER of Gartner's 2013, 2015 & 2017 Magic Quadrant for Strategic Sourcing Application Suites and The Forrester Wave-: eProcurement, Q2 2017

MARKET SIZEUSD 5 Billion and grows tremendously during economic downturns

Roles & Responsibilities:-
  • The incumbent will report to Manager/Sr. Manager - Business Development and will be Executive/Sr. Executive depending upon the relevance and no. of years of experience.-
  • Collaborating with Regional Sales Managers to accelerate the influx of business opportunities in the assigned territory.
  • Reaching out via Phone/Email/Social Media to connect with prospective customers.- Conducting secondary research into company financials, line of business, technology requirements, etc., to personalize prospecting efforts-
  • Staying up to date with the latest procurement industry trends & best practices- Competitor analysis & positioning against them-
  • Delivering first-level Corporate Overview Presentations- Pre-qualification of business opportunities (Gauging- Budget, Authority, Need & Timeline)-
  • Strategizing cross-functional campaigns to be conducted by BD & Marketing Necessary
Skill Sets:-

  • Educational Qualification:
  • Graduation is a must (B.E.and MBA referred)-
  • Excellent verbal & written communication skills in English (Public Speaking background is preferable)-
  • Soft Skills (Pitching, Probing, Objection Handling, Rapport Building)-
  • Strong, in-depth analytical skill-
  • Proficient in professional business conversational etiquette-
  • Account Mapping & Social Selling ability-
  • Networking & Team Management
  • Work Location: Mumbai
  • Education: Any Graduate/ Post Graduate

Benefits
  • LOCATION: Mumbai
  • COMPENSATION: Competitive
  • WHY ZYCUS? :- Be a part of one of the fastest-growing product Companies in India-
  • Come join a young, dynamic & enterprising team-
  • Work on the latest technologies-
  • Flexible working hours (As per business requirement).
Read more
Aryston Web Solution
Kolkata
0 - 1 yrs
₹1L - ₹1.5L / yr
Sales
Telesales
Business Development
Lead Generation
Communication Skills
+5 more
Immediate joiners required/ Spot Joining available/ Job vacancy for fresher

Designation: Web consultant/ web caller/ Website sales executive/ CCE.
Process: International B2B website process.
- Fresher candiates can also apply for the Job.
- Fresher candidates with flueny in english communication required.
- At least 3 to 6 months Exp. required in web selling,
Knowledge in Website sales, SEO/ GLL sales preferred.


#seo #insidesales #voiceprocess #kolkata #internationalbpo #cce #webconsultant #kolkatajobs #websitesales #websiteprocess #seosales #b2bcampaign #searchengineoptimization #gllsales #websitecampaign
Read more
Our client company is into Hospitality. (EM1)
Our client company is into Hospitality. (EM1)
Agency job
via Multi Recruit by savitha Rajesh
Bengaluru (Bangalore)
1 - 5 yrs
₹4L - ₹5L / yr
Digital Marketing
Search Engine Optimization (SEO)
Search Engine Marketing (SEM)
Google Adwords
On-page Optimization
+2 more
  • Design, maintain and supply content for the organization's website.
  • Formulate strategies to build a lasting digital connection with customers.
  • A monitor company presence on social media.
  • Launch advertisements to increase brand awareness.
Qualifications:
  • Bachelor's degree in Marketing or related field.
  • Excellent understanding of digital marketing concepts.
  • Experience with business to customer social media and content generation.
  • Strong creative and analytical skills.
Skills:
  • Analytical Skills
  • Social Media Marketing
  • Marketing Campaigns
  • Marketing Strategy
  • Online Marketing
  • Brand Awareness
  • Content Strategy
  • Content Management Systems (CMS)
  • SEO
  • Google Ads
  • English Proficiency (Verbal and Written)
Read more
Matellio Inc
at Matellio Inc
2 recruiters
Nikita Goyal
Posted by Nikita Goyal
Jaipur, Jodhpur
3 - 7 yrs
₹7L - ₹16L / yr
NDK
skill iconAndroid Development
1. Proficient in Java with strong object oriented design skills.
2. Experience of integrating any of the SDKs or third party libraries.
3. Knowledge of Android Software design patterns, Android/Media Frameworks.
4. Experience with NDK and JNI
5. Basic knowledge of C & C++ language/libraries (optional)
Read more
Media.net
at Media.net
21 recruiters
Shruti Doijode
Posted by Shruti Doijode
Mumbai, Pune, Bengaluru (Bangalore)
4 - 7 yrs
₹10L - ₹20L / yr
skill iconHTML/CSS
UXD
Webpack
What is the Job and Team like? Opportunity to develop game-changing, globally distributed, internet advertising products used by millions worldwide. Contribute hands-on, own, collaborate and deliver new features and services at a fast pace. Build platforms, frameworks, APIs, libraries & automated tools to facilitate rapid issue resolution and development & deployment of new features and changes. Own the prioritization, scoping and scheduling processes for new products and features. Work with and advise management on technical aspects, make well-informed decisions. Improve processes, technology & the applications you own continuously by showing the team better ways of doing things and help improve skills in the team. Prioritize tech-debt and ensure the platforms and applications meet the latest industry standards. Learn new technology & practices, coach the team and conduct training sessions – devote time for learning and continuous improvement of self and team. Imbibe and maintain a strong customer service oriented attitude while designing and building products. Recruit the best by owning the recruitment processes for your teams. Location: Mumbai (strong preference) and Pune Who should apply for this role? Requirements: Strong interpersonal communication skills, a positive attitude and a hunger for growth. Empathy and the ability to work well in a diverse, team-focused environment with other Engineers, Product Managers, etc. We like people with strong educational qualification (B.Tech, BE, BSC.IT, MCAs) but we won’t ding you if you’re self taught or just born brilliant. 5+ years of demonstrable experience developing software using modern Frontend technologies (ES6, React, Webpack/Babel etc) Ability to understand and resolve issues with dependencies. Expertise in building complex layouts with modern CSS (Grid/Flexbox) and HTML. Experience building and debugging complex, real-time systems in a team environment. Strong UX and design sensibilities, and a desire to sweat the small stuff. Self-­awareness and a desire to continually improve. Bonus Points: Experience working with Babel/Webpack/Gulp. Experience investigating and improving JavaScript performance. Experience with WebSockets, Local Storage and ES6. Familiarity with scalable, real-time distributed systems. We highly value contributions to open source projects; is there a public source repository or project that you are particularly proud of? Testing frameworks (Chai, Mocha, Enzyme, or Jasmine)
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos