Lead II - SE - AWS, Apache Spark (PySpark/Scala), Apache Kafka
at Global digital transformation solutions provider.
Amazon Web Services (AWS)
Scala
Jenkins
Data Analytics
Data Science
Java
MongoDBJOB DETAILS:
* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka
* Industry: Global digital transformation solutions provider
* Salary: Best in Industry
* Experience: 5-8 years
* Location: Hyderabad
Job Summary
We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.
Key Responsibilities
ETL Pipeline Development & Optimization
- Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
- Optimize data pipelines for performance, scalability, fault tolerance, and reliability.
Big Data Processing
- Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
- Ensure fault-tolerant, scalable, and high-performance data processing systems.
Cloud Infrastructure Development
- Build and manage scalable, cloud-native data infrastructure on AWS.
- Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.
Real-Time & Batch Data Integration
- Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
- Ensure consistency, data quality, and a unified view across multiple data sources and formats.
Data Analysis & Insights
- Partner with business teams and data scientists to understand data requirements.
- Perform in-depth data analysis to identify trends, patterns, and anomalies.
- Deliver high-quality datasets and present actionable insights to stakeholders.
CI/CD & Automation
- Implement and maintain CI/CD pipelines using Jenkins or similar tools.
- Automate testing, deployment, and monitoring to ensure smooth production releases.
Data Security & Compliance
- Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
- Implement data governance practices ensuring data integrity, security, and traceability.
Troubleshooting & Performance Tuning
- Identify and resolve performance bottlenecks in data pipelines.
- Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.
Collaboration & Cross-Functional Work
- Work closely with engineers, data scientists, product managers, and business stakeholders.
- Participate in agile ceremonies, sprint planning, and architectural discussions.
Skills & Qualifications
Mandatory (Must-Have) Skills
- AWS Expertise
- Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
- Strong understanding of cloud-native data architectures.
- Big Data Technologies
- Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
- Experience with Apache Spark and Apache Kafka in production environments.
- Data Frameworks
- Strong knowledge of Spark DataFrames and Datasets.
- ETL Pipeline Development
- Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.
- Database Modeling & Data Warehousing
- Expertise in designing scalable data models for OLAP and OLTP systems.
- Data Analysis & Insights
- Ability to perform complex data analysis and extract actionable business insights.
- Strong analytical and problem-solving skills with a data-driven mindset.
- CI/CD & Automation
- Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
- Familiarity with automated testing and deployment workflows.
Good-to-Have (Preferred) Skills
- Knowledge of Java for data processing applications.
- Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
- Familiarity with data governance frameworks and compliance tooling.
- Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
- Exposure to cost optimization strategies for large-scale cloud data platforms.
Skills: big data, scala spark, apache spark, ETL pipeline development
******
Notice period - 0 to 15 days only
Job stability is mandatory
Location: Hyderabad
Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer
F2F Interview: 14th Feb 2026
3 days in office, Hybrid model.
Shubham Vishwakarma
Full Stack Developer - Averlon
Similar jobs
JavaPosition: QA Automation
Mandatory Skills: Java, Selenium, Strong in Database Testing (SQL), BDD Cucumber
We are specifically looking for Automation Testing profiles, preferably SDET (Software Development Engineer in Test) roles.
Job Summary:
We are seeking a skilled 5+ Years experienced Automation Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and executing automated tests using Java, Selenium, and other testing frameworks. You will also play a key role in validating the data integrity and performance of our databases by integrating automated tests with database queries, ensuring the quality and reliability of our web applications and backend systems.
Required Skills and Qualifications:
· Programming Languages: Strong proficiency in Java.
· Test Automation: Experience with Selenium WebDriver for automating web applications.
Database Skills:
· Strong knowledge of SQL for database testing (MySQL, PostgreSQL, Oracle, or similar).
· Experience in validating data in the database and checking data integrity.
· Testing Frameworks: Experience with test frameworks like TestNG or JUnit.
· Version Control: Proficiency in version control tools like Git.
· CI/CD Tools: Familiarity with Jenkins, GitLab CI, or other CI/CD tools.
· Agile Environment: Experience working in an Agile/Scrum development environment.
· Debugging Skills: Ability to debug both frontend and backend issues.
· Problem Solving: Strong analytical and troubleshooting skills.
· Communication: Excellent verbal and written communication skills, with the ability to communicate effectively with cross-functional teams.
CJob Role: Optimization Engineer - C Programming
Experience: 3 to 8 Years
Location: Bangalore, Pune, Delhi
Were hiring an Optimization Engineer skilled in C Programming and Operations Research / Optimization to design and optimize algorithms solving complex business and engineering problems.
Key Responsibilities:
- Develop and maintain high-performance software using C.
- Build and implement optimization models (linear, integer, nonlinear).
- Collaborate with teams to deliver scalable, efficient solutions.
- Analyze and improve existing algorithms for performance and scalability.
Must-Have Skills:
- Expertise in C Programming and Operations Research / Optimization.
- Strong in data structures, algorithms, and memory management.
- Hands-on with tools like CPLEX, Gurobi, or COIN-OR.
- Python experience is an added advantage.
Preferred Skills:
- Knowledge of Python, C++, or Java.
- Familiarity with AMPL, GAMS, or solver APIs.
- Understanding of HPC, parallel computing, or multi-threading.
C#
.NETJob Overview:
We are seeking a skilled and experienced .NET Developer to join our development team. As a .NET Developer, you will be responsible for designing, coding, testing, and implementing software solutions using Microsoft technologies.
Responsibilities:
- Collaborate with cross-functional teams to gather and define software requirements.
- Design and develop robust, scalable, and high-performance applications using .NET technologies.
- Write clean, well-documented, and efficient code following established coding standards.
- Perform thorough testing of the developed software to ensure the reliability and stability of applications.
- Troubleshoot, debug, and resolve software defects and issues in a timely manner.
- Stay updated on the latest trends and technologies in .NET development to enhance team knowledge and productivity.
- Collaborate with other developers to review code, provide constructive feedback, and ensure code quality.
- Participate in code and design reviews to maintain high development standards.
Requirements:
- Bachelor's degree in Computer Science, Information Technology, or related field.
- Proven experience of 2-4 years as a .NET Developer or Software Developer.
- Strong proficiency in C#, ASP.NET, and other .NET technologies.
- Experience with front-end technologies such as HTML, CSS, and JavaScript.
- Familiarity with databases (SQL Server, Oracle, or others) and data modeling.
- Understanding of software development life cycle (SDLC) and best practices.
- Excellent problem-solving and communication skills.
- Ability to work both independently and collaboratively in a team environment.
- Familiarity with Agile/Scrum methodologies is a plus.
ROLE AND RESPONSIBILITIES
Should be able to work as an individual contributor and maintain good relationship with stakeholders. Should
be proactive to learn new skills per business requirement. Familiar with extraction of relevant data, cleanse and
transform data into insights that drive business value, through use of data analytics, data visualization and data
modeling techniques.
QUALIFICATIONS AND EDUCATION REQUIREMENTS
Technical Bachelor’s Degree.
Non-Technical Degree holders should have 1+ years of relevant experience.
Vue.js
AngularJS (1.x)
Angular (2+)
React.js
Javascript• Developing front end website architecture.
• Designing user interactions on web pages.
• Developing back end website applications.
• Creating servers and databases for functionality.
• Ensuring cross-platform optimization for mobile phones.
• Ensuring responsiveness of applications.
• Working alongside graphic designers for web design features.
• Seeing through a project from conception to finished product.
• Designing and developing APIs.
• Meeting both technical and consumer needs.
• Staying abreast of developments in web applications and programming languages
PHPReact.js
NodeJS (Node.js)
CodeIgniterIntegration of user-facing elements developed by front-end developers
Build efficient, testable, and reusable PHP modules
Solve complex performance problems and architectural challenges
work on multiple projects with Product Development team
Contribute in all phases of the development lifecycle
Follow industry best practices
Develop and deploy new features to facilitate related procedures and tools if necessary Skills And Qualifications
Strong knowledge of PHP web frameworks ( Magento 1/2, Wordpress, YII, Laravel, Prestashop)
Understanding of MVC design patterns
Basic understanding of front-end technologies, such as JavaScript, jQuery, HTML5, and CSS3
Knowledge of object oriented PHP programming Strong knowledge of the common PHP or web server exploits and their solutions
Understanding fundamental design principles behind a scalable application
User authentication and authorization between multiple systems, servers, and environments
Good knowledge of relational databases, version control tools (like GIT) and of developing web services
Experience in common third-party APIs (Google, Facebook, Ebay etc) (OPTIONAL)
HTML/CSSJavaAEM Developer - Job Description
Required Skills
- “AEM Lead Dev” - Minimum 3+ years of experience in AEM and at least 4 to 7 years of industry experience
- Design and develop AEM components and templates.
- Hands on experience in DAM, workflows, MSM, User management and Dispatcher.
- Strong in Maven configurations with respect to various AEM archetype projects and integrations.
- Basic understanding and implementation of Java/J2EE design patterns
- Knowledge on Portal or related technologies
- Experience with RESTful and SOAP based web services
- Intermediate experience with at least basic front-end skills - HTML, CSS, JS [Jquery]
Nice to have
- Adobe certification (AEM Developer/AEM Architect)
- Experience working with Adobe Cloud Manager for deployments
- Experience working with AEM as a Cloud Service (AEMaaCS)
- Experience using Adobe Target, Campaign, Analytics, Launch, and I/O, and how each of these integrates with AEM
- Experience with Adobe’s Commerce Integration Framework (CIF) components
- Experience, or at least exposure, to headless content delivery and presentation as single page applications
- Experience with server management in cloud ecosystems like Azure or AWS
- Prefer to have experience in third party integrations with AEM [Other products of Adobe or any other applications]
- Working with Adobe Marketing Cloud tools like Adobe Campaign, Target, Scene 7, etc
Javascript- Responsible for providing L1/L2 technical enablement to company customers/partners for any product issues
- Ensure that the customer issues are managed within target Service Level Agreements (SLA) and all the incoming technical and product inquiries are responded to, in a timely, professional and effective manner.
- Participate in solution development and publications for the company knowledge base resources
- Report common user issues, suggest product improvements and convey important product feedback
- Formulate action plans (technical and operational) for analyzing and resolving reported product issues
- Stay updated on all product features- current, new & in the pipeline. Work closely with the Product
Our Ideal Candidate:
- Comes with 2 to 6 years of Technical & Customer-Facing experience, preferably in a System Engineer, Technical Support, or Product Support role
- Preferably someone from Engineering / MCA background
- Possesses thorough knowledge of Javascript, CSS & HTML
- Gets a thrill in solving everyday problems.
- That zeal to not let go of a customer without a solution.
- Demonstrates impeccable communication and interpersonal skills, and the confidence to work directly with senior executives from large enterprise companies
- Flexible to align with customers from the North American (9 PM to 6 AM), European (1 to 10 PM), or A&Z (6 AM to 3 PM) time zones, on a rotational basis
DockerAmazon Web Services (AWS)MongoDB● Responsible for design, development, and implementation of Cloud solutions.
● Responsible for achieving automation & orchestration of tools(Puppet/Chef)
● Monitoring the product's security & health(Datadog/Newrelic)
● Managing and Maintaining databases(Mongo & Postgres)
● Automating Infrastructure using AWS services like CloudFormation
● Participating in Infrastructure Security Audits
● Migrating to Container technologies (Docker/Kubernetes)
● Should be able to work on serverless concepts (AWS Lambda)
● Should be able to work with AWS services like EC2, S3, Cloud-formation, EKS, IAM, RDS, ..etc
What you bring:
● Problem-solving skills that enable you to identify the best solutions.
● Team collaboration and flexibility at work.
● Strong verbal and written communication skills that will help in presenting complex ideas
in
● an accessible and engaging way.
● Ability to choose the best tools and technologies which best fits the business needs.
Aviso offers:
● Dynamic, diverse, inclusive startup environment driven by transparency and velocity
● Bright, open, sunny working environment and collaborative office space
● Convenient office locations in Redwood City, Hyderabad and Bangalore tech hubs
● Competitive salaries and company equity, and a focus on developing world class talent operations
● Comprehensive health insurance available (medical) for you and your family
● Unlimited leaves with manager approval and a 3 month paid sabbatical after 3 years of service
● CEO moonshots projects with cash awards every quarter
● Upskilling and learning support including via paid conferences, online courses, and certifications
● Every month Rupees 2,500 will be credited to Sudexo meal card
We are Hiring sales executive for India's Leading Mobile brand - Xiaomi
Location
Ahmedabad, Badodara, Surat, Bhavnagar, Surendernagar, Bhuj, Junagadh, Himmatnagar, Palanpur, Gandhi Nagar
Experience
Experience from Mobile industry is must interested candidates can share cv
Salary
13k to 17k CTC plus incentives
Note salary depends on bands of selection (basis on experience and product knowledge)
BR,
Ashok K
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon