Cutshort logo
Technology service company logo
Sr. Spark Software Engineer
Technology service company
Sr. Spark Software Engineer
Jobdost's logo

Sr. Spark Software Engineer

at Technology service company

Agency job
5 - 10 yrs
₹10L - ₹20L / yr
Remote only
Skills
skill iconJava
J2EE
skill iconSpring Boot
Hibernate (Java)
Ansible
skill iconGit
JIRA
Apache Spark
Spark
Apache Kafka
Microservices
skill iconKubernetes
Terraform
NOSQL Databases
API
skill iconDocker
  • Bachelor’s or master’s degree in Computer Engineering, Computer Science, Computer Applications, Mathematics, Statistics, or related technical field. Relevant experience of at least 3 years in lieu of above if from a different stream of education.

  • Well-versed in and 3+ hands-on demonstrable experience with: ▪ Stream & Batch Big Data Pipeline Processing using Apache Spark and/or Apache Flink.
    ▪ Distributed Cloud Native Computing including Server less Functions
    ▪ Relational, Object Store, Document, Graph, etc. Database Design & Implementation
    ▪ Micro services Architecture, API Modeling, Design, & Programming

  • 3+ years of hands-on development experience in Apache Spark using Scala and/or Java.

  • Ability to write executable code for Services using Spark RDD, Spark SQL, Structured Streaming, Spark MLLib, etc. with deep technical understanding of Spark Processing Framework.

  • In-depth knowledge of standard programming languages such as Scala and/or Java.

  • 3+ years of hands-on development experience in one or more libraries & frameworks such as Apache Kafka, Akka, Apache Storm, Apache Nifi, Zookeeper, Hadoop ecosystem (i.e., HDFS, YARN, MapReduce, Oozie & Hive), etc.; extra points if you can demonstrate your knowledge with working examples.

  • 3+ years of hands-on development experience in one or more Relational and NoSQL datastores such as PostgreSQL, Cassandra, HBase, MongoDB, DynamoDB, Elastic Search, Neo4J, etc.

  • Practical knowledge of distributed systems involving partitioning, bucketing, CAP theorem, replication, horizontal scaling, etc.

  • Passion for distilling large volumes of data, analyze performance, scalability, and capacity performance issues in Big Data Platforms.

  • Ability to clearly distinguish system and Spark Job performances and perform spark performance tuning and resource optimization.

  • Perform benchmarking/stress tests and document the best practices for different applications.

  • Proactively work with tenants on improving the overall performance and ensure the system is resilient, and scalable.

  • Good understanding of Virtualization & Containerization; must demonstrate experience in technologies such as Kubernetes, Istio, Docker, OpenShift, Anthos, Oracle VirtualBox, Vagrant, etc.

  • Well-versed with demonstrable working experience with API Management, API Gateway, Service Mesh, Identity & Access Management, Data Protection & Encryption.

    Hands-on experience with demonstrable working experience with DevOps tools and platforms viz., Jira, GIT, Jenkins, Code Quality & Security Plugins, Maven, Artifactory, Terraform, Ansible/Chef/Puppet, Spinnaker, etc.

  • Well-versed in AWS and/or Azure or and/or Google Cloud; must demonstrate experience in at least FIVE (5) services offered under AWS and/or Azure or and/or Google Cloud in any categories: Compute or Storage, Database, Networking & Content Delivery, Management & Governance, Analytics, Security, Identity, & Compliance (or) equivalent demonstrable Cloud Platform experience.

  • Good understanding of Storage, Networks and Storage Networking basics which will enable you to work in a Cloud environment.

  • Good understanding of Network, Data, and Application Security basics which will enable you to work in a Cloud as well as Business Applications / API services environment.

Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

NeoGenCode Technologies Pvt Ltd
Akshay Patil
Posted by Akshay Patil
Gurugram
3 - 8 yrs
₹2L - ₹12L / yr
skill iconNodeJS (Node.js)
skill iconMongoDB
Mongoose
skill iconExpress
TypeScript
+2 more

Job Title : Node.js Developer

Experience : 3+ Years

Location : Gurgaon

Interview Rounds : 3


Mandatory Key Skills :

  • Node.js, TypeScript
  • PostgreSQL, MongoDB
  • API Development, Microservices

Responsibilities :

  • Develop and maintain scalable backend applications.
  • Design and optimize database queries in PostgreSQL & MongoDB.
  • Collaborate with cross-functional teams for seamless integration.
Read more
IT Clients
IT Clients
Agency job
via kaldan soft by sathya priya
Chennai
4 - 8 yrs
₹1L - ₹12L / yr
skill iconJava
skill iconSpring Boot
Microservices

Job Description – Senior Java Developer (Spring Boot / Microservices)

Location: Chennai, India

Experience: 4+ Years

Availability: Immediate Joiner

About the Role:

We are looking for a highly skilled Java Developer with over 5 years of hands-on experience in building scalable backend systems using Java, Spring Boot, and Microservices architecture. The ideal candidate will have strong problem-solving skills, deep knowledge of backend frameworks, and a solid understanding of JUnit-based testing.

Key Responsibilities:

  • Design, develop, and maintain high-performance backend systems using Java and Spring Boot
  • Develop and deploy RESTful microservices in a distributed architecture
  • Collaborate with cross-functional teams to define, design, and ship new features
  • Write clean, scalable, and testable code following best practices
  • Implement unit and integration tests using JUnit and other testing frameworks
  • Participate in code reviews and contribute to improving team processes
  • Troubleshoot and resolve production issues in a timely manner

Key Skills:

  • Strong proficiency in Java
  • Expertise in Spring Boot and related modules (Spring Data, Spring Security, etc.)
  • Solid understanding of Microservices architecture, REST APIs, and inter-service communication
  • Experience with JUnit and writing automated test cases
  • Familiarity with CI/CD pipelines and version control (Git)
  • Knowledge of containerization (Docker, Kubernetes) is a plus
  • Exposure to messaging systems like Kafka or RabbitMQ is a plus
  • Familiarity with databases (SQL/NoSQL)


Read more
Hyderabad
3 - 6 yrs
₹10L - ₹16L / yr
SQL
Spark
Analytical Skills
Hadoop
Communication Skills
+4 more

The Sr. Analytics Engineer would provide technical expertise in needs identification, data modeling, data movement, and transformation mapping (source to target), automation and testing strategies, translating business needs into technical solutions with adherence to established data guidelines and approaches from a business unit or project perspective.


Understands and leverages best-fit technologies (e.g., traditional star schema structures, cloud, Hadoop, NoSQL, etc.) and approaches to address business and environmental challenges.


Provides data understanding and coordinates data-related activities with other data management groups such as master data management, data governance, and metadata management.


Actively participates with other consultants in problem-solving and approach development.


Responsibilities :


Provide a consultative approach with business users, asking questions to understand the business need and deriving the data flow, conceptual, logical, and physical data models based on those needs.


Perform data analysis to validate data models and to confirm the ability to meet business needs.


Assist with and support setting the data architecture direction, ensuring data architecture deliverables are developed, ensuring compliance to standards and guidelines, implementing the data architecture, and supporting technical developers at a project or business unit level.


Coordinate and consult with the Data Architect, project manager, client business staff, client technical staff and project developers in data architecture best practices and anything else that is data related at the project or business unit levels.


Work closely with Business Analysts and Solution Architects to design the data model satisfying the business needs and adhering to Enterprise Architecture.


Coordinate with Data Architects, Program Managers and participate in recurring meetings.


Help and mentor team members to understand the data model and subject areas.


Ensure that the team adheres to best practices and guidelines.


Requirements :


- Strong working knowledge of at least 3 years of Spark, Java/Scala/Pyspark, Kafka, Git, Unix / Linux, and ETL pipeline designing.


- Experience with Spark optimization/tuning/resource allocations


- Excellent understanding of IN memory distributed computing frameworks like Spark and its parameter tuning, writing optimized workflow sequences.


- Experience of relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., Redshift, Bigquery, Cassandra, etc).


- Familiarity with Docker, Kubernetes, Azure Data Lake/Blob storage, AWS S3, Google Cloud storage, etc.


- Have a deep understanding of the various stacks and components of the Big Data ecosystem.


- Hands-on experience with Python is a huge plus

Read more
Algorithma Digitech Pvt Ltd
Kalamassery
1 - 5 yrs
₹1L - ₹1.5L / yr
skill iconPython
skill iconDjango
skill iconFlask
skill iconGit
MySQL
+2 more

Title: Python Developer

Job Type: Full-time

Responsibilities:

  • Developing and maintaining Python-based applications.
  • Writing clean, maintainable, and efficient code.
  • Integrating user-facing elements developed by front-end developers with server-side logic.
  • Identifying and fixing bottlenecks and bugs.
  • Participating in code reviews.
  • Collaborating with cross-functional teams to design, develop and implement new features.

Requirements:

  • Bachelor's or Master's degree in Computer Science or a related field.
  • Strong proficiency in Python and its frameworks such as Django or Flask.
  • Understanding of fundamental design principles behind a scalable application.
  • Familiarity with code versioning tools such as Git.
  • Knowledge of databases such as MySQL, PostgreSQL or MongoDB.
  • Good problem-solving skills and attention to detail.
  • Excellent communication and collaboration skills.


Read more
Remote only
5 - 8 yrs
₹10L - ₹20L / yr
skill iconPython
skill iconDjango
skill iconFlask
Google Cloud Platform (GCP)
API
+2 more

Senior Python Developer (6 + Years Experience)


Core Skills:


  • Strong Python experience and understanding of modern design patterns, abstractions, object oriented programming at scale.
  • Strong understanding of event based/async architectures (Kafka, WebSockets)
  • Database interactions
  • Working understanding of infrastructure as code (terraform ) and how develop code that will be deployed by IAC.
  • Working understanding of Kubernetes including containerization, deploying, debugging services running on k8s.


Preferred:


  • Working GCP experience including GKE, GCE, Cloud functions/cloud run, IAM.
  • Git required
  • CICD and automation experience; stack includes Jenkins, Terraform, Argo, Harness.


Read more
Vauld
Vauld
Agency job
via Beiing Corporate Private Limited by Karen Fernandes
Remote, Bengaluru (Bangalore)
3 - 8 yrs
₹35L - ₹70L / yr
skill iconJava
skill iconPython
skill iconRuby
skill iconRuby on Rails (ROR)
skill iconGo Programming (Golang)
+4 more
Requirements:
You have 7 years or more of backend development experience with expertise in designing and
implementing high-performance applications.
Worked with and built high performance products using Node Js/GoLang/ Python and have seen
the product evolution cycle from PoC to Production
You have a deep understanding of design principles and can rightly judge/implement solutions to
complex problems
You are capable of writing high-quality code that conforms to the standards and best practices in a
particular language/framework.
You have experience building a product using a mainstream library or framework.
You love to own end to end responsibility, starting from defining the problem statement (either
yourself or alongside your peers), development (PoC if needed), testing, releasing in staging & then
production environment, and finally monitoring.
Read more
Appristine Technology
at Appristine Technology
1 recruiter
Prasad Jadhav
Posted by Prasad Jadhav
Pune
2 - 5 yrs
₹3L - ₹8L / yr
skill iconPHP
MySQL
skill iconJava
skill iconPython
skill iconLaravel

Appristine Technologies. We are based in Pune and we are hiring for the following role.

1. PHP Developer – 2+ Years Exp.

Skill:-

  • Strong skill level in PHP
  • Intermediate skill level in HTML and CSS
  • Intermediate skill level in JavaScript or JQuery
  • Intermediate skill level in at least one server-side language (i.e. Java, Python, etc)
  • Intermediate skill level with relational database systems (i.e. PostgreSQL, MySQL, etc)
  • Intermediate skill level with a source control software (i.e. Git, SVN, etc)
  • Good understanding of UI/UX design principles
  • Strong technical and non-technical communication skills
  • Good understanding of object-oriented programming
Read more
SmartDocs
at SmartDocs
4 recruiters
Ananda Bala
Posted by Ananda Bala
Hyderabad
3 - 6 yrs
₹6L - ₹12L / yr
skill iconJava
J2EE
Spring
skill iconJavascript
skill iconMongoDB
+3 more
Are you a fun loving and passionate to be a part of Global Innovator team? Are you planning to grow your career, which enhance your skills in technology? A career at SmartDocs can open many doors for you in the world of technology. If you are looking for a company that is dedicated to your ideas, recognizes you for your unique competency & contributions and provides a fun, flexible and delightful work atmosphere. Then, we are the right place to ignite your passion. We are totally committed about our employees, our Clients & Customers, our work culture and especially our technology. We are an organization who encourages new ideas of employees through collaboration and creativity. We are seeking for a dedicated individual to join SmartDocs Business Solutions. The candidate will work with the product development team and must be a subject matter expert in-order to deliver the best practices; an ideal candidate must possess excellent business skills with outstanding communication, professionalism & leadership and must have minimum 3 - 6 years of experience in JAVA as a full stack developer. Desirable Skills & Expertise:  Minimum 3 – 6 years of experience as a full stack developer (Preferable from product based entity). At least a year of experience in managing a team is mandatory.  Master’s or bachelor’s degree in Engineering (CSE / IT / MCA / Software) are preferred.  Work exp. in Java, Frame-works like : Spring, Spring boot; J2EE technologies like Servlets, JSP and Web Application Server is mandatory.  Strong skills in HTML, Java Script, XML, CSS and Mango DB (Optional).  Product development in Linux environment is an added advantage.  Possesses intellectual humility; smart, driven, creative and able to learn things from slipups, willing to raise others up.  Excellent Communication & Analytical skills with integrated professionalism at all levels. Key Responsibilities:  The role is responsible for designing, coding and deploying of high-performance applications.  Deep expertise and hands on experience as a Core JAVA Developer.  Team Management, Excellent interpersonal, Communication and effective organizational skills and solid technical skills.  Should be able to communicate effectively with both technical and non-technical personnel.  Excellent trouble shootings & problem-solving skills.  Experience with SDLC from requirement analysis through testing and deployment.  Must be an expert in technologies like J2EE, JSP, Servlets, Web Application, HTML, Java Script, XML, CSS, Mango DB and JAVA frameworks – Spring boot.  Develop and maintain strong product knowledge.  Guide the clients through various stages of the project and transition to support organization.  Review existing business processes and participate in the Process Improvement Program. SmartDocs, provides equal employment opportunity to all its employees and all qualified applicants for employment without regard to their race, caste, religion, color, ancestry, marital status, gender, sexual orientation, age, nationality, ethnic origin or disability. Our resource policies shall promote diversity and equality in the workplace while encouraging the adoption of international best practices.
Read more
Dataweave Pvt Ltd
at Dataweave Pvt Ltd
32 recruiters
Sanket Patil
Posted by Sanket Patil
Bengaluru (Bangalore)
7 - 10 yrs
Best in industry
skill iconPython
RESTful APIs
MySQL
Apache Kafka
NOSQL Databases
+3 more
Products@DataWeave:
We, the Products team at DataWeave, build data products that provide timely insights that are readily consumable and actionable, at scale. Our underpinnings are: scale, impact, engagement, and visibility. We help
businesses take data driven decisions everyday. We also give them insights for long term strategy. We are focused on creating value for our customers and help them succeed.

How we work
It's hard to tell what we love more, problems or solutions! Every day, we choose to address some of the hardest data problems that there are. We are in the business of making sense of messy public data on the web. At
serious scale! Read more on Become a DataWeaver

What do we offer?
- Opportunity to work on some of the most compelling data products that we are building for online retailers and brands.
- Ability to see the impact of your work and the value you are adding to our customers almost immediately.
- Opportunity to work on a variety of challenging problems and technologies to figure out what really excites you.
- A culture of openness. Fun work environment. A flat hierarchy. Organization wide visibility. Flexible working hours.
- Learning opportunities with courses, trainings, and tech conferences. Mentorship from seniors in the team.
- Last but not the least, competitive salary packages and fast paced growth opportunities.

Roles and Responsibilities:
● Build a low latency serving layer that powers DataWeave's Dashboards, Reports, and Analytics
functionality
● Build robust RESTful APIs that serve data and insights to DataWeave and other products
● Design user interaction workflows on our products and integrating them with data APIs
● Help stabilize and scale our existing systems. Help design the next generation systems.
● Scale our back end data and analytics pipeline to handle increasingly large amounts of data.
● Work closely with the Head of Products and UX designers to understand the product vision and design
philosophy
● Lead/be a part of all major tech decisions. Bring in best practices. Mentor younger team members and
interns.
● Constantly think scale, think automation. Measure everything. Optimize proactively.
● Be a tech thought leader. Add passion and vibrancy to the team. Push the envelope.

Skills and Requirements:
● 5-7 years of experience building and scaling APIs and web applications.
● Experience building and managing large scale data/analytics systems.
● Have a strong grasp of CS fundamentals and excellent problem solving abilities. Have a good understanding of software design principles and architectural best practices.
● Be passionate about writing code and have experience coding in multiple languages, including at least one scripting language, preferably Python.
● Be able to argue convincingly why feature X of language Y rocks/sucks, or why a certain design decision is right/wrong, and so on.
● Be a self-starter—someone who thrives in fast paced environments with minimal ‘management’.
● Have experience working with multiple storage and indexing technologies such as MySQL, Redis, MongoDB, Cassandra, Elastic.
● Good knowledge (including internals) of messaging systems such as Kafka and RabbitMQ.
● Use the command line like a pro. Be proficient in Git and other essential software development tools.
● Working knowledge of large-scale computational models such as MapReduce and Spark is a bonus.
● Exposure to one or more centralized logging, monitoring, and instrumentation tools, such as Kibana, Graylog, StatsD, Datadog etc.
● Working knowledge of building websites and apps. Good understanding of integration complexities and dependencies.
● Working knowledge linux server administration as well as the AWS ecosystem is desirable.
● It's a huge bonus if you have some personal projects (including open source contributions) that you work on during your spare time. Show off some of your projects you have hosted on GitHub.
Read more
ClearTrail Technologies
Nitin Gogia
Posted by Nitin Gogia
Indore
8 - 10 yrs
₹16L - ₹26L / yr
skill iconJava
Multithreading
Object Oriented Programming (OOPs)
JSON
RabbitMQ
+2 more

Technologies

Must have                         Core JAVA, J2EE, Multithreading, OOPS, and Multiprocessing

Great to have                   RabbitMQ, Netty, XML, JSON, Messagepack

Huge Advantage:             Good at mathematics, statistics, complex data structures, algorithms, and design patterns

 

Other skills

  • Ability to understand most complex and challenging problems and solve them from users’ view point
  • Ability to create and present software designs (as per the aligned solution) to the product owner
  • Ability to build and create modules from scratch.
  • Ability to write, understand and deal with large code base
  • Experience in understanding and implementing design patterns
  • Ability to provide technical leadership to a team of equally talented, efficient and motivated engineers

 

Roles and responsibilities

  • Responsible for solving the most complex and seemingly impossible problems related to variety of dimensions of software engineering. Dealing with Big Data to Complex Data Structures, From Inter Process Communication to sub 100 MilliSec server responses, From Distributed Systems to 99.999 Availability.
  • Responsible for providing technical leadership to a team of 3-4 Engineers
  • Responsible to drive delivery of a project/module and report to Product Owner
  • Responsible for functional/technical tracking of a project/module

 

Responsibility Split

  • 80% Technology, 20% Team management
  • With 80:20 split this is definitely a technology oriented role suitable for geeks.
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos