Cutshort logo
Technology service company logo
Sr. Spark Software Engineer
Technology service company
Sr. Spark Software Engineer
Technology service company's logo

Sr. Spark Software Engineer

at Technology service company

Agency job
5 - 10 yrs
₹10L - ₹20L / yr
Remote only
Skills
skill iconJava
J2EE
skill iconSpring Boot
Hibernate (Java)
Ansible
skill iconGit
JIRA
Apache Spark
Spark
Apache Kafka
Microservices
skill iconKubernetes
Terraform
NOSQL Databases
API
skill iconDocker
  • Bachelor’s or master’s degree in Computer Engineering, Computer Science, Computer Applications, Mathematics, Statistics, or related technical field. Relevant experience of at least 3 years in lieu of above if from a different stream of education.

  • Well-versed in and 3+ hands-on demonstrable experience with: ▪ Stream & Batch Big Data Pipeline Processing using Apache Spark and/or Apache Flink.
    ▪ Distributed Cloud Native Computing including Server less Functions
    ▪ Relational, Object Store, Document, Graph, etc. Database Design & Implementation
    ▪ Micro services Architecture, API Modeling, Design, & Programming

  • 3+ years of hands-on development experience in Apache Spark using Scala and/or Java.

  • Ability to write executable code for Services using Spark RDD, Spark SQL, Structured Streaming, Spark MLLib, etc. with deep technical understanding of Spark Processing Framework.

  • In-depth knowledge of standard programming languages such as Scala and/or Java.

  • 3+ years of hands-on development experience in one or more libraries & frameworks such as Apache Kafka, Akka, Apache Storm, Apache Nifi, Zookeeper, Hadoop ecosystem (i.e., HDFS, YARN, MapReduce, Oozie & Hive), etc.; extra points if you can demonstrate your knowledge with working examples.

  • 3+ years of hands-on development experience in one or more Relational and NoSQL datastores such as PostgreSQL, Cassandra, HBase, MongoDB, DynamoDB, Elastic Search, Neo4J, etc.

  • Practical knowledge of distributed systems involving partitioning, bucketing, CAP theorem, replication, horizontal scaling, etc.

  • Passion for distilling large volumes of data, analyze performance, scalability, and capacity performance issues in Big Data Platforms.

  • Ability to clearly distinguish system and Spark Job performances and perform spark performance tuning and resource optimization.

  • Perform benchmarking/stress tests and document the best practices for different applications.

  • Proactively work with tenants on improving the overall performance and ensure the system is resilient, and scalable.

  • Good understanding of Virtualization & Containerization; must demonstrate experience in technologies such as Kubernetes, Istio, Docker, OpenShift, Anthos, Oracle VirtualBox, Vagrant, etc.

  • Well-versed with demonstrable working experience with API Management, API Gateway, Service Mesh, Identity & Access Management, Data Protection & Encryption.

    Hands-on experience with demonstrable working experience with DevOps tools and platforms viz., Jira, GIT, Jenkins, Code Quality & Security Plugins, Maven, Artifactory, Terraform, Ansible/Chef/Puppet, Spinnaker, etc.

  • Well-versed in AWS and/or Azure or and/or Google Cloud; must demonstrate experience in at least FIVE (5) services offered under AWS and/or Azure or and/or Google Cloud in any categories: Compute or Storage, Database, Networking & Content Delivery, Management & Governance, Analytics, Security, Identity, & Compliance (or) equivalent demonstrable Cloud Platform experience.

  • Good understanding of Storage, Networks and Storage Networking basics which will enable you to work in a Cloud environment.

  • Good understanding of Network, Data, and Application Security basics which will enable you to work in a Cloud as well as Business Applications / API services environment.

Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs

Quant LegalTech
Quant LegalTech
Agency job
via wrackle by Naveen Taalanki
Bengaluru (Bangalore)
2 - 5 yrs
₹6L - ₹10L / yr
skill iconPython
skill iconDjango
skill iconFlask
Web2py
RESTful APIs
+11 more
Responsibilities:
• As a member of the development team, become deeply familiar with the product goals, and provide thought leadership on the technical aspects of realizing the product goals
• Contribute to the SDLC best practices, modeled after contemporary industry standards, and ensure the practices are diligently followed
• Encourage automation in various aspects of software development such as code reviews, unit tests, continuous integration, system testing. Identify appropriate tools, and create best practices around automation
• Participate in requirement gathering, analysis, designing proofs of concepts
• Team up with various stakeholders in aiding project planning, estimation, execution, and production support
• Identify the most appropriate platforms, frameworks, and third-party components to meet the product requirements
• Define and own the interfaces, such as services and APIs between product components
• Perform code walkthroughs, reviews. Promote code-reuse. Identify opportunities for refactoring code, and ensure code is modular and extensible.
• Ensure adequate coverage of functional and non-functional requirements in test plans
• Provide guidance on production support/bug-fixing issues. Own the technical resolution of high-priority fixes

Technical Skills:
• 0-3 years of hands-on experience with Python and Python-based frameworks such as Django, Flask, and web2py, etc.
• Hands-on experience with web-based front-end technologies is a plus
• Understanding of fundamental design principles for building a scalable web application and integration using RESTful APIs and JSON data
• Deep familiarity with n-tier and MVC patterns
• Strong unit testing and debugging skills
• Familiarity with CI tools such as Jenkins

Who are we looking for?
• Should be comfortable working in a team-oriented, fast-paced, collaborative environment
• Willingness to take on ownership and responsibility
• Resourceful, confident under pressure, ability to communicate with folks with varied experience and backgrounds
• Aptitude for problem-solving and analytical skills
Read more
Play Games24x7
at Play Games24x7
2 recruiters
Agency job
via Zyoin Web Private Limited by Vishali Vashnavi
Bengaluru (Bangalore)
8 - 12 yrs
₹40L - ₹50L / yr
skill iconJava
J2EE
skill iconPostgreSQL
MySQL
skill iconMongoDB
+19 more
Requirements:
• B. E. /B. Tech. in Computer Science or MCA from a reputed university.
• 3.5 plus years of experience in software development, with emphasis on JAVA/J2EE Server side
programming.
• Hands on experience in core Java, multithreading, RMI, socket programing, JDBC, NIO, webservices
and design patterns.
• Knowledge of distributed system, distributed caching, messaging frameworks, ESB etc.
• Experience in Linux operating system and PostgreSQL/MySQL/MongoDB/Cassandra database.
• Additionally, knowledge of HBase, Hadoop and Hive is desirable.
• Familiarity with message queue systems and AMQP and Kafka is desirable.
• Experience as a participant in agile methodologies.
• Excellent written and verbal communication skills and presentation skills.
• This is not a fullstack requirement, we are looking for a purely backend expert.
Read more
predictive analytics platform
predictive analytics platform
Agency job
via Hunt & Badge Consulting Pvt Ltd by Chandramohan Subramanian
Chennai
2 - 7 yrs
₹15L - ₹30L / yr
skill iconJava
  • Interest in our tech stack (Java or Scala, SQL Databases, NoSQL, Microservices, and AWS)
  • Strong computer science fundamentals with data structures, algorithms, various programming languages, distributed systems, etc. 
  • Understanding of engineering best practices—you constantly seek to improve engineering standards, tooling and processes
  • Ability to write “good code” - easily understandable, secure and testable! 
  • Comfortable helping drive technical decisions within our Engineering team
  • Experience building applications at scale
Read more
Elastic
Bengaluru (Bangalore)
5 - 10 yrs
Best in industry
skill iconJava
skill iconPython
skill iconRuby
skill iconRuby on Rails (ROR)
skill iconGo Programming (Golang)
+2 more

Elastic is a search company built on a free and open heritage. Anyone can use Elastic products and solutions to get started quickly and frictionlessly. Elastic offers three solutions for enterprise search, observability, and security, built on one technology stack that can be deployed anywhere. From finding documents to monitoring infrastructure to hunting for threats, Elastic makes data usable in real time and at scale. Thousands of organizations worldwide, including Cisco, eBay, Goldman Sachs, Microsoft, The Mayo Clinic, NASA, The New York Times, Wikipedia, and Verizon, use Elastic to power mission-critical systems. Founded in 2012, Elastic is a distributed company with Elasticians around the globe and is publicly traded on the NYSE under the symbol ESTC. Learn more at https://www.elastic.co/">elastic.co.

 

The Observability team is in charge of developing solutions that focus on application developers and engineers that run infrastructure and services supporting these applications. Elasticsearch is an efficient datastore for logs, metrics, and application traces, supporting the three pillars of observability. The Observability team builds and maintains solutions that make getting insights from this data turnkey and efficient, such as our https://www.elastic.co/solutions/apm">APM, https://www.elastic.co/solutions/metrics">Infrastructure Monitoring, and https://www.elastic.co/uptime-monitoring">Synthetics solutions. When developing these solutions, we think about the problem end-to-end: how do we automatically collect data from common data sources, how do we store it efficiently in Elasticsearch, how do we present this information to the user, what actions do we take on the insights from the data? All of these aspects are important in bringing a turnkey solution to the market. 

You will be working on Observability Integrations to collect logs and metrics from all the systems that matter to our users, using the current and next generation of data shippers: Beats and the new Elastic Agent.

Job Location : Bangalore, India

What you will be working on 

  • Work with product management to define the requirements for new https://github.com/elastic/integrations">integrations: data collection strategies, storage optimization, alerting and visualization options, refining requirements, translating requirements to stories and epics, and performing detailed scoping of features
  • Develop, maintain and evolve https://github.com/elastic/integrations">integrations for multiple services using the Elastic Stack and https://github.com/elastic/elastic-package">specific tooling. For some projects, contributions to the https://github.com/elastic/beats">beats platform will be needed (Go).
  • Work with our support team to help customers troubleshoot and solve their issues in production environments, ensuring SLAs are met.
  • Interact with the community and help them use our products.
  • Collaborate with other engineering, QA and documentation teams to execute on product deliverables.

What you will bring along

  • Experience operating or developing monitoring products, such as Elastic (ELK, Elastic Observability),  Prometheus, Grafana, ELK, Datadog, New Relic, etc.
  • Proficiency in at least one programming language (Go, Java, PHP, Python, Ruby, C++,  we use Go) with strong analytical, debugging, and troubleshooting skills.
  • Excellent verbal and written communication skills and ability to work in a distributed team.
  • Experience participating in technical decision-making across different engineering teams.

Bonus Skills

  • Interest in learning new technologies.
  • Experience contributing to open source projects.
  • Hands-on experience managing cloud services at scale.
  • Experience in SRE and / or DevOps roles.
  • Experience monitoring or operating container orchestrators (Kubernetes, etc).

 

 

Additional Information - We Take Care of Our People

As a distributed company, diversity drives our identity. Whether you’re looking to launch a new career or grow an existing one, Elastic is the type of company where you can balance great work with great life. Your age is only a number. It doesn’t matter if you’re just out of college or your children are; we need you for what you can do.

We strive to have parity of benefits across regions and while regulations differ from place to place, we believe taking care of our people is the right thing to do.

  • Competitive pay based on the work you do here and not your previous salary
  • Health coverage for you and your family in many locations
  • Ability to craft your calendar with flexible locations and schedules for many roles
  • Generous number of vacation days each year
  • Double your charitable giving - We match up to $1500 (or local currency equivalent)
  • Up to 40 hours each year to use toward volunteer projects you love
  • Embracing parenthood with minimum of 16 weeks of parental leave
Read more
CricStox Private Limited
at CricStox Private Limited
3 recruiters
Ishwar Sharma
Posted by Ishwar Sharma
Pune
2 - 3 yrs
₹8L - ₹10L / yr
skill iconNodeJS (Node.js)
Microservices
skill iconKubernetes
skill iconDocker
skill iconAmazon Web Services (AWS)
+1 more
Backend Cloud Engineer @ CricStox
CricStox is a Pune startup building a trading solution in the realm of gametech x fintech.
We intend to build a sport-agnostic platform to allow trading in stocks of sportspersons under any sport
through our mobile & web-based applications.
We’re currently hiring a Backend Cloud Engineer who will gather, refine specifications and requirements
based on technical needs and implement the same by using best software development practices.
Responsibilities?
● Mainly, but not limited to maintaining, expanding, and scaling our microservices/ app/ site.
● Integrate data from various back-end services and databases.
● Always be plugged into emerging technologies/industry trends and apply them into operations and
activities.
● Comfortably work and thrive in a fast-paced environment, learn rapidly and master diverse web
technologies and techniques.
● Juggle multiple tasks within the constraints of timelines and budgets with business acumen.
What skills do I need?
● Excellent programming skills in Javascript or Typescript.
● Excellent programming skills in Nodejs with Nestjs framework or equivalent.
● A solid understanding of how web applications work including security, session management, and
best development practices.
● Good working knowledge and experience of how AWS cloud infrastructure works including services
like APIGateway, Cognito, S3, EC2, RDS, SNS, MSK, EKS is a MUST.
● Solid understanding of distributed event streaming technologies like Kafka is a MUST.
● Solid understanding of microservices communication using Saga Design pattern is a MUST.
● Adequate knowledge of database systems, OOPs and web application development.
● Adequate knowledge to create well-designed, testable, efficient APIs using tools like Swagger (or
equivalent).
● Good functional understanding of ORMs like Prisma (or equivalent).
● Good functional understanding of containerising applications using Docker.
● Good functional understanding of how a distributed microservice architecture works.
● Basic understanding of setting up Github CI/CD pipeline to automate Docker images building,
pushing to AWS ECR & deploying to the cluster.
● Proficient understanding of code versioning tools, such as Git (or equivalent).
● Hands-on experience with network diagnostics, monitoring and network analytics tools.
● Aggressive problem diagnosis and creative problem-solving skills.
Read more
Rajasthan Studio
at Rajasthan Studio
1 recruiter
Amita Tambe Irani
Posted by Amita Tambe Irani
Mumbai, Jaipur, Jajpur
2 - 10 yrs
₹7L - ₹15L / yr
skill iconPHP
Yii
RESTful APIs
GraphQL
skill iconJavascript
+6 more

Rajasthan Studio is looking for young and bright minds for its upcoming innovative, never-experienced-before app platform for virtual art experiences.

 

We are looking for programmers with problem-solving skills and critical thinking abilities for the position of Back End developer. Back End developers are responsible for creating a bridge between customers and business logic, optimizing applications for maximum efficiency, creating dashboards for internal teams, and maintaining brand consistency across the application, among other duties.

 

Backend Web Developers are responsible for managing the interchange of data between the server and the users. The role's primary focus will be the development of all server-side logic, definition, and maintenance of the central database, and ensuring high performance and responsiveness to requests from the front-end. A basic understanding of front-end technologies is necessary as well.

 

Responsibilities -

  • Creating RESTful API/GraphQL to be consumed by Flutter developers
  • Building reusable code and libraries for future use
  • Optimization of the application for maximum speed and scalability
  • Implementation of security and data protection
  • Design and implementation of data storage solutions
  • Participate in the entire application lifecycle, focusing on coding and debugging
  • Write clean code to develop functional web applications
  • Troubleshoot and debug applications
  • Provide training and support to internal teams

 

Skill & Qualifications -

  • Basic understanding of front-end technologies and platforms, such as JavaScript, HTML5, and CSS3
  • Understanding accessibility and security compliances
  • User authentication and authorization between multiple systems, servers, and environments
  • Integration of multiple data sources and databases into one system
  • Management of hosting environment, including database administration and scaling an application to support load changes
  • Data migration, transformation, and scripting
  • Setup and administration of backups
  • Outputting data in different formats
  • Understanding differences between multiple delivery platforms such as mobile vs desktop, and optimizing output to match the specific platform
  • Creating database schemas that represent and support business processes
  • Implementing automated testing platforms and unit tests
  • Proficient understanding of code versioning tools, such as Git
  • Proficient understanding of OWASP security principles
  • Understanding of “session management” in a distributed server environment
  • Knowledge of Yii framework - version 2 (preferred)
Read more
Oust Labs
at Oust Labs
1 video
3 recruiters
Prabhakar Kulkarni
Posted by Prabhakar Kulkarni
Bengaluru (Bangalore)
5 - 8 yrs
₹7L - ₹11L / yr
RESTful APIs
Object Oriented Programming (OOPs)
Database Design
skill iconJava
skill iconAmazon Web Services (AWS)
+3 more
Job Description : PROFESSIONAL QUALIFICATIONS / SKILLS 1) 2 - 4 years of extensive experience in developing Client-Server (Web) applications using Java and J2EE Technologies 2) Expertise in developing applications using JDBC/DBCP2 3) Very Strong Core Java ( Collections, multithreading etc) 4) Expertise in Java Server programming 5) Must have work experience in RESTful Web services 6) Strong in OOPS and designing solutions 7) Good in MYSQL queries scripting and database concepts 8) Good problem solving ability 9) Ability to debug complex issues 10) Hands on in Git 11) Comfortable in working in Linux environment 12) Understanding of JSON 13) Experience in Tomcat 14) Experience in using tools like Eclipse, IntelliJ Good to have: 1) Caching concept and hands-on 2) AWS 3) Messaging Queue (JMS/SQS etc) 4) NoSQL database Thank you
Read more
CloudPanel Technologies Pvt Ltd
Sri Nilaya
Posted by Sri Nilaya
Bengaluru (Bangalore)
4 - 7 yrs
₹4L - ₹6L / yr
skill iconPython
skill iconDjango
MySQL
NOSQL Databases
skill iconMongoDB
+1 more
# In-depth knowledge in Core Python with Django building end to end applications development. # Experience in Web technologies-HTML, CSS, Javascript. # Database - SQL Server/PostGres/ No SQL database. # Good understanding of Algorithms, data structures. # Knowledge in ORM (Object Relational Mapper) libraries. # Experience in integrating multiple data sources and databases into one system. # Knowledge in version control tools like GIT. # Experience with various cloud technologies.
Read more
SabPaisa
at SabPaisa
3 recruiters
Suchismita  Satapathy
Posted by Suchismita Satapathy
NCR (Delhi | Gurgaon | Noida), NCR (Delhi | Gurgaon | Noida)
5 - 8 yrs
₹2L - ₹5L / yr
skill iconJava
JAAS
Spring Security
Apache Shiro
CISSP
+1 more
Roles & Responsibilities • Mini. work experience in following skills 4 to 6 year. • work experience using JAAS (Java Authentication and Authorization Services), Spring Security, Apache Shiro, HDIV, OACC • In-depth knowledge of application security vulnerabilities, testing techniques, and the OWASP framework. • In depth understanding of secure web application development, Java, Java development frameworks, web services and SOAP. • Experience of web application and Agile development methodologies. • Understanding of Apache web server and unix server operating systems. • Comprehensive knowledge of IT and information security subject matter. • Knowledge of security policies and standards and such as PCI-DSS and ISO 17799, 27001. Required Skills & Abilities: • Security exposure in – CISSP, CISA, CISM • Ability to work alone and build relationships across the organization. • Able to priorities workload and drive work to set deadlines • Exposure to methods of promoting security awareness. • Strong communication (verbal/written) and influencing skills, with an ability to manage internal and external relationships up to senior levels of management. • Anticipates problems and identifies long-term implications of decisions and actions. • Knowledge of technical security architectural principles • Security project management experience • Any Graduates/post graduate ideally with around 4-6 years of Java Application Security experience. • Good listening skills and active problem-solving skills • Computer Savvy, Ability to multitask. • Ability to handle pressure
Read more
2020 Imaging India Ltd
at 2020 Imaging India Ltd
1 recruiter
monica raisinghani
Posted by monica raisinghani
Mumbai
5 - 7 yrs
₹1L - ₹1L / yr
Technical Architecture
skill iconJava
skill iconPython
skill iconJavascript
skill iconAmazon Web Services (AWS)
+2 more
Managing tender related activities (Understanding the tender documents, preparing pre-bid queries, tracking important dates, checking for corrigendum’s, maintaining records for that tender etc.) Preparing tender synopsis Tender documentations (MAF, SAC etc.), solution documentations (Technical proposal, solution assumptions, hardware & OS requirements Interact with eco system partners, arriving at integration methodologies, preparing system architecture, preparing solution PPT, interaction with customer & consultant on the solution proposed and making presentations, supporting sales team Full Technical rating of the tender Understanding POC requirements, Interacting with sub system vendors and development team for integration of sub-systems in to CCC for POC, Coordinating with MSI for POC requirements, demonstrating the POC to customer/MSI/Consultant Setting up standard process and check list to speed up the response to RFP
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
21,01,133
Matches delivered
37,12,187
Network size
15,000
Companies hiring
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos