Sr. Spark Software Engineer at Technology service company · Remote only · 5 - 10 years · ₹10L - ₹20L / yr · Remote only · Posted 18 Jan 2022

-
Bachelor’s or master’s degree in Computer Engineering, Computer Science, Computer Applications, Mathematics, Statistics, or related technical field. Relevant experience of at least 3 years in lieu of above if from a different stream of education.
-
Well-versed in and 3+ hands-on demonstrable experience with: ▪ Stream & Batch Big Data Pipeline Processing using Apache Spark and/or Apache Flink.
▪ Distributed Cloud Native Computing including Server less Functions
▪ Relational, Object Store, Document, Graph, etc. Database Design & Implementation
▪ Micro services Architecture, API Modeling, Design, & Programming -
3+ years of hands-on development experience in Apache Spark using Scala and/or Java.
-
Ability to write executable code for Services using Spark RDD, Spark SQL, Structured Streaming, Spark MLLib, etc. with deep technical understanding of Spark Processing Framework.
-
In-depth knowledge of standard programming languages such as Scala and/or Java.
-
3+ years of hands-on development experience in one or more libraries & frameworks such as Apache Kafka, Akka, Apache Storm, Apache Nifi, Zookeeper, Hadoop ecosystem (i.e., HDFS, YARN, MapReduce, Oozie & Hive), etc.; extra points if you can demonstrate your knowledge with working examples.
-
3+ years of hands-on development experience in one or more Relational and NoSQL datastores such as PostgreSQL, Cassandra, HBase, MongoDB, DynamoDB, Elastic Search, Neo4J, etc.
-
Practical knowledge of distributed systems involving partitioning, bucketing, CAP theorem, replication, horizontal scaling, etc.
-
Passion for distilling large volumes of data, analyze performance, scalability, and capacity performance issues in Big Data Platforms.
-
Ability to clearly distinguish system and Spark Job performances and perform spark performance tuning and resource optimization.
-
Perform benchmarking/stress tests and document the best practices for different applications.
-
Proactively work with tenants on improving the overall performance and ensure the system is resilient, and scalable.
-
Good understanding of Virtualization & Containerization; must demonstrate experience in technologies such as Kubernetes, Istio, Docker, OpenShift, Anthos, Oracle VirtualBox, Vagrant, etc.
-
Well-versed with demonstrable working experience with API Management, API Gateway, Service Mesh, Identity & Access Management, Data Protection & Encryption.
Hands-on experience with demonstrable working experience with DevOps tools and platforms viz., Jira, GIT, Jenkins, Code Quality & Security Plugins, Maven, Artifactory, Terraform, Ansible/Chef/Puppet, Spinnaker, etc.
-
Well-versed in AWS and/or Azure or and/or Google Cloud; must demonstrate experience in at least FIVE (5) services offered under AWS and/or Azure or and/or Google Cloud in any categories: Compute or Storage, Database, Networking & Content Delivery, Management & Governance, Analytics, Security, Identity, & Compliance (or) equivalent demonstrable Cloud Platform experience.
-
Good understanding of Storage, Networks and Storage Networking basics which will enable you to work in a Cloud environment.
-
Good understanding of Network, Data, and Application Security basics which will enable you to work in a Cloud as well as Business Applications / API services environment.

Similar jobs (10)
We are seeking a skilled Java Developer with hands-on experience in Java and Spark to build scalable data processing solutions. You'll contribute to high-performance data pipelines and analytics platforms in a dynamic Agile environment.
Key Responsibilities
- Design and develop Java applications integrated with Apache Spark for ETL processes, data transformations, and analytics.
- Build and optimize Spark jobs (Spark SQL, DataFrames, Streaming) for large-scale data processing.
- Collaborate with data engineers and analysts to implement robust data workflows.
- Write clean, maintainable Java code following best practices (Spring Boot, Microservices preferred).
- Perform code reviews, unit testing, and contribute to CI/CD pipelines.
- Troubleshoot and optimize Spark performance for production workloads.
- Document technical solutions and mentor junior developers.
Required Skills & Qualifications
- 4-7 years of hands-on Java development experience.
- Strong expertise in Apache Spark (Spark Core, Spark SQL, PySpark basics).
- Proficiency in Java 8/11+ with multithreading and collections frameworks.
- Experience with data processing (ETL, data pipelines, big data).
- Familiarity with build tools (Maven/Gradle) and version control (Git).
- Strong problem-solving skills and Bangalore location availability.
- Excellent communication skills for cross-team collaboration.
Good to Have
- Experience with Snowflake for cloud data warehousing.
- Knowledge of DBT (Data Build Tool) for analytics engineering.
- Python scripting for data manipulation and automation.
- Exposure to AWS/GCP/Azure cloud platforms.
- Familiarity with Kafka, Airflow, or containerization (Docker/Kubernetes).
We are looking for a Java Backend Developer to join our team. You will be responsible for developing and maintaining the server-side logic and databases of our applications, ensuring they run smoothly and efficiently.
Responsibilities:
- Write and maintain server-side code using Java.
- Develop and integrate APIs to support frontend functionality.
- Work with databases to store, retrieve, and manipulate data.
- Troubleshoot, debug, and optimize backend performance.
- Collaborate with frontend developers to integrate backend and frontend systems.
- Write unit tests to ensure code quality.
Required Skills:
- Strong experience with Java.
- Familiarity with Spring Boot or other Java frameworks.
- Knowledge of relational databases (e.g., MySQL, PostgreSQL).
- Understanding of REST APIs.
- Experience with version control systems (e.g., Git).
- Basic knowledge of unit testing (e.g., JUnit).
Preferred Skills:
- Familiarity with Microservices and cloud platforms.
- Experience with Docker and containerization.
Skills Required:
- Experience of more than 4 to 6 years in software development.
- Very Strong Experience in Core Java.
- Excellent Java Programming skills Experience in Data Structures, Algorithms and Design Patterns.
- Strong in Problem solving, Analytical skill and logical thinking.
- Skill to be trained: java, spring, mango db, cassandra.
- Strong experience in Spring Boot, Restful API
- Looking for shorter notice period candidates only.
Wissen Technology is now hiring for Java Developer with hands on experience in Core Java, multithreading, algorithms, data structure and SQL skills .
Required Skills:
- Experience - 5 to 10 years.
- Experience in Core Java 5.0 and above, CXF, Spring.
- Extensive experience in developing enterprise-scale n-tier applications for financial domain. Should possess good architectural knowledge and be aware of enterprise application design patterns.
- Should have the ability to analyze, design, develop and test complex, low-latency client-facing applications.
- Good development experience with RDBMS, preferably Sybase database.
- Good knowledge of multi-threading and high-volume server-side development.
- Experience in sales and trading platforms in investment banking/capital markets.
- Basic working knowledge of Unix/Linux.
- Excellent problem solving and coding skills in Java.
- Strong interpersonal, communication and analytical skills.
- Should have the ability to express their design ideas and thoughts.
Java back end developer
Chennai
3+yrs
3 to 5 years Java,
Sprint Boot, Hibernate,
REST API Design Patterns SQL,
Oracle,
Table design
Hands on experience with Spring/Hibernate or .Net frame works.
Will have 3 months of training on ALM by the company.
Your Responsibilities :
1) Building highly scalable and secured applications/microservices and responsive APIs
2) Assessing the security risks associated with the platform architecture and suggesting/implementing suitable changes for the same.
3) Architect and implement back-end services to support our mobile and web clients.
4) Creating and implementing database schemas that represent and support business processes.
5) Use best software development practices and processes including unit testing, code reviews, and site monitoring post deployment.
6) Continue to research and evaluate new technologies that may improve performance and elegance of our software implementation.
7) Explore and Integrate third-party APIs when required for the project.
8) Work closely with the rest of the team to bring our features into production.
Your skills :
- Hands-on experience in Django Framework.
- Strong Knowledge of Python.
- Strong Knowledge of MVC design pattern and object oriented programming concepts.
- Strong knowledge of RDBMS like Postgresql and Database schema design.
- Experience in developing robust and secure RESTful web API and microservices using Django/Python.
- Experience in Test Driven Development.
- Experience in handling No-SQL MongoDB or similar.
- Hands-on experience in GitHub.
- Experience in CI, CD tools like jenkins.
- Experience in managing cloud-based web applications on Amazon Web Services or Google Cloud.
- You love code and up for the challenge!
Requires a bachelor's degree in area of specialty and experience in the field or in a related area. Familiar with standard concepts, practices, and procedures within a particular field. Relies on experience and judgment to plan and accomplish goals. Performs a variety of tasks. A degree of creativity and latitude is required. Typically reports to a supervisor or manager.
Designs, develops, and implements web-based Java applications to support business requirements. Follows approved life cycle methodologies, creates design documents, and performs program coding and testing. Resolves technical issues through debugging, research, and investigation.
Additional Job Details:
Strong in Java, Spring, Spring Boot, REST and developing MicroServices.
Knowledge or experience , Cassandra preferred
Knowledge or experience on Kafka
Good to have but not must
Good to know:
Reporting tools like Splunk/Grafana
Protobuf
Python/Ruby
Required Knowledge and technical skills:
- Expert level experience designing and building web services using Java
- Hands on experience in frameworks like struts , springs, hibernate
- Experience with multiple database platforms
- Experience with high volume, mission critical applications
- Experience building distributed systems at Internet scale
- Knowledge of commonly used design patterns
- Prior experience building internet platforms from the ground up
- Understanding Microservices patterns and architectures, enterprise design and industry best practices.
Required Qualities/ personal skills:
- Strong written and verbal communication skills
- Strong interpersonal skills and time management skills
- Strong analytical and troubleshooting skills
- Experience working in an Agile environment
- Experience working in a fast-paced delivery environment









