Spark Scala Developer

🚀 Hiring: Data Engineer | GCP + Spark + Python + .NET |

| 6–10 Yrs | Gurugram (Hybrid)

We’re looking for a skilled Data Engineer with strong hands-on experience in GCP, Spark-Scala, Python, and .NET.

📍 Location: Suncity, Sector 54, Gurugram (Hybrid – 3 days onsite)

💼 Experience: 6–10 Years

⏱️ Notice Period :- Immediate Joiner

Required Skills:

• 5+ years of experience in distributed computing (Spark) and software development.
• 3+ years of experience in Spark-Scala
• 5+ years of experience in Data Engineering.
• 5+ years of experience in Python.
• Fluency in working with databases (preferably Postgres).
• Have a sound understanding of object-oriented programming and development principles.
• Experience working in an Agile Scrum or Kanban development environment.
• Experience working with version control software (preferably Git).
• Experience with CI/CD pipelines.
• Experience with automated testing, including integration/delta, Load, and Performance

Key Responsibilities

• Provide technical leadership in the design, development, and delivery of scalable, high-performance software systems.
• Partner with product managers, architects, and cross-functional teams to define technical strategy and ensure alignment with business objectives.
• Lead by example in writing high-quality, testable, and maintainable code.
• Drive best practices in software engineering, including code reviews, system design, and performance optimization.
• Mentor and guide engineers across teams, fostering a culture of technical excellence and continuous learning.
• Evaluate and introduce new technologies, tools, and frameworks to improve productivity, scale and system robustness.

Required Skills & Qualifications

• Strong foundation in computer science fundamentals: data structures, algorithms, and functional programming techniques.
• Expertise in Scala, with strong preference for functional programming.
• Solid experience in software design, implementation, and debugging, including inter-process communication and multi-threading.
• Hands-on experience with distributed systems and event-driven architectures.
• Familiarity with databases (Postgres preferred).
• Proficiency with Apache Kafka for messaging and persistence.
• Working knowledge of Python for unit and integration testing.
• Basic to intermediate experience with Ansible for automation.
• Strong problem-solving, analytical, and communication skills.

Nice-to-Have / Bonus Skills

• Experience with modeling in YANG.
• Experience with Scala libraries such as Cats Effect (2/3), Monix, and Akka.
• Experience working in Agile/Scrum environments.

What We Offer

• Opportunity to work on cutting-edge technologies in a collaborative environment.
• A role with strong ownership, technical influence, and visibility across teams.
• Competitive compensation and benefits.

Roles & Responsibilities:

• 6+ Yrs. of development experience on Scala, Elastic Search and Mongo DB

• Should have experience of working on Responsive and high availability web platforms

• Knowledge on some of the key tech stack like Elastic Search, MongoDB, Play framework, Akka Framework and Java is a must.

• Deep insight into Rest API development • Should be up to date with industry best practices for API development using Scala

• Good team handling skills

• Good development and logical skills • Hands-on experience + Guiding Junior Scala Developers

• Good analytical skills

• Able to learn new technologies on the fly based on requirement

• Able to help the team members as and when needed to improve team capability 

About the Role-

Thinking big and executing beyond what is expected. The challenges cut across algorithmic problem solving, systems engineering, machine learning and infrastructure at a massive scale.

Reason to Join-

An opportunity for innovators, problem solvers & learners.  Working will be Innovative, empowering, rewarding & fun. Amazing Office, competitive pay along with excellent benefits package.

Requiremets and Responsibilities- (please read carefully before applying)

• The overall experience of 3-6 years in Java/Python Framework and Machine Learning.
• Develop Web Services, REST, XSD, XML technologies, Java, Python, AWS, API.
• Experience on Elastic Search or SOLR or Lucene -Search Engine, Text Mining, Indexing.
• Experience in highly scalable tools like Kafka, Spark, Aerospike, etc.
• Hands on experience in Design, Architecture, Implementation, Performance & Scalability, and Distributed Systems.
• Design, implement, and deploy highly scalable and reliable systems.
• Troubleshoot Solr indexing process and querying engine.
• Bachelors or Masters in Computer Science from Tier 1 Institutions

·      Core responsibilities to include analyze business requirements and designs for accuracy and completeness. Develops and maintains relevant product.

·      BlueYonder is seeking a Senior/Principal Architect in the Data Services department (under Luminate Platform ) to act as one of key technology leaders to build and manage BlueYonder’ s technology assets in the Data Platform and Services.

·      This individual will act as a trusted technical advisor and strategic thought leader to the Data Services department. The successful candidate will have the opportunity to lead, participate, guide, and mentor other people in the team on architecture and design in a hands-on manner. You are responsible for technical direction of Data Platform. This position reports to the Global Head, Data Services and will be based in Bangalore, India.

·      Core responsibilities to include Architecting and designing (along with counterparts and distinguished Architects) a ground up cloud native (we use Azure) SaaS product in Order management and micro-fulfillment

·      The team currently comprises of 60+ global associates across US, India (COE) and UK and is expected to grow rapidly. The incumbent will need to have leadership qualities to also mentor junior and mid-level software associates in our team. This person will lead the Data platform architecture – Streaming, Bulk with Snowflake/Elastic Search/other tools

Our current technical environment:

·      Software: Java, Springboot, Gradle, GIT, Hibernate, Rest API, OAuth , Snowflake

·      • Application Architecture: Scalable, Resilient, event driven, secure multi-tenant Microservices architecture

·      • Cloud Architecture: MS Azure (ARM templates, AKS, HD insight, Application gateway, Virtue Networks, Event Hub, Azure AD)

·      Frameworks/Others: Kubernetes, Kafka, Elasticsearch, Spark, NOSQL, RDBMS, Springboot, Gradle GIT, Ignite

Have you streamed a program on Disney+, watched your favorite binge-worthy series on Peacock or cheered your favorite team on during the World Cup from one of the 20 top streaming platforms around the globe? If the answer is yes, you’ve already benefitted from Conviva technology, helping the world’s leading streaming publishers deliver exceptional streaming experiences and grow their businesses. 

Conviva is the only global streaming analytics platform for big data that collects, standardizes, and puts trillions of cross-screen, streaming data points in context, in real time. The Conviva platform provides comprehensive, continuous, census-level measurement through real-time, server side sessionization at unprecedented scale. If this sounds important, it is! We measure a global footprint of more than 500 million unique viewers in 180 countries watching 220 billion streams per year across 3 billion applications streaming on devices. With Conviva, customers get a unique level of actionability and scale from continuous streaming measurement insights and benchmarking across every stream, every screen, every second.

As Conviva is expanding, we are building products providing deep insights into end user experience for our customers.

Platform and TLB Team

The vision for the TLB team is to build data processing software that works on terabytes of streaming data in real time. Engineer the next-gen Spark-like system for in-memory computation of large time-series dataset’s – both Spark-like backend infra and library based programming model. Build horizontally and vertically scalable system that analyses trillions of events per day within sub second latencies. Utilize the latest and greatest of big data technologies to build solutions for use-cases across multiple verticals. Lead technology innovation and advancement that will have big business impact for years to come. Be part of a worldwide team building software using the latest technologies and the best of software development tools and processes.

What You’ll Do

This is an individual contributor position. Expectations will be on the below lines:

• Design, build and maintain the stream processing, and time-series analysis system which is at the heart of Conviva's products
• Responsible for the architecture of the Conviva platform
• Build features, enhancements, new services, and bug fixing in Scala and Java on a Jenkins-based pipeline to be deployed as Docker containers on Kubernetes
• Own the entire lifecycle of your microservice including early specs, design, technology choice, development, unit-testing, integration-testing, documentation, deployment, troubleshooting, enhancements etc.
• Lead a team to develop a feature or parts of the product
• Adhere to the Agile model of software development to plan, estimate, and ship per business priority

What you need to succeed

• 9+ years of work experience in software development of data processing products.
• Engineering degree in software or equivalent from a premier institute.
• Excellent knowledge of fundamentals of Computer Science like algorithms and data structures. Hands-on with functional programming and know-how of its concepts
• Excellent programming and debugging skills on the JVM. Proficient in writing code in Scala/Java/Rust/Haskell/Erlang that is reliable, maintainable, secure, and performant
• Experience with big data technologies like Spark, Flink, Kafka, Druid, HDFS, etc.
• Deep understanding of distributed systems concepts and scalability challenges including multi-threading, concurrency, sharding, partitioning, etc.
• Experience/knowledge of Akka/Lagom framework and/or stream processing technologies like RxJava or Project Reactor will be a big plus. Knowledge of design patterns like event-streaming, CQRS and DDD to build large microservice architectures will be a big plus
• Excellent communication skills. Willingness to work under pressure. Hunger to learn and succeed. Comfortable with ambiguity. Comfortable with complexity

Underpinning the Conviva platform is a rich history of innovation. More than 60 patents represent award-winning technologies and standards, including first-of-its kind-innovations like time-state analytics and AI-automated data modeling, that surfaces actionable insights. By understanding real-world human experiences and having the ability to act within seconds of observation, our customers can solve business-critical issues and focus on growing their businesses ahead of the competition. Examples of the brands Conviva has helped fuel streaming growth for include DAZN, Disney+, HBO, Hulu, NBCUniversal, Paramount+, Peacock, Sky, Sling TV, Univision, and Warner Bros Discovery.  

Privately held, Conviva is headquartered in Silicon Valley, California with offices and people around the globe. For more information, visit us at www.conviva.com. Join us to help extend our leadership position in big data streaming analytics to new audiences and markets! 

Required Skills/Qualifications: 

∙         B.Tech/MCA in Comput

Required Skills/Qualifications: 

Any Backend Technology is acceptable

∙         B.Tech/MCA in Computer Science or equivalent with 1-3 years’ experience with server-side web application development.

∙         Extensive development experience using LAMP or MEAN stack, RESTful web services or Node.js, HTML and CSS.

∙         Good understanding of Data structures and Relational Databases like MySQL or NoSQL Databases like MongoDB.

∙         Experience working with services in AWS such as EC2, RDS, and ELBs and have knowledge of VPCs.

∙         Experience with server side and client side MVC frameworks, Kafka Ansilble,Jenkins.

∙         Exposure to Continuous Integration (CI) and Continuous Deployment (CD), automated testing and agile development methods.

∙         Understanding of Version Management Tools like GitHub.

er Science or equivalent with 1-3 years’ experience with server-side web application development.

∙         Extensive development experience using LAMP or MEAN stack, RESTful web services or Node.js, HTML and CSS.

∙         Good understanding of Data structures and Relational Databases like MySQL or NoSQL Databases like MongoDB.

∙         Experience working with services in AWS such as EC2, RDS, and ELBs and have knowledge of VPCs.

∙         Experience with server side and client side MVC frameworks, Kafka Ansilble,Jenkins.

∙         Exposure to Continuous Integration (CI) and Continuous Deployment (CD), automated testing and agile development methods.

∙         Understanding of Version Management Tools like GitHub.

Basic Qualifications

Candidates for this role must have:

• Bachelor’s degree in computer science or a related field
• At least three years of experience writing production code of increasing complexity
• Strong understanding of computer science fundamentals, including algorithms, complexity analysis, data structures, problem solving, and object-oriented analysis and design
• Proficiency in at least one of the following: Java, C, C++, C#, Ruby, Scala, Python
• Experience refactoring code and evolving architectures
• Experience crafting unit, integration, functional and regression tests

 Preferred Qualifications

• Master’s degree in computer science or a related field
• Experience developing in a Linux environment
• Experience applying service-oriented architecture techniques to production use cases
• Experience in developing secure, multi-platform mobile applications