The candidate should have extensive experience in designing and developing scalable data pipelines and real-time data processing solutions. As a key member of the team, the Senior Data Engineer will play a critical role in building end-to-end data workflows, supporting machine learning model deployment, and driving MLOps practices in a fast-paced, agile environment. Strong expertise in Apache Kafka, Apache Flink, AWS SageMaker, and Terraform is essential. Additional experience with infrastructure automation and CI/CD for ML models is a significant advantage.
Key Responsibilities
- Design, develop, and maintain high-performance ETL and real-time data pipelines using Apache Kafka and Apache Flink.
- Build scalable and automated MLOps pipelines for training, validation, and deployment of models using AWS SageMaker and associated services.
- Implement and manage Infrastructure as Code (IaC) using Terraform to provision and manage AWS environments.
- Collaborate with data scientists, ML engineers, and DevOps teams to streamline model deployment workflows and ensure reliable production delivery.
- Optimize data storage and retrieval strategies for large-scale structured and unstructured datasets.
- Develop data transformation logic and integrate data from various internal and external sources into data lakes and warehouses.
- Monitor, troubleshoot, and enhance performance of data systems in a cloud-native, fast-evolving production setup.
- Ensure adherence to data governance, privacy, and security standards across all data handling activities.
- Document data engineering solutions and workflows to facilitate cross-functional understanding and ongoing maintenance.

About Genspark
About
Empower Today,Excel Tomorrow
Candid answers by the company
Empower Today,
Excel Tomorrow
Company social profiles
Similar jobs
• Design, develop, code and test the solution
• Experience working with large datasets and NoSQL database technologies (e.g. Cassandra, ES)
• 5-10 yrs of industry experience
• Energetic self-starter, with a desire to work in a startup environment
• Proficient in advanced Java programming skills
• Expert in Application development cloud/on premise end to end. Middle layer, DB layer.
• Nice to have understanding on MQ (e.g. Kafka)
• Good hands on in Complex Event Processing systems
• Solved scale and performance issues while dealing with huge sets of data. Pre compute or data aggregation frameworks to achieve good response time
• Experience working with large datasets and NoSQL database technologies (e.g. Cassandra, ES)
• Experience of debugging applications running on Unix like systems (e.g. Ubuntu, CentOS)
• Experience developing RESTful APIs for complex data sets
• Knowledge of container based development & deployment (e.g. Dockers, K8s)
• Expertise in software security domain, a plus
About Us:
Avarna derived from Sanskrit means one with a fun loving nature but having no form, no color, or marks. We at Avarna Capital solve the most interesting questions in sports analytics. Diligence, productivity and transparency defines the work culture at Avarna. We are a team of
highly motivated and innovative people all working remotely so emphasis on communication, accessibility, and accountability are our key focus to build a strong culture at Avarna.
Resposibilities:
1. Integrate Kafka pipelines into the company's current infrastructure
2. Recommended appropriate solutions to enhance the current software stack
3. Utilizing the latest and most innovative technology in solving business problems
4. Take ownership of project delivery and the entire SDLC
Java Developer
As a senior technical member of the team, you will be expected to lead the team in the design, architecture, and implementation of high-quality, well-architected software products with a specific focus on cloud, distributed systems, scalability, elasticity and SLA rollups into each component. In this role, you are expected to interact with Senior Architects, Product Managers, Solution Architects and occasionally, with Partners and customers to discuss and collaborate on strategies for enhancing features and coming up with next-generation products. Critical thinking and strong leadership skills are a must as the candidate will be involved in leading a team and making crucial road map decisions.
The Work: What type of work will you be doing? What assignments, requirements, or skills will you be performing on a regular basis?
- 8+ years of hands-on experience with Java-based development projects using RESTful APIs, SpringBoot, horizontal scalability, microservices, high availability principles
- You master the entire life-cycle including research, design, application and deployment architecture, development, testing (dev owns quality), continuous deployment and continuous delivery up until to production.
- You will develop complex, highly available, distributed, cloud-based solutions, and maintain its high level of quality software.
- Solid background in OOP, object and data modelling using SQL and NoSQL databases
- Experience using containers, Kubernetes and developing services on AWS
- Monitor resource allocation, consumption, and performance.
- Engage in service software performance analysis and system tuning.
- Challenge yourself to walk in a different path and to deliver quickly while maintaining a high quality
- Experience with automation tools like Jenkins, GitLab CI
- Excellent communication skills essential
Must have,
- Fundamental of Python(Data types in python, scope of variable, asynchronous programming)
- Should be able to code in Python
- Basic Understanding of Database
- API designing
- Problem Solving skills
- Fundamentals of storage, caching
Good to have,
- AWS Knowledge,
- Python Framework(Django, Flask)
- Knowledge on Message Broker, Caching
- SQL and no-SQL database
Role
- A Golang developer role is to build optimized, scalable, and modular software using required technologies. You are tasked with developing and coding back-end components and connecting applications to other web services.
Authority
- Research and test new technologies
- Collaborating with others to build and develop a quality
- Monitoring and Oversee company’s data
- Managing users and user roles
- Detecting, announcing, and correcting errors
- Proposing alternative solutions
Responsibility
- To cooperate with other stakeholder to design, develop, test, release and improve services
- Maintain development standards, practices & principles
- Build scalable and maintainable software
- Analytical approach on what and how
Requirements
- At least 4 years of experience with Golang.
- Have expertise in implementing micro services (using tools and technologies for messaging, RPC, containerization, etc.)
- Experience working with SQL/NoSQL databases, ability to write complex queries and optimize them
- Understanding of containerization technologies (Docker, RKT, Kubernetes, etc.)
- Basic experience with CI/CD systems (Jenkins, TeamCity, GoCD, Concourse, etc.)
- Basic experience working with AWS/Google Cloud
At Kyt, we are building an online-first, global academy focused on extracurricular learning for kids. Our mission is to ensure that every kid around the world gets access to holistic education while being able to learn using content, tools and technologies that teaches them concepts - some even better than offline. Bringing in technology as a differentiator to teach kids concepts and help them practice is key to our product development path.
- Experience of 6+ years across product based companies
- Should have worked on building B2C products. Having experience in building edtech products would be a major plus.
- Should be an avid programmer with great problem solving skills.
- Should have a great grasp over NodeJs/Typescript/ AWS microservices
- Should have experience in building scalable server-side systems. Should have significant experience in cloud infrastructure and designing databases.
- Should have knowledge of software architecture, system thinking and design patterns.
About MoEngage
MoEngage is a fast-paced startup that’s helping companies run smart marketing efforts in their effort to reach the customer. We are a leading Marketing Technology Stack provider that is helping brands redefine their customer engagement in the mobile era. Brands use MoEngage to drive long-term, personalised and context-based engagement across channels to help achieve increased customer retention as well as customer LTV. Sitting at a conflux of diverse technologies like Artificial Intelligence, Big Data, Web & Mobile platforms, MoEngage technology analyses billions of data points generated by customers and their devices, to predict customer behavior and build marketing campaigns that proactively engage users.
In just four years since inception, MoEngage is working with leading brands across e-commerce, entertainment, travel, publishing and banking domains among others. With marquee clients like Vodafone, Oyo, Airtel, McAfee, MoEngage has over 125+ paying Customers in the Enterprise & Internet companies space in India, US, South East Asia & EU. With a global presence spanning 35 countries, MoEngage has offices in San Francisco, Berlin, Jakarta, and Bengaluru.
Today, MoEngage is an industry pioneer in the space and engages more than 350M devices. This includes approximately 40B events tracked per month, 30B+ messages sent, to millions of users across the globe.
As part of the Engineering team at MoEngage, here are some things you can expect:
- Take ownership and be responsible for what you build - no micro management
- Work with A players (some of the best talent in the country), and expedite your learning curve and career growth
- Make in India and build for the world at scale of 350M active users, which no other internet company in the country has seen
- Learn together from different teams on how they scale to millions of users and billions of messages.
- Explore the latest in topics like Data Pipeline, MongoDB, ElasticSearch, Kafka, Spark, Samza and share with the team
and more importantly have fun while you work on scaling MoEngage.
About Push team,
Push team is one of the core teams at MoEngage, responsible for sending close to a billion notifications everyday to help clients engage their users better. As a member of the Push team, you will be working on developing high performance solutions to deliver personalised and context-based notifications across various channels to help achieve increased customer retention as well as customer LTV. You will also be working on designing and building features to help clients to provide customised experience for end users allowing them to have a more personalised experience at scale. Here you will have a chance to own systems and develop features end to end i.e right from inception to deployment. Though we work at scale, reliability is of utmost importance for us and we build in house solutions like Campaign Watcher & AutoBatchRunner to ensure 100% transparency and delivery of notifications
- Scaling campaign sending system to ensure industry leading delivery times (40 Million notifications under 2 minutes)
- Rich campaign content delivery and templating support
- Build and develop features to have appealing and consistent experiences across channels which touch 200+ customers and 200+ million users!
Skill Requirements
- Proven experience in handling large infrastructure and distributed systems
- Familiarity with Python related technologies and frameworks like Django or Pyramid.
- Familiarity with at least one of the cloud computing infrastructure - GCP / Azure / AWS
- Familiarity with task queue frameworks like Celery or Pika is a plus.
- Tech Stack - Python, Falcon, Elastic Search, MongoDB, AWS (SQS S3), Linux, Map Reduce

It can be taking our own product and customizing as per the customer requirements or developing applications from the scratch.










