Data Engineer

Description

Job Summary

We are seeking a visionary Senior AI Engineer with a minimum of 3+ years of experience to lead the development of an advanced AI system leveraging CrewAI and LangChain tools to build intelligent, collaborative multi-agent systems for the logistics domain. The role includes designing AI agents capable of handling complex tasks, optimizing workflows, and conducting A/B and beta testing to ensure robust, scalable performance under diverse conditions. This is an opportunity to shape cutting-edge AI systems and revolutionize logistics operations.

Responsibilities:

1. Design & Architecture:

 Architect and implement a multi-agent AI system using CrewAI for task collaboration, communication, and real-time logistics optimization.

 Integrate LangChain tools for natural language understanding, contextual reasoning, and knowledge retrieval.

 Ensure the system is designed to allow comprehensive testing, iteration, and improvement.

2. AI Agent Development:

 Develop autonomous agents capable of dynamic decision-making, task delegation, and seamless inter-agent communication.

 Use LangChain to enhance agent capabilities, such as reasoning over unstructured data and interfacing with external knowledge bases.

3. A/B Testing & Experimentation:

 Design and execute A/B tests to compare AI agent strategies, workflows, and decision-making models under varying scenarios.

 Develop and implement metrics for evaluating system performance (e.g., latency, accuracy, scalability).

 Use experimentation frameworks to test hypotheses and gather insights on system improvements.

4. Beta Testing & Real-World Simulation:

 Set up controlled beta testing environments that mimic real-world logistics operations.

 Simulate edge cases, bottlenecks, and high-load scenarios to ensure system robustness and scalability.

 Gather feedback from beta users and iterate on the system to address discovered issues.

5. Progress Demonstration:

 Develop dashboards and visualizations to showcase key performance indicators (KPIs) and test results.

 Implement tools to log and track agent interactions, decision outcomes, and error rates in real-time.

 Regularly present progress to stakeholders, highlighting improvements and areas for refinement.

6. Optimization & Scalability:

 Optimize multi-agent interactions and resource allocation for real-world logistics challenges.

 Ensure the system scales seamlessly for complex operations involving high volumes of agents and data.

7. Integration:

 Seamlessly integrate the multi-agent system with logistics platforms, such as Transportation Management Systems (TMS) and Warehouse Management Systems (WMS).

 Use LangChain tools to connect agents to APIs, external knowledge graphs, and live data streams.

8. Collaboration & Leadership:

 Lead a team of engineers in developing advanced AI solutions, mentoring them on testing frameworks and innovative technologies like CrewAI and LangChain.

 Collaborate with cross-functional teams to align AI development with business needs.

9. Research & Innovation:

 Stay updated on the latest advancements in multi-agent systems, LangChain, CrewAI, and testing frameworks.

 Introduce cutting-edge techniques to continuously improve the AI system’s performance.

Required Qualifications:

Education: Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Machine Learning, or a related field (Ph.D. preferred).

Experience:

• 3+ years in AI development, with experience in multi-agent systems, logistics, or related fields.

• Proven experience in conducting A/B testing and beta testing for AI systems.

• Hands-on experience with CrewAI and LangChain tools.

• Should have hands-on experience working with end-to-end chatbot development, specifically with Agentic and RAG-based chatbots. It is essential that the candidate has been involved in the entire lifecycle of chatbot creation, from design to deployment.

• Should have practical experience with LLM application deployment.

Technical Skills:

• Proficiency in Python and machine learning frameworks (e.g., TensorFlow, PyTorch).

• Strong understanding of A/B testing platforms and methodologies for AI systems.

• Expertise in building beta testing pipelines and real-world simulation environments.

• Familiarity with distributed systems, reinforcement learning, and natural language processing (NLP).

• Proficiency in using LangChain tools for chaining tasks, knowledge retrieval, and reasoning.

• Proficiency with cloud platforms (AWS, Azure, GCP) and containerization tools (Docker, Kubernetes).

• Good to have experience in Cursor & MCP.

• Experience in setting up monitoring dashboards with tools like Grafana, Tableau, or similar.

Preferred Qualifications:

 Experience in logistics systems, such as route optimization, shipment tracking, and demand forecasting.

 Familiarity with graph theory, network optimization, and blockchain for agent security.

 Background in designing scalable systems tested under diverse operational conditions.

Soft Skills:

 Strong analytical and problem-solving abilities.

 Ability to communicate technical concepts effectively to stakeholders.

 Leadership skills for mentoring teams and guiding project execution.

Position Summary:

As a QA Engineer at LimeChat, you will play a crucial role in ensuring the quality and performance of our messaging and communication applications. You will primarily focus on manual testing to identify issues, validate product functionality, and provide feedback to our development teams. Additionally, you will be involved in automation testing to enhance efficiency and test coverage.

Key Responsibilities:

• Execute test cases, identify defects, and report issues accurately.
• Collaborate with cross-functional teams to ensure product requirements and user expectations are met.
• Conduct manual testing of LimeChat's messaging applications to verify product functionality and user experience.
• Develop and maintain test plans and test cases for both new features and regression testing.
• Participate in design and implementation reviews to provide input on potential issues.
• Perform exploratory testing to uncover hidden defects and ensure a robust product.
• Automate test cases using testing frameworks and tools.
• Maintain and update automated test scripts as necessary.
• Execute automated test suites to improve test efficiency and coverage.
• Continuously monitor the quality of our applications and participate in the release process.
• Collaborate with developers to isolate and reproduce defects for faster resolution.
• Maintain and document testing processes and procedures for reference.
• Contribute to a culture of continuous improvement in quality assurance.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).
• Proven experience as a QA Engineer, with a strong emphasis on manual testing.
• Familiarity with automation testing tools such as Selenium, Appium, or similar.
• Proficiency in writing and executing test cases.
• Strong analytical and problem-solving skills.
• Knowledge of software testing methodologies and best practices.
• Experience with mobile and web application testing.
• Ability to work independently and as part of a team.
• Excellent communication and documentation skills.
• Detail-oriented with a commitment to quality.
• Experience with testing APIs is a plus.

Core Responsibilities

• Apply appropriate analysis and optimization techniques to scale media campaigns on Facebook.
• Run both regular as well as retargeting campaigns on Facebook.
• Use data from Facebook as well as our internal data tools to expand, evolve, and optimize paid social media campaigns.
• Run multivariate tests in order to determine highest performing creatives.
• Staying current with best practices, strategies, and industry standards related to social media marketing.
• Stay on top of Facebook Ad Policies
• Optimising various campaigns.
• Managing and setting KPI’s for affiliates
• Reducing CPA and CPD through all acquisition sources.

Have you streamed a program on Disney+, watched your favorite binge-worthy series on Peacock or cheered your favorite team on during the World Cup from one of the 20 top streaming platforms around the globe? If the answer is yes, you’ve already benefitted from Conviva technology, helping the world’s leading streaming publishers deliver exceptional streaming experiences and grow their businesses. 

Conviva is the only global streaming analytics platform for big data that collects, standardizes, and puts trillions of cross-screen, streaming data points in context, in real time. The Conviva platform provides comprehensive, continuous, census-level measurement through real-time, server side sessionization at unprecedented scale. If this sounds important, it is! We measure a global footprint of more than 500 million unique viewers in 180 countries watching 220 billion streams per year across 3 billion applications streaming on devices. With Conviva, customers get a unique level of actionability and scale from continuous streaming measurement insights and benchmarking across every stream, every screen, every second.

As Conviva is expanding, we are building products providing deep insights into end user experience for our customers.

Platform and TLB Team

The vision for the TLB team is to build data processing software that works on terabytes of streaming data in real time. Engineer the next-gen Spark-like system for in-memory computation of large time-series dataset’s – both Spark-like backend infra and library based programming model. Build horizontally and vertically scalable system that analyses trillions of events per day within sub second latencies. Utilize the latest and greatest of big data technologies to build solutions for use-cases across multiple verticals. Lead technology innovation and advancement that will have big business impact for years to come. Be part of a worldwide team building software using the latest technologies and the best of software development tools and processes.

What You’ll Do

This is an individual contributor position. Expectations will be on the below lines:

• Design, build and maintain the stream processing, and time-series analysis system which is at the heart of Conviva's products
• Responsible for the architecture of the Conviva platform
• Build features, enhancements, new services, and bug fixing in Scala and Java on a Jenkins-based pipeline to be deployed as Docker containers on Kubernetes
• Own the entire lifecycle of your microservice including early specs, design, technology choice, development, unit-testing, integration-testing, documentation, deployment, troubleshooting, enhancements etc.
• Lead a team to develop a feature or parts of the product
• Adhere to the Agile model of software development to plan, estimate, and ship per business priority

What you need to succeed

• 9+ years of work experience in software development of data processing products.
• Engineering degree in software or equivalent from a premier institute.
• Excellent knowledge of fundamentals of Computer Science like algorithms and data structures. Hands-on with functional programming and know-how of its concepts
• Excellent programming and debugging skills on the JVM. Proficient in writing code in Scala/Java/Rust/Haskell/Erlang that is reliable, maintainable, secure, and performant
• Experience with big data technologies like Spark, Flink, Kafka, Druid, HDFS, etc.
• Deep understanding of distributed systems concepts and scalability challenges including multi-threading, concurrency, sharding, partitioning, etc.
• Experience/knowledge of Akka/Lagom framework and/or stream processing technologies like RxJava or Project Reactor will be a big plus. Knowledge of design patterns like event-streaming, CQRS and DDD to build large microservice architectures will be a big plus
• Excellent communication skills. Willingness to work under pressure. Hunger to learn and succeed. Comfortable with ambiguity. Comfortable with complexity

Underpinning the Conviva platform is a rich history of innovation. More than 60 patents represent award-winning technologies and standards, including first-of-its kind-innovations like time-state analytics and AI-automated data modeling, that surfaces actionable insights. By understanding real-world human experiences and having the ability to act within seconds of observation, our customers can solve business-critical issues and focus on growing their businesses ahead of the competition. Examples of the brands Conviva has helped fuel streaming growth for include DAZN, Disney+, HBO, Hulu, NBCUniversal, Paramount+, Peacock, Sky, Sling TV, Univision, and Warner Bros Discovery.  

Privately held, Conviva is headquartered in Silicon Valley, California with offices and people around the globe. For more information, visit us at www.conviva.com. Join us to help extend our leadership position in big data streaming analytics to new audiences and markets! 

Springworks is building the tools and products to enable the era of Remote/Hybrid work and our products have been used by over 350,000+ remote/hybrid employees across 10,000+ organisations and 55+ countries in the last couple of years.

The product stack from Springworks includes:

• SpringVerify — B2B verification platform,
• EngageWith — an employee recognition and rewards platform that enriches company culture,
• Trivia — a suite of real time, fun and interactive games platform for remote team-building,
• SpringRole — verified professional-profile platform backed by blockchain, and
• SpringRecruit — a forever-free applicant tracking system.

Role Overview

First and foremost, we're looking for people who are excited about what we're doing—you don't need to know anything about the HRTech ecosystem, but should be excited to learn. We are seeking a committed and result-oriented individual to join our team.

As an Operations Intern you will be responsible for:

• Helping the operations team in the day to day work.
• Preparing and sorting documents for data entry.
• Compile, verify the accuracy, and sort information according to priorities.
• Entering data into database software and checking to ensure the accuracy of the data that has been inputted. 
• Review data for deficiencies or errors.
• Resolving discrepancies in the information and obtaining further information for incomplete documents. 
• Generate reports, store completed work in designated locations and perform backup operations.
• Take follow-up calls for verification and maintaining of MIS.

Requirements

1. Attention to detail.
2. Organizing abilities.
3. Good communication skills.
4. Good understanding of Google docs, sheets and MS Office is a must.
5. Able to handle multiple tasks.
6. Ability to work to time constraints.
   
   Note: Please note this is a full time internship opportunity and a person should be available for a minimum of 6 months duration for internship.

Benefits

What’s in it for you?

We are a 100% remote company so work from anywhere in India. The intern will gain exciting real-world software engineering experience at a thriving company. We also emphasize heavily on learning - employees have access to one of the world's leading e-learning providers (Pluralsight) & an option to avail additional online learning courses if required & relevant to their role. 

• Paid Internship (Monthly stipend upto INR: 21,000)
• Permanent remote/work from anywhere
• Rewards and recognitions(Surprise gift/Voucher for Intern of the month)
• Monthly reimbursement for Internet expenses - Upto INR 1500
• Opportunity to attend networking events, Shadowing and training experiences with knowledgeable professionals
• Vaccination- You can get a vaccination for yourself and can nominate 5 people of your choice to get vaccinated
• Food- Rs. 500 added to your payroll - no bills to upload

See what our past interns have to say about their experience -https://www.youtube.com/watch?v=1G3a-fkMAtU"> Interns @Springworks