Principal Engineer

We're seeking an AI/ML Engineer to join our team-

As an AI/ML Engineer, you will be responsible for designing, developing, and implementing artificial intelligence (AI) and machine learning (ML) solutions to solve real world business problems. You will work closely with cross-functional teams, including data scientists, software engineers, and product managers, to deploy and integrate Applied AI/ML solutions into the products that are being built at NonStop. Your role will involve researching cutting-edge algorithms, data processing techniques, and implementing scalable solutions to drive innovation and improve the overall user experience.

Responsibilities

• Applied AI/ML engineering; Building engineering solutions on top of the AI/ML tooling available in the industry today. Eg: Engineering APIs around OpenAI
• AI/ML Model Development: Design, develop, and implement machine learning models and algorithms that address specific business challenges, such as natural language processing, computer vision, recommendation systems, anomaly detection, etc.
• Data Preprocessing and Feature Engineering: Cleanse, preprocess, and transform raw data into suitable formats for training and testing AI/ML models. Perform feature engineering to extract relevant features from the data
• Model Training and Evaluation: Train and validate AI/ML models using diverse datasets to achieve optimal performance. Employ appropriate evaluation metrics to assess model accuracy, precision, recall, and other relevant metrics
• Data Visualization: Create clear and insightful data visualizations to aid in understanding data patterns, model behavior, and performance metrics
• Deployment and Integration: Collaborate with software engineers and DevOps teams to deploy AI/ML models into production environments and integrate them into various applications and systems
• Data Security and Privacy: Ensure compliance with data privacy regulations and implement security measures to protect sensitive information used in AI/ML processes
• Continuous Learning: Stay updated with the latest advancements in AI/ML research, tools, and technologies, and apply them to improve existing models and develop novel solutions
• Documentation: Maintain detailed documentation of the AI/ML development process, including code, models, algorithms, and methodologies for easy understanding and future reference

Requirements

• Bachelor's, Master's or PhD in Computer Science, Data Science, Machine Learning, or a related field. Advanced degrees or certifications in AI/ML are a plus
• Proven experience as an AI/ML Engineer, Data Scientist, or related role, ideally with a strong portfolio of AI/ML projects
• Proficiency in programming languages commonly used for AI/ML. Preferably Python
• Familiarity with popular AI/ML libraries and frameworks, such as TensorFlow, PyTorch, scikit-learn, etc.
• Familiarity with popular AI/ML Models such as GPT3, GPT4, Llama2, BERT etc.
• Strong understanding of machine learning algorithms, statistics, and data structures
• Experience with data preprocessing, data wrangling, and feature engineering
• Knowledge of deep learning architectures, neural networks, and transfer learning
• Familiarity with cloud platforms and services (e.g., AWS, Azure, Google Cloud) for scalable AI/ML deployment
• Solid understanding of software engineering principles and best practices for writing maintainable and scalable code
• Excellent analytical and problem-solving skills, with the ability to think critically and propose innovative solutions
• Effective communication skills to collaborate with cross-functional teams and present complex technical concepts to non-technical stakeholders

We are seeking a highly skilled and motivated Python Developer with hands-on experience in AWS cloud services (Lambda, API Gateway, EC2), microservices architecture, PostgreSQL, and Docker. The ideal candidate will be responsible for designing, developing, deploying, and maintaining scalable backend services and APIs, with a strong emphasis on cloud-native solutions and containerized environments.

Key Responsibilities:

• Develop and maintain scalable backend services using Python (Flask, FastAPI, or Django).
• Design and deploy serverless applications using AWS Lambda and API Gateway.
• Build and manage RESTful APIs and microservices.
• Implement CI/CD pipelines for efficient and secure deployments.
• Work with Docker to containerize applications and manage container lifecycles.
• Develop and manage infrastructure on AWS (including EC2, IAM, S3, and other related services).
• Design efficient database schemas and write optimized SQL queries for PostgreSQL.
• Collaborate with DevOps, front-end developers, and product managers for end-to-end delivery.
• Write unit, integration, and performance tests to ensure code reliability and robustness.
• Monitor, troubleshoot, and optimize application performance in production environments.

Required Skills:

• Strong proficiency in Python and Python-based web frameworks.
• Experience with AWS services: Lambda, API Gateway, EC2, S3, CloudWatch.
• Sound knowledge of microservices architecture and asynchronous programming.
• Proficiency with PostgreSQL, including schema design and query optimization.
• Hands-on experience with Docker and containerized deployments.
• Understanding of CI/CD practices and tools like GitHub Actions, Jenkins, or CodePipeline.
• Familiarity with API documentation tools (Swagger/OpenAPI).
• Version control with Git.

• Complete experience in the Spring Framework
• proficient knowledge of SQL and NoSQL must
• Hands-on experience in designing and developing applications using Java EE platforms
• Designing and developing high-volume, low-latency applications for mission-critical systems and delivering high availability and performance
• Contributing in all phases of the development lifecycle
• Writing well designed, testable, efficient code
• Excellent Communication Skills
• Willingness to own a responsibility
• Ability to work in a team as well as an individual
• Ability to work under pressure and maintain deadlines
• Good to have worked on end to end in projects

Be Part Of Building The Future

Dremio is the Data Lake Engine company. Our mission is to reshape the world of analytics to deliver on the promise of data with a fundamentally new architecture, purpose-built for the exploding trend towards cloud data lake storage such as AWS S3 and Microsoft ADLS. We dramatically reduce and even eliminate the need for the complex and expensive workarounds that have been in use for decades, such as data warehouses (whether on-premise or cloud-native), structural data prep, ETL, cubes, and extracts. We do this by enabling lightning-fast queries directly against data lake storage, combined with full self-service for data users and full governance and control for IT. The results for enterprises are extremely compelling: 100X faster time to insight; 10X greater efficiency; zero data copies; and game-changing simplicity. And equally compelling is the market opportunity for Dremio, as we are well on our way to disrupting a $25BN+ market.

About the Role

The Dremio India team owns the DataLake Engine along with Cloud Infrastructure and services that power it. With focus on next generation data analytics supporting modern table formats like Iceberg, Deltalake, and open source initiatives such as Apache Arrow, Project Nessie and hybrid-cloud infrastructure, this team provides various opportunities to learn, deliver, and grow in career. We are looking for innovative minds with experience in leading and building high quality distributed systems at massive scale and solving complex problems.

Responsibilities & ownership

• Lead, build, deliver and ensure customer success of next-generation features related to scalability, reliability, robustness, usability, security, and performance of the product.
• Work on distributed systems for data processing with efficient protocols and communication, locking and consensus, schedulers, resource management, low latency access to distributed storage, auto scaling, and self healing.
• Understand and reason about concurrency and parallelization to deliver scalability and performance in a multithreaded and distributed environment.
• Lead the team to solve complex and unknown problems 
• Solve technical problems and customer issues with technical expertise
• Design and deliver architectures that run optimally on public clouds like  GCP, AWS, and Azure
• Mentor other team members for high quality and design 
• Collaborate with Product Management to deliver on customer requirements and innovation
• Collaborate with Support and field teams to ensure that customers are successful with Dremio

Requirements

• B.S./M.S/Equivalent in Computer Science or a related technical field or equivalent experience
• Fluency in Java/C++ with 8+ years of experience developing production-level software
• Strong foundation in data structures, algorithms, multi-threaded and asynchronous programming models, and their use in developing distributed and scalable systems
• 5+ years experience in developing complex and scalable distributed systems and delivering, deploying, and managing microservices successfully
• Hands-on experience  in query processing or optimization, distributed systems, concurrency control, data replication, code generation, networking, and storage systems
• Passion for quality, zero downtime upgrades, availability, resiliency, and uptime of the platform
• Passion for learning and delivering using latest technologies
• Ability to solve ambiguous, unexplored, and cross-team problems effectively
• Hands on experience of working projects on AWS, Azure, and Google Cloud Platform 
• Experience with containers and Kubernetes for orchestration and container management in private and public clouds (AWS, Azure, and Google Cloud) 
• Understanding of distributed file systems such as  S3, ADLS, or HDFS
• Excellent communication skills and affinity for collaboration and teamwork
• Ability to work individually and collaboratively with other team members
• Ability to scope and plan solution for  big problems and mentors others on the same
• Interested and motivated to be part of a fast-moving startup with a fun and accomplished team