Senior Quality Engineer

• Responsible for designing, storing, processing, and maintaining of large-scale data and related infrastructure.
• Can drive multiple projects both from operational and technical standpoint.
• Ideate and build PoV or PoC for new product that can help drive more business.
• Responsible for defining, designing, and implementing data engineering best practices, strategies, and solutions.
• Is an Architect who can guide the customers, team, and overall organization on tools, technologies, and best practices around data engineering.
• Lead architecture discussions, align with business needs, security, and best practices.
• Has strong conceptual understanding of Data Warehousing and ETL, Data Governance and Security, Cloud Computing, and Batch & Real Time data processing
• Has strong execution knowledge of Data Modeling, Databases in general (SQL and NoSQL), software development lifecycle and practices, unit testing, functional programming, etc.
• Understanding of Medallion architecture pattern
• Has worked on at least one cloud platform.
• Has worked as data architect and executed multiple end-end data engineering project.
• Has extensive knowledge of different data architecture designs and data modelling concepts.
• Manages conversation with the client stakeholders to understand the requirement and translate it into technical outcomes.

Required Tech Stack

• Strong proficiency in SQL
• Experience working on any of the three major cloud platforms i.e., AWS/Azure/GCP
• Working knowledge of an ETL and/or orchestration tools like IICS, Talend, Matillion, Airflow, Azure Data Factory, AWS Glue, GCP Composer, etc.
• Working knowledge of one or more OLTP databases (Postgres, MySQL, SQL Server, etc.)
• Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Azure Synapse, Hive, Big Query, etc.
• Proficient in at least one programming language used in data engineering, such as Python (or Scala/Rust/Java)
• Has strong execution knowledge of Data Modeling (star schema, snowflake schema, fact vs dimension tables)
• Proficient in Spark and related applications like Databricks, GCP DataProc, AWS Glue, EMR, etc.
• Has worked on Kafka and real-time streaming.
• Has strong execution knowledge of data architecture design patterns (lambda vs kappa architecture, data harmonization, customer data platforms, etc.)
• Has worked on code and SQL query optimization.
• Strong knowledge of version control systems like Git to manage source code repositories and designing CI/CD pipelines for continuous delivery.
• Has worked on data and networking security (RBAC, secret management, key vaults, vnets, subnets, certificates)

About the role

Checking quality is one of the most important tasks at Anakin. Our clients are pricing their products based on our data, and minor errors on our end can lead to our client's losses of millions of dollars. You would work with multiple tools and with people across various departments to ensure the accuracy of the data being crawled. You would setup manual and automated processes and make sure they run to ensure the highest possible data quality.

You are the engineer other engineers can count on. You embrace every problem with enthusiasm. You remove hurdles, are a self-starter, team player. You have the hunger to venture into unknown areas and make the system work.

Your Responsibilities would be to:

• Understand customer web scraping and data requirements; translate these into test approaches that include exploratory manual/visual testing and any additional automated tests deemed appropriate
• Take ownership of the end-to-end QA process in newly-started projects
• Draw conclusions about data quality by producing basic descriptive statistics, summaries, and visualisations
• Proactively suggest and take ownership of improvements to QA processes and methodologies by employing other technologies and tools, including but not limited to: browser add-ons, Excel add-ons, UI-based test automation tools etc.
• Ensure that project requirements are testable; work with project managers and/or clients to clarify ambiguities before QA begins
• Drive innovation and advanced validation and analytics techniques to ensure data quality for Anakin's customers
• Optimize data quality codebases and systems to monitor the Anakin family of app crawlers
• Configure and optimize the automated and manual testing and deployment systems used to check the quality of billions of data points of over 1000+ crawlers across the company
• Analyze data and bugs that require in-depth investigations
• Interface directly with external customers including managing relationships and steering requirements

Basic Qualifications:

• 2+ years of experience as a backend or a full-stack software engineer
• Web scraping experience with Python or Node.js
• 2+ years of experience with AWS services such as EC2, S3, Lambda, etc.
• Should have managed a team of software engineers
• Should be paranoid about data quality

Preferred Skills and Experience:

• Deep experience with network debugging across all OSI layers (Wireshark)
• Knowledge of networks or/and cybersecurity
• Broad understanding of the landscape of software engineering design patterns and principles
• Ability to work quickly and accurately in a highly stressful environment during removing bugs in run-time within minutes
• Excellent communicator, both written and verbal

Additional Requirements:

• Must be available to work extended hours and weekends when needed to meet critical deadlines
• Must have an aversion to politics and BS. Should let his/her work speak for him/her.
• Must be comfortable with uncertainty. In almost all the cases, your job will be to figure it out.
• Must not be bounded to comfort zone. Often, you will need to challenge yourself to go above and beyond.

• Design & Development of the architecture for multi-robot planning.
• Design & development of task allocation algorithms
• Design & development of conflict resolution approaches.
• Design & develop queuing strategies for multi-robot deployments.
• Research & collaboration on approaches to improve task allocation based on historical data
• Design & development of communication architecture for inter-robot and robot-server communications.
  
  Requirements
• B.Tech, M. Tech or higher qualification in Computer Science Engineering, Information
• Technology or related fields
• Proficiency with C++/Python programming language
• Experience of working in the robotics field for 3-5 years
• Skilled at general software development, bug analysis, and fixing
• Knowledge of networking/communication concepts
• Strong knowledge of Robot Operating System (ROS)
• Good knowledge of system design
• Excellent problem-solving skills
• Good project management skills
• Excellent verbal communication skills
• Good interpersonal skill