Data Engineer - Data Migration & Analysis - PostgreSQL/Oracle DB

• Act as primary day-to-day contact on analytics to agency-client leads
• Develop bespoke analytics proposals for presentation to agencies & clients, for delivery within the teams
• Ensure delivery of projects and services across the analytics team meets our stakeholder requirements (time, quality, cost)
• Hands on platforms to perform data pre-processing that involves data transformation as well as data cleaning
  
• Ensure data quality and integrity
  
• Interpret and analyse data problems
  
• Build analytic systems and predictive models
  
• Increasing the performance and accuracy of machine learning algorithms through fine-tuning and further
  
• Visualize data and create reports
  
• Experiment with new models and techniques
  
• Align data projects with organizational goals

• Min 6 - 7 years’ experience working in Data Science
• Prior experience as a Data Scientist within a digital media is desirable
• Solid understanding of machine learning
• A degree in a quantitative field (e.g. economics, computer science, mathematics, statistics, engineering, physics, etc.)
• Experience with SQL/ Big Query/GMP tech stack / Clean rooms such as ADH
• A knack for statistical analysis and predictive modelling
• Good knowledge of R, Python
• Experience with SQL, MYSQL, PostgreSQL databases
• Knowledge of data management and visualization techniques
• Hands-on experience on BI/Visual Analytics Tools like PowerBI or Tableau or Data Studio
• Evidence of technical comfort and good understanding of internet functionality desirable
• Analytical pedigree - evidence of having approached problems from a mathematical perspective and working through to a solution in a logical way
• Proactive and results-oriented
• A positive, can-do attitude with a thirst to continually learn new things
• An ability to work independently and collaboratively with a wide range of teams
• Excellent communication skills, both written and oral

Job Description

Job Responsibilities

• Design and implement robust database solutions including

  • Security, backup and recovery

  • Performance, scalability, monitoring and tuning,

  • Data management and capacity planning,

  • Planning, and implementing failover between database instances.

• Create data architecture strategies for each subject area of the enterprise data model.

• Communicate plans, status and issues to higher management levels.

• Collaborate with the business, architects and other IT organizations to plan a data strategy, sharing important information related to database concerns and constrains

• Produce all project data architecture deliverables..

• Create and maintain a corporate repository of all data architecture artifacts.

Skills Required:

• Understanding of data analysis, business principles, and operations

• Software architecture and design Network design and implementation

• Data visualization, data migration and data modelling

• Relational database management systems

• DBMS software, including SQL Server  

• Database and cloud computing design, architectures and data lakes

• Information management and data processing on multiple platforms 

• Agile methodologies and enterprise resource planning implementation

• Demonstrate database technical functionality, such as performance tuning, backup and recovery, monitoring.

• Excellent skills with advanced features such as database encryption, replication, partitioning, etc.

• Strong problem solving, organizational and communication skill.

JD Code: SHI-LDE-01 

Version#: 1.0 

Date of JD Creation: 27-March-2023 

Position Title: Lead Data Engineer 

Reporting to: Technical Director 

Location: Bangalore Urban, India (on-site) 

SmartHub.ai (www.smarthub.ai) is a fast-growing Startup headquartered in Palo Alto, CA, and with offices in Seattle and Bangalore. We operate at the intersection of AI, IoT & Edge Computing. With strategic investments from leaders in infrastructure & data management, SmartHub.ai is redefining the Edge IoT space. Our “Software Defined Edge” products help enterprises rapidly accelerate their Edge Infrastructure Management & Intelligence. We empower enterprises to leverage their Edge environment to increase revenue, efficiency of operations, manage safety and digital risks by using Edge and AI technologies. 

SmartHub is an equal opportunity employer and will always be committed to nurture a workplace culture that supports, inspires and respects all individuals, encourages employees to bring their best selves to work, laugh and share. We seek builders who hail from a variety of backgrounds, perspectives and skills to join our team.  

Summary 

This role requires the candidate to translate business and product requirements to build, maintain, optimize data systems which can be relational or non-relational in nature. The candidate is expected to tune and analyse the data including from a short and long-term trend analysis and reporting, AI/ML uses cases. 

We are looking for a talented technical professional with at least 8 years of proven experience in owning, architecting, designing, operating and optimising databases that are used for large scale analytics and reports. 

Responsibilities 

• Provide technical & architectural leadership for the next generation of product development. 
• Innovate, Research & Evaluate new technologies and tools for a quality output. 
• Architect, Design and Implement ensuring scalability, performance and security. 
• Code and implement new algorithms to solve complex problems. 
• Analyze complex data, develop, optimize and transform large data sets both structured and unstructured. 
• Ability to deploy and administrator the database and continuously tuning for performance especially container orchestration stacks such as Kubernetes  
• Develop analytical models and solutions Mentor Junior members technically in Architecture, Designing and robust Coding. 
• Work in an Agile development environment while continuously evaluating and improvising engineering processes 

Required 

• At least 8 years of experience with significant depth in designing and building scalable distributed database systems for enterprise class products, experience of working in product development companies. 
• Should have been feature/component lead for several complex features involving large datasets. 
• Strong background in relational and non-relational database like Postgres, MongoDB, Hadoop etl. 
• Deep exp database optimization, tuning ertise in SQL, Time Series Databases, Apache Drill, HDFS, Spark are good to have 
• Excellent analytical and problem-solving skill sets. 
• Experience in  for high throughput is highly desirable 
• Exposure to database provisioning in Kubernetes/non-Kubernetes environments, configuration and tuning in a highly available mode. 
• Demonstrated ability to provide technical leadership and mentoring to the team 

• 3+ years of Experience majoring in applying AI/ML/ NLP / deep learning / data-driven statistical analysis & modelling solutions.
• Programming skills in Python, knowledge in Statistics.
• Hands-on experience developing supervised and unsupervised machine learning algorithms (regression, decision trees/random forest, neural networks, feature selection/reduction, clustering, parameter tuning, etc.). Familiarity with reinforcement learning is highly desirable.
• Experience in the financial domain and familiarity with financial models are highly desirable.
• Experience in image processing and computer vision.
• Experience working with building data pipelines.
• Good understanding of Data preparation, Model planning, Model training, Model validation, Model deployment and performance tuning.
• Should have hands on experience with some of these methods: Regression, Decision Trees,CART, Random Forest, Boosting, Evolutionary Programming, Neural Networks, Support Vector Machines, Ensemble Methods, Association Rules, Principal Component Analysis, Clustering, ArtificiAl Intelligence
• Should have experience in using larger data sets using Postgres Database. 

Roles and Responsibilities

• Managing available resources such as hardware, data, and personnel so that deadlines are met.
• Analyzing the ML and Deep Learning algorithms that could be used to solve a given problem and ranking them by their success probabilities
• Exploring data to gain an understanding of it, then identifying differences in data distribution that could affect performance when deploying the model in the real world
• Defining validation framework and establish a process to ensure acceptable data quality criteria are met
• Supervising the data acquisition and partnership roadmaps to create stronger product for our customers.
• Defining feature engineering process to ensure usage of meaningful features given the business constraints which may vary by market
• Device self-learning strategies through analysis of errors from the models
• Understand business issues and context, devise a framework for solving unstructured problems and articulate clear and actionable solutions underpinned by analytics.
• Manage multiple projects simultaneously while demonstrating business leadership to collaborate & coordinate with different functions to deliver the solutions in a timely, efficient and effective manner.
• Manage project resources optimally to deliver projects on time; drive innovation using residual resources to create strong solution pipeline; provide direction, coaching & training, feedbacks to project team members to enhance performance, support development and encourage value aligned behaviour of the project team members; Provide inputs for periodic performance appraisal of project team members.

Preferred Technical & Professional expertise

• Undergraduate Degree in Computer Science / Engineering / Mathematics / Statistics / economics or other quantitative fields
• At least 2+ years of experience of managing Data Science projects with specializations in Machine Learning
• In-depth knowledge of cloud analytics tools.
• Able to drive Python Code optimization; ability review codes and provide inputs to improve the quality of codes
• Ability to evaluate hardware selection for running ML models for optimal performance
• Up to date with Python libraries and versions for machine learning; Extensive hands-on experience with Regressors; Experience working with data pipelines.
• Deep knowledge of math, probability, statistics and algorithms; Working knowledge of Supervised Learning, Adversarial Learning and Unsupervised learning
• Deep analytical thinking with excellent problem-solving abilities
• Strong verbal and written communication skills with a proven ability to work with all levels of management; effective interpersonal and influencing skills.
• Ability to manage a project team through effectively allocation of tasks, anticipating risks and setting realistic timelines for managing the expectations of key stakeholders
• Strong organizational skills and an ability to balance and handle multiple concurrent tasks and/or issues simultaneously.
• Ensure that the project team understand and abide by compliance framework for policies, data, systems etc. as per group, region and local standards

We are an early stage start-up, building new fintech products for small businesses. Founders are IIT-IIM alumni, with prior experience across management consulting, venture capital and fintech startups. We are driven by the vision to empower small business owners with technology and dramatically improve their access to financial services. To start with, we are building a simple, yet powerful solution to address a deep pain point for these owners: cash flow management. Over time, we will also add digital banking and 1-click financing to our suite of offerings.

We have developed an MVP which is being tested in the market. We have closed our seed funding from marquee global investors and are now actively building a world class tech team. We are a young, passionate team with a strong grip on this space and are looking to on-board enthusiastic, entrepreneurial individuals to partner with us in this exciting journey. We offer a high degree of autonomy, a collaborative fast-paced work environment and most importantly, a chance to create unparalleled impact using technology.

Reach out if you want to get in on the ground floor of something which can turbocharge SME banking in India!

Technology stack at Velocity comprises a wide variety of cutting edge technologies like, NodeJS, Ruby on Rails, Reactive Programming,, Kubernetes, AWS, NodeJS, Python, ReactJS, Redux (Saga) Redis, Lambda etc. 

Key Responsibilities

• Responsible for building data and analytical engineering pipelines with standard ELT patterns, implementing data compaction pipelines, data modelling and overseeing overall data quality

• Work with the Office of the CTO as an active member of our architecture guild

• Writing pipelines to consume the data from multiple sources

• Writing a data transformation layer using DBT to transform millions of data into data warehouses.

• Implement Data warehouse entities with common re-usable data model designs with automation and data quality capabilities

• Identify downstream implications of data loads/migration (e.g., data quality, regulatory)

What To Bring

• 3+ years of software development experience, a startup experience is a plus.

• Past experience of working with Airflow and DBT is preferred

• 2+ years of experience working in any backend programming language. 

• Strong first-hand experience with data pipelines and relational databases such as Oracle, Postgres, SQL Server or MySQL

• Experience with DevOps tools (GitHub, Travis CI, and JIRA) and methodologies (Lean, Agile, Scrum, Test Driven Development)

• Experienced with the formulation of ideas; building proof-of-concept (POC) and converting them to production-ready projects

• Experience building and deploying applications on on-premise and AWS or Google Cloud cloud-based infrastructure

• Basic understanding of Kubernetes & docker is a must.

• Experience in data processing (ETL, ELT) and/or cloud-based platforms

• Working proficiency and communication skills in verbal and written English.