Senior Data Engineer

Looking for freelance?

We are seeking a freelance Data Engineer with 7+ years of experience

Skills Required: Deep knowledge in any cloud (AWS, Azure , Google cloud), Data bricks, Data lakes, Data Ware housing Python/Scala , SQL, BI, and other analytics systems

What we are looking for

We are seeking an experienced Senior Data Engineer with experience in architecture, design, and development of highly scalable data integration and data engineering processes

• The Senior Consultant must have a strong understanding and experience with data & analytics solution architecture, including data warehousing, data lakes, ETL/ELT workload patterns, and related BI & analytics systems
• Strong in scripting languages like Python, Scala
• 5+ years of hands-on experience with one or more of these data integration/ETL tools.
• Experience building on-prem data warehousing solutions.
• Experience with designing and developing ETLs, Data Marts, Star Schema
• Designing a data warehouse solution using Synapse or Azure SQL DB
• Experience building pipelines using Synapse or Azure Data Factory to ingest data from various sources
• Understanding of integration run times available in Azure.
• Advanced working SQL knowledge and experience working with relational databases, and queries. authoring (SQL) as well as working familiarity with a variety of database

What you need to have:

Job Description

Want to make every line of code count? Tired of being a small cog in a big machine? Like a fast-paced environment where stuff get DONE? Wanna grow with a fast-growing company (both career and compensation)? Like to wear different hats? Join ThinkDeeply in our mission to create and apply Enterprise-Grade AI for all types of applications.

Seeking an M.L. Engineer with high aptitude toward development. Will also consider coders with high aptitude in M.L. Years of experience is important but we are also looking for interest and aptitude. As part of the early engineering team, you will have a chance to make a measurable impact in future of Thinkdeeply as well as having a significant amount of responsibility.

Experience

10+ Years

Location

Bozeman/Hyderabad

Skills

Required Skills:

Bachelors/Masters or Phd in Computer Science or related industry experience

3+ years of Industry Experience in Deep Learning Frameworks in PyTorch or TensorFlow

7+ Years of industry experience in scripting languages such as Python, R.

7+ years in software development doing at least some level of Researching / POCs, Prototyping, Productizing, Process improvement, Large-data processing / performance computing

Familiar with non-neural network methods such as Bayesian, SVM, Adaboost, Random Forests etc

Some experience in setting up large scale training data pipelines.

Some experience in using Cloud services such as AWS, GCP, Azure

Desired Skills:

Experience in building deep learning models for Computer Vision and Natural Language Processing domains

Experience in productionizing/serving machine learning in industry setting

Understand the principles of developing cloud native applications

Responsibilities

Collect, Organize and Process data pipelines for developing ML models

Research and develop novel prototypes for customers

Train, implement and evaluate shippable machine learning models

Deploy and iterate improvements of ML Models through feedback

Key Responsibilities:

• Partnering with clients and internal business owners (product, marketing, edit, etc.) to understand needs and develop models and products for Kaleidofin business line.
• Good understanding of the underlying business and workings of cross functional teams for successful execution
• Design and develop analyses based on business requirement needs and challenges.
• Leveraging statistical analysis on consumer research and data mining projects, including segmentation, clustering, factor analysis, multivariate regression, predictive modeling, hyperparameter tuning, ensembling etc.
• Providing statistical analysis on custom research projects and consult on A/B testing and other statistical analysis as needed. Other reports and custom analysis as required.
• Identify and use appropriate investigative and analytical technologies to interpret and verify results.
• Apply and learn a wide variety of tools and languages to achieve results
• Use best practices to develop statistical and/ or machine learning techniques to build models that address business needs.
• Collaborate with the team to improve the effectiveness of business decisions using data and machine learning/predictive modeling.
• Innovate on projects by using new modeling techniques or tools.
• Utilize effective project planning techniques to break down complex projects into tasks and ensure deadlines are kept.
• Communicate findings to team and leadership to ensure models are well understood and incorporated into business processes.

Skills:

• 2+ year experience in advanced analytics, model building, statistical modeling, optimization, and machine learning algorithms.
• Machine Learning Algorithms: Crystal clear understanding, coding, implementation, error analysis, model tuning knowledge on Linear Regression, Logistic Regression, SVM, shallow Neural Networks, clustering, Decision Trees, Random forest, Boosting trees, Recommender Systems, ARIMA and Anomaly Detection. Feature selection, hyper parameters tuning, model selection and error analysis, ensemble methods.
• Strong with programming languages like Python and data processing using SQL or equivalent and ability to experiment with newer open source tools
• Experience in normalizing data to ensure it is homogeneous and consistently formatted to enable sorting, query and analysis.
• Experience designing, developing, implementing and maintaining a database and programs to manage data analysis efforts.
• Experience with big data and cloud computing viz. Spark, Hadoop (MapReduce, PIG, HIVE)
• Experience in risk and credit scoring domains preferred

Job Overview

                         We are looking for a ML Architect to help us discover the information hidden in vast amounts of data, and help us make smarter decisions to deliver even better products. Your primary focus will be in applying data mining techniques, doing statistical analysis, and building high quality prediction systems integrated with our products. They must have strong experience using variety of data mining and data analysis methods, building and implementing models, using/creating algorithm’s and creating/running simulations.  They must be comfortable working with a wide range of stakeholders and functional teams. The right candidate will have a passion for discovering solutions hidden in large data sets and working with stakeholders to improve business outcomes. Automating to identify the textual data with their properties and structure form various type of document.

Responsibilities

• Selecting features, building and optimizing classifiers using machine learning techniques
• Data mining using state-of-the-art methods
• Enhancing data collection procedures to include information that is relevant for building analytic systems
• Processing, cleansing, and verifying the integrity of data used for analysis
• Creating automated anomaly detection systems and constant tracking of its performance
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Secure and manage when needed GPU cluster resources for events
• Write comprehensive internal feedback reports and find opportunities for improvements
• Manage GPU instances/machines to increase the performance and efficiency of the ML/DL model.

Skills and Qualifications

• Strong Hands-on experience in Python Programming
• Working experience with Computer Vision models - Object Detection Model, Image Classification
• Good experience in feature extraction, feature selection techniques and transfer learning
• Working Experience in building deep learning NLP Models for text classification, image analytics-CNN,RNN,LSTM.
• Working Experience in any of the AWS/GCP cloud platforms, exposure in fetching data from various sources.
• Good experience in exploratory data analysis, data visualisation, and other data preprocessing techniques.
• Knowledge in any one of the DL frameworks like Tensorflow, Pytorch, Keras, Caffe
• Good knowledge in statistics,distribution of data and in supervised and unsupervised machine learning algorithms.
• Exposure to OpenCV Familiarity with GPUs + CUDA
• Experience with NVIDIA software for cluster management and provisioning such as nvsm, dcgm and DeepOps.
• We are looking for a candidate with 14+ years of experience, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with AWS cloud services: EC2, RDS, AWS-Sagemaker(Added advantage)
• Experience with object-oriented/object function scripting languages in any: Python, Java, C++, Scala, etc.