Data Engineer

Data Scientist – Program Embedded 

Job Description:   

We are seeking a highly skilled and motivated senior data scientist to support a big data program. The successful candidate will play a pivotal role in supporting multiple projects in this program covering traditional tasks from revenue management, demand forecasting, improving customer experience to testing/using new tools/platforms such as Copilot Fabric for different purpose. The expected candidate would have deep expertise in machine learning methodology and applications. And he/she should have completed multiple large scale data science projects (full cycle from ideation to BAU). Beyond technical expertise, problem solving in complex set-up will be key to the success for this role. This is a data science role directly embedded into the program/projects, stake holder management and collaborations with patterner are crucial to the success on this role (on top of the deep expertise). 

What we are looking for: 

1. Highly efficient in Python/Pyspark/R. 
2. Understand MLOps concepts, working experience in product industrialization (from Data Science point of view). Experience in building product for live deployment, and continuous development and continuous integration. 
3. Familiar with cloud platforms such as Azure, GCP, and the data management systems on such platform. Familiar with Databricks and product deployment on Databricks. 
4. Experience in ML projects involving techniques: Regression, Time Series, Clustering, Classification, Dimension Reduction, Anomaly detection with traditional ML approaches and DL approaches. 
5. Solid background in statistics, probability distributions, A/B testing validation, univariate/multivariate analysis, hypothesis test for different purpose, data augmentation etc. 
6. Familiar with designing testing framework for different modelling practice/projects based on business needs. 
7. Exposure to Gen AI tools and enthusiastic about experimenting and have new ideas on what can be done. 
8. If they have improved an internal company process using an AI tool, that would be great (e.g. process simplification, manual task automation, auto emails) 
9. Ideally, 10+ years of experience, and have been on independent business facing roles. 
10. CPG or retail as a data scientist would be nice, but not number one priority, especially for those who have navigated through multiple industries. 
11. Being proactive and collaborative would be essential. 

Some projects examples within the program: 

1. Test new tools/platforms such as Copilo, Fabric for commercial reporting. Testing, validation and build trust. 
2. Building algorithms for predicting trend in category, consumptions to support dashboards. 
3. Revenue Growth Management, create/understand the algorithms behind the tools (can be built by 3rd parties) we need to maintain or choose to improve. Able to prioritize and build product roadmap. Able to design new solutions and articulate/quantify the limitation of the solutions. 
4. Demand forecasting, create localized forecasts to improve in store availability. Proper model monitoring for early detection of potential issues in the forecast focusing particularly on improving the end user experience. 

Big Data Engineer: 5+ yrs.
Immediate Joiner

• Expertise in building AWS Data Engineering pipelines with AWS Glue -> Athena -> Quick sight
• Experience in developing lambda functions with AWS Lambda
• Expertise with Spark/PySpark – Candidate should be hands on with PySpark code and should be able to do transformations with Spark
• Should be able to code in Python and Scala.
• Snowflake experience will be a plus
• We can start keeping Hadoop and Hive requirements as good to have or understanding of is enough rather than keeping it as a desirable requirement.

Specific Responsibilities

• Minimum of 2 years Experience in Google Big Query and Google Cloud Platform.
• Design and develop the ETL framework using BigQuery
• Expertise in Big Query concepts like Nested Queries, Clustering, Partitioning, etc.
• Working Experience of Clickstream database, Google Analytics/ Adobe Analytics.
• Should be able to automate the data load from Big Query using APIs or scripting language.
• Good experience in Advanced SQL concepts.
• Good experience with Adobe launch Web, Mobile & e-commerce tag implementation.
• Identify complex fuzzy problems, break them down in smaller parts, and implement creative, data-driven solutions
• Responsible for defining, analyzing, and communicating key metrics and business trends to the management teams
• Identify opportunities to improve conversion & user experience through data. Influence product & feature roadmaps.
• Must have a passion for data quality and be constantly looking to improve the system. Drive data-driven decision making through the stakeholders & drive Change Management
• Understand requirements to translate business problems & technical problems into analytics problems.
• Effective storyboarding and presentation of the solution to the client and leadership.
• Client engagement & management
• Ability to interface effectively with multiple levels of management and functional disciplines.
• Assist in developing/coaching individuals technically as well as on soft skills during the project and as part of Client Project’s training program.

• 2 to 3 years of working experience in Google Big Query & Google Cloud Platform
• Relevant experience in Consumer Tech/CPG/Retail industries
• Bachelor’s in engineering, Computer Science, Math, Statistics or related discipline
• Strong problem solving and web analytical skills. Acute attention to detail.
• Experience in analyzing large, complex, multi-dimensional data sets.
• Experience in one or more roles in an online eCommerce or online support environment.

• Expertise in Google Big Query & Google Cloud Platform
• Experience in Advanced SQL, Scripting language (Python/R)
• Hands-on experience in BI tools (Tableau, Power BI)
• Working Experience & understanding of Adobe Analytics or Google Analytics
• Experience in creating and debugging website & app tracking (Omnibus, Dataslayer, GA debugger, etc.)
• Excellent analytical thinking, analysis, and problem-solving skills.
• Knowledge of other GCP services is a plus

• Proficiency in shell scripting
• Proficiency in automation of tasks
• Proficiency in Pyspark/Python
• Proficiency in writing and understanding of sqoop
• Understanding of CloudEra manager
• Good understanding of RDBMS
• Good understanding of Excel

Datametica is looking for talented SQL engineers who would get training & the opportunity to work on Cloud and Big Data Analytics.

Mandatory Skills:

• Strong in SQL development
• Hands-on at least one scripting language - preferably shell scripting
• Development experience in Data warehouse projects

Opportunities:

• Selected candidates will be provided training opportunities on one or more of the following: Google Cloud, AWS, DevOps Tools, Big Data technologies like Hadoop, Pig, Hive, Spark, Sqoop, Flume, and KafkaWould get a chance to be part of the enterprise-grade implementation of Cloud and Big Data systems
• Will play an active role in setting up the Modern data platform based on Cloud and Big Data
• Would be part of teams with rich experience in various aspects of distributed systems and computing

Role and Responsibilities

• Execute data mining projects, training and deploying models over a typical duration of 2 -12 months.
• The ideal candidate should be able to innovate, analyze the customer requirement, develop a solution in the time box of the project plan, execute and deploy the solution.
• Integrate the data mining projects embedded data mining applications in the FogHorn platform (on Docker or Android).

Core Qualifications
Candidates must meet ALL of the following qualifications:

• Have analyzed, trained and deployed at least three data mining models in the past. If the candidate did not directly deploy their own models, they will have worked with others who have put their models into production. The models should have been validated as robust over at least an initial time period.
• Three years of industry work experience, developing data mining models which were deployed and used.
• Programming experience in Python is core using data mining related libraries like Scikit-Learn. Other relevant Python mining libraries include NumPy, SciPy and Pandas.
• Data mining algorithm experience in at least 3 algorithms across: prediction (statistical regression, neural nets, deep learning, decision trees, SVM, ensembles), clustering (k-means, DBSCAN or other) or Bayesian networks

Bonus Qualifications
Any of the following extra qualifications will make a candidate more competitive:

• Soft Skills
  • Sets expectations, develops project plans and meets expectations.
  • Experience adapting technical dialogue to the right level for the audience (i.e. executives) or specific jargon for a given vertical market and job function.
• Technical skills
  • Commonly, candidates have a MS or Ph.D. in Computer Science, Math, Statistics or an engineering technical discipline. BS candidates with experience are considered.
  • Have managed past models in production over their full life cycle until model replacement is needed. Have developed automated model refreshing on newer data. Have developed frameworks for model automation as a prototype for product.
  • Training or experience in Deep Learning, such as TensorFlow, Keras, convolutional neural networks (CNN) or Long Short Term Memory (LSTM) neural network architectures. If you don’t have deep learning experience, we will train you on the job.
  • Shrinking deep learning models, optimizing to speed up execution time of scoring or inference.
  • OpenCV or other image processing tools or libraries
  • Cloud computing: Google Cloud, Amazon AWS or Microsoft Azure. We have integration with Google Cloud and are working on other integrations.
  • Decision trees like XGBoost or Random Forests is helpful.
  • Complex Event Processing (CEP) or other streaming data as a data source for data mining analysis
  • Time series algorithms from ARIMA to LSTM to Digital Signal Processing (DSP).
  • Bayesian Networks (BN), a.k.a. Bayesian Belief Networks (BBN) or Graphical Belief Networks (GBN)
  • Experience with PMML is of interest (see www.DMG.org).
• Vertical experience in Industrial Internet of Things (IoT) applications:
  • Energy: Oil and Gas, Wind Turbines
  • Manufacturing: Motors, chemical processes, tools, automotive
  • Smart Cities: Elevators, cameras on population or cars, power grid
  • Transportation: Cars, truck fleets, trains

About FogHorn Systems
FogHorn is a leading developer of “edge intelligence” software for industrial and commercial IoT application solutions. FogHorn’s Lightning software platform brings the power of advanced analytics and machine learning to the on-premise edge environment enabling a new class of applications for advanced monitoring and diagnostics, machine performance optimization, proactive maintenance and operational intelligence use cases. FogHorn’s technology is ideally suited for OEMs, systems integrators and end customers in manufacturing, power and water, oil and gas, renewable energy, mining, transportation, healthcare, retail, as well as Smart Grid, Smart City, Smart Building and connected vehicle applications.

Press:  https://www.foghorn.io/press-room/">https://www.foghorn.io/press-room/

Awards: https://www.foghorn.io/awards-and-recognition/">https://www.foghorn.io/awards-and-recognition/

• 2019 Edge Computing Company of the Year – Compass Intelligence
• 2019 Internet of Things 50: 10 Coolest Industrial IoT Companies – CRN
• 2018 IoT Planforms Leadership Award & Edge Computing Excellence – IoT Evolution World Magazine
• 2018 10 Hot IoT Startups to Watch – Network World. (Gartner estimated 20 billion connected things in use worldwide by 2020)
• 2018 Winner in Artificial Intelligence and Machine Learning – Globe Awards
• 2018 Ten Edge Computing Vendors to Watch – ZDNet & 451 Research
• 2018 The 10 Most Innovative AI Solution Providers – Insights Success
• 2018 The AI 100 – CB Insights
• 2017 Cool Vendor in IoT Edge Computing – Gartner
• 2017 20 Most Promising AI Service Providers – CIO Review

Our Series A round was for $15 million.  Our Series B round was for $30 million October 2017.  Investors include: Saudi Aramco Energy Ventures, Intel Capital, GE, Dell, Bosch, Honeywell and The Hive.

About the Data Science Solutions team
In 2018, our Data Science Solutions team grew from 4 to 9.  We are growing again from 11. We work on revenue generating projects for clients, such as predictive maintenance, time to failure, manufacturing defects.  About half of our projects have been related to vision recognition or deep learning. We are not only working on consulting projects but developing vertical solution applications that run on our Lightning platform, with embedded data mining.

Our data scientists like our team because:

• We care about “best practices”
• Have a direct impact on the company’s revenue
• Give or receive mentoring as part of the collaborative process
• Questions and challenging the status quo with data is safe
• Intellectual curiosity balanced with humility
• Present papers or projects in our “Thought Leadership” meeting series, to support continuous learning