Data Engineer

About The Company

 The client is 17-year-old Multinational Company headquartered in Bangalore, Whitefield, and having another delivery center in Pune, Hinjewadi. It also has offices in US and Germany and are working with several OEM’s and Product Companies in about 12 countries and is a 200+ strong team worldwide. 

The Role

Power BI front-end developer in the Data Domain (Manufacturing, Sales & Marketing, Purchasing, Logistics, …).Responsible for the Power BI front-end design, development, and delivery of highly visible data-driven applications in the Compressor Technique. You always take a quality-first approach where you ensure the data is visualized in a clear, accurate, and user-friendly manner. You always ensure standards and best practices are followed and ensure documentation is created and maintained. Where needed, you take initiative and make

recommendations to drive improvements. In this role you will also be involved in the tracking, monitoring and performance analysis

of production issues and the implementation of bugfixes and enhancements.

Skills & Experience

• The ideal candidate has a degree in Computer Science, Information Technology or equal through experience.

• Strong knowledge on BI development principles, time intelligence, functions, dimensional modeling and data visualization is required.

• Advanced knowledge and 5-10 years experience with professional BI development & data visualization is preferred.

• You are familiar with data warehouse concepts.

• Knowledge on MS Azure (data lake, databricks, SQL) is considered as a plus.

• Experience and knowledge on scripting languages such as PowerShell and Python to setup and automate Power BI platform related activities is an asset.

• Good knowledge (oral and written) of English is required.

Title:- Data Scientist

Experience:-6 years

Work Mode:- Onsite

Primary Skills:- Data Science, SQL, Python, Data Modelling, Azure, AWS, Banking Domain (BFSI/NBFC)

Qualification:- Any

Roles & Responsibilities:-

1.  Acquiring, cleaning, and preprocessing raw data for analysis.

2.  Utilizing statistical methods and tools for analyzing and interpreting complex  datasets.

3.  Developing and implementing machine learning models for predictive analysis.

4.  Creating visualizations to effectively communicate insights to both technical and   non-technical stakeholders.

5.  Collaborating with cross-functional teams, including data engineers, business   analysts, and domain experts.

6.  Evaluating and optimizing the performance of machine learning models for   accuracy and efficiency.

7.  Identifying patterns and trends within data to inform business decision-making.

8.  Staying updated on the latest advancements in data science, machine learning, and  relevant technologies.

Requirement:- 

1.  Experience with modeling techniques such as Linear Regression, clustering, and classification techniques.

2.  Must have a passion for data, structured or unstructured.  0.6 – 5 years of hands-on experience with Python and SQL is a must.

3.   Should have sound experience in data mining, data analysis and machine learning techniques.

4.  Excellent critical thinking, verbal and written communications skills.

5.  Ability and desire to work in a proactive, highly engaging, high-pressure, client service environment.

6.   Good presentation skills.

What are we looking for:

1. Strong experience in MySQL and writing advanced queries
2. Strong experience in Bash and Python
3. Familiarity with ElasticSearch, Redis, Java, NodeJS, ClickHouse, S3
4. Exposure to cloud services such as AWS, Azure, or GCP
5. 2+ years of experience in the production support
6. Strong experience in log management and performance monitoring like ELK, Prometheus + Grafana, logging services on various cloud platforms
7. Strong understanding of Linux OSes like Ubuntu, CentOS / Redhat Linux
8. Interest in learning new languages / framework as needed
9. Good written and oral communications skills
10. A growth mindset and passionate about building things from the ground up, and most importantly, you should be fun to work with

As a product solutions engineer, you will:

1. Analyze recorded runtime issues, diagnose and do occasional code fixes of low to medium complexity
2. Work with developers to find and correct more complex issues
3. Address urgent issues quickly, work within and measure against customer SLAs
4. Using shell and python scripts, and use scripting to actively automate manual / repetitive activities
5. Build anomaly detectors wherever applicable
6. Pass articulated feedback from customers to the development and product team
7. Maintain ongoing record of the operation of problem analysis and resolution in a on call monitoring system
8. Offer technical support needed in development

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:

• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

Role and Responsibilities

• Execute data mining projects, training and deploying models over a typical duration of 2 -12 months.
• The ideal candidate should be able to innovate, analyze the customer requirement, develop a solution in the time box of the project plan, execute and deploy the solution.
• Integrate the data mining projects embedded data mining applications in the FogHorn platform (on Docker or Android).

Core Qualifications
Candidates must meet ALL of the following qualifications:

• Have analyzed, trained and deployed at least three data mining models in the past. If the candidate did not directly deploy their own models, they will have worked with others who have put their models into production. The models should have been validated as robust over at least an initial time period.
• Three years of industry work experience, developing data mining models which were deployed and used.
• Programming experience in Python is core using data mining related libraries like Scikit-Learn. Other relevant Python mining libraries include NumPy, SciPy and Pandas.
• Data mining algorithm experience in at least 3 algorithms across: prediction (statistical regression, neural nets, deep learning, decision trees, SVM, ensembles), clustering (k-means, DBSCAN or other) or Bayesian networks

Bonus Qualifications
Any of the following extra qualifications will make a candidate more competitive:

• Soft Skills
  • Sets expectations, develops project plans and meets expectations.
  • Experience adapting technical dialogue to the right level for the audience (i.e. executives) or specific jargon for a given vertical market and job function.
• Technical skills
  • Commonly, candidates have a MS or Ph.D. in Computer Science, Math, Statistics or an engineering technical discipline. BS candidates with experience are considered.
  • Have managed past models in production over their full life cycle until model replacement is needed. Have developed automated model refreshing on newer data. Have developed frameworks for model automation as a prototype for product.
  • Training or experience in Deep Learning, such as TensorFlow, Keras, convolutional neural networks (CNN) or Long Short Term Memory (LSTM) neural network architectures. If you don’t have deep learning experience, we will train you on the job.
  • Shrinking deep learning models, optimizing to speed up execution time of scoring or inference.
  • OpenCV or other image processing tools or libraries
  • Cloud computing: Google Cloud, Amazon AWS or Microsoft Azure. We have integration with Google Cloud and are working on other integrations.
  • Decision trees like XGBoost or Random Forests is helpful.
  • Complex Event Processing (CEP) or other streaming data as a data source for data mining analysis
  • Time series algorithms from ARIMA to LSTM to Digital Signal Processing (DSP).
  • Bayesian Networks (BN), a.k.a. Bayesian Belief Networks (BBN) or Graphical Belief Networks (GBN)
  • Experience with PMML is of interest (see www.DMG.org).
• Vertical experience in Industrial Internet of Things (IoT) applications:
  • Energy: Oil and Gas, Wind Turbines
  • Manufacturing: Motors, chemical processes, tools, automotive
  • Smart Cities: Elevators, cameras on population or cars, power grid
  • Transportation: Cars, truck fleets, trains

About FogHorn Systems
FogHorn is a leading developer of “edge intelligence” software for industrial and commercial IoT application solutions. FogHorn’s Lightning software platform brings the power of advanced analytics and machine learning to the on-premise edge environment enabling a new class of applications for advanced monitoring and diagnostics, machine performance optimization, proactive maintenance and operational intelligence use cases. FogHorn’s technology is ideally suited for OEMs, systems integrators and end customers in manufacturing, power and water, oil and gas, renewable energy, mining, transportation, healthcare, retail, as well as Smart Grid, Smart City, Smart Building and connected vehicle applications.

Press:  https://www.foghorn.io/press-room/">https://www.foghorn.io/press-room/

Awards: https://www.foghorn.io/awards-and-recognition/">https://www.foghorn.io/awards-and-recognition/

• 2019 Edge Computing Company of the Year – Compass Intelligence
• 2019 Internet of Things 50: 10 Coolest Industrial IoT Companies – CRN
• 2018 IoT Planforms Leadership Award & Edge Computing Excellence – IoT Evolution World Magazine
• 2018 10 Hot IoT Startups to Watch – Network World. (Gartner estimated 20 billion connected things in use worldwide by 2020)
• 2018 Winner in Artificial Intelligence and Machine Learning – Globe Awards
• 2018 Ten Edge Computing Vendors to Watch – ZDNet & 451 Research
• 2018 The 10 Most Innovative AI Solution Providers – Insights Success
• 2018 The AI 100 – CB Insights
• 2017 Cool Vendor in IoT Edge Computing – Gartner
• 2017 20 Most Promising AI Service Providers – CIO Review

Our Series A round was for $15 million.  Our Series B round was for $30 million October 2017.  Investors include: Saudi Aramco Energy Ventures, Intel Capital, GE, Dell, Bosch, Honeywell and The Hive.

About the Data Science Solutions team
In 2018, our Data Science Solutions team grew from 4 to 9.  We are growing again from 11. We work on revenue generating projects for clients, such as predictive maintenance, time to failure, manufacturing defects.  About half of our projects have been related to vision recognition or deep learning. We are not only working on consulting projects but developing vertical solution applications that run on our Lightning platform, with embedded data mining.

Our data scientists like our team because:

• We care about “best practices”
• Have a direct impact on the company’s revenue
• Give or receive mentoring as part of the collaborative process
• Questions and challenging the status quo with data is safe
• Intellectual curiosity balanced with humility
• Present papers or projects in our “Thought Leadership” meeting series, to support continuous learning