Data Engineer

Job Description: Data Engineer

We are looking for a curious Data Engineer to join our extremely fast-growing Tech Team at StanPlus

About RED.Health (Formerly Stanplus Technologies)

Get to know the team:

Join our team and help us build the world’s fastest and most reliable emergency response system using cutting-edge technology.

Because every second counts in an emergency, we are building systems and flows with 4 9s of reliability to ensure that our technology is always there when people need it the most. We are looking for distributed systems experts who can help us perfect the architecture behind our key design principles: scalability, reliability, programmability, and resiliency. Our system features a powerful dispatch engine that connects emergency service providers with patients in real-time

.

Key Responsibilities

●     Build Data ETL Pipelines

●     Develop data set processes

●     Strong analytic skills related to working with unstructured datasets

●     Evaluate business needs and objectives

●     Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery

●     Interpret trends and patterns

●     Work with data and analytics experts to strive for greater functionality in our data system

●     Build algorithms and prototypes

●     Explore ways to enhance data quality and reliability

●     Work with the Executive, Product, Data, and D   esign teams, to assist with data-related technical issues and support their data infrastructure needs.

●     Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.

Key Requirements

●     Proven experience as a data engineer, software developer, or similar of at least 3 years.

●     Bachelor's / Master’s degree in data engineering, big data analytics, computer engineering, or related field.

●     Experience with big data tools: Hadoop, Spark, Kafka, etc.

●     Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.

●     Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

●     Experience with Azure, AWS cloud services: EC2, EMR, RDS, Redshift

●     Experience with BigQuery

●     Experience with stream-processing systems: Storm, Spark-Streaming, etc.

●     Experience with languages: Python, Java, C++, Scala, SQL, R, etc.

●     Good hands-on with Hive, Presto.

• Play a critical role as a member of the leadership team in shaping and supporting our overall company vision, day-to-day operations, and culture.
• Set the technical vision and build the technical product roadmap from launch to scale; including defining long-term goals and strategies
• Define best practices around coding methodologies, software development, and quality assurance
• Define innovative technical requirements and systems while balancing time, feasibility, cost and customer experience
• Build and support production products
• Ensure our internal processes and services comply with privacy and security regulations
• Establish a high performing, inclusive engineering culture focused on innovation, execution, growth and development
• Set a high bar for our overall engineering practices in support of our mission and goals
• Develop goals, roadmaps and delivery dates to help us scale quickly and sustainably
• Collaborate closely with Product, Business, Marketing and Data Science
• Experience with financial and transactional systems
• Experience engineering for large volumes of data at scale
• Experience with financial audit and compliance is a plus
• Experience building a successful consumer facing web and mobile apps at scale

4-6 years of total experience in data warehousing and business intelligence

3+ years of solid Power BI experience (Power Query, M-Query, DAX, Aggregates)

2 years’ experience building Power BI using cloud data (Snowflake, Azure Synapse, SQL DB, data lake)

Strong experience building visually appealing UI/UX in Power BI

Understand how to design Power BI solutions for performance (composite models, incremental refresh, analysis services)

Experience building Power BI using large data in direct query mode

Expert SQL background (query building, stored procedure, optimizing performance)

Summary
Our Kafka developer has a combination of technical skills, communication skills and business knowledge. The developer should be able to work on multiple medium to large projects. The successful candidate will have excellent technical skills of Apache/Confluent Kafka, Enterprise Data WareHouse preferable GCP BigQuery or any equivalent Cloud EDW and also will be able to take oral and written business requirements and develop efficient code to meet set deliverables.

Must Have Skills

• Participate in the development, enhancement and maintenance of data applications both as an individual contributor and as a lead.
• Leading in the identification, isolation, resolution and communication of problems within the production environment.
• Leading developer and applying technical skills Apache/Confluent Kafka (Preferred) AWS Kinesis (Optional), Cloud Enterprise Data Warehouse Google BigQuery (Preferred) or AWS RedShift or SnowFlakes (Optional)
• Design recommending best approach suited for data movement from different sources to Cloud EDW using Apache/Confluent Kafka
• Performs independent functional and technical analysis for major projects supporting several corporate initiatives.
• Communicate and Work with IT partners and user community with various levels from Sr Management to detailed developer to business SME for project definition .
• Works on multiple platforms and multiple projects concurrently.
• Performs code and unit testing for complex scope modules, and projects
• Provide expertise and hands on experience working on Kafka connect using schema registry in a very high volume environment (~900 Million messages)

• Provide expertise in Kafka brokers, zookeepers, KSQL, KStream and Kafka Control center.
• Provide expertise and hands on experience working on AvroConverters, JsonConverters, and StringConverters.
• Provide expertise and hands on experience working on Kafka connectors such as MQ connectors, Elastic Search connectors, JDBC connectors, File stream connector,  JMS source connectors, Tasks, Workers, converters, Transforms.
• Provide expertise and hands on experience on custom connectors using the Kafka core concepts and API.
• Working knowledge on Kafka Rest proxy.
• Ensure optimum performance, high availability and stability of solutions.
• Create topics, setup redundancy cluster, deploy monitoring tools, alerts and has good knowledge of best practices.
• Create stubs for producers, consumers and consumer groups for helping onboard applications from different languages/platforms.  Leverage Hadoop ecosystem knowledge to design, and develop capabilities to deliver our solutions using Spark, Scala, Python, Hive, Kafka and other things in the Hadoop ecosystem. 
• Use automation tools like provisioning using Jenkins, Udeploy or relevant technologies
• Ability to perform data related benchmarking, performance analysis and tuning.
• Strong skills in In-memory applications, Database Design, Data Integration.

• Must have 5-8 years of experience in handling data
• Must have the ability to interpret large amounts of data and to multi-task
• Must have strong knowledge of and experience with programming (Python), Linux/Bash scripting, databases(SQL, etc)
• Must have strong analytical and critical thinking to resolve business problems using data and tech
•  Must have domain familiarity and interest of – Cloud technologies (GCP/Azure Microsoft/ AWS Amazon), open-source technologies, Enterprise technologies
• Must have the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.
• Must have good communication skills
• Working knowledge/exposure to ElasticSearch, PostgreSQL, Athena, PrestoDB, Jupyter Notebook

We’re looking to hire someone to help scale Machine Learning and NLP efforts at Episource. You’ll work with the team that develops the models powering Episource’s product focused on NLP driven medical coding. Some of the problems include improving our ICD code recommendations , clinical named entity recognition and information extraction from clinical notes.

This is a role for highly technical machine learning & data engineers who combine outstanding oral and written communication skills, and the ability to code up prototypes and productionalize using a large range of tools, algorithms, and languages. Most importantly they need to have the ability to autonomously plan and organize their work assignments based on high-level team goals.

You will be responsible for setting an agenda to develop and ship machine learning models that positively impact the business, working with partners across the company including operations and engineering. You will use research results to shape strategy for the company, and help build a foundation of tools and practices used by quantitative staff across the company.

What you will achieve:

• Define the research vision for data science, and oversee planning, staffing, and prioritization to make sure the team is advancing that roadmap

• Invest in your team’s skills, tools, and processes to improve their velocity, including working with engineering counterparts to shape the roadmap for machine learning needs

• Hire, retain, and develop talented and diverse staff through ownership of our data science hiring processes, brand, and functional leadership of data scientists

• Evangelise machine learning and AI internally and externally, including attending conferences and being a thought leader in the space

• Partner with the executive team and other business leaders to deliver cross-functional research work and models

Required Skills:

• Strong background in classical machine learning and machine learning deployments is a must and preferably with 4-8 years of experience

• Knowledge of deep learning & NLP

• Hands-on experience in TensorFlow/PyTorch, Scikit-Learn, Python, Apache Spark & Big Data platforms to manipulate large-scale structured and unstructured datasets.

• Experience with GPU computing is a plus.

• Professional experience as a data science leader, setting the vision for how to most effectively use data in your organization. This could be through technical leadership with ownership over a research agenda, or developing a team as a personnel manager in a new area at a larger company.

• Expert-level experience with a wide range of quantitative methods that can be applied to business problems.

• Evidence you’ve successfully been able to scope, deliver and sell your own research in a way that shifts the agenda of a large organization.

• Excellent written and verbal communication skills on quantitative topics for a variety of audiences: product managers, designers, engineers, and business leaders.

• Fluent in data fundamentals: SQL, data manipulation using a procedural language, statistics, experimentation, and modeling

Qualifications

• Professional experience as a data science leader, setting the vision for how to most effectively use data in your organization

• Expert-level experience with machine learning that can be applied to business problems

• Evidence you’ve successfully been able to scope, deliver and sell your own work in a way that shifts the agenda of a large organization

• Fluent in data fundamentals: SQL, data manipulation using a procedural language, statistics, experimentation, and modeling

• Degree in a field that has very applicable use of data science / statistics techniques (e.g. statistics, applied math, computer science, OR a science field with direct statistics application)

• 5+ years of industry experience in data science and machine learning, preferably at a software product company

• 3+ years of experience managing data science teams, incl. managing/grooming managers beneath you

• 3+ years of experience partnering with executive staff on data topics