Tech Lead

About DeepIntent:

DeepIntent is a marketing technology company that helps healthcare brands strengthen communication with patients and healthcare professionals by enabling highly effective and performant digital advertising campaigns. Our healthcare technology platform, MarketMatch™, connects advertisers, data providers, and publishers to operate the first unified, programmatic marketplace for healthcare marketers. The platform’s built-in identity solution matches digital IDs with clinical, behavioural, and contextual data in real-time so marketers can qualify 1.6M+ verified HCPs and 225M+ patients to find their most clinically-relevant audiences and message them on a one-to-one basis in a privacy-compliant way. Healthcare marketers use MarketMatch to plan, activate, and measure digital campaigns in ways that best suit their business, from managed service engagements to technical integration or self-service solutions. DeepIntent was founded by Memorial Sloan Kettering alumni in 2016 and acquired by Propel Media, Inc. in 2017. We proudly serve major pharmaceutical and Fortune 500 companies out of our offices in New York, Bosnia and India.

What You’ll Do:

• Establish formal data practice for the organisation.
• Build & operate scalable and robust data architectures.
• Create pipelines for the self-service introduction and usage of new data
• Implement DataOps practices
• Design, Develop, and operate Data Pipelines which support Data scientists and machine learning
• Engineers.
• Build simple, highly reliable Data storage, ingestion, and transformation solutions which are easy
• to deploy and manage.
• Collaborate with various business stakeholders, software engineers, machine learning
• engineers, and analysts.

Who You Are:

• Experience in designing, developing and operating configurable Data pipelines serving high
• volume and velocity data.
• Experience working with public clouds like GCP/AWS.
• Good understanding of software engineering, DataOps, data architecture, Agile and
• DevOps methodologies.
• Experience building Data architectures that optimize performance and cost, whether the
• components are prepackaged or homegrown
• Proficient with SQL, Java, Spring boot, Python or JVM-based language, Bash
• Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow
• etc. and big data databases like BigQuery, Clickhouse, etc
• Good communication skills with the ability to collaborate with both technical and non-technical
• people.
• Ability to Think Big, take bets and innovate, Dive Deep, Bias for Action, Hire and Develop the Best, Learn and be Curious

CORE RESPONSIBILITIES

• Create and manage cloud resources in AWS 
• Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies 
• Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform 
• Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations 
• Develop an infrastructure to collect, transform, combine and publish/distribute customer data.
• Define process improvement opportunities to optimize data collection, insights and displays.
• Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible 
• Identify and interpret trends and patterns from complex data sets 
• Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders. 
• Key participant in regular Scrum ceremonies with the agile teams  
• Proficient at developing queries, writing reports and presenting findings 
• Mentor junior members and bring best industry practices 

QUALIFICATIONS

• 5-7+ years’ experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales) 
• Strong background in math, statistics, computer science, data science or related discipline
• Advanced knowledge one of language: Java, Scala, Python, C# 
• Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake  
• Proficient with
• Data mining/programming tools (e.g. SAS, SQL, R, Python)
• Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
• Data visualization (e.g. Tableau, Looker, MicroStrategy)
• Comfortable learning about and deploying new technologies and tools. 
• Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines. 
• Good written and oral communication skills and ability to present results to non-technical audiences 
• Knowledge of business intelligence and analytical tools, technologies and techniques.

Mandatory Requirements 

• Experience in AWS Glue
• Experience in Apache Parquet 
• Proficient in AWS S3 and data lake 
• Knowledge of Snowflake
• Understanding of file-based ingestion best practices.
• Scripting language - Python & pyspark

Designation: Specialist - Cloud Service Developer (ABL_SS_600)

Position description:

• The person would be primary responsible for developing solutions using AWS services. Ex: Fargate, Lambda, ECS, ALB, NLB, S3 etc.
• Apply advanced troubleshooting techniques to provide Solutions to issues pertaining to Service Availability, Performance, and Resiliency
• Monitor & Optimize the performance using AWS dashboards and logs
• Partner with Engineering leaders and peers in delivering technology solutions that meet the business requirements 
• Work with the cloud team in agile approach and develop cost optimized solutions

Primary Responsibilities:

• Develop solutions using AWS services includiing Fargate, Lambda, ECS, ALB, NLB, S3 etc.

Reporting Team

• Reporting Designation: Head - Big Data Engineering and Cloud Development (ABL_SS_414)
• Reporting Department: Application Development (2487)

Required Skills:

• AWS certification would be preferred
• Good understanding in Monitoring (Cloudwatch, alarms, logs, custom metrics, Trust SNS configuration)
• Good experience with Fargate, Lambda, ECS, ALB, NLB, S3, Glue, Aurora and other AWS services. 
• Preferred to have Knowledge on Storage (S3, Life cycle management, Event configuration)
• Good in data structure, programming in (pyspark / python / golang / Scala)

Responsibilities

• Own the design, development, testing, deployment, and craftsmanship of the team’s infrastructure and systems capable of handling massive amounts of requests with high reliability and scalability
• Leverage the deep and broad technical expertise to mentor engineers and provide leadership on resolving complex technology issues
• Entrepreneurial and out-of-box thinking essential for a technology startup
• Guide the team for unit-test code for robustness, including edge cases, usability, and general reliability

Requirements

• In-depth understanding of image processing algorithms, pattern recognition methods, and rule-based classifiers
• Experience in feature extraction, object recognition and tracking, image registration, noise reduction, image calibration, and correction
• Ability to understand, optimize and debug imaging algorithms
• Understating and experience in openCV library
• Fundamental understanding of mathematical techniques involved in ML and DL schemas (Instance-based methods, Boosting methods, PGM, Neural Networks etc.)
• Thorough understanding of state-of-the-art DL concepts (Sequence modeling, Attention, Convolution etc.) along with knack to imagine new schemas that work for the given data.
• Understanding of engineering principles and a clear understanding of data structures and algorithms
• Experience in writing production level codes using either C++ or Java
• Experience with technologies/libraries such as python pandas, numpy, scipy
• Experience with tensorflow and scikit.

• Must have 5-8 years of experience in handling data
• Must have the ability to interpret large amounts of data and to multi-task
• Must have strong knowledge of and experience with programming (Python), Linux/Bash scripting, databases(SQL, etc)
• Must have strong analytical and critical thinking to resolve business problems using data and tech
•  Must have domain familiarity and interest of – Cloud technologies (GCP/Azure Microsoft/ AWS Amazon), open-source technologies, Enterprise technologies
• Must have the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.
• Must have good communication skills
• Working knowledge/exposure to ElasticSearch, PostgreSQL, Athena, PrestoDB, Jupyter Notebook