Senior Software Engineer

About DeepIntent:

DeepIntent is a marketing technology company that helps healthcare brands strengthen communication with patients and healthcare professionals by enabling highly effective and performant digital advertising campaigns. Our healthcare technology platform, MarketMatch™, connects advertisers, data providers, and publishers to operate the first unified, programmatic marketplace for healthcare marketers. The platform’s built-in identity solution matches digital IDs with clinical, behavioural, and contextual data in real-time so marketers can qualify 1.6M+ verified HCPs and 225M+ patients to find their most clinically-relevant audiences and message them on a one-to-one basis in a privacy-compliant way. Healthcare marketers use MarketMatch to plan, activate, and measure digital campaigns in ways that best suit their business, from managed service engagements to technical integration or self-service solutions. DeepIntent was founded by Memorial Sloan Kettering alumni in 2016 and acquired by Propel Media, Inc. in 2017. We proudly serve major pharmaceutical and Fortune 500 companies out of our offices in New York, Bosnia and India.

What You’ll Do:

• Establish formal data practice for the organisation.
• Build & operate scalable and robust data architectures.
• Create pipelines for the self-service introduction and usage of new data
• Implement DataOps practices
• Design, Develop, and operate Data Pipelines which support Data scientists and machine learning
• Engineers.
• Build simple, highly reliable Data storage, ingestion, and transformation solutions which are easy
• to deploy and manage.
• Collaborate with various business stakeholders, software engineers, machine learning
• engineers, and analysts.

Who You Are:

• Experience in designing, developing and operating configurable Data pipelines serving high
• volume and velocity data.
• Experience working with public clouds like GCP/AWS.
• Good understanding of software engineering, DataOps, data architecture, Agile and
• DevOps methodologies.
• Experience building Data architectures that optimize performance and cost, whether the
• components are prepackaged or homegrown
• Proficient with SQL, Java, Spring boot, Python or JVM-based language, Bash
• Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow
• etc. and big data databases like BigQuery, Clickhouse, etc
• Good communication skills with the ability to collaborate with both technical and non-technical
• people.
• Ability to Think Big, take bets and innovate, Dive Deep, Bias for Action, Hire and Develop the Best, Learn and be Curious

• Creating and managing ETL/ELT pipelines based on requirements
• Build PowerBI dashboards and manage datasets needed.
• Work with stakeholders to identify data structures needed for future and perform any transformations including aggregations.
• Build data cubes for real-time visualisation needs and CXO dashboards.

Required Tech Skills

• Microsoft PowerBI & DAX
• Python, Pandas, PyArrow, Jupyter Noteboks, ApacheSpark
• Azure Synapse, Azure DataBricks, Azure HDInsight, Azure Data Factory

• We are looking for : Data engineer
• Sprak
• Scala
• Hadoop

Job Description

The applicant must have a minimum of 5 years of hands-on IT experience, working on a full software lifecycle in Agile mode.

Good to have experience in data modeling and/or systems architecture.
Responsibilities will include technical analysis, design, development and perform enhancements.

You will participate in all/most of the following activities:
- Working with business analysts and other project leads to understand requirements.
- Modeling and implementing database schemas in DB2 UDB or other relational databases.
- Designing, developing, maintaining and Data processing using Python, DB2, Greenplum, Autosys and other technologies

Skills /Expertise Required :

Work experience in developing large volume database (DB2/Greenplum/Oracle/Sybase).

Good experience in writing stored procedures, integration of database processing, tuning and optimizing database queries.

Strong knowledge of table partitions, high-performance loading and data processing.
Good to have hands-on experience working with Perl or Python.
Hands on development using Spark / KDB / Greenplum platform will be a strong plus.
Designing, developing, maintaining and supporting Data Extract, Transform and Load (ETL) software using Informatica, Shell Scripts, DB2 UDB and Autosys.
Coming up with system architecture/re-design proposals for greater efficiency and ease of maintenance and developing software to turn proposals into implementations.

Need to work with business analysts and other project leads to understand requirements.
Strong collaboration and communication skills

• 1-5 years of experience in building and maintaining robust data pipelines, enriching data, low-latency/highly-performance data analytics applications.
• Experience handling complex, high volume, multi-dimensional data and architecting data products in streaming, serverless, and microservices-based Architecture and platform.
• Experience in Data warehousing, Data modeling, and Data architecture.
• Expert level proficiency with the relational and NoSQL databases.
• Expert level proficiency in Python, and PySpark.
• Familiarity with Big Data technologies and utilities (Spark, Hive, Kafka, Airflow).
• Familiarity with cloud services (preferable AWS)
• Familiarity with MLOps processes such as data labeling, model deployment, data-model feedback loop, data drift.

Key Roles/Responsibilities:

• Act as a technical leader for resolving problems, with both technical and non-technical audiences.
• Identifying and solving issues with data pipelines regarding consistency, integrity, and completeness.
• Lead data initiatives, architecture design discussions, and implementation of next-generation BI solutions.
• Partner with data scientists, tech architects to build advanced, scalable, efficient self-service BI infrastructure.
• Provide thought leadership and mentor data engineers in information presentation and delivery.