Analyst/Senior Analyst/Assistant Manager/Manager - Data Engineering

About DeepIntent:

DeepIntent is a marketing technology company that helps healthcare brands strengthen communication with patients and healthcare professionals by enabling highly effective and performant digital advertising campaigns. Our healthcare technology platform, MarketMatch™, connects advertisers, data providers, and publishers to operate the first unified, programmatic marketplace for healthcare marketers. The platform’s built-in identity solution matches digital IDs with clinical, behavioural, and contextual data in real-time so marketers can qualify 1.6M+ verified HCPs and 225M+ patients to find their most clinically-relevant audiences and message them on a one-to-one basis in a privacy-compliant way. Healthcare marketers use MarketMatch to plan, activate, and measure digital campaigns in ways that best suit their business, from managed service engagements to technical integration or self-service solutions. DeepIntent was founded by Memorial Sloan Kettering alumni in 2016 and acquired by Propel Media, Inc. in 2017. We proudly serve major pharmaceutical and Fortune 500 companies out of our offices in New York, Bosnia and India.

What You’ll Do:

• Establish formal data practice for the organisation.
• Build & operate scalable and robust data architectures.
• Create pipelines for the self-service introduction and usage of new data
• Implement DataOps practices
• Design, Develop, and operate Data Pipelines which support Data scientists and machine learning
• Engineers.
• Build simple, highly reliable Data storage, ingestion, and transformation solutions which are easy
• to deploy and manage.
• Collaborate with various business stakeholders, software engineers, machine learning
• engineers, and analysts.

Who You Are:

• Experience in designing, developing and operating configurable Data pipelines serving high
• volume and velocity data.
• Experience working with public clouds like GCP/AWS.
• Good understanding of software engineering, DataOps, data architecture, Agile and
• DevOps methodologies.
• Experience building Data architectures that optimize performance and cost, whether the
• components are prepackaged or homegrown
• Proficient with SQL, Java, Spring boot, Python or JVM-based language, Bash
• Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow
• etc. and big data databases like BigQuery, Clickhouse, etc
• Good communication skills with the ability to collaborate with both technical and non-technical
• people.
• Ability to Think Big, take bets and innovate, Dive Deep, Bias for Action, Hire and Develop the Best, Learn and be Curious

LogiNext is looking for a technically savvy and passionate Software Engineer - Data Science to analyze large amounts of raw information to find patterns that will help improve our company. We will rely on you to build data products to extract valuable business insights.

In this role, you should be highly analytical with a knack for analysis, math and statistics. Critical thinking and problem-solving skills are essential for interpreting data. We also want to see a passion for machine-learning and research.

Your goal will be to help our company analyze trends to make better decisions. Without knowledge of how the software works, data scientists might have difficulty in work. Apart from experience in developing R and Python, they must know modern approaches to software development and their impact. DevOps continuous integration and deployment, experience in cloud computing are everyday skills to manage and process data.

Responsibilities:

Identify valuable data sources and automate collection processes Undertake preprocessing of structured and unstructured data Analyze large amounts of information to discover trends and patterns Build predictive models and machine-learning algorithms Combine models through ensemble modeling Present information using data visualization techniques Propose solutions and strategies to business challenges Collaborate with engineering and product development teams

Requirements:

Bachelors degree or higher in Computer Science, Information Technology, Information Systems, Statistics, Mathematics, Commerce, Engineering, Business Management, Marketing or related field from top-tier school 2 to 3 year experince in in data mining, data modeling, and reporting. Understading of SaaS based products and services. Understanding of machine-learning and operations research Experience of R, SQL and Python; familiarity with Scala, Java or C++ is an asset Experience using business intelligence tools (e.g. Tableau) and data frameworks (e.g. Hadoop) Analytical mind and business acumen and problem-solving aptitude Excellent communication and presentation skills Proficiency in Excel for data management and manipulation Experience in statistical modeling techniques and data wrangling Able to work independently and set goals keeping business objectives in mind

• Creating and managing ETL/ELT pipelines based on requirements
• Build PowerBI dashboards and manage datasets needed.
• Work with stakeholders to identify data structures needed for future and perform any transformations including aggregations.
• Build data cubes for real-time visualisation needs and CXO dashboards.

Required Tech Skills

• Microsoft PowerBI & DAX
• Python, Pandas, PyArrow, Jupyter Noteboks, ApacheSpark
• Azure Synapse, Azure DataBricks, Azure HDInsight, Azure Data Factory

 Job DescriptionPosition: Sr Data Engineer – Databricks & AWS

Experience: 4 - 5 Years

Company Profile:

Exponentia.ai is an AI tech organization with a presence across India, Singapore, the Middle East, and the UK. We are an innovative and disruptive organization, working on cutting-edge technology to help our clients transform into the enterprises of the future. We provide artificial intelligence-based products/platforms capable of automated cognitive decision-making to improve productivity, quality, and economics of the underlying business processes. Currently, we are transforming ourselves and rapidly expanding our business.

Exponentia.ai has developed long-term relationships with world-class clients such as PayPal, PayU, SBI Group, HDFC Life, Kotak Securities, Wockhardt and Adani Group amongst others.

One of the top partners of Cloudera (leading analytics player) and Qlik (leader in BI technologies), Exponentia.ai has recently been awarded the ‘Innovation Partner Award’ by Qlik in 2017.

Get to know more about us on our website: http://www.exponentia.ai/ and Life @Exponentia.

​Role Overview: 

·         A Data Engineer understands the client requirements and develops and delivers the data engineering solutions as per the scope.

·         The role requires good skills in the development of solutions using various services required for data architecture on Databricks Delta Lake, streaming, AWS, ETL Development, and data modeling.

Job Responsibilities

•         Design of data solutions on Databricks including delta lake, data warehouse, data marts and other data solutions to support the analytics needs of the organization.

•         Apply best practices during design in data modeling (logical, physical) and ETL pipelines (streaming and batch) using cloud-based services.

•         Design, develop and manage the pipelining (collection, storage, access), data engineering (data quality, ETL, Data Modelling) and understanding (documentation, exploration) of the data.

•         Interact with stakeholders regarding data landscape understanding, conducting discovery exercises, developing proof of concepts and demonstrating it to stakeholders.

Technical Skills 

•         Has more than 2 Years of experience in developing data lakes, and datamarts on the Databricks platform.

•         Proven skill sets in AWS Data Lake services such as - AWS Glue, S3, Lambda, SNS, IAM, and skills in Spark, Python, and SQL.

•         Experience in Pentaho

•         Good understanding of developing a data warehouse, data marts etc.

•         Has a good understanding of system architectures, and design patterns and should be able to design and develop applications using these principles.

Personality Traits

•         Good collaboration and communication skills

•         Excellent problem-solving skills to be able to structure the right analytical solutions.

•         Strong sense of teamwork, ownership, and accountability

•         Analytical and conceptual thinking 

•         Ability to work in a fast-paced environment with tight schedules.

•         Good presentation skills with the ability to convey complex ideas to peers and management.

Education:

BE / ME / MS/MCA.

• 5+ years of experience in software development.
• At least 2 years of relevant work experience on large scale Data applications
• Good attitude, strong problem-solving abilities, analytical skills, ability to take ownership as appropriate
• Should be able to do coding, debugging, performance tuning, and deploying the apps to Prod.
• Should have good working experience Hadoop ecosystem (HDFS, Hive, Yarn, File formats like Avro/Parquet)
• Kafka
• J2EE Frameworks (Spring/Hibernate/REST)
• Spark Streaming or any other streaming technology.
• Java programming language is mandatory.
• Good to have experience with Java
• Ability to work on the sprint stories to completion along with Unit test case coverage.
• Experience working in Agile Methodology
• Excellent communication and coordination skills
• Knowledgeable (and preferred hands-on) - UNIX environments, different continuous integration tools.
• Must be able to integrate quickly into the team and work independently towards team goals

• Take the complete responsibility of the sprint stories’ execution
• Be accountable for the delivery of the tasks in the defined timelines with good quality
• Follow the processes for project execution and delivery.
• Follow agile methodology
• Work with the team lead closely and contribute to the smooth delivery of the project.
• Understand/define the architecture and discuss the pros-cons of the same with the team
• Involve in the brainstorming sessions and suggest improvements in the architecture/design.
• Work with other team leads to get the architecture/design reviewed.
• Work with the clients and counterparts (in US) of the project.
• Keep all the stakeholders updated about the project/task status/risks/issues if there are any.

• Drive and participate in requirements gathering workshops, estimation discussions, design meetings and status review meetings
• Participate and contribute in Solution Design and Solution Architecture for implementing Big Data Projects on-premise and on cloud
• Technical Hands on experience in design, coding, development and managing Large Hadoop implementation
• Proficient in SQL, Hive, PIG, Spark SQL, Shell Scripting, Kafka, Flume, Scoop with large Big Data and Data Warehousing projects with either Java, Python or Scala based Hadoop programming background
• Proficient with various development methodologies like waterfall, agile/scrum and iterative
• Good Interpersonal skills and excellent communication skills for US and UK based clients

• Strong Python Coding skills and OOP skills
• Should have worked on Big Data product Architecture
• Should have worked with any one of the SQL-based databases like MySQL, PostgreSQL and any one of
• NoSQL-based databases such as Cassandra, Elasticsearch etc.
• Hands on experience on frameworks like Spark RDD, DataFrame, Dataset
• Experience on development of ETL for data product
• Candidate should have working knowledge on performance optimization, optimal resource utilization, Parallelism and tuning of spark jobs
• Working knowledge on file formats: CSV, JSON, XML, PARQUET, ORC, AVRO
• Good to have working knowledge with any one of the Analytical Databases like Druid, MongoDB, Apache Hive etc.
• Experience to handle real-time data feeds (good to have working knowledge on Apache Kafka or similar tool)

• Python and Scala (Optional), Spark / PySpark, Parallel programming