Data Analyst

Who We Are:

DeepIntent is leading the healthcare advertising industry with data-driven solutions built for the future. From day one, our mission has been to improve patient outcomes through the artful use of advertising, data science, and real-world clinical data.

What You’ll Do:

We are looking for a Senior Software Engineer based in Pune, India who can master both DeepIntent’s data architectures and pharma research and analytics methodologies to make significant contributions to how health media is analyzed by our clients. This role requires an Engineer who not only understands DBA functions but also how they impact research objectives and can work with researchers and data scientists to achieve impactful results.  

This role will be in the Analytics Organization and will require integration and partnership with the Engineering Organization. The ideal candidate is a self-starter who is inquisitive who is not afraid to take on and learn from challenges and will constantly seek to improve the facets of the business they manage. The ideal candidate will also need to demonstrate the ability to collaborate and partner with others.  

• Serve as the Engineering interface between Analytics and Engineering teams
• Develop and standardized all interface points for analysts to retrieve and analyze data with a focus on research methodologies and data based decisioning
• Optimize queries and data access efficiencies, serve as expert in how to most efficiently attain desired data points
• Build “mastered” versions of the data for Analytics specific querying use cases
• Help with data ETL, table performance optimization
• Establish formal data practice for the Analytics practice in conjunction with rest of DeepIntent
• Build & operate scalable and robust data architectures
• Interpret analytics methodology requirements and apply to data architecture to create standardized queries and operations for use by analytics teams
• Implement DataOps practices
• Master existing and new Data Pipelines and develop appropriate queries to meet analytics specific objectives
• Collaborate with various business stakeholders, software engineers, machine learning engineers, analysts
• Operate between Engineers and Analysts to unify both practices for analytics insight creation

Who You Are:

• Adept in market research methodologies and using data to deliver representative insights
• Inquisitive, curious, understands how to query complicated data sets, move and combine data between databases
• Deep SQL experience is a must
• Exceptional communication skills with ability to collaborate and translate with between technical and non technical needs
• English Language Fluency and proven success working with teams in the U.S.
• Experience in designing, developing and operating configurable Data pipelines serving high volume and velocity data
• Experience working with public clouds like GCP/AWS
• Good understanding of software engineering, DataOps, and data architecture, Agile and DevOps methodologies
• Experience building Data architectures that optimize performance and cost, whether the components are prepackaged or homegrown
• Proficient with SQL,Python or JVM based language, Bash
• Experience with any of Apache open source projects such as Spark, Druid, Beam, Airflow etc.and big data databases like BigQuery, Clickhouse, etc         
• Ability to think big, take bets and innovate, dive deep, hire and develop the best talent, learn and be curious
• Comfortable to work in EST Time Zone

·       Honest, transparent Team player and go-getter.

·       Friendly and positive work culture that's based on respect for individual.

·       Work in large-scale projects with onshore and offshore models.

·       Health care knowledge at least 6+ years of experience.

·       Knowledge of Health care Data domains and supported Data initiatives, like Data Governance, Data modelling and build data driven use cases.

·       Knowledge in understanding Source to Target mapping

·       Work well with Optimizers and help on performance tuning, with SQL server and Azure objects.

·       Has a can-do attitude and has used cutting-edge technologies in real projects. 

·       Support and perform daily tasks around SQL server and ETL.

·       Ability to integrate with Azure and ADF

·       Very process focused and had good knowledge in the Error/Exception handling.

·       Ability to code with modifications history and ability to do through unit test before passing to

·       Ability to review code, trustworthy and is a leader in finishing ETL processes from Start to end without errors.

·       Work in a environment with pair-programming and Agile.

Key responsibilities:

·       Develop and maintain databases in Azure environment.

·       Build ETL processes with Azure Data Factory, Azure Databricks, Data queries

·       Perform migration processes to or from Azure environment​.

·       Azure Data Engineering: Azure Data Factory, Azure Data Lake, Azure Databricks, Azure Synapse​

·       Developing using Python/Scala and SQL​ server components

·       Work with SQL server on a daily basis and problem solving.

·       Work with RDBMS and NoSQL​

·       Work in the team following Agile practices​.

·       Support exploration and sales of new market trends to ensure growth with interesting projects that leverage the latest technologies​

·       Actively participate in building specialty group skills together with others

Job Title: AWS-Azure Data Engineer with Snowflake

Location: Bangalore, India

Experience: 4+ years

Budget: 15 to 20 LPA

Notice Period: Immediate joiners or less than 15 days

Job Description:

We are seeking an experienced AWS-Azure Data Engineer with expertise in Snowflake to join our team in Bangalore. As a Data Engineer, you will be responsible for designing, implementing, and maintaining data infrastructure and systems using AWS, Azure, and Snowflake. Your primary focus will be on developing scalable and efficient data pipelines, optimizing data storage and processing, and ensuring the availability and reliability of data for analysis and reporting.

Responsibilities:

1. Design, develop, and maintain data pipelines on AWS and Azure to ingest, process, and transform data from various sources.
2. Optimize data storage and processing using cloud-native services and technologies such as AWS S3, AWS Glue, Azure Data Lake Storage, Azure Data Factory, etc.
3. Implement and manage data warehouse solutions using Snowflake, including schema design, query optimization, and performance tuning.
4. Collaborate with cross-functional teams to understand data requirements and translate them into scalable and efficient technical solutions.
5. Ensure data quality and integrity by implementing data validation, cleansing, and transformation processes.
6. Develop and maintain ETL processes for data integration and migration between different data sources and platforms.
7. Implement and enforce data governance and security practices, including access control, encryption, and compliance with regulations.
8. Collaborate with data scientists and analysts to support their data needs and enable advanced analytics and machine learning initiatives.
9. Monitor and troubleshoot data pipelines and systems to identify and resolve performance issues or data inconsistencies.
10. Stay updated with the latest advancements in cloud technologies, data engineering best practices, and emerging trends in the industry.

Requirements:

1. Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
2. Minimum of 4 years of experience as a Data Engineer, with a focus on AWS, Azure, and Snowflake.
3. Strong proficiency in data modelling, ETL development, and data integration.
4. Expertise in cloud platforms such as AWS and Azure, including hands-on experience with data storage and processing services.
5. In-depth knowledge of Snowflake, including schema design, SQL optimization, and performance tuning.
6. Experience with scripting languages such as Python or Java for data manipulation and automation tasks.
7. Familiarity with data governance principles and security best practices.
8. Strong problem-solving skills and ability to work independently in a fast-paced environment.
9. Excellent communication and interpersonal skills to collaborate effectively with cross-functional teams and stakeholders.
10. Immediate joiner or notice period less than 15 days preferred.

If you possess the required skills and are passionate about leveraging AWS, Azure, and Snowflake to build scalable data solutions, we invite you to apply. Please submit your resume and a cover letter highlighting your relevant experience and achievements in the AWS, Azure, and Snowflake domains.

You will:

• Create highly scalable AWS micro-services utilizing cutting edge cloud technologies.
• Design and develop Big Data pipelines handling huge geospatial data.
• Bring clarity to large complex technical challenges.
• Collaborate with Engineering leadership to help drive technical strategy.
• Project scoping, planning and estimation.
• Mentor and coach team members at different levels of experience.
• Participate in peer code reviews and technical meetings.
• Cultivate a culture of engineering excellence.
• Seek, implement and adhere to standards, frameworks and best practices in the industry.
• Participate in on-call rotation.

You have:

• Bachelor’s/Master’s degree in computer science, computer engineering or relevant field.
• 5+ years of experience in software design, architecture and development.
• 5+ years of experience using object-oriented languages (Java, Python).
• Strong experience with Big Data technologies like Hadoop, Spark, Map Reduce, Kafka, etc.
• Strong experience in working with different AWS technologies.
• Excellent competencies in data structures & algorithms.

Nice to have:

• Proven track record of delivering large scale projects, and an ability to break down large tasks into smaller deliverable chunks
• Experience in developing high throughput low latency backend services
• Affinity to spatial data structures and algorithms.
• Familiarity with Postgres DB, Google Places or Mapbox APIs

Technical/Core skills

1. Minimum 3 yrs of exp in Informatica Big data Developer(BDM) in Hadoop environment.
2. Have knowledge of informatica Power exchange (PWX).
3. Minimum 3 yrs of exp in big data querying tool like Hive and Impala.
4. Ability to designing/development of complex mappings using informatica Big data Developer.
5. Create and manage Informatica power exchange and CDC real time implementation
6. Strong Unix knowledge skills for writing shell scripts and troubleshoot of existing scripts.
7. Good knowledge of big data platforms and its framework.
8. Good to have an experience in cloudera data platform (CDP)
9. Experience with building stream processing systems using Kafka and spark
10. Excellent SQL knowledge

Soft skills :

1. Ability to work independently 
2. Strong analytical and problem solving skills
3. Attitude of learning new technology
4. Regular interaction with vendors, partners and stakeholders

Role: Data Analytics Lead / Manager

1. Lead cross-functional projects using advanced data modelling and analysis techniques to discover insights that will guide strategic decisions and uncover optimization opportunities.
2. Build, develop and maintain data models, reporting systems, data automation systems, dashboards and performance metrics support that support key business decisions.
3. Coordinate with different teams to determine requirements for data warehousing, reporting and analytical solutions, and ensure customer satisfaction on deliverables
4. Oversee analytics projects to extract, manage, and analyse data from multiple applications, ensuring that deadlines are met.
5. Apply statistics and data modelling to gain actionable business insights and boost customer productivity and revenue.
6. Enforce company policies and procedures to ensure quality and prevent discrepancies.
7. Communicate and track key performance metrics across departments.
8. Keep abreast of industry best practices and policies.
9. Research Latest trends, analyse data, identify opportunities and incorporate changes into business strategies.
10. Manage and optimize processes for data intake, validation, mining and engineering as well as modelling, visualization and communication deliverables.
11. Examine, interpret and report results of analytical initiatives to stakeholders in leadership, technology, sales, marketing and product teams.
12. Oversee the data/report requests process: tracking requests submitted, prioritization, approval, etc.
13. Develop and implement MLDevOps, quality controls and departmental standards to ensure quality standards, organizational expectations, and regulatory requirements.
14. Anticipate future demands of initiatives related to people, technology, budget and business within your department and design/implement solutions to meet these needs.
15. Organize and drive successful completion of data insight initiatives through effective management of data analyst and effective collaboration with stakeholders.

Essential Skills

1. Working knowledge of data mining principles: predictive analytics, mapping, collecting data from multiple data systems on premises and cloud-based data sources.
2. Strong SQL skills, ability to perform effective querying involving multiple tables and subqueries.
3. Understanding of and experience using analytical concepts and statistical techniques: hypothesis development, designing tests/experiments, analyzing data, drawing conclusions, and developing actionable recommendations for business units.
4. Experience and knowledge of statistical modelling techniques: multiple regression, logistic regression, log-linear regression, variable selection, etc.

 We’re looking for someone with at least 10+ years of experience in a position monitoring, managing, transforming and drawing insights from data, and someone with at least 5 years of experience leading a data analyst team.

Experience in Azure is preferred