Hiring for GCP compliant cloud data lake solutions for clinical trials for US based pharmaceutical company.
Summary
This is a key position within Data Sciences and Systems organization responsible for data systems and related technologies. The role will part of Amazon Web service (AWS) Data Lake strategy, roadmap, and AWS architecture for data systems and technologies.
Essential/Primary Duties, Functions and Responsibilities
The essential duties and responsibilities of this position are as follows:
- Collaborate with data science and systems leaders and other stakeholders to roadmap, structure, prioritize and execute on AWS data engineering requirements.
- Works closely with the IT organization and other functions to make sure that business needs and requirements, IT processes, and regulatory compliance requirements are met.
- Build the AWS infrastructure required for optimal extraction, transformation, and loading of data from a vendor site clinical data sources using AWS big data technologies
- Create and maintain optimal AWS data pipeline architecture
- Assemble large, complex data sets that meet functional / non-functional business requirements
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics
- Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs
- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product
- Work with data and analytics experts to strive for greater functionality in our data systems
Requirements
- A minimum of a bachelors degree in a Computer Science, Mathematics, Statistics or related discipline is required. A Master's degree is preferred. A minimum of 6-8 years technical management experience is required. Equivalent experience may be accepted.
- Experience with data lake and/or data warehouse implementation is required
- Minimum Bachelors Degree in Computer Science, Computer Engineering, Mathematical Engineering, Information Systems or related fields
- Project experience with visualization tools (AWS, Tableau, R Studio, PowerBI, R shiny, D3js) and databases. Experience with python, R or SAS coding is a big plus.
- Experience with AWS based S3, Lambda, Step functions.
- Strong team player and you can work effectively in a collaborative, fast-paced, multi-tasking environment
- Solid analytical and technical skill and the ability to exchange innovative ideas
- Quick learner and passionate about continuously developing your skills and knowledge
- Ability to solve problems by using AWS in data acquisitions
- Ability to work in an interdisciplinary environment. You are able to interpret and translate very abstract and technical approaches into a healthcare and business-relevant solution
About MEDTEK DOT AI
Similar jobs
About Company
Our client is a well-funded construction Tech Start-up by a renowned group.
Responsibilities
- Gather intelligence from key business leaders about needs and future growth
- Partner with the internal IT team to ensure each project meets a specific need and resolves successfully
- Assume responsibility for project tasks and ensure they are completed in a timely fashion
- Evaluate, test and recommend new opportunities for enhancing our software, hardware and IT processes
- Compile and distribute reports on application development and deployment
- Design and execute A/B testing procedures to extract data from test runs
- Evaluate and conclude data related to customer behavior
- Consult with the executive team and the IT department on the newest technology and its implications in the industry
Requirements :
- Bachelor's Degree in Software Development, Computer Engineering, Project Management or a related field
- 3+ years experience in technology development and deployment
We are looking out for a technically driven "Full-Stack Engineer" for one of our premium client
COMPANY DESCRIPTION:
Qualifications
• Bachelor's degree in computer science or related field; Master's degree is a plus
• 3+ years of relevant work experience
• Meaningful experience with at least two of the following technologies: Python, Scala, Java
• Strong proven experience on distributed processing frameworks (Spark, Hadoop, EMR) and SQL is very
much expected
• Commercial client-facing project experience is helpful, including working in close-knit teams
• Ability to work across structured, semi-structured, and unstructured data, extracting information and
identifying linkages across disparate data sets
• Confirmed ability in clearly communicating complex solutions
• Understandings on Information Security principles to ensure compliant handling and management of
client data
• Experience and interest in Cloud platforms such as: AWS, Azure, Google Platform or Databricks
• Extraordinary attention to detail
1. Core Responsibilities
· Leading solutions for data engineering
· Maintain the integrity of both the design and the data that is held within the architecture
· Champion and educate people in the development and use of data engineering best practises
· Support the Head of Data Engineering and lead by example
· Contribute to the development of database management services and associated processes relating to the delivery of data solutions
· Provide requirements analysis, documentation, development, delivery and maintenance of data platforms.
· Develop database requirements in a structured and logical manner ensuring delivery is aligned with business prioritisation and best practise
· Design and deliver performance enhancements, application migration processes and version upgrades across a pipeline of BI environments.
· Provide support for the scoping and delivery of BI capability to internal users.
· Identify risks and issues and escalate to Line / Project manager.
· Work with clients, existing asset owners & their service providers and non BI development staff to clarify and deliver work stream objectives in timescales that deliver to the overall project expectations.
· Develop and maintain documentation in support of all BI processes.
· Proactively identify cost-justifiable improvements to data manipulation processes.
· Research and promote relevant BI tools and processes that contribute to increased efficiency and capability in support of corporate objectives.
· Promote a culture that embraces change, continuous improvement and a ‘can do’ attitude.
· Demonstrate enthusiasm and self-motivation at all times.
· Establish effective working relationships with other internal teams to drive improved efficiency and effective processes.
· Be a champion for high quality data and use of strategic data repositories, associated relational model, and Data Warehouse for optimising the delivery of accurate, consistent and reliable business intelligence
· Ensure that you fully understand and comply with the organisation’s Risk Management Policies as they relate to your area of responsibility and demonstrate in your day to day work that you put customers at the heart of everything you do.
· Ensure that you fully understand and comply with the organisation’s Data Governance Policies as they relate to your area of responsibility and demonstrate in your day to day work that you treat data as an important corporate asset which must be protected and managed.
· Maintain the company’s compliance standards and ensure timely completion of all mandatory on-line training modules and attestations.
2. Experience Requirements
· 5 years Data Engineering / ETL development experience is essential
· 5 years data design experience in an MI / BI / Analytics environment (Kimball, lake house, data lake) is essential
· 5 years experience of working in a structured Change Management project lifecycle is essential
· Experience of working in a financial services environment is desirable
· Experience of dealing with senior management within a large organisation is desirable
· 5 years experience of developing in conjunction with large complex projects and programmes is desirable
· Experience mentoring other members of the team on best practise and internal standards is essential
· Experience with cloud data platforms desirable (Microsoft Azure) is desirable
3. Knowledge Requirements
· A strong knowledge of business intelligence solutions and an ability to translate this into data solutions for the broader business is essential
· Strong demonstrable knowledge of data warehouse methodologies
· Robust understanding of high level business processes is essential
· Understanding of data migration, including reconciliation, data cleanse and cutover is desirable
WHAT YOU WILL DO:
-
● Create and maintain optimal data pipeline architecture.
-
● Assemble large, complex data sets that meet functional / non-functional business requirements.
-
● Identify, design, and implement internal process improvements: automating manual processes,
optimizing data delivery, re-designing infrastructure for greater scalability, etc.
-
● Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide
variety of data sources using Spark,Hadoop and AWS 'big data' technologies.(EC2, EMR, S3, Athena).
-
● Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition,
operational efficiency and other key business performance metrics.
-
● Work with stakeholders including the Executive, Product, Data and Design teams to assist with
data-related technical issues and support their data infrastructure needs.
-
● Keep our data separated and secure across national boundaries through multiple data centers and AWS
regions.
-
● Create data tools for analytics and data scientist team members that assist them in building and
optimizing our product into an innovative industry leader.
-
● Work with data and analytics experts to strive for greater functionality in our data systems.
REQUIRED SKILLS & QUALIFICATIONS:
-
● 5+ years of experience in a Data Engineer role.
-
● Advanced working SQL knowledge and experience working with relational databases, query authoring
(SQL) as well as working familiarity with a variety of databases.
-
● Experience building and optimizing 'big data' data pipelines, architectures and data sets.
-
● Experience performing root cause analysis on internal and external data and processes to answer
specific business questions and identify opportunities for improvement.
-
● Strong analytic skills related to working with unstructured datasets.
-
● Build processes supporting data transformation, data structures, metadata, dependency and workload
management.
-
● A successful history of manipulating, processing and extracting value from large disconnected datasets.
-
● Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
-
● Strong project management and organizational skills.
-
● Experience supporting and working with cross-functional teams in a dynamic environment
-
● Experience with big data tools: Hadoop, Spark, Pig, Vetica, etc.
-
● Experience with AWS cloud services: EC2, EMR, S3, Athena
-
● Experience with Linux
-
● Experience with object-oriented/object function scripting languages: Python, Java, Shell, Scala, etc.
PREFERRED SKILLS & QUALIFICATIONS:
● Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
We have a requirement for Collibra Developer
Experience required- 5-12 yrs
Having experience in Data Governence , Data Quality management
Role: Talend Production Support Consultant
Brief Job Description:
- Involve in release deployment and monitoring of the ETL pipelines.
- Closely work with the development team and business team to provide operational support.
- Candidate should have good knowledge and hands on experience on below tools/technologies:
Talend (Talend Studio, TAC, TMC),SAP BODS,SQL,HIVE & Azure(Azure fundamentals, ADB,ADF)
- Hands on experience in CI/CD is an added advantage.
As discussed, please provide your Linkedin ID URL & a valid ID proof of yours.
Please confirm as well, you will relocate to Bangalore once required.