Data Engineer

Publicis Sapient Overview:

The Senior Associate People Senior Associate L1 in Data Engineering, you will translate client requirements into technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution 

.

Job Summary:

As Senior Associate L1 in Data Engineering, you will do technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution

The role requires a hands-on technologist who has strong programming background like Java / Scala / Python, should have experience in Data Ingestion, Integration and data Wrangling, Computation, Analytics pipelines and exposure to Hadoop ecosystem components. Having hands-on knowledge on at least one of AWS, GCP, Azure cloud platforms will be preferable.

Role & Responsibilities:

Job Title: Senior Associate L1 – Data Engineering

Your role is focused on Design, Development and delivery of solutions involving:

• Data Ingestion, Integration and Transformation

• Data Storage and Computation Frameworks, Performance Optimizations

• Analytics & Visualizations

• Infrastructure & Cloud Computing

• Data Management Platforms

• Build functionality for data ingestion from multiple heterogeneous sources in batch & real-time

• Build functionality for data analytics, search and aggregation

Experience Guidelines:

Mandatory Experience and Competencies:

# Competency

1.Overall 3.5+ years of IT experience with 1.5+ years in Data related technologies

2.Minimum 1.5 years of experience in Big Data technologies

3.Hands-on experience with the Hadoop stack – HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data pipeline. Working knowledge on real-time data pipelines is added advantage.

4.Strong experience in at least of the programming language Java, Scala, Python. Java preferable

5.Hands-on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDb, Cassandra, AWS Redshift, Azure SQLDW, GCP BigQuery etc

Preferred Experience and Knowledge (Good to Have):

# Competency

1.Good knowledge of traditional ETL tools (Informatica, Talend, etc) and database technologies (Oracle, MySQL, SQL Server, Postgres) with hands on experience

2.Knowledge on data governance processes (security, lineage, catalog) and tools like Collibra, Alation etc

3.Knowledge on distributed messaging frameworks like ActiveMQ / RabbiMQ / Solace, search & indexing and Micro services architectures

4.Performance tuning and optimization of data pipelines

5.CI/CD – Infra provisioning on cloud, auto build & deployment pipelines, code quality

6.Working knowledge with data platform related services on at least 1 cloud platform, IAM and data security

7.Cloud data specialty and other related Big data technology certifications

Job Title: Senior Associate L1 – Data Engineering

Personal Attributes:

• Strong written and verbal communication skills

• Articulation skills

• Good team player

• Self-starter who requires minimal oversight

• Ability to prioritize and manage multiple tasks

• Process orientation and the ability to define and set up processes

Data driven decision-making is core to advertising technology at AdElement. We are looking for sharp, disciplined, and highly quantitative machine learning/ artificial intellignce engineers with big data experience and a passion for digital marketing to help drive informed decision-making. You will work with top-talent and cutting edge technology and have a unique opportunity to turn your insights into products influencing billions. The potential candidate will have an extensive background in distributed training frameworks, will have experience to deploy related machine learning models end to end, and will have some experience in data-driven decision making of machine learning infrastructure enhancement. This is your chance to leave your legacy and be part of a highly successful and growing company.

Required Skills

- 3+ years of industry experience with Java/ Python in a programming intensive role

- 3+ years of experience with one or more of the following machine learning topics: classification, clustering, optimization, recommendation system, graph mining, deep learning

- 3+ years of industry experience with distributed computing frameworks such as Hadoop/Spark, Kubernetes ecosystem, etc

- 3+ years of industry experience with popular deep learning frameworks such as Spark MLlib, Keras, Tensorflow, PyTorch, etc

- 3+ years of industry experience with major cloud computing services

- An effective communicator with the ability to explain technical concepts to a non-technical audience

- (Preferred) Prior experience with ads product development (e.g., DSP/ad-exchange/SSP)

- Able to lead a small team of AI/ML Engineers to achieve business objectives

Responsibilities

- Collaborate across multiple teams - Data Science, Operations & Engineering on unique machine learning system challenges at scale

- Leverage distributed training systems to build scalable machine learning pipelines including ETL, model training and deployments in Real-Time Bidding space. 

- Design and implement solutions to optimize distributed training execution in terms of model hyperparameter optimization, model training/inference latency and system-level bottlenecks  

- Research state-of-the-art machine learning infrastructures to improve data healthiness, model quality and state management during the lifecycle of ML models refresh.

- Optimize integration between popular machine learning libraries and cloud ML and data processing frameworks. 

- Build Deep Learning models and algorithms with optimal parallelism and performance on CPUs/ GPUs.

- Work with top management on defining teams goals and objectives.

Education

- MTech or Ph.D. in Computer Science, Software Engineering, Mathematics or related fields

Lead Data Engineer

Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On other projects, you might be acting as the architect, leading the design of technical solutions, or perhaps overseeing a program inception to build a new product. It could also be a software delivery project where you're equally happy coding and tech-leading the team to implement the solution.

Job responsibilities

·      You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems

·      You will partner with teammates to create complex data processing pipelines in order to solve our clients' most ambitious challenges

·      You will collaborate with Data Scientists in order to design scalable implementations of their models

·      You will pair to write clean and iterative code based on TDD

·      Leverage various continuous delivery practices to deploy, support and operate data pipelines

·      Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available

·      Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions

·      Create data models and speak to the tradeoffs of different modeling approaches

·      On other projects, you might be acting as the architect, leading the design of technical solutions, or perhaps overseeing a program inception to build a new product

·      Seamlessly incorporate data quality into your day-to-day work as well as into the delivery process

·      Assure effective collaboration between Thoughtworks' and the client's teams, encouraging open communication and advocating for shared outcomes

Job qualifications Technical skills

·      You are equally happy coding and leading a team to implement a solution

·      You have a track record of innovation and expertise in Data Engineering

·      You're passionate about craftsmanship and have applied your expertise across a range of industries and organizations

·      You have a deep understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop

·      You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting

·      Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions

·      You are comfortable taking data-driven approaches and applying data security strategy to solve business problems

·      You're genuinely excited about data infrastructure and operations with a familiarity working in cloud environments

·      Working with data excites you: you have created Big data architecture, you can build and operate data pipelines, and maintain data storage, all within distributed systems

Professional skills

·      Advocate your data engineering expertise to the broader tech community outside of Thoughtworks, speaking at conferences and acting as a mentor for more junior-level data engineers

·      You're resilient and flexible in ambiguous situations and enjoy solving problems from technical and business perspectives

·      An interest in coaching others, sharing your experience and knowledge with teammates

·      You enjoy influencing others and always advocate for technical excellence while being open to change when needed

Job Description: Data Scientist

At Propellor.ai, we derive insights that allow our clients to make scientific decisions. We believe in demanding more from the fields of Mathematics, Computer Science, and Business Logic. Combine these and we show our clients a 360-degree view of their business. In this role, the Data Scientist will be expected to work on Procurement problems along with a team-based across the globe.

We are a Remote-First Company.

Read more about us here: https://www.propellor.ai/consulting" target="_blank">https://www.propellor.ai/consulting

What will help you be successful in this role

• Articulate
• High Energy
• Passion to learn
• High sense of ownership
• Ability to work in a fast-paced and deadline-driven environment
• Loves technology
• Highly skilled at Data Interpretation
• Problem solver
• Ability to narrate the story to the business stakeholders
• Generate insights and the ability to turn them into actions and decisions

Skills to work in a challenging, complex project environment

• Need you to be naturally curious and have a passion for understanding consumer behavior
• A high level of motivation, passion, and high sense of ownership
• Excellent communication skills needed to manage an incredibly diverse slate of work, clients, and team personalities
• Flexibility to work on multiple projects and deadline-driven fast-paced environment
• Ability to work in ambiguity and manage the chaos

Key Responsibilities

• Analyze data to unlock insights: Ability to identify relevant insights and actions from data.  Use regression, cluster analysis, time series, etc. to explore relationships and trends in response to stakeholder questions and business challenges.   
• Bring in experience for AI and ML:  Bring in Industry experience and apply the same to build efficient and optimal Machine Learning solutions.
• Exploratory Data Analysis (EDA) and Generate Insights: Analyse internal and external datasets using analytical techniques, tools, and visualization methods. Ensure pre-processing/cleansing of data and evaluate data points across the enterprise landscape and/or external data points that can be leveraged in machine learning models to generate insights. 
• DS and ML Model Identification and Training: Identity, test, and train machine learning models that need to be leveraged for business use cases. Evaluate models based on interpretability, performance, and accuracy as required. Experiment and identify features from datasets that will help influence model outputs.  Determine what models will need to be deployed, data points that need to be fed into models, and aid in the deployment and maintenance of models.

Technical Skills

An enthusiastic individual with the following skills. Please do not hesitate to apply if you do not match all of them. We are open to promising candidates who are passionate about their work, fast learners and are team players.

• Strong experience with machine learning and AI including regression, forecasting, time series, cluster analysis, classification, Image recognition, NLP, Text Analytics and Computer Vision.
• Strong experience with advanced analytics tools for Object-oriented/object function scripting using languages such as Python, or similar.
• Strong experience with popular database programming languages including SQL.
• Strong experience in Spark/Pyspark
• Experience in working in Databricks

What are the company benefits you get, when you join us as?

• Permanent Work from Home Opportunity
• Opportunity to work with Business Decision Makers and an internationally based team
• The work environment that offers limitless learning
• A culture void of any bureaucracy, hierarchy
• A culture of being open, direct, and with mutual respect
• A fun, high-caliber team that trusts you and provides the support and mentorship to help you grow
• The opportunity to work on high-impact business problems that are already defining the future of Marketing and improving real lives

To know more about how we work: https://bit.ly/3Oy6WlE" target="_blank">https://bit.ly/3Oy6WlE

Whom will you work with?

You will closely work with other Senior Data Scientists and Data Engineers.

Immediate to 15-day Joiners will be preferred.

• Does analytics to extract insights from raw historical data of the organization. 
• Generates usable training dataset for any/all MV projects with the help of Annotators, if needed.
• Analyses user trends, and identifies their biggest bottlenecks in Hammoq Workflow.
• Tests the short/long term impact of productized MV models on those trends.
• Skills - Numpy, Pandas, SPARK, APACHE SPARK, PYSPARK, ETL mandatory. 

• Expert software implementation and automated testing
• Promoting development standards, code reviews, mentoring, knowledge sharing
• Improving our Agile methodology maturity
• Product and feature design, scrum story writing
• Build, release, and deployment automation
• Product support & troubleshooting

Who we have in mind: 

• Demonstrated experience as a Java
• Should have a deep understanding of Enterprise/Distributed Architecture patterns and should be able to demonstrate the relevant usage of the same
• Turn high-level project requirements into application-level architecture and collaborate with the team members to implement the solution
• Strong experience and knowledge in Spring boot framework and microservice architecture
• Experience in working with Apache Spark
• Solid demonstrated object-oriented software development experience with Java, SQL, Maven, relational/NoSQL databases and testing frameworks 
• Strong working experience with developing RESTful services
• Should have experience working on Application frameworks such as Spring, Spring Boot, AOP
• Exposure to tools – Jira, Bamboo, Git, Confluence would be an added advantage
• Excellent grasp of the current technology landscape, trends and emerging technologies

We are looking for an outstanding Big Data Engineer with experience setting up and maintaining Data Warehouse and Data Lakes for an Organization. This role would closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Roles and Responsibilities:

• Develop and maintain scalable data pipelines and build out new integrations and processes required for optimal extraction, transformation, and loading of data from a wide variety of data sources using 'Big Data' technologies.
• Develop programs in Scala and Python as part of data cleaning and processing.
• Assemble large, complex data sets that meet functional / non-functional business requirements and fostering data-driven decision making across the organization.  
• Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems.
• Implement processes and systems to validate data, monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
• Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Provide high operational excellence guaranteeing high availability and platform stability.
• Closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Skills:

• Experience with Big Data pipeline, Big Data analytics, Data warehousing.
• Experience with SQL/No-SQL, schema design and dimensional data modeling.
• Strong understanding of Hadoop Architecture, HDFS ecosystem and eexperience with Big Data technology stack such as HBase, Hadoop, Hive, MapReduce.
• Experience in designing systems that process structured as well as unstructured data at large scale.
• Experience in AWS/Spark/Java/Scala/Python development.
• Should have Strong skills in PySpark (Python & SPARK). Ability to create, manage and manipulate Spark Dataframes. Expertise in Spark query tuning and performance optimization.
• Experience in developing efficient software code/frameworks for multiple use cases leveraging Python and big data technologies.
• Prior exposure to streaming data sources such as Kafka.
• Should have knowledge on Shell Scripting and Python scripting.
• High proficiency in database skills (e.g., Complex SQL), for data preparation, cleaning, and data wrangling/munging, with the ability to write advanced queries and create stored procedures.
• Experience with NoSQL databases such as Cassandra / MongoDB.
• Solid experience in all phases of Software Development Lifecycle - plan, design, develop, test, release, maintain and support, decommission.
• Experience with DevOps tools (GitHub, Travis CI, and JIRA) and methodologies (Lean, Agile, Scrum, Test Driven Development).
• Experience building and deploying applications on on-premise and cloud-based infrastructure.
• Having a good understanding of machine learning landscape and concepts. 

Qualifications and Experience:

Engineering and post graduate candidates, preferably in Computer Science, from premier institutions with proven work experience as a Big Data Engineer or a similar role for 3-5 years.

Certifications:

Good to have at least one of the Certifications listed here:

    AZ 900 - Azure Fundamentals

    DP 200, DP 201, DP 203, AZ 204 - Data Engineering

    AZ 400 - Devops Certification

We’re looking to hire someone to help scale Machine Learning and NLP efforts at Episource. You’ll work with the team that develops the models powering Episource’s product focused on NLP driven medical coding. Some of the problems include improving our ICD code recommendations , clinical named entity recognition and information extraction from clinical notes.

This is a role for highly technical machine learning & data engineers who combine outstanding oral and written communication skills, and the ability to code up prototypes and productionalize using a large range of tools, algorithms, and languages. Most importantly they need to have the ability to autonomously plan and organize their work assignments based on high-level team goals.

You will be responsible for setting an agenda to develop and ship machine learning models that positively impact the business, working with partners across the company including operations and engineering. You will use research results to shape strategy for the company, and help build a foundation of tools and practices used by quantitative staff across the company.

What you will achieve:

• Define the research vision for data science, and oversee planning, staffing, and prioritization to make sure the team is advancing that roadmap

• Invest in your team’s skills, tools, and processes to improve their velocity, including working with engineering counterparts to shape the roadmap for machine learning needs

• Hire, retain, and develop talented and diverse staff through ownership of our data science hiring processes, brand, and functional leadership of data scientists

• Evangelise machine learning and AI internally and externally, including attending conferences and being a thought leader in the space

• Partner with the executive team and other business leaders to deliver cross-functional research work and models

Required Skills:

• Strong background in classical machine learning and machine learning deployments is a must and preferably with 4-8 years of experience

• Knowledge of deep learning & NLP

• Hands-on experience in TensorFlow/PyTorch, Scikit-Learn, Python, Apache Spark & Big Data platforms to manipulate large-scale structured and unstructured datasets.

• Experience with GPU computing is a plus.

• Professional experience as a data science leader, setting the vision for how to most effectively use data in your organization. This could be through technical leadership with ownership over a research agenda, or developing a team as a personnel manager in a new area at a larger company.

• Expert-level experience with a wide range of quantitative methods that can be applied to business problems.

• Evidence you’ve successfully been able to scope, deliver and sell your own research in a way that shifts the agenda of a large organization.

• Excellent written and verbal communication skills on quantitative topics for a variety of audiences: product managers, designers, engineers, and business leaders.

• Fluent in data fundamentals: SQL, data manipulation using a procedural language, statistics, experimentation, and modeling

Qualifications

• Professional experience as a data science leader, setting the vision for how to most effectively use data in your organization

• Expert-level experience with machine learning that can be applied to business problems

• Evidence you’ve successfully been able to scope, deliver and sell your own work in a way that shifts the agenda of a large organization

• Fluent in data fundamentals: SQL, data manipulation using a procedural language, statistics, experimentation, and modeling

• Degree in a field that has very applicable use of data science / statistics techniques (e.g. statistics, applied math, computer science, OR a science field with direct statistics application)

• 5+ years of industry experience in data science and machine learning, preferably at a software product company

• 3+ years of experience managing data science teams, incl. managing/grooming managers beneath you

• 3+ years of experience partnering with executive staff on data topics