Big Data Engineer

About UPS:

Moving our world forward by delivering what matters! UPS is a company with a proud past and an even brighter future. Our values define us. Our culture differentiates us. Our strategy drives us. At UPS we are customer first, people led and innovation driven. UPS’s India based Technology Development Centers will bring UPS one step closer to creating a global technology workforce that will help accelerate our digital journey and help us engineer technology solutions that drastically improve our competitive advantage in the field of Logistics.

‘Future You’ grows as a visible and valued Technology professional with UPS, driving us towards an exciting tomorrow. As a global Technology organization we can put serious resources behind your development. If you are solutions orientated, UPS Technology is the place for you. ‘Future You’ delivers ground-breaking solutions to some of the biggest logistics challenges around the globe. You’ll take technology to unimaginable places and really make a difference for UPS and our customers.

Job Summary:

This position participates in the support of batch and real-time data pipelines utilizing various data analytics processing frameworks in support of data science practices for Marketing and Finance business units. This position supports the integration of data from various data sources, as well as performs extract, transform, load (ETL) data conversions, and facilitates data cleansing and enrichment. This position performs full systems life cycle management activities, such as analysis, technical requirements, design, coding, testing, implementation of systems and applications software. This position participates and contributes to synthesizing disparate data sources to support reusable and reproducible data assets.

RESPONSIBILITIES

• Supervises and supports data engineering projects and builds solutions by leveraging a strong foundational knowledge in software/application development. He/she is hands on.

• Develops and delivers data engineering documentation.

• Gathers requirements, defines the scope, and performs the integration of data for data engineering projects.

• Recommends analytic reporting products/tools and supports the adoption of emerging technology.

• Performs data engineering maintenance and support.

• Provides the implementation strategy and executes backup, recovery, and technology solutions to perform analysis.

• Performs ETL tool capabilities with the ability to pull data from various sources and perform a load of the transformed data into a database or business intelligence platform.

• Codes using programming language used for statistical analysis and modeling such as Python/Spark

REQUIRED QUALIFICATIONS:

• Literate in the programming languages used for statistical modeling and analysis, data warehousing and Cloud solutions, and building data pipelines.

• Proficient in developing notebooks in Data bricks using Python and Spark and Spark SQL.

• Strong understanding of a cloud services platform (e.g., GCP, or AZURE, or AWS) and all the data life cycle stages. Azure is preferred

• Proficient in using Azure Data Factory and other Azure features such as LogicApps.

• Preferred to have knowledge of Delta lake, Lakehouse and Unity Catalog concepts.

• Strong understanding of cloud-based data lake systems and data warehousing solutions.

• Has used AGILE concepts for development, including KANBAN and Scrums

• Strong understanding of the data interconnections between organizations’ operational and business functions.

• Strong understanding of the data life cycle stages - data collection, transformation, analysis, storing the data securely, providing data accessibility

• Strong understanding of the data environment to ensure that it can scale for the following demands: Throughput of data, increasing data pipeline throughput, analyzing large amounts of data, Real-time predictions, insights and customer feedback, data security, data regulations, and compliance.

• Strong knowledge of algorithms and data structures, as well as data filtering and data optimization.

• Strong understanding of analytic reporting technologies and environments (e.g., Power BI, Looker, Qlik, etc.)

• Understanding of distributed systems and the underlying business problem being addressed, as well as guides team members on how their work will assist by performing data analysis and presenting findings to the stakeholders. •

REQUIRED SKILLS:

 3 years of experience with Databricks, Apache Spark, Python, and SQL

Preferred SKILLS:

 DeltaLake Unity Catalog, R, Scala, Azure Logic Apps, Cloud Services Platform (e.g., GCP, or AZURE, or AWS), and AGILE concepts.

Publicis Sapient Overview:

The Senior Associate People Senior Associate L1 in Data Engineering, you will translate client requirements into technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution 

.

Job Summary:

As Senior Associate L1 in Data Engineering, you will do technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution

The role requires a hands-on technologist who has strong programming background like Java / Scala / Python, should have experience in Data Ingestion, Integration and data Wrangling, Computation, Analytics pipelines and exposure to Hadoop ecosystem components. Having hands-on knowledge on at least one of AWS, GCP, Azure cloud platforms will be preferable.

Role & Responsibilities:

Job Title: Senior Associate L1 – Data Engineering

Your role is focused on Design, Development and delivery of solutions involving:

• Data Ingestion, Integration and Transformation

• Data Storage and Computation Frameworks, Performance Optimizations

• Analytics & Visualizations

• Infrastructure & Cloud Computing

• Data Management Platforms

• Build functionality for data ingestion from multiple heterogeneous sources in batch & real-time

• Build functionality for data analytics, search and aggregation

Experience Guidelines:

Mandatory Experience and Competencies:

# Competency

1.Overall 3.5+ years of IT experience with 1.5+ years in Data related technologies

2.Minimum 1.5 years of experience in Big Data technologies

3.Hands-on experience with the Hadoop stack – HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data pipeline. Working knowledge on real-time data pipelines is added advantage.

4.Strong experience in at least of the programming language Java, Scala, Python. Java preferable

5.Hands-on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDb, Cassandra, AWS Redshift, Azure SQLDW, GCP BigQuery etc

Preferred Experience and Knowledge (Good to Have):

# Competency

1.Good knowledge of traditional ETL tools (Informatica, Talend, etc) and database technologies (Oracle, MySQL, SQL Server, Postgres) with hands on experience

2.Knowledge on data governance processes (security, lineage, catalog) and tools like Collibra, Alation etc

3.Knowledge on distributed messaging frameworks like ActiveMQ / RabbiMQ / Solace, search & indexing and Micro services architectures

4.Performance tuning and optimization of data pipelines

5.CI/CD – Infra provisioning on cloud, auto build & deployment pipelines, code quality

6.Working knowledge with data platform related services on at least 1 cloud platform, IAM and data security

7.Cloud data specialty and other related Big data technology certifications

Job Title: Senior Associate L1 – Data Engineering

Personal Attributes:

• Strong written and verbal communication skills

• Articulation skills

• Good team player

• Self-starter who requires minimal oversight

• Ability to prioritize and manage multiple tasks

• Process orientation and the ability to define and set up processes

What are the Key Responsibilities:

• Design NLP applications
• Select appropriate annotated datasets for Supervised Learning methods
• Use effective text representations to transform natural language into useful features
• Find and implement the right algorithms and tools for NLP tasks
• Develop NLP systems according to requirements
• Train the developed model and run evaluation experiments
• Perform statistical analysis of results and refine models
• Extend ML libraries and frameworks to apply in NLP tasks
• Remain updated in the rapidly changing field of machine learning

What are we looking for:

• Proven experience as an NLP Engineer or similar role
• Understanding of NLP techniques for text representation, semantic extraction techniques, data structures, and modeling
• Ability to effectively design software architecture
• Deep understanding of text representation techniques (such as n-grams, a bag of words, sentiment analysis etc), statistics and classification algorithms
• Knowledge of Python, Java, and R
• Ability to write robust and testable code
• Experience with machine learning frameworks (like Keras or PyTorch) and libraries (like sci-kit-learn)
• Strong communication skills
• An analytical mind with problem-solving abilities
• Degree in Computer Science, Mathematics, Computational Linguistics, or similar field

• Editorialist YX is looking for a Technical Architect - Search. As part of this role, you will work with a team that builds a unified search platform to power various searches for our .com website,IOS app, and internal support tools. This search impacts thousands of customers in a day and will also become pivotal to our tech efforts as we continue to grow 30x YoY.

• You will own the technical direction for the team, and you will be leading key search projects from ideation all the way to deployment. You will be working closely with both technical and business leaders to fulfill your mission.
• Salary is no bar for the relevant candidate. 

• 6+ years of experience working in java and web services.
• 6+ years of experience working in the Search domain.
• Proven skills in designing scalable, highly available distributed systems which can handle high data volumes.
• Strong understanding of software engineering principles and fundamentals including data structures and algorithms.
• Solid understanding of concurrency and multi-threading, multiple design patterns, and debugging and analytical methodologies.
• Hands-on experience on Solr Cloud or ElasticSearch.
• Deep understanding of information retrieval concepts.
• Deep understanding of Linguistic processing like tokenizers, spellers, and stemmers.
• Hands-on experience on big data tech stacks, like Hadoop, Hive, Cassandra, and Spark is a plus.
• Self-directed, self-motivated, and detail-oriented with the ability to come up with good design proposals and thorough analysis of production issues.
• Excellent written and oral communication skills on both technical and non-technical topics.

• Designing and building a search engine using elastic search with engineers in the team for overall success for Search and other ML-based systems.
• Collaborate with peers from other Engineering groups to tackle complex and meaningful problems with efficient and scalable delivery of Search solutions.
• You are expected to be self-motivated, dedicated, and a solution-oriented individual. The main responsibilities for this position include:Leading effort to build large-scale, distributed, and highly available systems and pipelines.
• Leading effort to build large scale and highly available information retrieval systems
• Design and develop solutions using Java tech stack.
• Design and implement as per secure guidelines
• Work with QA to identify issues and fix them.

• B.Tech. in Computer Science or equivalent experience
• 6+ Yrs of experience in Java, Web services & Search Domain
• Experience working in Product based company

• Retiral Benefits
• Medical Insurance
• Remote Working Opportunity for the time being
• MacBook
• Stock Options
• Gym Membership

Introduction

http://www.synapsica.com/">Synapsica is a https://yourstory.com/2021/06/funding-alert-synapsica-healthcare-ivycap-ventures-endiya-partners/">series-A funded HealthTech startup founded by alumni from IIT Kharagpur, AIIMS New Delhi, and IIM Ahmedabad. We believe healthcare needs to be transparent and objective while being affordable. Every patient has the right to know exactly what is happening in their bodies and they don't have to rely on cryptic 2 liners given to them as a diagnosis. 

Towards this aim, we are building an artificial intelligence enabled cloud based platform to analyse medical images and create v2.0 of advanced radiology reporting.  We are backed by IvyCap, Endia Partners, YCombinator and other investors from India, US, and Japan. We are proud to have GE and The Spinal Kinetics as our partners. Here’s a small sample of what we’re building: https://www.youtube.com/watch?v=FR6a94Tqqls">https://www.youtube.com/watch?v=FR6a94Tqqls

Your Roles and Responsibilities

Synapsica is looking for a Principal AI Researcher to lead and drive AI based research and development efforts. Ideal candidate should have extensive experience in Computer Vision and AI Research, either through studies or industrial R&D projects and should be excited to work on advanced exploratory research and development projects in computer vision and machine learning to create the next generation of advanced radiology solutions.

The role involves computer vision tasks including development customization and training of Convolutional Neural Networks (CNNs); application of ML techniques (SVM, regression, clustering etc.), and traditional Image Processing (OpenCV, etc.). The role is research-focused and would involve going through and implementing existing research papers, deep dive of problem analysis, frequent review of results, generating new ideas, building new models from scratch, publishing papers, automating and optimizing key processes. The role will span from real-world data handling to the most advanced methods such as transfer learning, generative models, reinforcement learning, etc., with a focus on understanding quickly and experimenting even faster. Suitable candidate will collaborate closely both with the medical research team, software developers and AI research scientists. The candidate must be creative, ask questions, and be comfortable challenging the status quo. The position is based in our Bangalore office.

Primary Responsibilities

• Interface between product managers and engineers to design, build, and deliver AI models and capabilities for our spine products.
• Formulate and design AI capabilities of our stack with special focus on computer vision.
• Strategize end-to-end model training flow including data annotation, model experiments, model optimizations, model deployment and relevant automations
• Lead teams, engineers, and scientists to envision and build new research capabilities and ensure delivery of our product roadmap.
• Organize regular reviews and discussions.
• Keep the team up-to-date with latest industrial and research updates.
• Publish research and clinical validation papers

Requirements

• 6+ years of relevant experience in solving complex real-world problems at scale using computer vision-based deep learning.
• Prior experience in leading and managing a team.
• Strong problem-solving ability
• Prior experience with Python, cuDNN, Tensorflow, PyTorch, Keras, Caffe (or similar Deep Learning frameworks).
• Extensive understanding of computer vision/image processing applications like object classification, segmentation, object detection etc
• Ability to write custom Convolutional Neural Network Architecture in Pytorch (or similar)
• Background in publishing research papers and/or patents 
• Computer Vision and AI Research background in medical domain will be a plus
• Experience of GPU/DSP/other Multi-core architecture programming
• Effective communication with other project members and project stakeholders
• Detail-oriented, eager to learn, acquire new skills
• Prior Project Management and Team Leadership experience
• Ability to plan work and meet the deadline

• Key responsibility is to design & develop a data pipeline for real-time data integration, processing, executing of the model (if required), and exposing output via MQ / API / No-SQL DB for consumption
• Provide technical expertise to design efficient data ingestion solutions to store & process unstructured data, such as Documents, audio, images, weblogs, etc
• Developing API services to provide data as a service
• Prototyping Solutions for complex data processing problems using AWS cloud-native solutions
• Implementing automated Audit & Quality assurance Checks in Data Pipeline
• Document & maintain data lineage from various sources to enable data governance
• Coordination with BIU, IT, and other stakeholders to provide best-in-class data pipeline solutions, exposing data via APIs, loading in down streams, No-SQL Databases, etc

Skills

• Programming experience using Python & SQL
• Extensive working experience in Data Engineering projects, using AWS Kinesys,  AWS S3, DynamoDB, EMR, Lambda, Athena, etc for event processing
• Experience & expertise in implementing complex data pipeline
• Strong Familiarity with AWS Toolset for Storage & Processing. Able to recommend the right tools/solutions available to address specific data processing problems
• Hands-on experience in Unstructured (Audio, Image, Documents, Weblogs, etc) Data processing.
• Good analytical skills with the ability to synthesize data to design and deliver meaningful information
• Know-how on any No-SQL DB (DynamoDB, MongoDB, CosmosDB, etc) will be an advantage.
• Ability to understand business functionality, processes, and flows
• Good combination of technical and interpersonal skills with strong written and verbal communication; detail-oriented with the ability to work independently

Functional knowledge

• Real-time Event Processing
• Data Governance & Quality assurance
• Containerized deployment
• Linux
• Unstructured Data Processing
• AWS Toolsets for Storage & Processing
• Data Security

• Extract and present valuable information from data
• Understand business requirements and generate insights
• Build mathematical models, validate and work with them
• Explain complex topics tailored to the audience
• Validate and follow up on results
• Work with large and complex data sets
• Establish priorities with clear goals and responsibilities to achieve a high level of performance.
• Work in an agile and iterative manner on solving problems
• Evaluate different options proactively and the ability to solve problems in an innovative way. Develop new solutions or combine existing methods to create new approaches.
• Good understanding of Digital & analytics
• Strong communication skills, orally and in writing