Data Engineer

About UPS:

Moving our world forward by delivering what matters! UPS is a company with a proud past and an even brighter future. Our values define us. Our culture differentiates us. Our strategy drives us. At UPS we are customer first, people led and innovation driven. UPS’s India based Technology Development Centers will bring UPS one step closer to creating a global technology workforce that will help accelerate our digital journey and help us engineer technology solutions that drastically improve our competitive advantage in the field of Logistics.

‘Future You’ grows as a visible and valued Technology professional with UPS, driving us towards an exciting tomorrow. As a global Technology organization we can put serious resources behind your development. If you are solutions orientated, UPS Technology is the place for you. ‘Future You’ delivers ground-breaking solutions to some of the biggest logistics challenges around the globe. You’ll take technology to unimaginable places and really make a difference for UPS and our customers.

Job Summary:

This position participates in the support of batch and real-time data pipelines utilizing various data analytics processing frameworks in support of data science practices for Marketing and Finance business units. This position supports the integration of data from various data sources, as well as performs extract, transform, load (ETL) data conversions, and facilitates data cleansing and enrichment. This position performs full systems life cycle management activities, such as analysis, technical requirements, design, coding, testing, implementation of systems and applications software. This position participates and contributes to synthesizing disparate data sources to support reusable and reproducible data assets.

RESPONSIBILITIES

• Supervises and supports data engineering projects and builds solutions by leveraging a strong foundational knowledge in software/application development. He/she is hands on.

• Develops and delivers data engineering documentation.

• Gathers requirements, defines the scope, and performs the integration of data for data engineering projects.

• Recommends analytic reporting products/tools and supports the adoption of emerging technology.

• Performs data engineering maintenance and support.

• Provides the implementation strategy and executes backup, recovery, and technology solutions to perform analysis.

• Performs ETL tool capabilities with the ability to pull data from various sources and perform a load of the transformed data into a database or business intelligence platform.

• Codes using programming language used for statistical analysis and modeling such as Python/Spark

REQUIRED QUALIFICATIONS:

• Literate in the programming languages used for statistical modeling and analysis, data warehousing and Cloud solutions, and building data pipelines.

• Proficient in developing notebooks in Data bricks using Python and Spark and Spark SQL.

• Strong understanding of a cloud services platform (e.g., GCP, or AZURE, or AWS) and all the data life cycle stages. Azure is preferred

• Proficient in using Azure Data Factory and other Azure features such as LogicApps.

• Preferred to have knowledge of Delta lake, Lakehouse and Unity Catalog concepts.

• Strong understanding of cloud-based data lake systems and data warehousing solutions.

• Has used AGILE concepts for development, including KANBAN and Scrums

• Strong understanding of the data interconnections between organizations’ operational and business functions.

• Strong understanding of the data life cycle stages - data collection, transformation, analysis, storing the data securely, providing data accessibility

• Strong understanding of the data environment to ensure that it can scale for the following demands: Throughput of data, increasing data pipeline throughput, analyzing large amounts of data, Real-time predictions, insights and customer feedback, data security, data regulations, and compliance.

• Strong knowledge of algorithms and data structures, as well as data filtering and data optimization.

• Strong understanding of analytic reporting technologies and environments (e.g., Power BI, Looker, Qlik, etc.)

• Understanding of distributed systems and the underlying business problem being addressed, as well as guides team members on how their work will assist by performing data analysis and presenting findings to the stakeholders. •

REQUIRED SKILLS:

 3 years of experience with Databricks, Apache Spark, Python, and SQL

Preferred SKILLS:

 DeltaLake Unity Catalog, R, Scala, Azure Logic Apps, Cloud Services Platform (e.g., GCP, or AZURE, or AWS), and AGILE concepts.

Role: Principal Software Engineer

We looking for a passionate Principle Engineer - Analytics to build data products that extract valuable business insights for efficiency and customer experience. This role will require managing, processing and analyzing large amounts of raw information and in scalable databases. This will also involve developing unique data structures and writing algorithms for the entirely new set of products. The candidate will be required to have critical thinking and problem-solving skills. The candidates must be experienced with software development with advanced algorithms and must be able to handle large volume of data. Exposure with statistics and machine learning algorithms is a big plus. The candidate should have some exposure to cloud environment, continuous integration and agile scrum processes.

Responsibilities:

• Lead projects both as a principal investigator and project manager, responsible for meeting project requirements on schedule

• Software Development that creates data driven intelligence in the products which deals with Big Data backends

• Exploratory analysis of the data to be able to come up with efficient data structures and algorithms for given requirements

• The system may or may not involve machine learning models and pipelines but will require advanced algorithm development

• Managing, data in large scale data stores (such as NoSQL DBs, time series DBs, Geospatial DBs etc.)

• Creating metrics and evaluation of algorithm for better accuracy and recall

• Ensuring efficient access and usage of data through the means of indexing, clustering etc.

• Collaborate with engineering and product development teams.

Requirements:

• Master’s or Bachelor’s degree in Engineering in one of these domains - Computer Science, Information Technology, Information Systems, or related field from top-tier school

• OR Master’s degree or higher in Statistics, Mathematics, with hands on background in software development.

• Experience of 8 to 10 year with product development, having done algorithmic work

• 5+ years of experience working with large data sets or do large scale quantitative analysis

• Understanding of SaaS based products and services.

• Strong algorithmic problem-solving skills

• Able to mentor and manage team and take responsibilities of team deadline.

Skill set required:

• In depth Knowledge Python programming languages

• Understanding of software architecture and software design

• Must have fully managed a project with a team

• Having worked with Agile project management practices

• Experience with data processing analytics and visualization tools in Python (such as pandas, matplotlib, Scipy, etc.)

• Strong understanding of SQL and querying to NoSQL database (eg. Mongo, Casandra, Redis

Power BI Developer(Azure Developer )

Job Description:

Senior visualization engineer with understanding in Azure Data Factory & Databricks to develop and deliver solutions that enable delivery of information to audiences in support of key business processes.

Ensure code and design quality through execution of test plans and assist in development of standards & guidelines working closely with internal and external design, business and technical counterparts.

Desired Competencies:

• Strong designing concepts of data visualization centered on business user and a knack of communicating insights visually.
• Ability to produce any of the charting methods available with drill down options and action-based reporting. This includes use of right graphs for the underlying data with company themes and objects.
• Publishing reports & dashboards on reporting server and providing role-based access to users.
• Ability to create wireframes on any tool for communicating the reporting design.
• Creation of ad-hoc reports & dashboards to visually communicate data hub metrics (metadata information) for top management understanding.
• Should be able to handle huge volume of data from databases such as SQL Server, Synapse, Delta Lake or flat files and create high performance dashboards.
• Should be good in Power BI development
• Expertise in 2 or more BI (Visualization) tools in building reports and dashboards.
• Understanding of Azure components like Azure Data Factory, Data lake Store, SQL Database, Azure Databricks
• Strong knowledge in SQL queries
• Must have worked in full life-cycle development from functional design to deployment
• Intermediate understanding to format, process and transform data
• Should have working knowledge of GIT, SVN
• Good experience in establishing connection with heterogeneous sources like Hadoop, Hive, Amazon, Azure, Salesforce, SAP, HANA, API’s, various Databases etc.
• Basic understanding of data modelling and ability to combine data from multiple sources to create integrated reports

Preferred Qualifications:

• Bachelor's degree in Computer Science or Technology
• Proven success in contributing to a team-oriented environment

Job Sector: IT, Software

Job Type: Permanent

Location: Chennai

Experience: 10 - 20 Years

Salary: 12 – 40 LPA

Education: Any Graduate

Notice Period: Immediate

Key Skills: Python, Spark, AWS, SQL, PySpark

Contact at triple eight two zero nine four two double seven

Job Description:

Requirements

• Minimum 12 years experience
• In depth understanding and knowledge on distributed computing with spark.
• Deep understanding of Spark Architecture and internals
• Proven experience in data ingestion, data integration and data analytics with spark, preferably PySpark.
• Expertise in ETL processes, data warehousing and data lakes.
• Hands on with python for Big data and analytics.
• Hands on in agile scrum model is an added advantage.
• Knowledge on CI/CD and orchestration tools is desirable.
• AWS S3, Redshift, Lambda knowledge is preferred

InViz is Bangalore Based Startup helping Enterprises simplifying the Search and Discovery experiences for both their end customers as well as their internal users. We use state-of-the-art technologies in Computer Vision, Natural Language Processing, Text Mining, and other ML techniques to extract information/concepts from data of different formats- text, images, videos and make them easily discoverable through simple human-friendly touchpoints. 

TSDE - Data 

Data Engineer:

• Should have total 3-6 Yrs of experience in Data Engineering.
• Person should have experience in coding data pipeline on GCP. 
• Prior experience on Hadoop systems is ideal as candidate may not have total GCP experience. 
• Strong on programming languages like Scala, Python, Java. 
• Good understanding of various data storage formats and it’s advantages. 
• Should have exposure on GCP tools to develop end to end data pipeline for various scenarios (including ingesting data from traditional data bases as well as integration of API based data sources). 
• Should have Business mindset to understand data and how it will be used for BI and Analytics purposes. 
• Data Engineer Certification preferred 

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Author data services using a variety of programming languages
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and Azure regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Work in an Agile environment with Scrum teams.
• Ensure data quality and help in achieving data governance.

• 2+ years of experience in a Data Engineer role
• Undergraduate degree required (Graduate degree preferred) in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
• Experience using the following software/tools:
  • Experience with big data tools: Hadoop, Spark, Kafka, etc.
  • Experience with relational SQL and NoSQL databases
  • Experience with data pipeline and workflow management tools
  • Experience with Azure cloud services: ADLS, ADF, ADLA, AAS
  • Experience with stream-processing systems: Storm, Spark-Streaming, etc.
  • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

REQUIREMENT:

•  Previous experience of working in large scale data engineering
•  4+ years of experience working in data engineering and/or backend technologies with cloud experience (any) is mandatory.
•  Previous experience of architecting and designing backend for large scale data processing.
•  Familiarity and experience of working in different technologies related to data engineering – different database technologies, Hadoop, spark, storm, hive etc.
•  Hands-on and have the ability to contribute a key portion of data engineering backend.
•  Self-inspired and motivated to drive for exceptional results.
•  Familiarity and experience working with different stages of data engineering – data acquisition, data refining, large scale data processing, efficient data storage for business analysis.
•  Familiarity and experience working with different DB technologies and how to scale them.

RESPONSIBILITY:

•  End to end responsibility to come up with data engineering architecture, design, development and then implementation of it.
•  Build data engineering workflow for large scale data processing.
•  Discover opportunities in data acquisition.
•  Bring industry best practices for data engineering workflow.
•  Develop data set processes for data modelling, mining and production.
•  Take additional tech responsibilities for driving an initiative to completion
•  Recommend ways to improve data reliability, efficiency and quality
•  Goes out of their way to reduce complexity.
•  Humble and outgoing - engineering cheerleaders.