Data Engineer

Main Responsibilities:

 Work closely with different Front Office and Support Function stakeholders including but not restricted to Business
Management, Accounts, Regulatory Reporting, Operations, Risk, Compliance, HR on all data collection and reporting use cases.
 Collaborate with Business and Technology teams to understand enterprise data, create an innovative narrative to explain, engage and enlighten regular staff members as well as executive leadership with data-driven storytelling
 Solve data consumption and visualization through data as a service distribution model
 Articulate findings clearly and concisely for different target use cases, including through presentations, design solutions, visualizations
 Perform Adhoc / automated report generation tasks using Power BI, Oracle BI, Informatica
 Perform data access/transfer and ETL automation tasks using Python, SQL, OLAP / OLTP, RESTful APIs, and IT tools (CFT, MQ-Series, Control-M, etc.)
 Provide support and maintain the availability of BI applications irrespective of the hosting location
 Resolve issues escalated from Business and Functional areas on data quality, accuracy, and availability, provide incident-related communications promptly
 Work with strict deadlines on high priority regulatory reports
 Serve as a liaison between business and technology to ensure that data related business requirements for protecting sensitive data are clearly defined, communicated, and well understood, and considered as part of operational
prioritization and planning
 To work for APAC Chief Data Office and coordinate with a fully decentralized team across different locations in APAC and global HQ (Paris).

General Skills:
 Excellent knowledge of RDBMS and hands-on experience with complex SQL is a must, some experience in NoSQL and Big Data Technologies like Hive and Spark would be a plus
 Experience with industrialized reporting on BI tools like PowerBI, Informatica
 Knowledge of data related industry best practices in the highly regulated CIB industry, experience with regulatory report generation for financial institutions
 Knowledge of industry-leading data access, data security, Master Data, and Reference Data Management, and establishing data lineage
 5+ years experience on Data Visualization / Business Intelligence / ETL developer roles
 Ability to multi-task and manage various projects simultaneously
 Attention to detail
 Ability to present to Senior Management, ExCo; excellent written and verbal communication skills

Roles and Responsibilities

● Research and develop advanced statistical and machine learning models for
analysis of large-scale, high-dimensional data.

● Dig deeper into data, understand characteristics of data, evaluate alternate
models and validate hypotheses through theoretical and empirical approaches.

● Productize has proven or working models into production-quality code.

● Collaborate with product management, marketing, and engineering teams in
Business Units to elicit & understand their requirements & challenges and
develop potential solutions

● Stay current with the latest research and technology ideas; share knowledge by
clearly articulating results and ideas to key decision-makers.

● File patents for innovative solutions that add to the company's IP portfolio

Requirements

● 4 to 6 years of strong experience in data mining, machine learning and
statistical analysis.

● BS/MS/Ph.D. in Computer Science, Statistics, Applied Math, or related areas
from Premier institutes ( only IITs / IISc / BITS / Top NITs or top US university
should apply)

● Experience in productizing models to code in a fast-paced start-up
environment.

● Fluency in analytical tools such as Matlab, R, Weka etc.

● Strong intuition for data and Keen aptitude on large scale data analysis

● Strong communication and collaboration skills.

Job Description:

The data science team is responsible for solving business problems with complex data. Data complexity could be characterized in terms of volume, dimensionality and multiple touchpoints/sources. We understand the data, ask fundamental-first-principle questions, apply our analytical and machine learning skills to solve the problem in the best way possible. 

 

Our ideal candidate

The role would be a client facing one, hence good communication skills are a must. 

The candidate should have the ability to communicate complex models and analysis in a clear and precise manner. 

 

The candidate would be responsible for:

• Comprehending business problems properly - what to predict, how to build DV, what value addition he/she is bringing to the client, etc.
• Understanding and analyzing large, complex, multi-dimensional datasets and build features relevant for business
• Understanding the math behind algorithms and choosing one over another
• Understanding approaches like stacking, ensemble and applying them correctly to increase accuracy

Desired technical requirements

• Proficiency with Python and the ability to write production-ready codes. 
• Experience in pyspark, machine learning and deep learning
• Big data experience, e.g. familiarity with Spark, Hadoop, is highly preferred
• Familiarity with SQL or other databases.

Roles and

Responsibilities

Seeking AWS Cloud Engineer /Data Warehouse Developer for our Data CoE team to

help us in configure and develop new AWS environments for our Enterprise Data Lake,

migrate the on-premise traditional workloads to cloud. Must have a sound

understanding of BI best practices, relational structures, dimensional data modelling,

structured query language (SQL) skills, data warehouse and reporting techniques.

 Extensive experience in providing AWS Cloud solutions to various business

use cases.

 Creating star schema data models, performing ETLs and validating results with

business representatives

 Supporting implemented BI solutions by: monitoring and tuning queries and

data loads, addressing user questions concerning data integrity, monitoring

performance and communicating functional and technical issues.

Job Description: -

This position is responsible for the successful delivery of business intelligence

information to the entire organization and is experienced in BI development and

implementations, data architecture and data warehousing.

Requisite Qualification

Essential

-

AWS Certified Database Specialty or -

AWS Certified Data Analytics

Preferred

Any other Data Engineer Certification

Requisite Experience

Essential 4 -7 yrs of experience

Preferred 2+ yrs of experience in ETL & data pipelines

Skills Required

Special Skills Required

 AWS: S3, DMS, Redshift, EC2, VPC, Lambda, Delta Lake, CloudWatch etc.

 Bigdata: Databricks, Spark, Glue and Athena

 Expertise in Lake Formation, Python programming, Spark, Shell scripting

 Minimum Bachelor’s degree with 5+ years of experience in designing, building,

and maintaining AWS data components

 3+ years of experience in data component configuration, related roles and

access setup

 Expertise in Python programming

 Knowledge in all aspects of DevOps (source control, continuous integration,

deployments, etc.)

 Comfortable working with DevOps: Jenkins, Bitbucket, CI/CD

 Hands on ETL development experience, preferably using or SSIS

 SQL Server experience required

 Strong analytical skills to solve and model complex business requirements

 Sound understanding of BI Best Practices/Methodologies, relational structures,

dimensional data modelling, structured query language (SQL) skills, data

warehouse and reporting techniques

Preferred Skills

Required

 Experience working in the SCRUM Environment.

 Experience in Administration (Windows/Unix/Network/Database/Hadoop) is a

plus.

 Experience in SQL Server, SSIS, SSAS, SSRS

 Comfortable with creating data models and visualization using Power BI

 Hands on experience in relational and multi-dimensional data modelling,

including multiple source systems from databases and flat files, and the use of

standard data modelling tools

 Ability to collaborate on a team with infrastructure, BI report development and

business analyst resources, and clearly communicate solutions to both

technical and non-technical team members

Design and implement cloud solutions, build MLOps on cloud (GCP) Build CI/CD pipelines orchestration by GitLab CI, GitHub Actions, Circle CI, Airflow or similar tools; Data science model review, run the code refactoring and optimization, containerization, deployment, versioning, and monitoring of its quality. Data science models testing, validation and test automation. Communicate with a team of data scientists, data engineers and architects, and document the processes. Required Qualifications: Ability to design and implement cloud solutions and ability to build MLOps pipelines on cloud solutions (GCP) Experience with MLOps Frameworks like Kubeflow, MLFlow, DataRobot, Airflow etc., experience with Docker and Kubernetes, OpenShift. Programming languages like Python, Go, Ruby or Bash, a good understanding of Linux, and knowledge of frameworks such as sci-kit-learn, Keras, PyTorch, Tensorflow, etc. Ability to understand tools used by data scientists and experience with software development and test automation. Fluent in English, good communication skills and ability to work in a team. Desired Qualifications: Bachelor’s degree in Computer Science or Software Engineering Experience in using GCP services. Good to have Google Cloud Certification

Bigdata with cloud:

 

Experience : 5-10 years

 

Location : Hyderabad/Chennai

 

Notice period : 15-20 days Max

 

1.  Expertise in building AWS Data Engineering pipelines with AWS Glue -> Athena -> Quick sight

2.  Experience in developing lambda functions with AWS Lambda

3.  Expertise with Spark/PySpark – Candidate should be hands on with PySpark code and should be able to do transformations with Spark

4.  Should be able to code in Python and Scala.

5.  Snowflake experience will be a plus

We are looking for a skilled Senior/Lead Bigdata Engineer to join our team. The role is part of the research and development team, where you with enthusiasm and knowledge are going to be our technical evangelist for the development of our inspection technology and products.

 

At Elop we are developing product lines for sustainable infrastructure management using our own patented technology for ultrasound scanners and combine this with other sources to see holistic overview of the concrete structure. At Elop we will provide you with world-class colleagues highly motivated to position the company as an international standard of structural health monitoring. With the right character you will be professionally challenged and developed.

This position requires travel to Norway.

 

Elop is sister company of Simplifai and co-located together in all geographic locations.

https://elop.no/

https://www.simplifai.ai/en/

Roles and Responsibilities

• Define technical scope and objectives through research and participation in requirements gathering and definition of processes
• Ingest and Process data from data sources (Elop Scanner) in raw format into Big Data ecosystem
• Realtime data feed processing using Big Data ecosystem
• Design, review, implement and optimize data transformation processes in Big Data ecosystem
• Test and prototype new data integration/processing tools, techniques and methodologies
• Conversion of MATLAB code into Python/C/C++.
• Participate in overall test planning for the application integrations, functional areas and projects.
• Work with cross functional teams in an Agile/Scrum environment to ensure a quality product is delivered.

Desired Candidate Profile

• Bachelor's degree in Statistics, Computer or equivalent
• 7+ years of experience in Big Data ecosystem, especially Spark, Kafka, Hadoop, HBase.
• 7+ years of hands-on experience in Python/Scala is a must.
• Experience in architecting the big data application is needed.
• Excellent analytical and problem solving skills
• Strong understanding of data analytics and data visualization, and must be able to help development team with visualization of data.
• Experience with signal processing is plus.
• Experience in working on client server architecture is plus.
• Knowledge about database technologies like RDBMS, Graph DB, Document DB, Apache Cassandra, OpenTSDB
• Good communication skills, written and oral, in English

We can Offer

• An everyday life with exciting and challenging tasks with the development of socially beneficial solutions
• Be a part of companys research and Development team to create unique and innovative products
• Colleagues with world-class expertise, and an organization that has ambitions and is highly motivated to position the company as an international player in maintenance support and monitoring of critical infrastructure!
• Good working environment with skilled and committed colleagues an organization with short decision paths.
• Professional challenges and development

Designation: Specialist - Cloud Service Developer (ABL_SS_600)

Position description:

• The person would be primary responsible for developing solutions using AWS services. Ex: Fargate, Lambda, ECS, ALB, NLB, S3 etc.
• Apply advanced troubleshooting techniques to provide Solutions to issues pertaining to Service Availability, Performance, and Resiliency
• Monitor & Optimize the performance using AWS dashboards and logs
• Partner with Engineering leaders and peers in delivering technology solutions that meet the business requirements 
• Work with the cloud team in agile approach and develop cost optimized solutions

 

Primary Responsibilities:

• Develop solutions using AWS services includiing Fargate, Lambda, ECS, ALB, NLB, S3 etc.

 

Reporting Team

• Reporting Designation: Head - Big Data Engineering and Cloud Development (ABL_SS_414)
• Reporting Department: Application Development (2487)

Required Skills:

• AWS certification would be preferred
• Good understanding in Monitoring (Cloudwatch, alarms, logs, custom metrics, Trust SNS configuration)
• Good experience with Fargate, Lambda, ECS, ALB, NLB, S3, Glue, Aurora and other AWS services. 
• Preferred to have Knowledge on Storage (S3, Life cycle management, Event configuration)
• Good in data structure, programming in (pyspark / python / golang / Scala)

Build data systems and pipelines using Apache Flink (or similar) pipelines.
Understand various raw data input formats, build consumers on Kafka/ksqldb for them and ingest large amounts of raw data into Flink and Spark.
Conduct complex data analysis and report on results.
Build various aggregation streams for data and convert raw data into various logical processing streams.
Build algorithms to integrate multiple sources of data and create a unified data model from all the sources.
Build a unified data model on both SQL and NO-SQL databases to act as data sink.
Communicate the designs effectively with the fullstack engineering team for development.
Explore machine learning models that can be fitted on top of the data pipelines.

Mandatory Qualifications Skills:

Deep knowledge of Scala and Java programming languages is mandatory
Strong background in streaming data frameworks (Apache Flink, Apache Spark) is mandatory
Good understanding and hands on skills on streaming messaging platforms such as Kafka
Familiarity with R, C and Python is an asset
Analytical mind and business acumen with strong math skills (e.g. statistics, algebra)
Problem-solving aptitude
Excellent communication and presentation skills

Couture.ai is building a patent-pending AI platform targeted towards vertical-specific solutions. The platform is already licensed by Reliance Jio and few European retailers, to empower real-time experiences for their combined >200 million end users. For this role, credible display of innovation in past projects (or academia) is a must. We are looking for a candidate who lives and talks Data & Algorithms, love to play with BigData engineering, hands-on with Apache Spark, Kafka, RDBMS/NoSQL DBs, Big Data Analytics and handling Unix & Production Server. Tier-1 college (BE from IITs, BITS-Pilani, top NITs, IIITs or MS in Stanford, Berkley, CMU, UW–Madison) or exceptionally bright work history is a must. Let us know if this interests you to explore the profile further.