PythonSenior Data Engineer
Responsibilities:
● Clean, prepare and optimize data at scale for ingestion and consumption by machine learning models
● Drive the implementation of new data management projects and re-structure of the current data architecture
● Implement complex automated workflows and routines using workflow scheduling tools
● Build continuous integration, test-driven development and production deployment frameworks
● Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards
● Anticipate, identify and solve issues concerning data management to improve data quality
● Design and build reusable components, frameworks and libraries at scale to support machine learning products
● Design and implement product features in collaboration with business and Technology stakeholders
● Analyze and profile data for the purpose of designing scalable solutions
● Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues
● Mentor and develop other data engineers in adopting best practices
● Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
Qualifications:
● 8+ years of experience developing scalable Big Data applications or solutions on distributed platforms
● Experience in Google Cloud Platform (GCP) and good to have other cloud platform tools
● Experience working with Data warehousing tools, including DynamoDB, SQL, and Snowflake
● Experience architecting data products in Streaming, Serverless and Microservices Architecture and platform.
● Experience with Spark (Scala/Python/Java) and Kafka
● Work experience with using Databricks (Data Engineering and Delta Lake components)
● Experience working with Big Data platforms, including Dataproc, Data Bricks etc
● Experience working with distributed technology tools including Spark, Presto, Databricks, Airflow
● Working knowledge of Data warehousing, Data modeling
● Experience working in Agile and Scrum development process
● Bachelor's degree in Computer Science, Information Systems, Business, or other relevant subject area
Role:
Senior Data Engineer
Total No. of Years:
8+ years of relevant experience
To be onboarded by:
Immediate
Notice Period:
Skills
Mandatory / Desirable
Min years (Project Exp)
Max years (Project Exp)
GCP Exposure
Mandatory Min 3 to 7
BigQuery, Dataflow, Dataproc, AI Building Blocks, Looker, Cloud Data Fusion, Dataprep .Spark and PySpark
Mandatory Min 5 to 9
Relational SQL
Mandatory Min 4 to 8
Shell scripting language
Mandatory Min 4 to 8
Python /scala language
Mandatory Min 4 to 8
Airflow/Kubeflow workflow scheduling tool
Mandatory Min 3 to 7
Kubernetes
Desirable 1 to 6
Scala
Mandatory Min 2 to 6
Databricks
Desirable Min 1 to 6
Google Cloud Functions
Mandatory Min 2 to 6
GitHub source control tool
Mandatory Min 4 to 8
Machine Learning
Desirable 1 to 6
Deep Learning
Desirable Min 1to 6
Data structures and algorithms
Mandatory Min 4 to 8
Shubham Vishwakarma
Full Stack Developer - Averlon
About Kloud9 Technologies
About
Kloud9 was founded with the vision of enabling our customers to transform into an intelligent enterprise with our “AI-First” approach. We help our customers in their data transformation and insights transformation journeys and enable them to make smart business decisions.
At Kloud9, we know AI & ML is one of the key technologies that can help organizations to significantly improve their customer experiences and transform their business operations to enable them to survive & thrive in this global competitive market.
Photos
Company social profiles
Similar jobs
Description of Role:
We are looking for a career-minded professional with global perspective to join the Mumbai based Data & Analytics Group (DAG).
Key responsibilities:
As part of the Data & Analytics Group (DAG) and reporting to the Head of the India DAG locally, the individual is responsible for the following –
1.Review, analyze, and resolve data quality issues for enterprise core data in the IM Data Warehouse.
2.Coordinate with data owners and other teams to identify root-cause of data quality issues and implement solutions.
3.Coordinate the onboarding of data from various internal / external sources into the central repository.
4.Work closely with Data Owners/Owner delegates on data analysis and development data quality (DQ) rules. Work with IT on enhancing DQ controls.
5.End-to-end analysis of business processes, data flows, and data usage to improve business productivity through re-engineering and data governance.
6.Interact with business stakeholders to identify, prioritize, and address data-related needs and issues.
7.Document business requirements and use cases for data-related projects.
8.Manage change control process and participate in user acceptance testing (UAT) activities.
9.Support other DAG initiatives.
Key Skills:
1.Ability to interact with business and technology teams to understand processes and data usage.
2.Ability to do data analysis - to trace data from source to consumption.
3.Capable of working across organization and as part of a cross-functional virtual team. Ability to work and think independently, but within a team-based approach.
4.Problem solver, self-starter, ability to work through an entire issue lifecycle. Ability to effectively prioritize and multi-task.
5.Strong communication skills
Key qualifications:
1.Bachelor’s Degree required and any other relevant academic course a plus.
2.Strong domain knowledge of investment data
3.5+ years of data management, data analytics, or data governance experience in financial services
4.Experience in data analysis, exploratory analysis using SQL and formulating data quality rules.
5.Experience working with BI reporting tools like Tableau, Power BI is preferred.
6.Knowledge in coding, Python is a plus.
7.Prior experience working with Data Platforms like Aladdin, FactSet, Bloomberg, MDM platforms preferred.
tailwindcss
AngularJS (1.x)
Angular (2+)
React.jsProficiency in Tailwind CSS for development.
Experience in troubleshooting and debugging frontend applications.
Solid understanding and hands-on experience with either Angular or React.
Strong communication skills with the ability to collaborate effectively within a team.
A keen eye for detail and a passion for creating seamless user experiences.
Skills: to UI Automation, Selenium, JAVA Rest Assured, TestNG, BDD, Cucumber, GIT, Jenkins
Requirments:
- Hands on of working across multiple projects
- Hands-on knowledge of BDD Cucumber Framework
- Hands-on knowledge of Maven
- Hands-on knowledge of Selenium - Appium
- Hands-on knowledge of Jenkins
- Basic Knowledge about CI/CD Concepts
- Exposure to UI Automation, Selenium, JAVA Rest Assured, TestNG
- Experience using at least one defect management tool – JIRA, HP ALM, ADO etc.
.NET
Javascript- Strong knowledge of .NET web frameworkwith .Net 3.5/4.0 Version.
- Experience on web-based projects using NET, C# and MS-SQL Server, JavaScript.
- Strong understanding of object-oriented programming.
- Skill for writing reusable libraries.
- Familiar with various design and architectural patterns.
- Knowledge of concurrency patterns.
- Familiarity with Microsoft SQL Server.
- Experience with popular web application frameworks.
- Familiarity with Windows Presentation Framework.
- Knack for writing clean, readable, and easily maintainable code.
- Understanding of fundamental design principles for building a scalable application
- Experience creating database schemas that represent and support business processes.
Python
Django- Analyze and organize raw data
- Build data systems and pipelines
- Evaluate business needs and objectives
- Interpret trends and patterns
- Conduct complex data analysis and report on results
- Build algorithms and prototypes
- Combine raw information from different sources
- Explore ways to enhance data quality and reliability
- Identify opportunities for data acquisition
- Should have experience in Python, Django Micro Service Senior developer with Financial Services/Investment Banking background.
- Develop analytical tools and programs
- Collaborate with data scientists and architects on several projects
- Should have 5+ years of experience as a data engineer or in a similar role
- Technical expertise with data models, data mining, and segmentation techniques
- Should have experience programming languages such as Python
- Hands-on experience with SQL database design
- Great numerical and analytical skills
- Degree in Computer Science, IT, or similar field; a Master’s is a plus
- Data engineering certification (e.g. IBM Certified Data Engineer) is a plus
React.js
Redux/Flux
React NativeJavascript
HTML/CSS- 4+ years experience working as a react.js developer.
- Knowledge of REACT tools including React.js, Webpack, Enzyme, Redux, and Flux.
- In-depth knowledge of JavaScript, CSS, HTML, and front-end languages.
- Experience with user interface design.
- Knowledge of performance testing frameworks including Mocha and Jest.
- Experience with browser-based debugging and performance testing software.
- Excellent troubleshooting skills.
- Good project management skills.
- Need At least 2-year experience in fast paced staffing environment working on IT, Non-IT and Techno-Functional Requirement like Business Analyst, IT Project Manager, Marketing Manager, Digital Marketing, Program Manager, Software QA Engineer, Software Developer, Accountant, Accounting Manager etc.
- Good in communication and can work independently with minimal supervision.
- Should be well aware of the recruitment process and related terms like MSP/Markup/Rate Negotiation.
- Team Player.
- Ability to learn and grasp new skills.
- Positive Attitude towards Work.
- Not looking for a specialist recruiter for a particular domain, we need someone who can work on all types of requirements as per the business need.
- Qualification: Any Graduate.
PythonDjango
Flask- Need to develop the new script using Perl & Python
- Need to analyse the existing script and do new changes
- Need to interact with QA team, Deployment team
- Need to interact with in-house and external customers
- Need to interact with internal team members for integrated development
- Need to have good communication skills within the team members
Skills required:
- Need to have experience in developing projects using PERL and Python
- Familiarity in Unix/Linux development environments and tools including
- scripting and process management
- Need to have experience in database(Mysql) concepts
- Need to have a experience in Elastic Search
- Have a knowledge in Git,Svn commands
- Need to have experience in implementing OOPS concepts
- Need to have a experience in XML functionality (read,create.,etc)
- Have a knowledge in creating a csv,xlsx files, json format
- Need to have a experience in PDF Functionality & FTP,SFTP Modules
- Need experience in unit testing
- Develop best practices to ensure coding efficiency and quality
- Experience in test driven development and Agile methodologies
Preference of Educational background:
- B.E
Preference of Professional background:
- Experience in handling modules
- Experience in PERL,Python,Mysql,Linux,Elastic search,Xml,PDF,FTP functionality
- 8+ years of experience in software development
- 4+ years of recent hands-on experience in architecting and building complex solutions that run in SaaS/PaaS environments, especially on AWS cloud leveraging SaaS based Microservices development coupled with Distributed Caching & Message Queuing.
- Should have experience in developing solution architecture and/or evaluate architectural alternatives for private, public and hybrid cloud models, including IaaS, PaaS, and other cloud services
- Should have excellent knowledge of cloud architecture and implementation features (OS, multi-tenancy, virtualization, orchestration, elastic scalability)
- Should have experience with Full Stack development with experience in technology stacks/frameworks like JAVA, Springboot, Python, Redis, SQL, NoSQL and Graph DBs
- Good to have Architected solutions that handle Big Data and should be proficient in data analytics
- Must have Expert level proficiency in Design / Architectural patterns, data structures and algorithms
- Must demonstrate knowledge of DevOps tool chains and processes
- Experience in web-based application migration from on-premise to SaaS model is a big plus.
- Experience of Integration patterns and associated best practice(e.g. Web Services, REST API's, Pub/Sub, MOM)
- Excellent knowledge and hands-on experience in Web services related, functionally decomposed architecture, Load Balancing of Web Services and applications, designing multi-tenant systems, Clustering and sharding of data, microservices architecture / design patterns, and throttling and performance management of such services
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon