Title:- Senior Data Engineer
Experience: 4-6 yrs
Budget: 24-28 lpa
Location: Bangalore
Work of Mode: Work from office
Primary Skills: Data Bricks, Spark, Pyspark,Sql, Python, AWS
Qualification: Any Engineering degree
Responsibilities:
∙Design and build reusable components, frameworks and libraries at scale to support analytics products.
∙Design and implement product features in collaboration with business and Technology
stakeholders.
∙Anticipate, identify and solve issues concerning data management to improve data quality.
∙Clean, prepare and optimize data at scale for ingestion and consumption.
∙Drive the implementation of new data management projects and re-structure of the current data architecture.
∙Implement complex automated workflows and routines using workflow scheduling tools.
∙Build continuous integration, test-driven development and production deployment
frameworks.
∙Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards.
∙Analyze and profile data for the purpose of designing scalable solutions.
∙Troubleshoot complex data issues and perform root cause analysis to proactively resolve
product and operational issues.
∙Mentor and develop other data engineers in adopting best practices.
Qualifications:
Primary skillset:
∙Experience working with distributed technology tools for developing Batch and Streaming pipelines using
o SQL, Spark, Python, PySpark [4+ years],
o Airflow [3+ years],
o Scala [2+ years].
∙Able to write code which is optimized for performance.
∙Experience in Cloud platform, e.g., AWS, GCP, Azure, etc.
∙Able to quickly pick up new programming languages, technologies, and frameworks.
∙Strong skills building positive relationships across Product and Engineering.
∙Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
∙Experience with creating/ configuring Jenkins pipeline for smooth CI/CD process for Managed Spark jobs, build Docker images, etc.
∙Working knowledge of Data warehousing, Data modelling, Governance and Data Architecture
Good to have:
∙Experience working with Data platforms, including EMR, Airflow, Databricks (Data
Engineering & Delta Lake components, and Lakehouse Medallion architecture), etc.
∙Experience working in Agile and Scrum development process.
∙Experience in EMR/ EC2, Databricks etc.
∙Experience working with Data warehousing tools, including SQL database, Presto, and
Snowflake
∙Experience architecting data product in Streaming, Serverless and Microservices Architecture and platform.
Similar jobs
Project Manager – Team Lead (Appian System Analyst)
About the position:-
The Manager will be working in Intelligent Automation domain in a techno functional role and develop proprietary solutions on Appian BPM, IBM BPM, Java and RPA platforms. This Position will be required to work extensively with Audit business Teams for developing solutions using Appian BPM, Java and RPA platforms. Candidate should be able to work as an individual contributor as well as in leading the team on various technology aspect from programming and development point of view.
Broadly, the key responsibilities are:
- Design, Development, Architecture, Implementation and Support on end to end Business applications with Appian BPM platform.
- Adept to leading technologies preferably in BPM and RPA Domain.
- Collaborating with Business teams for process discussions and working closely with IT team for infrastructure and deployment.
Roles and responsibilities:
The Manager will be predominantly responsible to:
- Extensively involve at all phase of Technical planning & requirements gathering.
- Work with business and technology teams to analyze and assess automation feasibility.
- Manage Projects end-to-end using relevant development framework either SDLC or Agile.
- Lead team of Appian developers and making sure development is moving
- Ensure quality of development by closely working with developers and testing team..
- Work closely with IT technology and infrastructure team.
- Able to support and drive change management of existing processes as per new requirements using a structured change control process.
- Work with project leads on technology platform related issues and provide plausible suggestion for improvements.
- Ensure the work is performed with high quality and completed within agreed turnaround time.
- Strong experience in Developing Web services and exposure to Enterprise application integration.
- Timely raise any issues identified to the Managers so that the quality and turnaround time are not impacted
- Plan and organize the work in such a way that the Quality and efficiency are not impacted on the deliverables
- Lead / participate in firm's strategic initiatives and corporate social responsibility programs
Essential skills:-
Education and Experience Requirements:
- B.E or B. Tech with IT/Computer Science/ related specialization.
- 10+ year of post qualification experience last 5 years need to be on Appian.
- Strong knowledge on BPM and having worked on any other BPM product is definite plus.
- Strong knowledge of Appian BPM and IBM BPM.
- Good knowledge on Software applications, SQL, Python, JAVA/J2EE programming and C++
- Good understanding on BPM Fundamentals, Architecture Components, Technical Features as design feature, configuration in Appian etc.
- Well versed with AGILE and SDLC based development approach.
- Expertise on Process Design documents & Technical Design documents
About the Idea
Company's Mission: Company is changing the way India discovers and shops new brands. How? We match brands to passionate creators who create authentic content enabling discovery of brand’s products. This also helps the creators earn sustainably and get recognised as curators in their fields!
Current Focus: To achieve our mission, we start by enabling brands and influencers to collaborate for mutual benefit.
- Creators on Youtube get access to D2C brand campaigns to boost income.
- D2C brands gain sales through RoI-based marketing via authentic content from creators.
Progress:
- Business: We’ve already worked with 35+ influencers and 5 brands generating INR 85L+ in GMV and growing fast
- Funding: MavX has raised seed funding from top-tier global VC funds and leading industry veteran angels
About the Role
Company is looking for a Tech Leader to oversee the tech org and own the entire building of the V1 product.
What will you do:
- Creating the marketplace product that helps brands and influencers
- Designing the tech architecture, roadmap and vision
- Hiring the right team to achieve product-market fit and the vision
- Get involved in brainstorming and problem solving to solve critical product and business problems
What will you need:
Skills:
- Strong and demonstrated technical skills
- Ability to hire and manage a team of coders
- Cross-functional collaboration with product, business, design folks
Attitude:
- Bias for quick action - progress over process
- High empathy for small creators and small D2C brands
- First principles thinking and a problem solving mindset
- Strong accountability and ownership mindset
What will you gain:
- Creating tech vision from scratch
- Build in a fast-paced environment and test impact directly with users
- Experience the thrills of 0-1 and PMF discovery
- Work with a smart, young team that is highly committed to the mission
Rejolut is one of the award-winning fastest-growing tech company which is working on leading technology like Blockchain, ML & AI, Complex mobile & web apps, IoT, etc.
The ideal candidate will be responsible for managing a development team and client coordination.
Roles & Responsibilities:-
• Collaborate with business and functional partners and technology leadership in defining features backlog and specifying requirements
• Taking Ownership of Delivery
• Drive technology discussion and strategy in line with business needs and company reference architecture
• Define execution plan and approach based on project scope, the expected timeline,
• Support execution team with resolution of roadblocks and inter-dependencies Partner with technical and functional teams external to the project to ensure their tasks are completed as required
• Ensure quality of deliverable is verified and matching stakeholder expectations
• Conducting daily scrum calls for smooth delivery.
Desired Candidate Profile:-
- Sprint Planning, Coordinate sprints, retrospective meetings and daily stand-ups, Direct Client Interaction
- Organize and Manage Daily Scrum, Create various types of Project status reports, Manage the scrum board with various graphs, chart
- Experience in managing only Technical projects
- Excellent communication skill
How we are different
We are a young and dynamic team who are obsessed with solving the business problem at scale. We are a technology agnostic company that focus more on finding the optimal solution to the business problem rather than technology.
Why to join us
- We provide huge learning curve. Spending 1.5-2 years with us will put you 3 years ahead in the career.
- We work only on cutting edge technology like React, React Native, Flutter, NodeJS, Python, Go, Svelte, ELM
- There are two technology Co-founder in the company who are pretty much hands-on and have earlier bult the Consumer application which is downloaded millions of time and are in leadership position in companies like Samsung, Purplle, Loyalty Rewardz.
- Get mentored by the best in the industry
- Flexible working hours
- Perks best in the industry
• 2 - 5 years of experience building React and/or Mobile Applications
• 5-8 years working with microservices, API servers, databases, cloud-native development, observability,
alerting, and monitoring
• Deep exposure to cloud services, preferably Azure
• Preferably worked in the Finance/Retail domain or other similar domains with complex business
requirements.
• Hands-on skills combined with leadership qualities to guide teams.
Location – Bangalore, Mumbai, Gurgaon
Functional / Technical Skills:
• Strong understanding of networking fundamentals
o OSI Stack, DNS, TCP protocols
o Browser rendering and various stages of execution
• Good understanding of RESTful APIs, GraphQL and Web Sockets
• Ability to debug and profile Web/Mobile applications with Chrome DevTools or Native profilers
• Strong understanding of Distributed Systems, Fault Tolerance and Resiliency. Exposure to setting up
and managing Chaos is a plus.
• Exposure to Domain Driven Design (DDD), SOLID principles, and Data Modelling on various RDBMS,
NoSQL databases.
• Ability to define and document performance goals, SLAs, and volumetrics. Creating a framework for
measuring and validating the goals. Work with teams to implement and meet them.
• Create automation scripts to measure performance. Making this part of the CI/CD process.
• Good understanding of CNCF projects with a specific focus on Observability, Monitoring, Tracing,
Sidecars, Kubernetes
• Tuning of Cloud-native deployments with a focus on Cost Optimization.
• Participate in architecture reviews to identify potential issues, and bottlenecks and provide early guidance.
• Deep knowledge of at least 2 different programming languages and runtimes. Any two of Ruby, Python,
Swift, Go, Rust, C#, Dart, Kotlin, Java, Haskell, OCaml
• Excellent verbal and written communication
• A mindset to constantly learn new things and challenge the Status Quo.
Job Description:
- Responsible for helping in data preparation and data pull
- Should be able to cut and slice data to extract interesting insights from the data
- Work closely with cross-functional teams, TPMs & PMs to execute deliverables
- Identify key requirements needed from cross-functional teams and external vendors
- Accountable for delivering against established business goals/objectives
- Analyse, evaluate, and overcome program risks, and produce program reports for senior management and stakeholders
- Working on strategy development to enhance user experience
- Work with development teams to learn existing projects and different use cases.
- Hands-on experience in Excel & MS Office - Expert, SQL will be preferred
Technical Skills:
- Sales & acquisition experience, Customer Service, Customer Relationship, Relationship Management
- Be self-driven, and show the ability to deliver on ambiguous projects with incomplete data
Desired Skills:
- An ability and interest in working in a fast-paced, ambiguous, and rapidly changing environment
- Should have worked on Business Projects for an organization, Ex: customer acquisition, and customer retention.
1-3 Years’ experience (maximum 3 years) in Business Analyst/Program Manager/Project Manager role having good knowledge of SQL, Excel, and MS Office
- 15+ years of Hands-on technical application architecture experience and Application build/ modernization experience
- 15+ years of experience as a technical specialist in Customer-facing roles.
- Ability to travel to client locations as needed (25-50%)
- Extensive experience architecting, designing and programming applications in an AWS Cloud environment
- Experience with designing and building applications using AWS services such as EC2, AWS Elastic Beanstalk, AWS OpsWorks
- Experience architecting highly available systems that utilize load balancing, horizontal scalability and high availability
- Hands-on programming skills in any of the following: Python, Java, Node.js, Ruby, .NET or Scala
- Agile software development expert
- Experience with continuous integration tools (e.g. Jenkins)
- Hands-on familiarity with CloudFormation
- Experience with configuration management platforms (e.g. Chef, Puppet, Salt, or Ansible)
- Strong scripting skills (e.g. Powershell, Python, Bash, Ruby, Perl, etc.)
- Strong practical application development experience on Linux and Windows-based systems
- Extra curricula software development passion (e.g. active open source contributor)
This person MUST have:
- B.E Computer Science or equivalent
- Cloud architect with hands on development background.
- Build a collaborative relationship with different stakeholders
- Experience on React JS and Python Programming
- Exposure to Python/Django + NodeJS + ReactJS
- Ensure and drive completions of all committed tasks and deliverables
- Run daily stand-ups and bring teams into an agile cadence of frequent iterations
- Increase team productivity by removing obstacles and keeping processes lean.
- Able to prioritise different tasks and communicate it to the different verticals of the organisation.
- Regularly assess risks and opportunities; understand best practices, drive for process improvements, and find new ways of operating,
- with a focus on simplification and productivity.
- Excellent written and oral business communication and people skills
Experience:
- Min 7 year experience
- Not more than 15 year experience.
- Startup experience is a must.
Location:
- Remote
Timings:
- 40 hours a week but with 4 hours a day overlapping with client timezone. Typically clients are in California PST Timezone.
Position:
- Full time/Direct
- We have great benefits such as PF, medical insurance, 12 annual company holidays, 12 PTO leaves per year, annual increments, Diwali bonus, spot bonuses and other incentives etc.
- We dont believe in locking in people with large notice periods. You will stay here because you love the company. We have only a 15 days notice period.
This profile will include following responsibilities:
– Provide First Level of Support to Enterprise Customers
– Understand and Respond to Client Queries/Service Request
– Report Application Bugs to the Development Team and Follow-up
– Install Product Upgrades on the client premises/Cloud environment
– Follow Product Deployment Process
– Provide Product Demos and Trainings to Clients/Team Members
– Maintain Support Tickets on the Service Desk
– Write Scripts to automate installation tasks or other manual tasks
– Participate in Quality initiativesRequirements:
- Educational Qualification: BCS, MCS, BCA, MCA or equivalent degree in trade/commerce stream with domain knowledge in indirect taxation.
- Prior work experience in tax software solution or tax advisory companies preferred.
- Ability to effectively service large portfolio of customers
- Excellence in Interpersonal Skills with strong command on spoken and written communication skills
- Ability to thrive in a multi-tasking environment
Roles and Responsibilities:
- Guide a team of customer support associates on customer issue investigation and resolution by applying understanding of GST compliance requirements, GSTN APIs, Eway Bill APIs and E-Invoicing APIs etc.
- Logging and keeping records of customer/employee queries
- Analysing call logs so you can spot common trends and underlying problems, to guide support team
- Updating self-help documents or preparing FAQ’s, to build knowledge base of support team on issues raised by ASP Partners and Corporates.
- Ensure timely response to customers with accurate guidance for issue resolution, by measuring the performance of support team on response SLA and quality of response.
- Manage ongoing interaction with key client relationships to build the relationship to level of being the preferred and only service provider for GST compliance requirements.
- Manage monthly billing operations across customer portfolio – including a good control on receivable ageing and reporting to management on realization delays which require management intervention.
- Manage regular interaction with GST Network for customer issues. Build a strong relationship with GST Network to achieve quicker turnaround from GSTN on deliverables.
- Keep a close watch on any updates from GSTN and/or the ASP-GSP ecosystem, to convey relevant updates to our customer portfolio.
Azine Technologies is known for our innovative technologies, products and services - and for the people behind them. At Azine, we create identity verification products which are essentially apps, APIs, SDKs which have different modules that undergo OCR, Face Recognition, and other AI and machine learning.
As Project Manager - Software Development you will be responsible for managing engineering and developing cohesive codes with high proficiency in multiple languages and be comfortable on both front end and back end programming.
Role and Responsibilities
- Report directly to the CEO
- Lead, motivate, and manage a multidisciplinary IT team.
- Provide coaching, mentoring, performance management, and technical guidance that encourages growth within the team with the ability to manage teams in Ahmedabad.
- Increase the technology aspect of product development throughput 20x.
- Review code, lead technical discussions and promote best practices.
- Guide the team to deliver clean code and architecture and enable the business to scale
- Be able to jump in and lead by example; writing high-quality, highly-testable code.
- Promote lean and agile ways of delivering value through the technology team.
- Be the steward of the company's technology resources and technology spend.
- Creating timelines for the development and deployment of all technological services.
- Integrate the technology element with the company's business partnerships.
- Collaborate with the executive team members on strategic planning and management.
- Provide visibility into the development roadmap, sprints, throughput and expected timelines.
- Develop and design new scalable, loosely coupled and high-performance architecture.
- Work closely with the CEO on effective product management practices.
Qualifications and Education Requirements
B.E./B. Tech
6+ years of experience in a digital product development role in a software engineering business
6+ years of experience in Java for a commercial software company.
UX/UI, product development and project management experience.
Understanding of emerging technologies like Artificial Intelligence, Machine Learning, Optical Character Recognition, Anti-Spoofing, etc.
Proven experience in growing markets for technology products & services.
An ability to collaborate internally and externally with business partners.
Self-starter, independent and proactive.
Excellent interpersonal and communication skills.
Preferred Skills
Java, Python, Artificial Intelligence, Machine Learning, Optical character recognition, Self-organizing maps, Artificial Intelligence (AI), Machine Learning (ML), Computer Vision,
Additional Notes
We are offering amazing career growth opportunities, great compensation, and benefits. This begins with: - weekly lunches, alternate weekends-off, annual medical checkups, social engagement, financial assistance and more.
Job Type: Full-time
Office Hours: 10 a.m. to 7 p.m. Mon TO Fri; 9 a.m. to 2.30 a.m. Saturday (alternate Saturdays are off)
Location: Iscon, Ahmedabad