41+ Data engineering Jobs in Bangalore (Bengaluru) | Data engineering Job openings in Bangalore (Bengaluru)

About the role:

 Hopscotch is looking for a passionate Data Engineer to join our team. You will work closely with other teams like data analytics, marketing, data science and individual product teams to specify, validate, prototype, scale, and deploy data pipelines features and data architecture.

Here’s what will be expected out of you:

➢ Ability to work in a fast-paced startup mindset. Should be able to manage all aspects of data extraction transfer and load activities.

➢ Develop data pipelines that make data available across platforms.

➢ Should be comfortable in executing ETL (Extract, Transform and Load) processes which include data ingestion, data cleaning and curation into a data warehouse, database, or data platform.

➢ Work on various aspects of the AI/ML ecosystem – data modeling, data and ML pipelines.

➢ Work closely with Devops and senior Architect to come up with scalable system and model architectures for enabling real-time and batch services.

What we want:

➢ 5+ years of experience as a data engineer or data scientist with a focus on data engineering and ETL jobs.

➢ Well versed with the concept of Data warehousing, Data Modelling and/or Data Analysis.

➢ Experience using & building pipelines and performing ETL with industry-standard best practices on Redshift (more than 2+ years).

➢ Ability to troubleshoot and solve performance issues with data ingestion, data processing & query execution on Redshift.

➢ Good understanding of orchestration tools like Airflow.

 ➢ Strong Python and SQL coding skills.

➢ Strong Experience in distributed systems like spark.

➢ Experience with AWS Data and ML Technologies (AWS Glue,MWAA, Data Pipeline,EMR,Athena, Redshift,Lambda etc).

➢ Solid hands on with various data extraction techniques like CDC or Time/batch based and the related tools (Debezium, AWS DMS, Kafka Connect, etc) for near real time and batch data extraction.

Note :

Product based companies, Ecommerce companies is added advantage

Requirements:

• 2+ years of experience (4+ for Senior Data Engineer) with system/data integration, development or implementation of enterprise and/or cloud software Engineering degree in Computer Science, Engineering or related field.
• Extensive hands-on experience with data integration/EAI technologies (File, API, Queues, Streams), ETL Tools and building custom data pipelines.
• Demonstrated proficiency with Python, JavaScript and/or Java
• Familiarity with version control/SCM is a must (experience with git is a plus).
• Experience with relational and NoSQL databases (any vendor) Solid understanding of cloud computing concepts.
• Strong organisational and troubleshooting skills with attention to detail.
• Strong analytical ability, judgment and problem-solving techniques Interpersonal and communication skills with the ability to work effectively in a cross functional team.

We are looking for a passionate technologist with experience in building SaaS tech experience and products for a once-in-a-lifetime opportunity to lead Engineering for an AI powered Financial Operations platform to seamlessly monitor, optimize, reconcile and forecast cashflow with ease.

Background

An incredible rare opportunity for a VP Engineering to join a top tier incubated VC SaaS startup and outstanding management team. Product is currently in the build stage with a solid design partners pipeline of ~$250K and soon raising a pre-seed/seed round with marquee investors.

Responsibilities

• Develop and implement the company's technical strategy and roadmap, ensuring that it aligns with the overall business objectives and is scalable, reliable, and secure.

• Manage and optimize the company's technical resources, including staffing, software, hardware, and infrastructure, to ensure that they are being used effectively and efficiently.

• Work with the founding team and other executives to identify opportunities for innovation and new technology solutions, and evaluate the feasibility and impact of these solutions on the business.

• Lead the engineering function in developing and deploying high-quality software products and solutions, ensuring that they meet or exceed customer requirements and industry standards.

• Analyze and evaluate technical data and metrics, identifying areas for improvement and implementing changes to drive efficiency and effectiveness.

• Ensure that the company is in compliance with all legal and regulatory requirements, including data privacy and security regulations.

Eligibility criteria:

• 6+ years of experience in developing scalable SaaS products.

• Strong technical background with 6+ years of experience with a strong focus on SaaS, AI, and finance software.

• Prior experience in leadership roles.

• Entrepreneurial mindset, with a strong desire to innovate and grow a startup from the ground up.

Perks:

• Vested Equity.

• Ownership in the company.

• Build alongside passionate and smart individuals.

1. Flink Sr. Developer

Location: Bangalore(WFO)

Mandatory Skills & Exp -10+ Years : Must have Hands on Experience on FLINK, Kubernetes , Docker, Microservices, any one of Kafka/Pulsar, CI/CD and Java.

Job Responsibilities:

As the Data Engineer lead, you are expected to engineer, develop, support, and deliver real-time

streaming applications that model real-world network entities, and have a good understanding of the

Telecom Network KPIs to improve the customer experience through automation of operational network

data. Real-time application development will include building stateful in-memory backends, real-time

streaming APIs , leveraging real-time databases such as Apache Druid.

 Architecting and creating the streaming data pipelines that will enrich the data and support

the use cases for telecom networks

 Collaborating closely with multiple stakeholders, gathering requirements and seeking

iterative feedback on recently delivered application features.

 Participating in peer review sessions to provide teammates with code review as well as

architectural and design feedback.

 Composing detailed low-level design documentation, call flows, and architecture diagrams

for the solutions you build.

 Running to a crisis anytime the Operations team needs help.

 Perform duties with minimum supervision and participate in cross-functional projects as

scheduled.

Skills:

 Flink Sr. Developer, who has implemented and dealt with failure scenarios of

processing data through Flink.

 Experience with Java, K8S, Argo CD/Workflow, Prometheus, and Aether.

 Familiarity with object-oriented design patterns.

 Experience with Application Development DevOps Tools.

 Experience with distributed cloud-native application design deployed on Kubernetes

platforms.

 Experience with PostGres, Druid, and Oracle databases.

 Experience with Messaging Bus - Kafka/Pulsar

 Experience with AI/ML - Kubeflow, JupyterHub

 Experience with building real-time applications which leverage streaming data.

 Experience with streaming message bus platforming, either Kafka or Pulsar.

 Experience with Apache Spark applications and Hadoop platforms.

 Strong problem solving skills.

 Strong written and oral communication skills.

Job description Position: Data Engineer Experience: 6+ years Work Mode: Work from Office Location: Bangalore Please note: This position is focused on development rather than migration. Experience in Nifi or Tibco is mandatory.Mandatory Skills: ETL, DevOps platform, Nifi or Tibco We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will play a crucial role in developing and maintaining our data infrastructure and ensuring the smooth operation of our data platforms. The ideal candidate should have a strong background in advanced data engineering, scripting languages, cloud and big data technologies, ETL tools, and database structures.

Responsibilities: •  Utilize advanced data engineering techniques, including ETL (Extract, Transform, Load), SQL, and other advanced data manipulation techniques. •   Develop and maintain data-oriented scripting using languages such as Python. •   Create and manage data structures to ensure efficient and accurate data storage and retrieval. •   Work with cloud and big data technologies, specifically AWS and Azure stack, to process and analyze large volumes of data. •   Utilize ETL tools such as Nifi and Tibco to extract, transform, and load data into various systems. •   Have hands-on experience with database structures, particularly MSSQL and Vertica, to optimize data storage and retrieval. •   Manage and maintain the operations of data platforms, ensuring data availability, reliability, and security. •   Collaborate with cross-functional teams to understand data requirements and design appropriate data solutions. •   Stay up-to-date with the latest industry trends and advancements in data engineering and suggest improvements to enhance our data infrastructure.

Requirements: •  A minimum of 6 years of relevant experience as a Data Engineer. •  Proficiency in ETL, SQL, and other advanced data engineering techniques. •   Strong programming skills in scripting languages such as Python. •   Experience in creating and maintaining data structures for efficient data storage and retrieval. •   Familiarity with cloud and big data technologies, specifically AWS and Azure stack. •   Hands-on experience with ETL tools, particularly Nifi and Tibco. •   In-depth knowledge of database structures, including MSSQL and Vertica. •   Proven experience in managing and operating data platforms. •   Strong problem-solving and analytical skills with the ability to handle complex data challenges. •   Excellent communication and collaboration skills to work effectively in a team environment. •   Self-motivated with a strong drive for learning and keeping up-to-date with the latest industry trends.

I am looking for Mulesoft Developer for a reputed MNC

Experience: 6+ Years

Relevant experience: 4 Years

Location : Pune, Mumbai, Bangalore, Indore, Kolkata

Skills:

Mulesoft

Experience: 6+ Years

Relevant experience: 4 Years

Location : Pune, Mumbai, Bangalore, Indore, Kolkata

JOB DESCRIPTION:. THE IDEAL CANDIDATE WILL:

• Ensure new features and subject areas are modelled to integrate with existing structures and provide a consistent view. Develop and maintain documentation of the data architecture, data flow and data models of the data warehouse appropriate for various audiences. Provide direction on adoption of Cloud technologies (Snowflake) and industry best practices in the field of data warehouse architecture and modelling.

• Providing technical leadership to large enterprise scale projects. You will also be responsible for preparing estimates and defining technical solutions to proposals (RFPs). This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project Roles & Responsibilities.

ELIGIBILITY CRITERIA: Desired Experience/Skills:
• Must have total 5+ yrs. in IT and 2+ years' experience working as a snowflake Data Architect and 4+ years in Data warehouse, ETL, BI projects.
• Must have experience at least two end to end implementation of Snowflake cloud data warehouse and 3 end to end data warehouse implementations on-premise preferably on Oracle.

• Expertise in Snowflake – data modelling, ELT using Snowflake SQL, implementing complex stored Procedures and standard DWH and ETL concepts
• Expertise in Snowflake advanced concepts like setting up resource monitors, RBAC controls, virtual warehouse sizing, query performance tuning, Zero copy clone, time travel and understand how to use these features
• Expertise in deploying Snowflake features such as data sharing, events and lake-house patterns
• Hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, Big Data model techniques using Python
• Experience in Data Migration from RDBMS to Snowflake cloud data warehouse
• Deep understanding of relational as well as NoSQL data stores, methods and approaches (star and snowflake, dimensional modelling)
• Experience with data security and data access controls and design
• Experience with AWS or Azure data storage and management technologies such as S3 and ADLS
• Build processes supporting data transformation, data structures, metadata, dependency and workload management
• Proficiency in RDBMS, complex SQL, PL/SQL, Unix Shell Scripting, performance tuning and troubleshoot
• Provide resolution to an extensive range of complicated data pipeline related problems, proactively and as issues surface
• Must have expertise in AWS or Azure Platform as a Service (PAAS)
• Certified Snowflake cloud data warehouse Architect (Desirable)
• Should be able to troubleshoot problems across infrastructure, platform and application domains.
• Must have experience of Agile development methodologies
• Strong written communication skills. Is effective and persuasive in both written and oral communication

Nice to have Skills/Qualifications:Bachelor's and/or master’s degree in computer science or equivalent experience.
• Strong communication, analytical and problem-solving skills with a high attention to detail.

About you:
• You are self-motivated, collaborative, eager to learn, and hands on
• You love trying out new apps, and find yourself coming up with ideas to improve them
• You stay ahead with all the latest trends and technologies
• You are particular about following industry best practices and have high standards regarding quality

Since 1988 Quicken has been the top personal management software for millions of consumers. We pioneered a radically easier and faster way for people to manage their household finances. Since then we’ve continued to focus on delighting customers and making the Quicken product and experience better than ever.

Job Title:        Senior Data Engineer

Location:        Bangalore, India

Department:  Product Development

Quicken is the #1 personal finance management software with a 30-year heritage of helping millions of individuals and families stay on top of their finances. We are working on a strategy to deliver awesome personal finance experience to our customers across Windows, Mac, iOS, Android, and Web.

The successful candidate will join a fast-paced software development team building the next generation of Quicken Data Platform. The team uses the latest software development technology and tools.

If you are looking to be part of a high-performing team at the heart of a fun, energetic, and innovative company, come join the Quicken Team!

Responsibilities:

This is an opportunity to be a highly visible and key contributor on a small and passionate team delivering innovative data platform solutions across the company. You’ll help shape and deliver on an aggressive and innovative roadmap in areas key to Quicken’s continued success and growth.

• Senior Data Engineer role is a technical hands-on role. The responsibilities range from being at the vanguard of solving technical problems to venturing into unchartered areas of technologies to solve complex problems.
• Implement, or operate comprehensive data platform components to balance optimization of data access with batch loading and resource utilization factors, per customer requirements.
• Develop robust data platform components for sourcing, loading, transformation, and extracting data from various sources.
• Build metadata processes and frameworks.
• Create supporting documentation, such as metadata and diagrams of entity relationships, business processes, and process flow.
• Maintain standards, such as organization, structure, or nomenclature, for data platform elements, such as data architectures, pipelines, frameworks, models, tools, and databases.
• Implement business rules via scripts, middleware, or other technologies.
• Map data between source systems and data lake
• Ability to be independent and product high quality code on components related to the Data Platform. Should also possess Creativity, Responsibility, and Autonomy.
• Participate in the planning, design, and implementation of features, working with small teams that consist of engineers, product managers, and marketing.
• Demonstrate strong technical talent throughout the organization and engineer products that meet future scalability, performance, security, and quality goals while maintaining a cohesive user experience across different components and products.
• Adopt and share best practices of software development methodology and frameworks used in data platform.
• Passion for continuous learning, experimenting and applying cutting edge technology and software paradigms. Also responsible for fostering this culture across the team.

Qualifications: 

• 3+ years of hands-on experience with data platform technologies and tools
• Extensive experience in Python
• Should be comfortable with using REST APIs
• Experience in  at  least  2  of  the  3  stages  of  any  big  data  pipeline  -  data

ingestion/acquisition, data processing/transformation and data visualization

• Experience in  database  user  interface  and  query  software  -  Structured  query language (SQL)
• Experience in one or more structured DBMS along with data modelling – MySQL
• Nice to have working experience in big data processing frameworks like Spark
• Nice to have working knowledge on visualization tools like Tableau, Kibana, Amazon QuickSight
• Nice to have familiarity in AWS EMR, Kinesis, EC2, S3, AWS Glue
• Experience working with geographically distributed teams across different time zones
• Strong communication skills, both oral and written whether in-person or virtual
• Experience with Agile methodologies
• Bachelor’s degree in computer science or other technical discipline, or equivalent experience

What we offer:

• Competitive salary and performance bonus
• Amazing culture, strong believers in Autonomy/Mastery/Purpose
• Customer-driven, we make money by building the best products for our users. No confusion about how to win – build amazing products!
• Ability to work with and learn from some incredible talent
• Highly recognizable brand

We are an early stage start-up, building new fintech products for small businesses. Founders are IIT-IIM alumni, with prior experience across management consulting, venture capital and fintech startups. We are driven by the vision to empower small business owners with technology and dramatically improve their access to financial services. To start with, we are building a simple, yet powerful solution to address a deep pain point for these owners: cash flow management. Over time, we will also add digital banking and 1-click financing to our suite of offerings.

We have developed an MVP which is being tested in the market. We have closed our seed funding from marquee global investors and are now actively building a world class tech team. We are a young, passionate team with a strong grip on this space and are looking to on-board enthusiastic, entrepreneurial individuals to partner with us in this exciting journey. We offer a high degree of autonomy, a collaborative fast-paced work environment and most importantly, a chance to create unparalleled impact using technology.

Reach out if you want to get in on the ground floor of something which can turbocharge SME banking in India!

Technology stack at Velocity comprises a wide variety of cutting edge technologies like, NodeJS, Ruby on Rails, Reactive Programming,, Kubernetes, AWS, NodeJS, Python, ReactJS, Redux (Saga) Redis, Lambda etc. 

Key Responsibilities

• Responsible for building data and analytical engineering pipelines with standard ELT patterns, implementing data compaction pipelines, data modelling and overseeing overall data quality

• Work with the Office of the CTO as an active member of our architecture guild

• Writing pipelines to consume the data from multiple sources

• Writing a data transformation layer using DBT to transform millions of data into data warehouses.

• Implement Data warehouse entities with common re-usable data model designs with automation and data quality capabilities

• Identify downstream implications of data loads/migration (e.g., data quality, regulatory)

What To Bring

• 5+ years of software development experience, a startup experience is a plus.

• Past experience of working with Airflow and DBT is preferred

• 5+ years of experience working in any backend programming language. 

• Strong first-hand experience with data pipelines and relational databases such as Oracle, Postgres, SQL Server or MySQL

• Experience with DevOps tools (GitHub, Travis CI, and JIRA) and methodologies (Lean, Agile, Scrum, Test Driven Development)

• Experienced with the formulation of ideas; building proof-of-concept (POC) and converting them to production-ready projects

• Experience building and deploying applications on on-premise and AWS or Google Cloud cloud-based infrastructure

• Basic understanding of Kubernetes & docker is a must.

• Experience in data processing (ETL, ELT) and/or cloud-based platforms

• Working proficiency and communication skills in verbal and written English.

Job Description - Sr Azure Data Engineer

Roles & Responsibilities:

1. Hands-on programming in C# / .Net,
2. Develop serverless applications using Azure Function Apps.
3. Writing complex SQL Queries, Stored procedures, and Views. 
4. Creating Data processing pipeline(s).
5. Develop / Manage large-scale Data Warehousing and Data processing solutions.
6. Provide clean, usable data and recommend data efficiency, quality, and data integrity.

Skills

1. Should have working experience on C# /.Net.
2. Proficient with writing SQL queries, Stored Procedures, and Views
3. Should have worked on Azure Cloud Stack.
4. Should have working experience ofin developing serverless code.
5. Must have MANDATORILY worked on Azure Data Factory.

Experience 

1. 4+ years of relevant experience

• We are looking for : Data engineer
• Sprak
• Scala
• Hadoop

Looking Data Enginner for our OWn organization-

Notice Period- 15-30 days
CTC- upto 15 lpa

Preferred Technical Expertise 

1. Expertise in Python programming.
2. Proficient in Pandas/Numpy Libraries. 
3. Experience with Django framework and API Development.
4. Proficient in writing complex queries using SQL
5. Hands on experience with Apache Airflow.
6. Experience with source code versioning tools such as GIT, Bitbucket etc.

 Good to have Skills:

1. Create and maintain Optimal Data Pipeline Architecture
2. Experienced in handling large structured data.
3. Demonstrated ability in solutions covering data ingestion, data cleansing, ETL, Data mart creation and exposing data for consumers.
4. Experience with any cloud platform (GCP is a plus)
5. Experience with JQuery, HTML, Javascript, CSS is a plus.

Location: Bangalore

Function: Software Engineering → Backend Development

• Play a critical role as a member of the leadership team in shaping and supporting our overall company vision, day-to-day operations, and culture.
• Set the technical vision and build the technical product roadmap from launch to scale; including defining long-term goals and strategies
• Define best practices around coding methodologies, software development, and quality assurance
• Define innovative technical requirements and systems while balancing time, feasibility, cost and customer experience
• Build and support production products
• Ensure our internal processes and services comply with privacy and security regulations
• Establish a high performing, inclusive engineering culture focused on innovation, execution, growth and development
• Set a high bar for our overall engineering practices in support of our mission and goals
• Develop goals, roadmaps and delivery dates to help us scale quickly and sustainably
• Collaborate closely with Product, Business, Marketing and Data Science
• Experience with financial and transactional systems
• Experience engineering for large volumes of data at scale
• Experience with financial audit and compliance is a plus
• Experience building a successful consumer facing web and mobile apps at scale

Key  Responsibilities :

• Development of proprietary processes and procedures designed to process various data streams around critical databases in the org
• Manage technical resources around data technologies, including relational databases, NO SQL DBs, business intelligence databases, scripting languages, big data tools and technologies, visualization tools.
• Creation of a project plan including timelines and critical milestones to success in support of the project
• Identification of the vital skill sets/staff required to complete the project
• Identification of crucial sources of the data needed to achieve the objective.

Skill Requirement :

• Experience with data pipeline processes and tools
• Well versed in the Data domains (Data Warehousing, Data Governance, MDM, Data Quality, Data Catalog, Analytics, BI, Operational Data Store, Metadata, Unstructured Data, ETL, ESB)
• Experience with an existing ETL tool e.g Informatica and Ab initio etc
• Deep understanding of big data systems like Hadoop, Spark, YARN, Hive, Ranger, Ambari
• Deep knowledge of Qlik ecosystems like  Qlikview, Qliksense, and Nprinting
• Python, or a similar programming language
• Exposure to data science and machine learning
• Comfort working in a fast-paced environment

Soft attributes :

• Independence: Must have the ability to work on his/her own without constant direction or supervision. He/she must be self-motivated and possess a strong work ethic to strive to put forth extra effort continually
• Creativity: Must be able to generate imaginative, innovative solutions that meet the needs of the organization. You must be a strategic thinker/solution seller and should be able to think of integrated solutions (with field force apps, customer apps, CCT solutions etc.). Hence, it would be best to approach each unique situation/challenge in different ways using the same tools.
• Resilience: Must remain effective in high-pressure situations, using both positive and negative outcomes as an incentive to move forward toward fulfilling commitments to achieving personal and team goals.

• Expertise in designing and implementing enterprise scale database (OLTP) and Data warehouse solutions.
• Hands on experience in implementing Azure SQL Database, Azure SQL Date warehouse (Azure Synapse Analytics) and big data processing using Azure Databricks and Azure HD Insight.
• Expert in writing T-SQL programming for complex stored procedures, functions, views and query optimization.
• Should be aware of Database development for both on-premise and SAAS Applications using SQL Server and PostgreSQL.
• Experience in ETL and ELT implementations using Azure Data Factory V2 and SSIS.
• Experience and expertise in building machine learning models using Logistic and linear regression, Decision tree  and Random forest Algorithms.
• PolyBase queries for exporting and importing data into Azure Data Lake.
• Building data models both tabular and multidimensional using SQL Server data tools.
• Writing data preparation, cleaning and processing steps using Python, SCALA, and R.
• Programming experience using python libraries NumPy, Pandas and Matplotlib.
• Implementing NOSQL databases and writing queries using cypher.
• Designing end user visualizations using Power BI, QlikView and Tableau.
• Experience working with all versions of SQL Server 2005/2008/2008R2/2012/2014/2016/2017/2019
• Experience using the expression languages MDX and DAX.
• Experience in migrating on-premise SQL server database to Microsoft Azure.
• Hands on experience in using Azure blob storage, Azure Data Lake Storage Gen1 and Azure Data Lake Storage Gen2.
• Performance tuning complex SQL queries, hands on experience using SQL Extended events.
• Data modeling using Power BI for Adhoc reporting.
• Raw data load automation using T-SQL and SSIS
• Expert in migrating existing on-premise database to SQL Azure.
• Experience in using U-SQL for Azure Data Lake Analytics.
• Hands on experience in generating SSRS reports using MDX.
• Experience in designing predictive models using Python and SQL Server.
• Developing machine learning models using Azure Databricks and SQL Server

• Bring in industry best practices around creating and maintaining robust data pipelines for complex data projects with/without AI component
  • programmatically ingesting data from several static and real-time sources (incl. web scraping)
  • rendering results through dynamic interfaces incl. web / mobile / dashboard with the ability to log usage and granular user feedbacks
  • performance tuning and optimal implementation of complex Python scripts (using SPARK), SQL (using stored procedures, HIVE), and NoSQL queries in a production environment

• Industrialize ML / DL solutions and deploy and manage production services; proactively handle data issues arising on live apps
• Perform ETL on large and complex datasets for AI applications - work closely with data scientists on performance optimization of large-scale ML/DL model training
• Build data tools to facilitate fast data cleaning and statistical analysis
• Ensure data architecture is secure and compliant
• Resolve issues escalated from Business and Functional areas on data quality, accuracy, and availability
• Work closely with APAC CDO and coordinate with a fully decentralized team across different locations in APAC and global HQ (Paris).

•  Expert in structured and unstructured data in traditional and Big data environments – Oracle / SQLserver, MongoDB, Hive / Pig, BigQuery, and Spark
• Have excellent knowledge of Python programming both in traditional and distributed models (PySpark)
• Expert in shell scripting and writing schedulers
• Hands-on experience with Cloud - deploying complex data solutions in hybrid cloud / on-premise environment both for data extraction/storage and computation
• Hands-on experience in deploying production apps using large volumes of data with state-of-the-art technologies like Dockers, Kubernetes, and Kafka
• Strong knowledge of data security best practices
• 5+ years experience in a data engineering role
• Science / Engineering graduate from a Tier-1 university in the country
• And most importantly, you must be a passionate coder who really cares about building apps that can help people do things better, smarter, and faster even when they sleep

Recko Inc. is looking for data engineers to join our kick-ass engineering team. We are looking for smart, dynamic individuals to connect all the pieces of the data ecosystem.

What are we looking for:

1. 3+  years of development experience in at least one of MySQL, Oracle, PostgreSQL or MSSQL and experience in working with Big Data technologies like Big Data frameworks/platforms/data stores like Hadoop, HDFS, Spark, Oozie, Hue, EMR, Scala, Hive, Glue, Kerberos etc.

2. Strong experience setting up data warehouses, data modeling, data wrangling and dataflow architecture on the cloud

3. 2+ experience with public cloud services such as AWS, Azure, or GCP and languages like Java/ Python etc

4. 2+ years of development experience in Amazon Redshift, Google Bigquery or Azure data warehouse platforms preferred

5. Knowledge of statistical analysis tools like R, SAS etc 

6. Familiarity with any data visualization software

7. A growth mindset and passionate about building things from the ground up and most importantly, you should be fun to work with

As a data engineer at Recko, you will:

1. Create and maintain optimal data pipeline architecture,

2. Assemble large, complex data sets that meet functional / non-functional business requirements.

3. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

4. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.

5. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.

6. Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

7. Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.

8. Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.

9. Work with data and analytics experts to strive for greater functionality in our data systems.

About Recko: 

Recko was founded in 2017 to organise the world’s transactional information and provide intelligent applications to finance and product teams to make sense of the vast amount of data available. With the proliferation of digital transactions over the past two decades, Enterprises, Banks and Financial institutions are finding it difficult to keep a track on the money flowing across their systems. With the Recko Platform, businesses can build, integrate and adapt innovative and complex financial use cases within the organization and  across external payment ecosystems with agility, confidence and at scale.  . Today, customer-obsessed brands such as Deliveroo, Meesho, Grofers, Dunzo, Acommerce, etc use Recko so their finance teams can optimize resources with automation and prioritize growth over repetitive and time-consuming tasks around day-to-day operations. 

Recko is a Series A funded startup, backed by marquee investors like Vertex Ventures, Prime Venture Partners and Locus Ventures. Traditionally enterprise software is always built around functionality. We believe software is an extension of one’s capability, and it should be delightful and fun to use.

Working at Recko: 

We believe that great companies are built by amazing people. At Recko, We are a group of young Engineers, Product Managers, Analysts and Business folks who are on a mission to bring consumer tech DNA to enterprise fintech applications. The current team at Recko is 60+ members strong with stellar experience across fintech, e-commerce, digital domains at companies like Flipkart, PhonePe, Ola Money, Belong, Razorpay, Grofers, Jio, Oracle etc. We are growing aggressively across verticals.

• Total Experience of 7-10 years and should be interested in teaching and research
• 3+ years’ experience in data engineering which includes data ingestion, preparation, provisioning, automated testing, and quality checks.
• 3+ Hands-on experience in Big Data cloud platforms like AWS and GCP, Data Lakes and Data Warehouses
• 3+ years of Big Data and Analytics Technologies. Experience in SQL, writing code in spark engine using python, scala or java Language. Experience in Spark, Scala
• Experience in designing, building, and maintaining ETL systems
• Experience in data pipeline and workflow management tools like Airflow
• Application Development background along with knowledge of Analytics libraries, opensource Natural Language Processing, statistical and big data computing libraries
• Familiarity with Visualization and Reporting Tools like Tableau, Kibana.
• Should be good at storytelling in Technology

• Insurance P&C and Specialty domain experience a plus
• Experience in a cloud-based architecture preferred, such as Databricks, Azure Data Lake, Azure Data Factory, etc.
• Strong understanding of ETL fundamentals and solutions. Should be proficient in writing advanced / complex SQL, expertise in performance tuning and optimization of SQL queries required.
• Strong experience in Python/PySpark and Spark SQL
• Experience in troubleshooting data issues, analyzing end to end data pipelines, and working with various teams in resolving issues and solving complex problems.
• Strong experience developing Spark applications using PySpark and SQL for data extraction, transformation, and aggregation from multiple formats for analyzing & transforming the data to uncover insights and actionable intelligence for internal and external use

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Author data services using a variety of programming languages
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and Azure regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Work in an Agile environment with Scrum teams.
• Ensure data quality and help in achieving data governance.

• 2+ years of experience in a Data Engineer role
• Undergraduate degree required (Graduate degree preferred) in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
• Experience using the following software/tools:
  • Experience with big data tools: Hadoop, Spark, Kafka, etc.
  • Experience with relational SQL and NoSQL databases
  • Experience with data pipeline and workflow management tools
  • Experience with Azure cloud services: ADLS, ADF, ADLA, AAS
  • Experience with stream-processing systems: Storm, Spark-Streaming, etc.
  • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

REQUIREMENT:

•  Previous experience of working in large scale data engineering
•  4+ years of experience working in data engineering and/or backend technologies with cloud experience (any) is mandatory.
•  Previous experience of architecting and designing backend for large scale data processing.
•  Familiarity and experience of working in different technologies related to data engineering – different database technologies, Hadoop, spark, storm, hive etc.
•  Hands-on and have the ability to contribute a key portion of data engineering backend.
•  Self-inspired and motivated to drive for exceptional results.
•  Familiarity and experience working with different stages of data engineering – data acquisition, data refining, large scale data processing, efficient data storage for business analysis.
•  Familiarity and experience working with different DB technologies and how to scale them.

RESPONSIBILITY:

•  End to end responsibility to come up with data engineering architecture, design, development and then implementation of it.
•  Build data engineering workflow for large scale data processing.
•  Discover opportunities in data acquisition.
•  Bring industry best practices for data engineering workflow.
•  Develop data set processes for data modelling, mining and production.
•  Take additional tech responsibilities for driving an initiative to completion
•  Recommend ways to improve data reliability, efficiency and quality
•  Goes out of their way to reduce complexity.
•  Humble and outgoing - engineering cheerleaders.