Apache Flume Jobs in Bangalore (Bengaluru)

Role description

• Work with the application development team to implement database schema, data strategies, build data flows
• Should have expertise across the database. Responsible for gathering and analyzing data requirements and data flows to other systems
• Perform database modeling and SQL scripting

 

Background/ Experience

• 8+ years of experience in Database modeling and SQL scripting
• Experience in OLTP and OLAP database modeling
• Experience in Data modeling for Relational (PostgreSQL) and Non-relational Databases
• Strong scripting experience P-SQL (Views, Store procedures), Python, Spark.
• Develop notebooks & jobs using Python language
• Big Data stack: Spark, Hadoop, Sqoop, Pig, Hive, Hbase, Flume, Kafka, Storm
• Should have good understanding on Azure Cosmos DB, Azure DW

Education

• Bachelors Master’s degree in information technology/Computer Science, Engineering (or equivalent)

 

1. Design and development of data ingestion pipelines.
2. Perform data migration and conversion activities.
3. Develop and integrate software applications using suitable development
methodologies and standards, applying standard architectural patterns, taking
into account critical performance characteristics and security measures.
4. Collaborate with Business Analysts, Architects and Senior Developers to
establish the physical application framework (e.g. libraries, modules, execution
environments).
5. Perform end to end automation of ETL process for various datasets that are
being ingested into the big data platform.