Ab>initio, Big Data, Informatica, Tableau, Data Architect, Cognos, Microstrategy, Healther Business Analysts, Cloud etc.

Profile: Big Data Engineer (System Design)

Experience: 5+ years

Location: Bangalore

Work Mode: Hybrid

About the Role

We're looking for an experienced Big Data Engineer with system design expertise to architect and build scalable data pipelines and optimize big data solutions.

Key Responsibilities

• Design, develop, and maintain data pipelines and ETL processes using Python, Hive, and Spark
• Architect scalable big data solutions with strong system design principles
• Build and optimize workflows using Apache Airflow
• Implement data modeling, integration, and warehousing solutions
• Collaborate with cross-functional teams to deliver data solutions

Must-Have Skills

• 5+ years as a Data Engineer with Python, Hive, and Spark
• Strong hands-on experience with Java
• Advanced SQL and Hadoop experience
• Expertise in Apache Airflow
• Strong understanding of data modeling, integration, and warehousing
• Experience with relational databases (PostgreSQL, MySQL)
• System design knowledge
• Excellent problem-solving and communication skills

Good to Have

• Docker and containerization experience
• Knowledge of Apache Beam, Apache Flink, or similar frameworks
• Cloud platform experience.

We are looking for Senior Software Engineers responsible for designing, developing, and maintaining large scale distributed ad technology systems. This would entail working on several different systems, platforms and technologies.Collaborate with various engineering teams to meet a range of technological challenges. You will work with our product team to contribute and influence the roadmap of our products and technologies and also influence and inspire team members.

Experience

• 3 - 10 Years

Required Skills

• 3+ years of work experience and a degree in computer science or a similar field
• Knowledgeable about computer science fundamentals including data structures, algorithms, and coding
• Enjoy owning projects from creation to completion and wearing multiple hats
• Product focused mindset
• Experience building distributed systems capable of handling large volumes of traffic
• Fluency with Java, Vertex, Redis, Relational Databases
• Possess good communication skills
• Enjoy working in a team-oriented environment that values excellence
• Have a knack for solving very challenging problems
• (Preferred) Previous experience in advertising technology or gaming apps
• (Preferred) Hands-on experience with Spark, Kafka or similar open-source software

Responsibilities

• Creating design and architecture documents
• Conducting code reviews
• Collaborate with others in the engineering teams to meet a range of technological challenges
• Build, Design and Develop large scale advertising technology system capable of handling tens of billions of events daily

Education

• UG - B.Tech/B.E. - Computers; PG - M.Tech - Computer

What We Offer:

• Competitive salary and benefits package.
• Opportunities for professional growth and development.
• A collaborative and inclusive work environment.

Salary budget upto 50 LPA or hike20% on current ctc

you can text me over linkedin for quick response

Company Description

At Bungee Tech, we help retailers and brands meet customers everywhere and, on every occasion, they are in. We believe that accurate, high-quality data matched with compelling market insights empowers retailers and brands to keep their customers at the center of all innovation and value they are delivering. 

We provide a clear and complete omnichannel picture of their competitive landscape to retailers and brands. We collect billions of data points every day and multiple times in a day from publicly available sources. Using high-quality extraction, we uncover detailed information on products or services, which we automatically match, and then proactively track for price, promotion, and availability. Plus, anything we do not match helps to identify a new assortment opportunity.

Empowered with this unrivalled intelligence, we unlock compelling analytics and insights that once blended with verified partner data from trusted sources such as Nielsen, paints a complete, consolidated picture of the competitive landscape.

We are looking for a Big Data Engineer who will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them.

You will also be responsible for integrating them with the architecture used in the company.

We're working on the future. If you are seeking an environment where you can drive innovation, If you want to apply state-of-the-art software technologies to solve real world problems, If you want the satisfaction of providing visible benefit to end-users in an iterative fast paced environment, this is your opportunity.

Responsibilities

As an experienced member of the team, in this role, you will:

• Contribute to evolving the technical direction of analytical Systems and play a critical role their design and development

• You will research, design and code, troubleshoot and support. What you create is also what you own.

• Develop the next generation of automation tools for monitoring and measuring data quality, with associated user interfaces.

• Be able to broaden your technical skills and work in an environment that thrives on creativity, efficient execution, and product innovation.

BASIC QUALIFICATIONS

• Bachelor’s degree or higher in an analytical area such as Computer Science, Physics, Mathematics, Statistics, Engineering or similar.
• 5+ years relevant professional experience in Data Engineering and Business Intelligence
• 5+ years in with Advanced SQL (analytical functions), ETL, Data Warehousing.
• Strong knowledge of data warehousing concepts, including data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures, data modeling and performance tuning.
• Ability to effectively communicate with both business and technical teams.
• Excellent coding skills in Java, Python, C++, or equivalent object-oriented programming language
• Understanding of relational and non-relational databases and basic SQL
• Proficiency with at least one of these scripting languages: Perl / Python / Ruby / shell script

PREFERRED QUALIFICATIONS

• Experience with building data pipelines from application databases.
• Experience with AWS services - S3, Redshift, Spectrum, EMR, Glue, Athena, ELK etc.
• Experience working with Data Lakes.
• Experience providing technical leadership and mentor other engineers for the best practices on the data engineering space
• Sharp problem solving skills and ability to resolve ambiguous requirements
• Experience on working with Big Data
• Knowledge and experience on working with Hive and the Hadoop ecosystem
• Knowledge of Spark
• Experience working with Data Science teams

Responsibilities:

* 3+ years of Data Engineering Experience - Design, develop, deliver and maintain data infrastructures.

* SQL Specialist – Strong knowledge and Seasoned experience with SQL Queries

* Languages: Python

* Good communicator, shows initiative, works well with stakeholders.

* Experience working closely with Data Analysts and provide the data they need and guide them on the issues.

* Solid ETL experience and Hadoop/Hive/Pyspark/Presto/ SparkSQL

* Solid communication and articulation skills

* Able to handle stakeholders independently with less interventions of reporting manager.

* Develop strategies to solve problems in logical yet creative ways.

* Create custom reports and presentations accompanied by strong data visualization and storytelling

We would be excited if you have:

* Excellent communication and interpersonal skills

* Ability to meet deadlines and manage project delivery

* Excellent report-writing and presentation skills

* Critical thinking and problem-solving capabilities

Your Opportunity

• Own and drive business features into tech requirements
• Design & develop large scale real time server side systems
• Quickly create quality prototypes
• Staying updated on emerging technologies
• Ensuring that all deliverables adhere to our world class standards
• Promote coding best practices
• Mentor and develop junior developers in the team

Required Experience:

• 4+ years of relevant experience as described below
• Excellent grasp of Core Java, Multi Threading and OO design patterns
• Experience with Scala, functional, reactive programming and Akka/Play is a plus
• Excellent understanding of data structures and algorithms
• Solid grasp of large scale distributed real time systems
• Prior experience on building a scalable and resilient micro service
• Solid understanding of relational databases, NoSQL databases and Caching systems
• Good understanding of Big Data technologies such as Spark, Hadoop is a plus
• Experience on one of AWS, Azure or GCP

Who you are :

• You have excellent and effective communication and collaborative skills
• You love problem solving
• You stay up to date with the latest technologies and then apply them in real life
• You love paying attention to detail
• You thrive in meeting tight deadlines and prioritising workloads
• Ability to collaborate across multiple functions

Education:

Bachelor’s degree in Engineering or equivalent experience within the field

Job Description for :

Role: Data/Integration Architect

Experience – 8-10 Years

Notice Period: Under 30 days

Key Responsibilities: Designing, Developing frameworks for batch and real time jobs on Talend. Leading migration of these jobs from Mulesoft to Talend, maintaining best practices for the team, conducting code reviews and demos.

Core Skillsets:

Talend Data Fabric - Application, API Integration, Data Integration. Knowledge on Talend Management Cloud, deployment and scheduling of jobs using TMC or Autosys.

Programming Languages - Python/Java
Databases: SQL Server, Other Databases, Hadoop

Should have worked on Agile

Sound communication skills

Should be open to learning new technologies based on business needs on the job

Additional Skills:

Awareness of other data/integration platforms like Mulesoft, Camel

Awareness Hadoop, Snowflake, S3

Be Part Of Building The Future

Dremio is the Data Lake Engine company. Our mission is to reshape the world of analytics to deliver on the promise of data with a fundamentally new architecture, purpose-built for the exploding trend towards cloud data lake storage such as AWS S3 and Microsoft ADLS. We dramatically reduce and even eliminate the need for the complex and expensive workarounds that have been in use for decades, such as data warehouses (whether on-premise or cloud-native), structural data prep, ETL, cubes, and extracts. We do this by enabling lightning-fast queries directly against data lake storage, combined with full self-service for data users and full governance and control for IT. The results for enterprises are extremely compelling: 100X faster time to insight; 10X greater efficiency; zero data copies; and game-changing simplicity. And equally compelling is the market opportunity for Dremio, as we are well on our way to disrupting a $25BN+ market.

About the Role

The Dremio India team owns the DataLake Engine along with Cloud Infrastructure and services that power it. With focus on next generation data analytics supporting modern table formats like Iceberg, Deltalake, and open source initiatives such as Apache Arrow, Project Nessie and hybrid-cloud infrastructure, this team provides various opportunities to learn, deliver, and grow in career. We are looking for innovative minds with experience in leading and building high quality distributed systems at massive scale and solving complex problems.

Responsibilities & ownership

• Lead, build, deliver and ensure customer success of next-generation features related to scalability, reliability, robustness, usability, security, and performance of the product.
• Work on distributed systems for data processing with efficient protocols and communication, locking and consensus, schedulers, resource management, low latency access to distributed storage, auto scaling, and self healing.
• Understand and reason about concurrency and parallelization to deliver scalability and performance in a multithreaded and distributed environment.
• Lead the team to solve complex and unknown problems 
• Solve technical problems and customer issues with technical expertise
• Design and deliver architectures that run optimally on public clouds like  GCP, AWS, and Azure
• Mentor other team members for high quality and design 
• Collaborate with Product Management to deliver on customer requirements and innovation
• Collaborate with Support and field teams to ensure that customers are successful with Dremio

Requirements

• B.S./M.S/Equivalent in Computer Science or a related technical field or equivalent experience
• Fluency in Java/C++ with 8+ years of experience developing production-level software
• Strong foundation in data structures, algorithms, multi-threaded and asynchronous programming models, and their use in developing distributed and scalable systems
• 5+ years experience in developing complex and scalable distributed systems and delivering, deploying, and managing microservices successfully
• Hands-on experience  in query processing or optimization, distributed systems, concurrency control, data replication, code generation, networking, and storage systems
• Passion for quality, zero downtime upgrades, availability, resiliency, and uptime of the platform
• Passion for learning and delivering using latest technologies
• Ability to solve ambiguous, unexplored, and cross-team problems effectively
• Hands on experience of working projects on AWS, Azure, and Google Cloud Platform 
• Experience with containers and Kubernetes for orchestration and container management in private and public clouds (AWS, Azure, and Google Cloud) 
• Understanding of distributed file systems such as  S3, ADLS, or HDFS
• Excellent communication skills and affinity for collaboration and teamwork
• Ability to work individually and collaboratively with other team members
• Ability to scope and plan solution for  big problems and mentors others on the same
• Interested and motivated to be part of a fast-moving startup with a fun and accomplished team