Python
Java
Scala
Data Analytics
Data Science
Machine Learning (ML)Senior Data Engineer Job Description
Overview
The Senior Data Engineer will design, develop, and maintain scalable data pipelines and
infrastructure to support data-driven decision-making and advanced analytics. This role requires deep
expertise in data engineering, strong problem-solving skills, and the ability to collaborate with
cross-functional teams to deliver robust data solutions.
Key Responsibilities
Data Pipeline Development: Design, build, and optimize scalable, secure, and reliable data
pipelines to ingest, process, and transform large volumes of structured and unstructured data.
Data Architecture: Architect and maintain data storage solutions, including data lakes, data
warehouses, and databases, ensuring performance, scalability, and cost-efficiency.
Data Integration: Integrate data from diverse sources, including APIs, third-party systems,
and streaming platforms, ensuring data quality and consistency.
Performance Optimization: Monitor and optimize data systems for performance, scalability,
and cost, implementing best practices for partitioning, indexing, and caching.
Collaboration: Work closely with data scientists, analysts, and software engineers to
understand data needs and deliver solutions that enable advanced analytics, machine
learning, and reporting.
Data Governance: Implement data governance policies, ensuring compliance with data
security, privacy regulations (e.g., GDPR, CCPA), and internal standards.
Automation: Develop automated processes for data ingestion, transformation, and validation
to improve efficiency and reduce manual intervention.
Mentorship: Guide and mentor junior data engineers, fostering a culture of technical
excellence and continuous learning.
Troubleshooting: Diagnose and resolve complex data-related issues, ensuring high
availability and reliability of data systems.
Required Qualifications
Education: Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science,
or a related field.
Experience: 5+ years of experience in data engineering or a related role, with a proven track
record of building scalable data pipelines and infrastructure.
Technical Skills:
Proficiency in programming languages such as Python, Java, or Scala.
Expertise in SQL and experience with NoSQL databases (e.g., MongoDB, Cassandra).
Strong experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services
(e.g., Redshift, BigQuery, Snowflake).
Hands-on experience with ETL/ELT tools (e.g., Apache Airflow, Talend, Informatica) and
data integration frameworks.
Familiarity with big data technologies (e.g., Hadoop, Spark, Kafka) and distributed
systems.
Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes) is a
plus.
Soft Skills:
Excellent problem-solving and analytical skills.
Strong communication and collaboration abilities.
Ability to work in a fast-paced, dynamic environment and manage multiple priorities.
Certifications (optional but preferred): Cloud certifications (e.g., AWS Certified Data Analytics,
Google Professional Data Engineer) or relevant data engineering certifications.
Preferred Qualifica
Experience with real-time data processing and streaming architectures.
Familiarity with machine learning pipelines and MLOps practices.
Knowledge of data visualization tools (e.g., Tableau, Power BI) and their integration with data
pipelines.
Experience in industries with high data complexity, such as finance, healthcare, or
e-commerce.
Work Environment
Location: Hybrid/Remote/On-site (depending on company policy).
Team: Collaborative, cross-functional team environment with data scientists, analysts, and
business stakeholders.
Hours: Full-time, with occasional on-call responsibilities for critical data systems.
Shubham Vishwakarma
Full Stack Developer - Averlon
About KJBN labs
About
With a strong foundation in technology and a deep understanding of the ever-evolving business landscape, we recognized the pressing need for user-friendly, scalable, and cost-effective SaaS solutions. Driven by this realisation, we embarked on a mission to craft software that seamlessly integrates into diverse industries, catering to their unique requirements and challenges.
Candid answers by the company
Agri tech product
Company social profiles
Similar jobs
Go Programming (Golang)Key Responsibilities:
Design, develop, and maintain efficient, reusable, and reliable Go code.
Implement and integrate with back-end services, databases, and APIs.
Write clean, scalable, and testable code following best practices and design patterns.
Collaborate with cross-functional teams to define, design, and ship new features.
Optimize application performance for maximum speed and scalability.
Identify and address bottlenecks and bugs, and devise solutions to these problems.
Stay up-to-date with the latest industry trends, technologies, and best practices.
Required Qualifications:
Proven experience as a Golang Developer or similar role in software development.
Proficiency in Go programming language, paradigms, constructs, and idioms.
Experience with server-side development, microservices architecture, and RESTful APIs.
Familiarity with common Go frameworks and tools such as Gin.
Knowledge implementing monitoring, logging, and alerting systems
Experience with SQL and NoSQL databases (e.g., PostgreSQL, MySQL, MongoDB).
Understanding of code versioning tools, such as Git.
Strong understanding of concurrency and parallelism in Go.
Experience with cloud platforms (AWS, GCP, Azure) and containerization (Docker, Kubernetes) is a plus.
Excellent problem-solving skills and attention to detail.
Ability to work effectively both independently and as part of a team.
Join our Cybersecurity team to protect real-world systems from evolving threats through hands-on ethical hacking and defensive security.
What you'll do:
- Conduct vulnerability assessments using Nmap, Nessus, OpenVAS
- Perform penetration testing on web apps (Burp Suite, OWASP Top 10)
- Analyze network traffic with Wireshark, set up SIEM alerts
- Develop Python scripts for automated security scanning
- Document findings in professional pentest reports
- Work on LIVE client security audits (e-commerce, fintech)
What we need:
- CS/IT/ECE background (1st-3rd year students welcome)
- Basic Linux commands, networking fundamentals (TCP/IP)
- Passion for cybersecurity; GitHub projects a plus
- Tools exposure: Kali Linux, Metasploit (we'll teach advanced)
You'll gain:
- Industry-recognized completion certificate
- Portfolio-ready pentest reports
- Live project experience for placements
- Mentorship from certified ethical hackers
Roles and Responsibilities :-
● Test the overall system using appropriate testing techniques and
instruments.
● Applying the appropriate test measurements and metrics in the product.
● Execute all levels of testing (Integration, Regression and Sanity).
● Design and develop automation scripts when needed.
● Detect and track software defects and inconsistencies.
Skill set required :-
● critical thinking and analytical skills to examine bug reports and prioritize
necessary tests
● Logical and creative mindset.
● Should be able to work on testing tools and technologies.
● Knowledge of Software Development Life Cycle
● Knowledge of Software Test Life Cycle
iOS App Development
Objective C
Swift● Proficiency in Swift and Objective C, a novice in backend development.
● Solid fundamentals of data structures, Algorithms, System Design.
● Good understanding of internal and external libraries and write code with useful
abstraction and separation of concerns.
● Concurrency and multithreading are friends.
● Prefer to reduce the 3rd party dependencies and opt only if it’s essential.
● GateKeeper for Master and maintain strategies to code integration.
● Pivot for a good degree of predictability (estimation, planning) in deliverables.
● Proficient in CI/CD Pipelines, Fastlane tools.
Document Credit: iOS Team
● Inclination towards reactive programming.
● Exposure in mentoring SD1, SD2, Interns within the team.
● Prepare and Maintain the hiring process in any capacity.
● Drives productivity within the organization and makes sure processes and protocols are
followed (JIRA, Confluence, Blogs, etc.)
React NativeJava
NodeJS (Node.js)Objective CWe're looking for a Fullstack Engineer to work remotely for a product-based HealthTech Startup based in Mumbai.
Requirements
4+ years of professional full-stack software development experience.
[ React Native (2+ years) + Typescript is required ]
Experience working with NodeJS, Swift, Objective C, and Java is a plus but not required. Must have excellent English written and verbal communication.
- Work mode - Remote / Mumbai
- Salary upto 30 LPA + Esops
- Notice Period - 45 days or lesser will be preferred
- Interview process - 1 Online Test > Founder discussion 1 > Founder discussion 2 > Offer
Interested candidates can reach out and apply here
The ability and desire to sell
Excellent communication skills
A positive, confident and determined approach
High degree of self-motivation and ambition
The skills to work both independently and as part of a team
Active listening
Taking Initiatives
Verbal communication skills
Time management
Critical thinking
Self-regulation
Java
Elastic Search
Spring BootJava Elastic Search
- Java Microservices ELK stack(Elasticsearch, Logstash, Kibana, Beats, X-Pack, APM),
- Build Custom Reports
Java + NodeJS
- Java with Node.js- Microservices Development,
- Using graphql with node.js,
- Backend development with Node.JS
Java Microservices
- Working experience in Java and spring boot (microservices)
- Strong understanding of OOPs concepts
- Proficient in API design and development using RESTful services
- Knowledge and understanding of design principles behind building a scalable application
- Source code management using git
Java
GitResponsibilities:
- Your primary responsibility as a senior backend engineer will be to architect and develop a scalable and robust micro-services backend with strong Java, Spring(Boot), SQL, AWS/GCP.
- Experience being part of a software development team in an Agile/Lean/Continuous Delivery environment
- Be a key performer in a high-performance product engineering team
Qualifications:
- 2 to 4 years of overall IT experience. Most of this experience in Java (Core Java, Spring boot, Java collections, Java Multithreading)
- Should have experience designing database schemas - SQL and NoSQL.
- Exposure to frameworks like Spring, Hibernate, Play would be a plus
- Experience with microservices architecture would be beneficial.
- Working knowledge of any public cloud (AWS, GCP or Azure)
- Broad understanding and experience of real-time analytics, NoSQL data stores, data modeling and data management, analytical tools, languages, or libraries
- Knowledge of container tech like Docker, Kubernetes would be a plus.
- Bachelor's Degree in Computer Science or Engineering.
Java
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon