Senior Data Engineer
at Heavy Manufacturing, Automotive/Commercial Vehicles, Industr
About the Role
The Smart Operations (Smart Ops) division is seeking a seasoned Senior Data Engineer / Lead Data Engineer to design and scale our next-generation industrial data platform. In this role, you will architect the real-time data backbone that processes high-frequency iHistorian telemetry, IoT sensor feeds, and complex manufacturing data across global operations.
You will be the core technical owner of our Databricks Lakehouse ecosystem, responsible for building optimized, cost-efficient streaming and batch pipelines that deliver high-trust data to our Advanced Analytics, MLOps, and Business Intelligence teams.
Key Responsibilities
- End-to-End Lakehouse Architecture: Design, implement, and scale a robust Medallion Architecture (Bronze-Silver-Gold) on Azure Databricks and Delta Lake.
- Real-Time IoT Ingestion: Build fault-tolerant, low-latency streaming pipelines to ingest high-frequency telemetry and machine sensor data.
- Enterprise Governance: Enforce data security, fine-grained access controls, and comprehensive lineage tracking using Unity Catalog.
- Performance Tuning & Cost Optimization: Actively profile, tune, and optimize Spark workloads (utilizing partitioning, liquid clustering, AQE, and Photon) to drastically minimize cloud compute costs and pipeline runtimes.
- CI/CD Automation: Modernize data platform deployments across Dev/QA/Prod environments using Infrastructure as Code (IaC) and Databricks Asset Bundles (DABs).
- Cross-Functional Collaboration: Partner closely with manufacturing stakeholders, supply chain analysts, and data scientists to provide curated datasets for predictive maintenance, anomaly detection, and operational KPI reporting.
Technical Skills Breakdown
Mandatory Skills (Must-Haves)
- Domain Context: Minimum 5+ years of Data Engineering experience with direct exposure to Heavy Manufacturing, Automotive/Commercial Vehicles, Industrial Automation, or Global Logistics data environments.
- Databricks Ecosystem: Deep architectural mastery of Azure Databricks, PySpark, Spark SQL, Delta Lake, and Unity Catalog governance.
- Streaming & CDC: Proven hands-on experience engineering real-time data pipelines using Apache Kafka, Azure Event Hubs, or Spark Structured Streaming, alongside Change Data Capture (CDC) tools (e.g., Debezium).
- Cloud Infrastructure: Strong proficiency within the Microsoft Azure stack—specifically Azure Data Factory (ADF), Azure Data Lake Storage Gen2 (ADLS Cores), Azure Key Vault, and Azure Monitor.
- Advanced SQL & Python: Expert-level Python programming and complex SQL optimisation (advanced indexing, partition pruning, performance tuning of massive datasets).
- DevOps Automation: Hands-on experience setting up production CI/CD pipelines using Git, Azure DevOps, GitHub Actions, or Databricks Asset Bundles (DABs).
Good to Have Skills (Nice-to-Haves)
- Professional Certifications: Databricks Certified Data Engineer Professional (highly preferred), Databricks Certified Associate Developer for Apache Spark, or Microsoft Certified: Azure Data Engineer Associate (DP-203).
- Industrial & ERP Systems: Familiarity with industrial data systems (SCADA, PLC, iHistorian), heavy vehicle telemetry standards (MDF/MF4 files), or deep integration experience with enterprise ERPs (SAP S/4HANA, SAP BW).
- MLOps & GenAI: Exposure to working alongside Machine Learning pipelines, dataset feature engineering, or implementing Generative AI frameworks (RAG, Vector Databases like FAISS/Weaviate) for semantic query self-service.
- Infrastructure as Code (IaC): Basic proficiency with Terraform for provisioning cloud data infrastructure.
Education & Qualifications
• Bachelor’s or Master’s Degree in Computer Science, Information Technology, Electrical/Electronics Engineering, or a closely related technical field.
Shubham Vishwakarma
Full Stack Developer - Averlon
Similar jobs
Mandatory (Total Experience): Must have 5+ years of overall experience in Testing/QA
Mandatory (Experience 1): Must have 2+ years of experience in testing AI/ML models and data-driven applications, across NLP, recommendation engines, fraud detection, and advanced analytics models
Mandatory (Experience 2): Must have expertise in validating AI/ML models for accuracy, bias, explainability, and performance, ensuring decisions are fair, reliable, and transparent
Mandatory (Experience 3): Must have strong experience to design AI/ML test strategies, including boundary testing, adversarial input simulation, and anomaly monitoring to detect manipulation attempts by marketplace users (buyers/sellers)
Mandatory (Tools & Frameworks): Proficiency in AI/ML testing frameworks and tools (like PyTest, TensorFlow Model Analysis, MLflow, Python-based data validation libraries, Jupyter) with the ability to integrate into CI/CD pipelines
Mandatory (Business Awareness): Must understand marketplace misuse scenarios, such as manipulating recommendation algorithms, biasing fraud detection systems, or exploiting gaps in automated scoring
Mandatory (Communication Skills): Must have strong verbal and written communication skills, able to collaborate with data scientists, engineers, and business stakeholders to articulate testing outcomes and issues.
Mandatory (Education):Degree in Engineering, Computer Science, IT, Data Science, or a related discipline (B.E./B.Tech/M.Tech/MCA/MS or equivalent)
Mandatory Location: Candidate must be based within Delhi NCR (100 km radius)
Preferred
Preferred (Certifications): Certifications such as ISTQB AI Testing, TensorFlow, Cloud AI, or equivalent applied AI credentials are an added advantage.
We are seeking a results-driven Business Development professional to grow our IT staffing and staff augmentation business by acquiring new clients, managing accounts, and driving revenue through contract and permanent staffing solutions.
- Identify, prospect, and onboard new clients for IT staffing and staff augmentation services
- Understand client requirements and coordinate with recruitment teams for timely closures
- Manage end-to-end sales cycle from lead generation to contract signing (MSA/SOW)
- Build and maintain strong client relationships to ensure repeat business
- Negotiate commercial terms, rate cards, and service agreements
- Track sales pipeline and report on revenue targets using CRM tools
AngularJS (1.x)
Angular (2+)
React.js
NodeJS (Node.js)
MongoDBJob Title: MERN Stack Developer
Location: Bangalore, India
Job Type: Full Time
Experience Level: 4+ Years
About the Role:
We are looking for an experienced MERN Stack Developer to join our team in Bangalore. The ideal candidate will have over 4 years of hands-on experience in developing robust, scalable, and high-performance web applications using the MERN (MongoDB, Express.js, React.js, Node.js) stack. You will play a key role in building and maintaining our web applications, working closely with our cross-functional team to deliver exceptional software solutions.
Key Responsibilities:
- Full-Stack Development:
- Design, develop, and maintain scalable web applications using the MERN stack.
- Collaborate with UI/UX designers to translate wireframes and mockups into responsive, high-quality code.
- Develop and implement RESTful APIs to support frontend functionalities.
- Database Management:
- Design and optimize MongoDB databases to ensure high performance and scalability.
- Implement data storage solutions, ensuring security and efficiency.
- Frontend Development:
- Build user-friendly interfaces using React.js, ensuring seamless user experiences.
- Optimize frontend performance, including loading times and responsiveness across devices.
- Backend Development:
- Develop and maintain server-side logic using Node.js and Express.js.
- Implement security and data protection measures.
- Integrate third-party services and APIs.
- Testing and Debugging:
- Conduct thorough testing of applications, including unit tests, integration tests, and end-to-end tests.
- Identify and fix bugs, performance issues, and other technical problems.
- Collaboration and Communication:
- Work closely with product managers, designers, and other developers to deliver high-quality features.
- Participate in code reviews and contribute to a culture of continuous improvement.
- Continuous Learning and Improvement:
- Stay updated with the latest industry trends, tools, and technologies.
- Contribute to team knowledge sharing and mentoring of junior developers.
Qualifications:
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- Minimum of 4 years of experience in full-stack development with expertise in the MERN stack.
- Strong proficiency in JavaScript, HTML5, CSS3, and related frontend technologies.
- Experience with MongoDB, including schema design and query optimization.
- Proficiency in building RESTful APIs using Node.js and Express.js.
- Familiarity with version control systems, particularly Git.
- Knowledge of cloud services (e.g., AWS, Azure) is a plus.
- Excellent problem-solving skills and attention to detail.
- Strong communication and collaboration skills.
Preferred Qualifications:
- Experience with Agile/Scrum methodologies.
- Knowledge of Typescript, Redux, or similar state management tools.
- Familiarity with CI/CD pipelines and DevOps practices.
- Experience with Docker or containerization.
Robotics trainer ---
* Hands on experience in technical content writing.
* Embedded C, microcontrollers like ARM, PIC, AVR.
* Communication protocols like CAN, I2C, SPI.
* Basics of Robo control mechanism.
* ADC nad DAC interfacing with sensors.
* Good Communication skills.
AngularJS (1.x)
Vue.jsRequirement :- Java Tech Lead (with knowledge of Angular)
- 5+ years of experience into Java 6/7/8, J2EE, EJB, Servlets, JSP, JDBC, LOG4J, JSON, XML
- 3+ years of experience in Spring Boot, Sprint Rest, Spring MVC.
- 5+ years of experience in REST, Micro service Development
- Knowledge of Design Patterns (Good to Have).
- Proven Experience in building complex web systems that have been successfully delivered to customers
- Experience in writing SQL-queries and knowledge of PL/SQL
- Experience in working with databases Oracle 10g, 11g, Postgres, Cassandra etc
- Exposure/Experience in working/setting up CI and CD environments (DevOps).
- Knowledge of development lifecycle, standards, approaches, utilities and environment: shells, version control systems, IDEs, project assembling tools;
Responsibilities:
•Participate in requirement, design, implementation, testing and support of custom banking solutions, which include features enhancements, new functionalities, and back-end systems integration.
•Define and manage resource plan together with the Project Manager aligning to the project schedule.
•Plan and drive the project's technical execution to achieve the committed scope and timeline.
•Monitor and report technical execution progress as well as all technical issues, risk or mitigation plan in a timely manner.
•Ability to coordinate multi-teams (e.g. customers, internal functional and technical SMEs), to understand business requirements, technical design and preparing of technical specifications.
•Assist in estimation of efforts and timescales for tasks delegated to the team.
•Contribute to design/code reviews for quality assurance and conformance to Company standards / guidelines.
•Hands-on participate in development and bugs fixing.
•Commit to continuous improvement and refinement of delivery.
•Strong technical knowledge and skills, consistently provide technical guidance and develop team members.
•Responsible for forward resource management, team building and staff career growth.
Requirements:
•Degree in Computer Sciences / Software Engineering / Information Technology.
•At least 5 years of working experience in Java EE development (preferably in the financial industry).
•Individual that is optimistic, proactive, good initiative, passionate, strives to achieve more as a team, and is able to motivate others.
•Understand design documentation standards such as Unified Modelling Language.
•Experience in various Java EE frameworks and design patterns. Preferably on Micro services architecture.
•Exposure in Java MVC frameworks such as Spring Boot.
Work Location: Hyderabad
Experience:3 to 5 Years
Package:Upto 10 LPA
Product Based Company
Notice Period:Immediate Joiners.
Its a Full Time Opportunity with Our Client
Mandatory Skills: Sterling SI B2B,EDI,ANSI X12/EDIFACT,XML & Sterling B2B Integrator (SI)
Job Description:
•Over 3 -5 years of work experience in Sterling SI B2B version 5.2.6 and above.
•Build and configure Sterling B2B integrator.
•Proficiency on B2B/EDI concepts, ANSI X12/EDIFACT standards.
•Thorough understanding and hands on experience with XML, Schema, XPATH and XML parsers is required.
•Good understanding of flat, csv data processing.
•Understand trading partners onboarding process.
•Experience in IBM Sterling SI Map Editor and Business Processes development.
•Proficient in various EDI transactions related to Retail Energy flow like 814, 810, 867,820, 568, 997 etc.
•Knowledge of various connectivity protocols like FTP, SFTP, AS2, HTTP protocols etc.
•Experience in Mapping various formats like X12 to XML, Flat files to XML and vice-versa.
•Should be able to understand and create mapping specifications/layouts.
•Good in Business Process development or troubleshooting, have good knowledge and understanding on adapters and services configuration/development.
•Architecture design and development in EDI, standard document and communication protocols, utilizing Sterling B2B Integrator (SI).
•Experience using NAESB and GISB protocol will be advantage.
•Experience using any type of Framework on top of SI will be advantage.
•Experience of using EDI systems installed on Linux/Unix platform.
•Good understanding of cloud platforms like MS Azure, AWS.
Android Development
KotlinKeepTruckin is on a mission to modernize the trucking industry. With the leading fleet management platform, we are bringing trucks online and fundamentally changing the way freight is moved on our roads.We see our hard work rewarded in tangible ways every day and we believe that intelligence is most powerful when paired with humility. We're motivated by the opportunity to impact and improve every facet of a trillion-dollar industry that touches everyone's lives. KeepTruckin is proud to be a Forbes Cloud 100 company, a 2020 Career-Launching Company by Wealthfront and named a Forbes Best Startup Employer 2020.About the
Role:
As one of our first mobile engineers in Bangalore, this is a great opportunity to make a significant impact in designing and building our new KeepTruckin Fleet Manager Android app and as well as grow by working directly with experienced technology professionals with backgrounds from Silicon Valley companies like Google, Twitter, Facebook and Uber (to name a few). We are looking for candidates who have a familiarity with Java and Kotlin, and a good understanding of system architectures (MVC, MVVM, Redux) and a forward thinking mindset aimed at high readability (declarative) code, as well as identifying problems or bottlenecks before they become issues.
What You'll Do:
Develop from scratch our new Android KeepTruckin Fleet Manager AppWork with product team to refine feature specs, and work with cross functional teams like IoT, Embedded, Data Science/ML, Computer Vision & Motion teams to create fluid user experiences with interactive visualization for a multitude of data from these different teamsCommunicate with backend engineers about API requirements or refinements, and engage with the senior engineers for mentorship to learn & grow in your areas of interestAnalyze our user's data to identify bugs, and help support team to diagnose user problemsBe able to work as part of a small, tightly knit team
What We're Looking For:
2-4 years of experience working on Android apps with a deep understanding of Android SDKsProficiency in Java, Kotlin, Android lifecycle, custom views, database/persistenceAn understanding of Android UI/UX and application design paradigms. An appreciation and understanding of automated app testingKnowledge in RxJava2, Dagger2, and knowledge or interest in commercial vehicles/trucking is a plus!
Creating a diverse and inclusive workplace is one of KeepTruckin's core values. We are an equal opportunity employer and welcome people of different backgrounds, experiences, abilities and perspectives.Please review our Candidate Privacy Notice here. ( https://drive.google.com/file/d/1Xs7GivrHCj05XC6-mn6MsRGG3V5AcgZ5/view?usp=sharing )
C#
JavascriptAngular (2+)NodeJS (Node.js)
PostgreSQLJavascript
Amazon Web Services (AWS)
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon