
Data Engineer
at consulting & implementation services in the area of Oil & Gas, Mining and Manufacturing Industry
- Data Engineer
Required skill set: AWS GLUE, AWS LAMBDA, AWS SNS/SQS, AWS ATHENA, SPARK, SNOWFLAKE, PYTHON
Mandatory Requirements
- Experience in AWS Glue
- Experience in Apache Parquet
- Proficient in AWS S3 and data lake
- Knowledge of Snowflake
- Understanding of file-based ingestion best practices.
- Scripting language - Python & pyspark
CORE RESPONSIBILITIES
- Create and manage cloud resources in AWS
- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies
- Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform
- Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations
- Develop an infrastructure to collect, transform, combine and publish/distribute customer data.
- Define process improvement opportunities to optimize data collection, insights and displays.
- Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible
- Identify and interpret trends and patterns from complex data sets
- Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders.
- Key participant in regular Scrum ceremonies with the agile teams
- Proficient at developing queries, writing reports and presenting findings
- Mentor junior members and bring best industry practices
QUALIFICATIONS
- 5-7+ years’ experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales)
- Strong background in math, statistics, computer science, data science or related discipline
- Advanced knowledge one of language: Java, Scala, Python, C#
- Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake
- Proficient with
- Data mining/programming tools (e.g. SAS, SQL, R, Python)
- Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
- Data visualization (e.g. Tableau, Looker, MicroStrategy)
- Comfortable learning about and deploying new technologies and tools.
- Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines.
- Good written and oral communication skills and ability to present results to non-technical audiences
- Knowledge of business intelligence and analytical tools, technologies and techniques.
Familiarity and experience in the following is a plus:
- AWS certification
- Spark Streaming
- Kafka Streaming / Kafka Connect
- ELK Stack
- Cassandra / MongoDB
- CI/CD: Jenkins, GitLab, Jira, Confluence other related tools

Similar jobs
At BigThinkCode, we build innovative technology solutions that solve complex enterprise challenges. We are looking for a highly skilled Go/Golang Engineer who thrives in building low-latency, distributed systems and enjoys working on security sensitive platforms.
This role offers the opportunity to work on cutting-edge SaaS platforms, building scalable backend systems that operate in demanding environments requiring high reliability, security, and performance.
Please find below our job description, if interested apply / reply sharing your profile to connect and discuss.
Company: BigThinkCode Technologies
URL: https://www.bigthinkcode.com/
Experience required: 3-6 years
Work location: Chennai
Joining time: Immediate – 4 weeks
Work Mode: Work from office / Hybrid
Must have skills:
- Hands on expertise in Go / Golang, low-latency routing, distributed systems.
- Experience designing and troubleshooting low-latency applications.
- Experience in enterprise software deploy / developing SaaS application is a plus
- Event drive architecture (EDA) microservices project experience is a plus.
- Additional programming knowledge: Typescript or NodeJS or Python
- Hands on experience using docket containers and Kubernetes.
- Aware about Observability, Asynchronous and Loose Coupling.
- Familiar with deep knowledge on infrastructure and customer application deployment
- Work with message brokers and streaming technologies such as Kafka, or similar event-driven platforms.
A skilled Go/Golang Developer with 3–6 years of experience in building scalable applications using event-driven architecture within distributed systems. The ideal candidate should be comfortable in low level programming, high-performance backend services and have a strong understanding of Observability, asynchronous programming patterns.
Experience or exposure to knowledge embedding and vectorization techniques is a strong plus, especially in the context of modern data-driven or AI-powered systems.
Required Skills
- 4+ years of hands-on experience with Go / Golang
- Strong engineer with security-sensitive, correctness-critical code paths
- Engineer worked on SAAS / On-prem environment
- Experience with containerization tools like Docker and orchestration tools like Kubernetes
Ideal Candidate
We are looking for engineers who:
- Have strong ownership mindset and platform thinking.
- Can balance correctness, performance, and simplicity under pressure.
- Are comfortable making architectural decisions in distributed environments.
- Use AI-assisted development tools effectively while maintaining engineering judgment.
- Can distinguish robust production-grade solutions from generated boilerplate code.
- Have experience supporting SaaS, hybrid-cloud, or on-premise deployments.
Why Join Us:
- Collaborative work environment.
- Exposure to modern tools and scalable application architectures.
- Medical cover for employee and eligible dependents.
- Tax beneficial salary structure.
- Comprehensive leave policy
- Competency development training programs.
Job Title: Senior Software Engineer
Location: Bangalore
Experience: 3–5 Years
About the Role
We are looking for a highly skilled Senior Sotware Engineer to design, develop, and scale robust backend systems for enterprise-grade applications. The ideal candidate should have strong experience in microservices architecture, cloud-native applications, distributed systems, and API development while also being capable of mentoring engineering teams and driving technical decisions.
Key Responsibilities
• Design and develop scalable backend systems and microservices. • Lead architecture discussions and technical decision-making across projects. • Build high-performance REST APIs and event-driven systems. • Collaborate with frontend, product, and DevOps teams to deliver end-to-end solutions. • Perform code reviews and establish engineering best practices. • Design database schemas and optimize application performance. • Build real-time systems using WebSockets, Redis, and messaging patterns. • Manage deployments, CI/CD pipelines, and cloud infrastructure. • Mentor junior and mid-level developers and contribute to sprint planning and delivery.
Required Skills
Backend Technologies • Node.js • TypeScript / JavaScript • Express.js • NestJS • Python • Django / FastAPI • Go (Golang) • Laravel (PHP)
Frontend Exposure • React.js • Next.js
Databases • PostgreSQL • MongoDB • Redis • Elasticsearch
Cloud & DevOps • AWS • GCP • Docker • Nginx • GitHub Actions • CI/CD Pipelines
Architecture & Design • Microservices Architecture • Event-Driven Systems • Distributed Systems • REST APIs • WebSockets • Caching Strategies • Scalability & Performance Optimization
Leadership Responsibilities
• Lead a team of developers across multiple projects. • Conduct code reviews and technical mentoring. • Drive engineering standards, architecture patterns, and development processes. • Collaborate with stakeholders for technical planning and estimation. • Ensure timely delivery while maintaining code quality.
Preferred Qualifications
• Experience building systems serving high user volumes. • Experience in fintech, banking, SaaS, hospitality, or IoT domains. • Hands-on experience with cloud cost optimization and performance tuning. • Exposure to AI integrations and modern software architecture patterns.
Education
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
Why Join Us
• Opportunity to work on large-scale enterprise applications. • Exposure to modern cloud-native technologies. • Leadership opportunities with direct impact on product and engineering decisions. • Collaborative and growth-oriented work environment.
Responsibilities
- Developing user-facing applications using Vue.js
- Building modular and reusable components and libraries
- Optimizing your application for performance
- Implementing automated testing integrated into development and maintenance workflows
- Staying up-to-date with all recent developments in the JavaScript and Vue.js space
- Keeping an eye on security updates and issues found with Vue.js and all project dependencies
- Proposing any upgrades and updates necessary for keeping up with modern security and development best practices
Skills
- Highly proficient with the JavaScript language and its modern ES6 syntax and features
- Highly proficient with Vue.js framework and its core principles such as components, reactivity, and the virtual DOM
- Familiarity with the Vue.js ecosystem, including Vue CLI, Vuex, Vue Router, and Nuxt.js
- Understanding of server-side rendering and its benefits and use cases
- Knowledge of functional programming and object-oriented programming paradigms
- Ability to write efficient, secure, well-documented, and clean JavaScript code
- Familiarity with automated JavaScript testing, specifically testing frameworks such as Jest or Mocha
- Proficiency with modern development tools, like Babel, Webpack, and Git
- Experience with both consuming and designing RESTful APIs
● 11–15 years of marketing experience, preferably in B2B SaaS.
● Minimum 5–6 years of people management experience, with a demonstrated ability to build and lead
high-performing teams.
● Strong experience in customer marketing and lifecycle marketing.
● Deep understanding of modern marketing tools (e.g., HubSpot, Salesforce, Marketo, Pardot, etc.).
● Strong project management skills with the ability to handle multiple priorities in a fast-paced
environment.
● Excellent communication skills — both written and verbal.
● Analytical mindset with a data-driven approach to decision-making.
● SaaS experience at the SMB, Mid Market, and Enterprise level strongly preferred.
● Leadership experience (managed teams)
● Knowledge of customer success platforms and retention analytics.
● Prior experience working in remote and globally distributed teams.
● Passion for storytelling and showcasing customer success.
● Demonstrated experience using AI tools for marketing tasks such as content ideation and creation.
● Experience with Canva templates or similar tools to streamline visual content creation.
● Experience developing and aligning marketing initiatives with OKRs.
Mandatory (Experience 1) - Must have a minimum 4+ years of experience in backend software development.
Mandatory (Experience 2) -Must have 4+ years of experience in backend development using Python (Highly preferred), Java, or Node.js.
Mandatory (Experience 3) - Must have experience with Cloud platforms like AWS (highly preferred), gcp or azure
Mandatory (Experience 4) - Must have Experience in any databases - MySQL / PostgreSQL / Postgres / Oracle / SQL Server / DB2 / SQL / MongoDB / Ne
Recruiterflow is an AI-first RecOps platform built for ambitious recruiting agencies. We offer an integrated ATS & CRM with powerful automation features to streamline recruiting operations. Our mission is to transform how recruiting agencies function by turning hiring into a structured sales process—helping them close roles faster, engage top talent better, and scale with confidence. We’re already among the top 5 players in the industry, and our next milestone is to break into the top 3 within the next three years.
Key responsibilities:
- Research, ideate and develop long-form and short-form content such as articles, blogs, white papers, case studies, press communication, thought-leadership insights, marketing campaigns and collateral.
- Develop a content calendar and manage content generation targets in line with the organisation wide content strategy.
- Deliver content for various stages of the Buyer’s Journey – TOFU, MOFU and BOFU
- Help the marketing team to devise and execute the marketing activities by providing content for social media, nurture campaigns, videos, etc.
- Build and own Reddit as a distribution channel for brand content
- Ensure all content is in line with SEO standards and practices.
- Quality check the content to ensure it is free of grammatical or factual errors and follows all brand editorial guidelines for consistency.
- Ensure targeted delivery of content and meet predefined organic KRAs.
Ideal Candidate:
- 4+ years of overall writing experience, including at least 2 years specifically in B2B content writing or as a SaaS content writer.
- Strong verbal and written communication skills.
- Excellent copywriting skills.
- Exceptional ideation, research, and writing abilities.
- Ability to work under tight deadlines.
- Hustler attitude to work
Overview of the Job:
We are looking for developers who are passionate about working in a highly entrepreneurial setup
developing products for enterprise clients and vendors in the Supply chain finance paradigm.
You will join a dynamic and fast-paced environment and work with cross-functional teams to design, develop and build products that deliver the company’s vision and strategy.
Overview of the Function:
The incumbent will work closely with the engineering leads and engineering managers along
with the product team on deploying and testing existing and potential products.
The Product and Engineering team conceptualizes designing, develop and customize
products.
Reporting to:
The incumbent will be working closely with the Engineering Lead.
Best things about the job:
• Working in a highly entrepreneurial setup with a visionary team passionate to help scale new
heights of business success.
• Exposure to exploring limitless possibilities and ideas no matter how impossible they may
seem today.
Your Role:
• Manage design projects across the entire product lifecycle and multiple product launches
• Prioritizing and articulate product and product features, and working cross-functionally with
engineers, product manager and Design Team to bring these products and features to reality
• Hands-on with design and challenges in interfaces
• Rapidly produce multiple concepts and prototypes; knowing when to apply pixel-perfect
attention to detail, and when to make low-fi sketches and prototypes.
• Making things happen execute good ideas that come from team, and make sure that the
whole team is doing its best work.
• Comfortable in working for interface designing
Skill Set:
• Expert Knowledge on Angluar Framework (8 and above)
• Hands On expertise on HTMS , CSS and JS.
- Strong knowledge in OOPS Concept,Data Structures and Algorithms
- Expertise in Mobile Automation with XCUI ( XCUI Tests Scripting, Test Framework Knowledge, Scheme Creation, Cloud Device Management, XCUI Performance and Profiling, XCUI Coverage and SonarCube) and Xcode
- Experience in swift and ios life cycle
- Experience in Appium Framework
- Understanding of Backend architecture, mysql/nosql, mongo, Redis, Zookeeper

Skills required:
•4- 8 + Years of experience in automation with Selenium Cucumber framework and C# /
Python
• Candidates must be able to write test plans, test scenarios, test cases and do
exploratory testing, be process-oriented and above all, do a good job at documenting
the work done.
• Demonstrate strong grip on understanding business requirements, technical aspects of
software testing like - security and performance in addition to the general software
testing best practices.
• Hands-on & experienced with web and mobile applications testing using manual and
automation tools.
• Should be strong in setting up Selenium Test environment (Selenium Webdriver, C#,
MSTest, Nunit and Azure Devops pipeline)
• Must have CI/CD tool exposure (Preferably Azure Devops)
• Hands on experience in mobile app automation (iOS and Android)
• Hands on experience in Web Service testing using RestAssured/SoapUI/Postman
• Experience with load testing and knowledge of load testing tools like Jmeter would be
a plus.
• Experience with Security testing using ZAP tool will be plus
• Exposure to reporting Framework – Extent Report, Allure, etc.
• Building Automation Framework, adapt to new Languages and Tools, Solution
Provider, also wider thought process in understanding of Service Virtualization and
Performance Testing.
• Understanding on upcoming Technologies and Tools, should be able to provide
hybrid / Custom Solutions. Should have built Custom Tools for Automation.
• Awareness in creating Test Automation Frameworks for large-scale distributed
systems\cloud infrastructure.
• Familiarity with configuration management tools for software development (Azure
Devops server)
• Responsible for driving automation test strategy
• Should be strong in DB concepts
• Vast experience in testing with external APIs and SDKs
Education
Bachelor or Masters Education in electronics, computer, software engineering or other
relevant discipline.
JD:
Required Skills:
- Intermediate to Expert level hands-on programming using one of programming language- Java or Python or Pyspark or Scala.
- Strong practical knowledge of SQL.
Hands on experience on Spark/SparkSQL - Data Structure and Algorithms
- Hands-on experience as an individual contributor in Design, Development, Testing and Deployment of Big Data technologies based applications
- Experience in Big Data application tools, such as Hadoop, MapReduce, Spark, etc
- Experience on NoSQL Databases like HBase, etc
- Experience with Linux OS environment (Shell script, AWK, SED)
- Intermediate RDBMS skill, able to write SQL query with complex relation on top of big RDMS (100+ table)










