Data Engineer

Mandatory (Experience 1) - Must have a minimum 4+ years of experience in backend software development.

Mandatory (Experience 2) -Must have 4+ years of experience in backend development using Python (Highly preferred), Java, or Node.js.

Mandatory (Experience 3) - Must have experience with Cloud platforms like AWS (highly preferred), gcp or azure

Mandatory (Experience 4) - Must have Experience in any databases - MySQL / PostgreSQL / Postgres / Oracle / SQL Server / DB2 / SQL / MongoDB / Ne

Role & Responsibilities:

We are seeking a Software Developer with 2-10 year’s experience with strong foundations in Python, databases, and AI technologies. The ideal candidate will support the development of AI-powered solutions, focusing on LLM integration, prompt engineering, and database-driven workflows. This is a hands-on role with opportunities to learn and grow into advanced AI engineering responsibilities.

Key Responsibilities

• Develop, test, and maintain Python-based applications and APIs.

• Design and optimize prompts for Large Language Models (LLMs) to improve accuracy and performance.

• Work with JSON-based data structures for request/response handling. • Integrate and manage PostgreSQL (pgSQL) databases, including writing queries and handling data pipelines.

• Collaborate with the product and AI teams to implement new features. • Debug, troubleshoot, and optimize performance of applications and workflows.

• Stay updated on advancements in LLMs, AI frameworks, and generative AI tools.

Required Skills & Qualifications

• Strong knowledge of Python (scripting, APIs, data handling).

• Basic understanding of Large Language Models (LLMs) and prompt engineering techniques.

• Experience with JSON data parsing and transformations.

• Familiarity with PostgreSQL or other relational databases.

• Ability to write clean, maintainable, and well-documented code.

• Strong problem-solving skills and eagerness to learn.

• Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent practical experience).

Nice-to-Have (Preferred)

• Exposure to AI/ML frameworks (e.g., LangChain, Hugging Face, OpenAI APIs).

• Experience working in startups or fast-paced environments.

• Familiarity with version control (Git/GitHub) and cloud platforms (AWS, GCP, or Azure).

What We Offer

• The opportunity to define the future of GovTech through AI-powered solutions.

• A strategic leadership role in a fast-scaling startup with direct impact on product direction and market success.

• Collaborative and innovative environment with cross-functional exposure.

• Growth opportunities backed by a strong leadership team.

• Remote flexibility and work-life balance.

• Build campaign generation services which can send app notifications at a speed of 10 million a minute
• Dashboards to show Real time key performance indicators to clients
• Develop complex user segmentation engines which creates segments on Terabytes of data within few seconds
• Building highly available & horizontally scalable platform services for ever growing data
• Use cloud based services like AWS Lambda for blazing fast throughput & auto scalability
• Work on complex analytics on terabytes of data like building Cohorts, Funnels, User path analysis, Recency Frequency & Monetary analysis at blazing speed
• You will build backend services and APIs to create scalable engineering systems.
• As an individual contributor, you will tackle some of our broadest technical challenges that requires deep technical knowledge, hands-on software development and seamless collaboration with all functions.
• You will envision and develop features that are highly reliable and fault tolerant to deliver a superior customer experience.
• Collaborating various highly-functional teams in the company to meet deliverables throughout the software development lifecycle.
• Identify and improvise areas of improvement through data insights and research.

• 2-5 years of experience in backend development and must have worked on Java/shell/Perl/python scripting.
• Solid understanding of engineering best practices, continuous integration, and incremental delivery.
• Strong analytical skills, debugging and troubleshooting skills, product line analysis.
• Follower of agile methodology (Sprint planning, working on JIRA, retrospective etc).
• Proficiency in usage of tools like Docker, Maven, Jenkins and knowledge on frameworks in Java like spring, spring boot, hibernate, JPA.
• Ability to design application modules using various concepts like object oriented, multi-threading, synchronization, caching, fault tolerance, sockets, various IPCs, database interfaces etc.
• Hands on experience on Redis, MySQL and streaming technologies like Kafka producer consumers and NoSQL databases like mongo dB/Cassandra.
• Knowledge about versioning like Git and deployment processes like CICD.

Level of skills and experience:

5 years of hands-on experience in using Python, Spark,Sql.

Experienced in AWS Cloud usage and management.

Experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow).

Experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch.

Experience with orchestrators such as Airflow and Kubeflow.

Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).

Fundamental understanding of Parquet, Delta Lake and other data file formats.

Proficiency on an IaC tool such as Terraform, CDK or CloudFormation.

Strong written and verbal English communication skill and proficient in communication with non-technical stakeholderst

Experience:

The candidate should have about 2+ years of experience with design and development in Java/Scala. Experience in algorithm, data-structure, database and distributed System is mandatory.

Required Skills:

Mandatory: -

1. Core Java or Scala
2. Experience in Big Data, Spark
3. Extensive experience in developing spark job. Should possess good Oops knowledge and be aware of enterprise application design patterns.
4. Should have the ability to analyze, design, develop and test complexity of spark job.
5. Working knowledge of Unix/Linux.
6. Hands on experience in Spark, creating RDD, applying operation - transformation-action

Good To have: -

1. Python
2. Spark streaming
3. Py Spark
4. Azure/AWS Cloud Knowledge of Data Storage and Compute side