Data Engineer

Job Summary

We are looking for a skilled and experienced Data Engineer to join our growing data team. The ideal candidate will have strong expertise in Python, PySpark, Data Modeling, and Power BI, with hands-on experience in designing, developing, and optimizing scalable data solutions. The role requires working closely with business stakeholders, data architects, and analytics teams to build robust data pipelines and semantic models that enable data-driven decision-making.

Technical Skills

• Strong hands-on experience in Python and PySpark development.
• Expertise in building and optimizing Data Engineering solutions and ETL pipelines.
• Strong understanding of Data Modeling concepts (Star Schema, Snowflake Schema, Dimensional Modeling).
• Experience with Power BI Data Modeling and Semantic Layer development.
• Proficiency in DAX (Data Analysis Expressions).
• Experience designing and managing Semantic Models in Power BI.
• Strong SQL skills and experience working with large datasets.
• Knowledge of data warehousing concepts and best practices.

Preferred Skills

• Experience with cloud platforms such as Azure, AWS, or GCP.
• Exposure to modern data platforms like Databricks.
• Understanding of data governance and data quality frameworks.

About the Role

We are seeking motivated Data Engineering Interns to join our team remotely for a 3-month internship. This role is designed for students or recent graduates interested in working with data pipelines, ETL processes, and big data tools. You will gain practical experience in building scalable data solutions. While this is an unpaid internship, interns who successfully complete the program will receive a Completion Certificate and a Letter of Recommendation.

Responsibilities

• Assist in designing and building data pipelines for structured and unstructured data.
• Support ETL (Extract, Transform, Load) processes to prepare data for analytics.
• Work with databases (SQL/NoSQL) for data storage and retrieval.
• Help optimize data workflows for performance and scalability.
• Collaborate with data scientists and analysts to ensure data quality and consistency.
• Document workflows, schemas, and technical processes.

Requirements

• Strong interest in data engineering, databases, and big data systems.
• Basic knowledge of SQL and relational database concepts.
• Familiarity with Python, Java, or Scala for data processing.
• Understanding of ETL concepts and data pipelines.
• Exposure to cloud platforms (AWS, Azure, or GCP) is a plus.
• Familiarity with big data frameworks (Hadoop, Spark, Kafka) is an advantage.
• Good problem-solving skills and ability to work independently in a remote setup.

What You’ll Gain

• Hands-on experience in data engineering and ETL pipelines.
• Exposure to real-world data workflows.
• Mentorship and guidance from experienced engineers.
• Completion Certificate upon successful completion.
• Letter of Recommendation based on performance.

Internship Details

• Duration: 3 months
• Location: Remote (Work from Home)
• Stipend: Unpaid
• Perks: Completion Certificate + Letter of Recommendation

Data Engineer Short Hiring Post

🚨 Hiring: Data Engineer

🔹 Experience: 5–9 Years

🔹 Location: Bangalore / Hyderabad

🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling

🔹 Process: L1 Virtual → L2 F2F Karat Test

🔹 F2F: Bangalore / Hyderabad Location

🔹 Positions: Immediate requirement

⚠️ Note: Candidates must be available for F2F Karat immediately after L1.

#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners

Must-Have Skills

• Minimum 3 years of experience in Data Engineering / Analytics Engineering / Fintech Data roles
• Must have worked on SMS Parsing, intelligent platform, converting RAW customer SMS data into structured actionable financial signals and enabling downstream usage of SMS derived variables
• Must have established a continuous learning cycle to expand parser coverage
• Experience in Lending / NBFC / Fintech domain
• Experience working with Bureau, SMS, Device, or Banking data
• Strong Python and SQL (production level)
• Experience handling unstructured data (SMS, logs, JSON, APIs)
• Experience building data pipelines, schedulers, and cron jobs
• Strong database design and data modelling skills
• Ability to work in a startup environment with high ownership
• Familiarity with modern platforms like AWS, Snowflake, Google BigQuery, Redshift

Good to Have

• Experience in STPL, especially less than 25K ticket size
• Experience with streaming (Kafka/Kinesis) and orchestration (Airflow or Step Functions)
• Experience with feature stores and risk analytics datasets
• Knowledge of regex, NLP basics for SMS parsing
• Experience supporting real-time decision engines/underwriting systems

Role Summary

This role will be responsible for owning the end-to-end data-structuring layer across the organisation. The individual will transform large volumes of raw, unstructured, and semi-structured data (such as SMS, device, bureau, and app data) into clean, standardised, and analysis-ready datasets. These structured datasets will directly power risk analytics, fraud detection, marketing insights, collections strategy, and policy decisioning.

Key Objective of the Role

Ensure all raw lending data (SMS, Bureau, Device, AA, App logs) is captured, parsed, structured, and stored in a clean analytics-ready format inside databases (PostgreSQL, DynamoDB, AWS stack) so that the Risk and Data Science team can directly use it for feature creation, policy building, and portfolio monitoring.

Core Responsibilities

1. End-to-End Data Ownership

• Design, build, and maintain end-to-end data pipelines (batch + streaming) using AWS native services (Glue, Lambda, Step Functions, Kinesis, S3, Athena, Redshift, EMR/Spark, etc.): ingestion

→ parsing → structuring → storage

• Work closely with Tech, Product, and Data Science to define what data should be captured
• Maintain data documentation, data dictionaries, and schema governance
• Ensure data quality, consistency, and version control

1. Unstructured Data Processing (Highest Priority)

• Parse raw SMS dumps and categorise into salary, EMI, loan apps, collections, credits, debits, OTP, etc.
• Process device fingerprint, behavioural logs, and vendor data (FinBox, AA, Bureau APIs)
• 
  
• 
  
• Convert JSON, logs, and raw API responses into structured feature tables
• Build regex/keyword-based parsers for financial SMS classification

1. Feature Implementation (From Risk & Data Science Team)

• Implement feature creation logic provided by Risk/Data Science team
• Translate business and policy logic into SQL/Python pipelines
• Create reusable feature layers for underwriting, fraud, collections, and monitoring
• Maintain a feature store for consistent model and policy usage

1. Lending Data Understanding (Domain-Specific Requirement)

• Work with Bureau data
• Structure SMS-derived financial variables (income, stress, EMI signals)
• Work with Account Aggregator and bank transaction datasets
• Understand fintech alternate data used in underwriting and fraud detection

1. Data Pipelines & Automation

• Build and maintain ETL/ELT pipelines using Python & SQL
• Create cron jobs for automated data ingestion and feature refresh
• Automate vendor data pulls (Bureau, SMS SDK, AA, device data)
• Ensure low-latency pipelines for real-time underwriting use cases

1. Database Structuring & Storage Architecture

• Structure clean datasets in PostgreSQL (analytics layer)
• Manage raw data storage in DynamoDB / S3 data lake
• Design normalized and denormalised tables for risk analytics
• Optimise database performance for large-scale query workloads

1. Dashboards & Readable Data Layer

• Create analytics-ready datasets, implement & write Metabase queries and convert into dashboards (Metabase / Power BI)
• Enable self-serve data access for Risk, Business, and Founders
• Support ad-hoc analysis requirements from leadership

1. Cross-Functional Collaboration (Very Important)

• The role requires close collaboration with data science, tech, product, and business teams to ensure reliable data pipelines, well-defined schemas, API integrations, logging architecture and high data quality, enabling faster and more accurate decision-making across lending workflows.

Tech Stack (Current Environment)

• AWS Services
• PostgreSQL (Primary analytics DB)
• DynamoDB (Raw/NoSQL storage)
• Python (Pandas, NumPy, ETL frameworks)
• Advanced SQL
• APIs, JSON, and Log Data Handling

About Us

Corporate Web Solutions works on technology-driven digital solutions involving data, automation, web technologies, and artificial intelligence. Our internship programs focus on practical learning and real-world project exposure.

Role Overview

As a Data Science Intern, you'll work with datasets to perform analysis, visualization, and machine learning tasks while learning modern AI-assisted workflows.

Key Responsibilities

• Collect, clean, and analyze datasets.
• Perform exploratory data analysis.
• Create data visualizations and reports.
• Assist in developing machine learning models.
• Work with Python-based data science tools.
• Explore AI tools for data analysis and productivity.

Requirements

• Basic knowledge of Python.
• Understanding of data analysis fundamentals.
• Familiarity with Pandas and NumPy is a plus.
• Basic understanding of statistics.
• Analytical and problem-solving skills.

Perks

• Certificate of Internship
• Flexible work hours
• Mentorship and real project exposure
• Potential for PPO
• Letter of Recommendation
• Performance-Based Stipend available up to ₹18,000/month

Role Summary

We are hiring a Data Engineer / ML Data Pipeline Engineer to build and operate the data backbone of the Enterprise AI platform:

What You'll Own

• Ingestion & ETL/ELT pipelines for heterogeneous project folders (PDF drawings, SVG files, IFC models, BBS.json bar-bending-schedule data, Excel exports, and AI agent output JSON).
• AWS-based data architecture: S3 raw/staging/curated/outputs structuring, partitioning, versioning, and lifecycle management; querying via Athena/Glue and warehousing via Redshift or Snowflake as needed.
• Data validation frameworks: GUID cross-referencing between SVG and BBS data, schema enforcement, duplicate/orphan detection, reference integrity checks, and structured validation reporting.
• Agent run logging & observability: designing the database schema and pipelines that track every AI agent run (inputs, outputs, status, errors, cost, retries, reviewer feedback).
• AI Factory monitoring dashboards: operational dashboards (failure rates, retries, latency, data quality) and business dashboards (throughput, cost per run, rework rate) for Power BI/QuickSight or equivalent.
• ML data pipeline support: dataset preparation, labeling/annotation workflows, human-in-the-loop review tooling, and dataset versioning for models that classify or QC drawing issues.
• APIs: designing and building FastAPI/Flask endpoints to trigger validation runs and expose agent processing status to internal tools.
• Data quality & testing discipline: idempotent pipelines, quarantine/reject handling, regression and reconciliation testing, and root-cause debugging when pipelines or query performance degrade in production.

Key Skills — Non-Negotiable (Must-Have, Strong Level)

• Python — production-grade scripting: file/folder handling, JSON/schema processing, clean error handling, not just notebook-level scripting.
• SQL — strong hands-on ability, including GROUP BY/HAVING for duplicate detection, window functions, and daily aggregate/rate calculations (e.g., success-rate queries).
• AWS S3 data handling — practical experience structuring buckets for raw/staging/curated data, versioning, and avoiding overwrite issues at scale.
• Data validation — demonstrable experience building validation logic (set comparisons, duplicate/missing detection, structured pass/fail reporting), not just "I write assertions."
• ETL/ELT pipeline design — end-to-end ownership of at least one pipeline: source → transform → storage → validation → monitoring → business outcome, with clear articulation of what they personally built.
• Query/warehouse engine judgment — working knowledge of when to use Athena vs. Redshift vs. Snowflake (or equivalent), partitioning, clustering, sort/distribution keys, and storage format trade-offs (Parquet vs. JSON vs. CSV).

Key Skills — Good to Have

• Dashboarding — Power BI / QuickSight (or equivalent) fact/dimension table design, KPI cards, drill-downs; medium-to-strong level is a plus but trainable.
• FastAPI / Flask — building real endpoints with request/response schemas and basic error handling; especially valuable for validation-trigger and agent-status APIs.
• ML data pipeline experience — dataset labeling, annotation platform design, train/test/validation splitting, dataset versioning; strong on the pipeline/data side rather than model training itself.
• Human-in-the-loop / review tooling — experience building or contributing to browser-based labeling/review platforms (session persistence, label schema, export formats).
• Large-scale metadata querying — experience making file discovery fast across large volumes (1,000+ projects, thousands of files each) via metadata index tables, event-based ingestion, or catalog tools like AWS Glue.

Job Summary

Role Overview

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, Advanced SQL, CI/CD, DevOps, and Data Analytics. The ideal candidate should have hands-on experience designing and developing scalable data pipelines, transforming large datasets, and supporting data-driven applications.

Experience with Google Cloud Platform (GCP) will be an added advantage.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
• Develop complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain reliable data integration workflows across multiple data sources.
• Perform data cleansing, validation, transformation, and quality checks.
• Analyze data and provide insights to support business and technical requirements.
• Implement and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
• Troubleshoot data pipeline failures, performance issues, and production incidents.
• Optimize data processing workflows for performance, scalability, and reliability.
• Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
• Follow best practices for version control, testing, documentation, and deployment.
• Contribute to cloud-based data engineering initiatives, preferably on GCP.

Required Skills

• 5–7 years of hands-on experience in Data Engineering.
• Strong programming skills in Python.
• Strong expertise in Advanced SQL and database concepts.
• Hands-on experience with ETL/ELT processes and data pipelines.
• Good understanding of Data Warehousing and Data Modeling concepts.
• Experience with CI/CD practices and tools.
• Strong understanding of DevOps principles, automation, and deployment processes.
• Strong data analytics and problem-solving skills.
• Experience working with large datasets and performance optimization.
• Good understanding of Git/version control and software development best practices.

Good to Have

• Hands-on experience with Google Cloud Platform (GCP).
• Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
• Experience with containerization/orchestration technologies such as Docker/Kubernetes.
• Experience with workflow orchestration tools such as Airflow.
• Knowledge of cloud-based data architecture and distributed data processing.

Preferred Candidate Profile

• Strong analytical and problem-solving abilities.
• Good communication and stakeholder management skills.
• Ability to work independently as well as in a collaborative team environment.
• Strong ownership of data pipelines and production systems.
• Candidates who can join at short notice are preferred.

Mandatory Skills

 Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops

About Nexora Group

Nexora Group is a forward-thinking technology and innovation company focused on leveraging Artificial Intelligence, Data Science, and emerging technologies to solve real-world business challenges. We provide opportunities for aspiring professionals to gain hands-on experience, work on impactful projects, and develop industry-relevant skills in a collaborative environment.

Internship Overview

We are looking for enthusiastic and motivated Data Science with AI Interns to join our growing team. This internship is designed for students and recent graduates who are passionate about data analytics, machine learning, artificial intelligence, and data-driven decision-making.

The selected candidates will work alongside experienced professionals on real-world datasets, AI models, and business intelligence projects while gaining practical exposure to industry-standard tools and technologies.

Key Responsibilities

• Collect, clean, and preprocess structured and unstructured datasets.
• Perform exploratory data analysis (EDA) and generate actionable insights.
• Assist in developing and deploying machine learning and AI models.
• Work with Python, SQL, and data visualization tools.
• Create dashboards, reports, and data-driven presentations.
• Support predictive analytics and model evaluation activities.
• Collaborate with cross-functional teams on AI-driven projects.
• Research emerging trends in Data Science, Machine Learning, and Generative AI.
• Document project findings and maintain technical reports.

Required Skills

• Basic understanding of Data Science and Machine Learning concepts.
• Knowledge of Python and data analysis libraries (Pandas, NumPy, Matplotlib, Scikit-learn).
• Familiarity with SQL and database concepts.
• Understanding of AI, Generative AI, and Large Language Models (LLMs) is a plus.
• Strong analytical and problem-solving skills.
• Good communication and teamwork abilities.
• Eagerness to learn and adapt to new technologies.

Eligibility

• Undergraduate or postgraduate students pursuing Computer Science, Data Science, AI, IT, Statistics, Mathematics, or related fields.
• Recent graduates looking to gain practical industry experience.
• Candidates with personal projects, certifications, or relevant coursework will be preferred.

What You'll Gain

• Hands-on experience with real-world AI and Data Science projects.
• Mentorship from industry professionals.
• Exposure to modern AI tools and technologies.
• Internship Certificate upon successful completion.
• Letter of Recommendation (based on performance).
• Opportunity for a Pre-Placement Offer (PPO) for outstanding performers.
• Professional networking and career development opportunities.

Hiring for Data Scientist / Senior Data Scientist

Exp : 4 - 12 yrs

Edu : BE/B.tech/MCA

Work Location : Pune

Notice Period : Immediate - 15 days

Skills :

4+ years of experience in data engineering, data science, or related domains.

Hands-on experience with SQL, Python, and distributed data systems.

Knowledge of machine learning techniques and statistical analysis.

Experience with cloud data platforms (Azure Data Factory, AWS Glue, GCP BigQuery).

Familiarity with DevOps practices and CI/CD for data pipelines.

Platforms & Operations Experience (Preferred)

- Experience working with Azure, AWS, or Google Cloud data tools.

Operational experience with data orchestration tools (Airflow, ADF, Glue).

Understanding of Kubernetes, Docker, or containerized environments.

Hands-on experience with data warehousing platforms (Snowflake, Redshift, BigQuery).

Experience in monitoring, logging, and alerting operations for data workflows.

Job Description

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop data processing solutions using Python.
• Write complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain data ingestion and integration workflows.
• Implement data quality, validation, monitoring, and error-handling processes.
• Develop and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
• Collaborate with data analysts, data scientists, software engineers, and business teams.
• Optimize data pipelines for performance, reliability, and scalability.
• Troubleshoot production data issues and ensure timely resolution.
• Follow best practices for version control, code quality, testing, and deployment.

Mandatory Skills

• Python
• ETL
• SQL
• CI/CD
• DevOps
• Git / Version Control
• Strong problem-solving and debugging skills