Technical Lead

We are looking for a skilled and motivated Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for designing, developing, and maintaining robust and scalable ETL (Extract, Transform, Load) data pipelines. The role involves working with various GCP services, implementing data ingestion and transformation logic, and ensuring data quality and consistency across systems.

Key Responsibilities:

• Design, develop, test, and maintain scalable ETL data pipelines using Python.
• Work extensively on Google Cloud Platform (GCP) services such as:
• Dataflow for real-time and batch data processing
• Cloud Functions for lightweight serverless compute
• BigQuery for data warehousing and analytics
• Cloud Composer for orchestration of data workflows (based on Apache Airflow)
• Google Cloud Storage (GCS) for managing data at scale
• IAM for access control and security
• Cloud Run for containerized applications
• Perform data ingestion from various sources and apply transformation and cleansing logic to ensure high-quality data delivery.
• Implement and enforce data quality checks, validation rules, and monitoring.
• Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver efficient data solutions.
• Manage version control using GitHub and participate in CI/CD pipeline deployments for data projects.
• Write complex SQL queries for data extraction and validation from relational databases such as SQL Server, Oracle, or PostgreSQL.
• Document pipeline designs, data flow diagrams, and operational support procedures.
• 
  

Required Skills:

• 4–8 years of hands-on experience in Python for backend or data engineering projects.
• Strong understanding and working experience with GCP cloud services (especially Dataflow, BigQuery, Cloud Functions, Cloud Composer, etc.).
• Solid understanding of data pipeline architecture, data integration, and transformation techniques.
• Experience in working with version control systems like GitHub and knowledge of CI/CD practices.
• Strong experience in SQL with at least one enterprise database (SQL Server, Oracle, PostgreSQL, etc.).

Must have exp in Django. [ Mandatory]

• The ability to problem-solve and critically think.
• High level of knowledge of Python and the Django framework.
• Familiarity with event driven programming as well as the MVC.
• Good understanding of SQL databases.
• Good understanding of REST APIs.

• Develop, record, and maintain cutting edge web-based PHP applications on portal plus premium service platforms
• Build innovative, state-of-the-art applications and collaborate with the User Experience (UX) team
• Ensure HTML, CSS, and shared JavaScript is valid and consistent across applications
• Prepare and maintain all applications utilizing standard development tools
• Utilize backend data services and contribute to increasing existing data services API
• Lead the entire web application development life cycle right from concept stage to delivery and post-launch support

The Sr. Analytics Engineer would provide technical expertise in needs identification, data modeling, data movement, and transformation mapping (source to target), automation and testing strategies, translating business needs into technical solutions with adherence to established data guidelines and approaches from a business unit or project perspective.

Understands and leverages best-fit technologies (e.g., traditional star schema structures, cloud, Hadoop, NoSQL, etc.) and approaches to address business and environmental challenges.

Provides data understanding and coordinates data-related activities with other data management groups such as master data management, data governance, and metadata management.

Actively participates with other consultants in problem-solving and approach development.

Responsibilities :

Provide a consultative approach with business users, asking questions to understand the business need and deriving the data flow, conceptual, logical, and physical data models based on those needs.

Perform data analysis to validate data models and to confirm the ability to meet business needs.

Assist with and support setting the data architecture direction, ensuring data architecture deliverables are developed, ensuring compliance to standards and guidelines, implementing the data architecture, and supporting technical developers at a project or business unit level.

Coordinate and consult with the Data Architect, project manager, client business staff, client technical staff and project developers in data architecture best practices and anything else that is data related at the project or business unit levels.

Work closely with Business Analysts and Solution Architects to design the data model satisfying the business needs and adhering to Enterprise Architecture.

Coordinate with Data Architects, Program Managers and participate in recurring meetings.

Help and mentor team members to understand the data model and subject areas.

Ensure that the team adheres to best practices and guidelines.

Requirements :

- Strong working knowledge of at least 3 years of Spark, Java/Scala/Pyspark, Kafka, Git, Unix / Linux, and ETL pipeline designing.

- Experience with Spark optimization/tuning/resource allocations

- Excellent understanding of IN memory distributed computing frameworks like Spark and its parameter tuning, writing optimized workflow sequences.

- Experience of relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., Redshift, Bigquery, Cassandra, etc).

- Familiarity with Docker, Kubernetes, Azure Data Lake/Blob storage, AWS S3, Google Cloud storage, etc.

- Have a deep understanding of the various stacks and components of the Big Data ecosystem.

- Hands-on experience with Python is a huge plus

We are hiring for a Software Engineer Team Lead - Backend, to join our education-focused fintech lending startup, GrayQuest. Our core product is a combination of a lending product, a rewards product and an insurance product to redefine the education fees payment across the country.

• Participate in all stages of the Software Development Lifecycle
• Develop back-end components, services, and APIs. Support integration of user-facing elements, and collaborate closely with the design and front-end development team.
• Write elegant, effective, and scalable code while maintaining the highest level of coding standards.
• Take complete ownership of projects and ensure timely deliveries
• Ability to debug, refine, review, and deploy code
• Work closely with the QA team to test, and document project features.
• Ability to interact with all stakeholders. Understand business requirements, and effectively convert them into technical specifications
• Mentor junior developers and participate in peer Review

Requirements:

• Demonstrable experience as a Python Developer (3+ years)
• Strong knowledge of Python frameworks (Django Rest, Flask)
• Experience working on REST APIs
• Strong knowledge of SQL and JavaScript
• Experience with Amazon Web Services (AWS)
• Working knowledge of React
• Familiarity with Agile development methodology
• Good understanding of task breakup, milestone definition, and work delegation.
• Past experience of working in a fintech/financial services domain or a consumer internet startup will be preferable but not required
• Work experience in early-stage startups (0 ->1) is a big plus
• Bachelors or equivalent degree from premier institutes is a plus
• Most importantly, the biggest skill we look for our prospective candidates is "Learnability" - how quickly can you learn. 

Why is this a great opportunity for the right candidate:

1. Work with a young dynamic core team backed by marquee investors from the financial services space
2. Work on a product/business that has achieved product market fit and take it to scale
3. Equity compensation and competitive market salaries
4. Work environment that prioritizes talent, hustle and dedication
   
   

We will also ensure we give a preference to applicants who directly email us with their background/context and why they think they should be a strong candidate for this position at GrayQuest. Email us at work (at the rate) grayquest (dot) com and cc Sonal (at the rate)grayquest (dot) com