PostgreSQLKey Responsibilities:
● Architect and Design Data Systems: Design and develop scalable data pipelines and systems to manage large volumes of structured and unstructured data.
● DataPipeline Development: Develop scalable end-to-end ETL (Extract, Transform, Load) data pipelines to ingest data from various diverse sources, perform transformations, and load data into target systems to meet both functional and non-functional requirements.
● DataModeling and Optimization: Develop and optimize data models (including relational Data) for efficient storage and retrieval, ensuring data quality and integrity.
● Technical Oversight: Manage technical scope, ensuring alignment with business objectives and delivering high-quality solutions. Provided required technical guidance and oversight to various cross-functional teams and team members on databases like PostgreSQL/ Vector DB and various queries, functions, and datasets.
● Configuration Management: Manage the configurations for various tools and databases like postgresDB, vectorDB, etc. for both on-premises and cloud platforms for optimized performance and redundancy
● Performance Tuning: Monitor and optimize data pipelines, SQL/queries, functions, procedures, and triggers for performance and scalability.
● Stakeholder Collaboration: Work closely with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and deliver solutions ensuring alignment with organizational goals.
● Promoting Best Practices: Define and promote reusable, scalable, and maintainable solutions, emphasizing software engineering best practices and continuous improvement.
● Communication: Communicate effectively at all levels about the importance of solution design, conducting technical walk-throughs to ensure a clear understanding of system architecture.
● Continuous Improvement: Work with data and analytics experts to enhance functionality in data systems and products, driving improvements and growth within the data team.
● CloudExperience: Experience of working with AWS RDS databases is preferred.
Required Experience, Skills & Qualifications:
● Experience: 5-7 years in High-Performance Data Products or Data Systems as a Data Architect/Engineer.
● Technical Skills: Proficiency in orchestration tools like Airflow. Proficiency in SQL and relational databases (e.g., PostgreSQL, Aurora) and NoSQL databases (e.g., MongoDB, Cassandra).
● Knowledge of Vector databases like pgvector etc.
● Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes).
● Experience with agile development methodologies (e.g., Scrum, Kanban).
● Software Engineering: Proficient in software engineering best practices, unit testing, integration testing, and tools like Git and Docker. ● Security and Compliance: Familiarity with security compliances and design practices.
● Communication: Exceptional interpersonal, analytical, and communication skills. Ability to explain and discuss concepts with colleagues and teams effectively.
● CI/CDPipeline: Fully adhere to and promote an entire CI/CD pipeline. ● APIDevelopment: Familiarity with API development and data formats like JSON/XML.
Subodh Popalwar
Software Engineer, Memorres
About Foundation AI
Similar jobs
Data Science
Machine Learning (ML)
Deep Learning• 6+ years of data science experience.
• Demonstrated experience in leading programs.
• Prior experience in customer data platforms/finance domain is a plus.
• Demonstrated ability in developing and deploying data-driven products.
• Experience of working with large datasets and developing scalable algorithms.
• Hands-on experience of working with tech, product, and operation teams.
Technical Skills:
• Deep understanding and hands-on experience of Machine learning and Deep
learning algorithms. Good understanding of NLP and LLM concepts and fair
experience in developing NLU and NLG solutions.
• Experience with Keras/TensorFlow/PyTorch deep learning frameworks.
• Proficient in scripting languages (Python/Shell), SQL.
• Good knowledge of Statistics.
• Experience with big data, cloud, and MLOps.
Soft Skills:
• Strong analytical and problem-solving skills.
• Excellent presentation and communication skills.
• Ability to work independently and deal with ambiguity.
Continuous Learning:
• Stay up to date with emerging technologies.
Qualification.
A degree in Computer Science, Statistics, Applied Mathematics, Machine Learning, or any related field / B. Tech.
Technical Skills:
- Ability to understand and translate business requirements into design.
- Proficient in AWS infrastructure components such as S3, IAM, VPC, EC2, and Redshift.
- Experience in creating ETL jobs using Python/PySpark.
- Proficiency in creating AWS Lambda functions for event-based jobs.
- Knowledge of automating ETL processes using AWS Step Functions.
- Competence in building data warehouses and loading data into them.
Responsibilities:
- Understand business requirements and translate them into design.
- Assess AWS infrastructure needs for development work.
- Develop ETL jobs using Python/PySpark to meet requirements.
- Implement AWS Lambda for event-based tasks.
- Automate ETL processes using AWS Step Functions.
- Build data warehouses and manage data loading.
- Engage with customers and stakeholders to articulate the benefits of proposed solutions and frameworks.
Python Professional experience in Python – Mandatory experience
Basic knowledge of any BI Tool (Microsoft Power BI, Tableau etc.) and experience in R
will be an added advantage
Proficient in Excel
Good verbal and written communication skills
Key Responsibilities:
Analyze data trends and provide intelligent business insights, monitor operational and
business metrics
Complete ownership of business excellence dashboard and preparation of reports for
senior management stating trends, patterns, and predictions using relevant data
Review, validate and analyse data points and implement new data analysis
methodologies
Perform data profiling to identify and understand anomalies
Perform analysis to assess quality and meaning of data
Develop policies and procedures for the collection and analysis of data
Analyse existing process with the help of data and propose process change and/or lead
process re-engineering initiatives
Use BI Tools (Microsoft Power BI/Tableau) and develop and manage BI solutions
Data AnalyticsWhat you will do:
- Understand the process of CaaStle business teams, KPIs, and pain points
- Build scalable data products, self-service tools, data cubes to analyze and present data associated with acquisition, retention, product performance, operations, client services, etc.
- Closely partner with data engineering, product, and business teams and participate in requirements capture, research design, data collection, dashboard generation, and translation of results into actionable insights that can add value for business stakeholders
- Leverage advanced analytics to drive key success metrics for business and revenue generation
- Operationalize, implement, and automate changes to drive data-driven decisions
- Attend and play an active role in answering questions from the executive and/or business teams through data mining and analysis
We would love for you to have:
- Education: Advanced degree in Computer Science, Statistics, Mathematics, Engineering, Economics, Business Analytics or related field is required
- Experience: 2-4 years of professional experience
- Proficiency in data visualization/reporting tools (i.e. Tableau, Qlikview, etc.)
- Experience in A/B testing and measure performance of experiments
- Strong proficiency with SQL-based languages. Experience with large scale data analytics technologies (i.e., Hadoop and Spark)
- Strong analytical skills and business mindset with the ability to translate complex concepts and analysis into clear and concise takeaways to drive insights and strategies
- Excellent communication, social, and presentation skills with meticulous attention to detail
- Programming experience in Python, R, or other languages
- Knowledge of Data mining, statistical modeling approaches, and techniques
CaaStle is committed to equality of opportunity in employment. It has been and will continue to be the policy of CaaStle to provide full and equal employment opportunities to all employees and candidates for employment without regard to race, color, religion, national or ethnic origin, veteran status, age, sexual orientation, gender identity, or physical or mental disability. This policy applies to all terms, conditions and privileges of employment, such as those pertaining to training, transfer, promotion, compensation and recreational programs.
Must Have Skills:
• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skills
PythonRole : Senior Customer Scientist
Experience : 6-8 Years
Location : Chennai (Hybrid)
Who are we?
A young, fast-growing AI and big data company, with an ambitious vision to simplify the world’s choices. Our clients are top-tier enterprises in the banking, e-commerce and travel spaces. They use our core AI-based choice engine http://maya.ai/">maya.ai, to deliver personal digital experiences centered around taste. The http://maya.ai/">maya.ai platform now touches over 125M customers globally. You’ll find Crayon Boxes in Chennai and Singapore. But you’ll find Crayons in every corner of the world. Especially where our client projects are – UAE, India, SE Asia and pretty soon the US.
Life in the Crayon Box is a little chaotic, largely dynamic and keeps us on our toes! Crayons are a diverse and passionate bunch. Challenges excite us. Our mission drives us. And good food, caffeine (for the most part) and youthful energy fuel us. Over the last year alone, Crayon has seen a growth rate of 3x, and we believe this is just the start.
We’re looking for young and young-at-heart professionals with a relentless drive to help Crayon double its growth. Leaders, doers, innovators, dreamers, implementers and eccentric visionaries, we have a place for you all.
Can you say “Yes, I have!” to the below?
- Experience with exploratory analysis, statistical analysis, and model development
- Knowledge of advanced analytics techniques, including Predictive Modelling (Logistic regression), segmentation, forecasting, data mining, and optimizations
- Knowledge of software packages such as SAS, R, Rapidminer for analytical modelling and data management.
- Strong experience in SQL/ Python/R working efficiently at scale with large data sets
- Experience in using Business Intelligence tools such as PowerBI, Tableau, Metabase for business applications
Can you say “Yes, I will!” to the below?
- Drive clarity and solve ambiguous, challenging business problems using data-driven approaches. Propose and own data analysis (including modelling, coding, analytics) to drive business insight and facilitate decisions.
- Develop creative solutions and build prototypes to business problems using algorithms based on machine learning, statistics, and optimisation, and work with engineering to deploy those algorithms and create impact in production.
- Perform time-series analyses, hypothesis testing, and causal analyses to statistically assess the relative impact and extract trends
- Coordinate individual teams to fulfil client requirements and manage deliverable
- Communicate and present complex concepts to business audiences
- Travel to client locations when necessary
Crayon is an equal opportunity employer. Employment is based on a person's merit and qualifications and professional competences. Crayon does not discriminate against any employee or applicant because of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, marital status, pregnancy or related.
More about Crayon: https://www.crayondata.com/">https://www.crayondata.com/
More about http://maya.ai/">maya.ai: https://maya.ai/">https://maya.ai/
Python-5+ years hands on experience with penetration testing would be added plus
-Strong Knowledge of programming or scripting languages, such as Python, PowerShell, Bash
-Industry certifications like OSCP and AWS are highly desired for this role
-Well-rounded knowledge in security tools, software and processes
Data SciencePython
R LanguageRole :
- Understand and translate statistics and analytics to address business problems
- Responsible for helping in data preparation and data pull, which is the first step in machine learning
- Should be able to do cut and slice data to extract interesting insights from the data
- Model development for better customer engagement and retention
- Hands on experience in relevant tools like SQL(expert), Excel, R/Python
- Working on strategy development to increase business revenue
Requirements:
- Hands on experience in relevant tools like SQL(expert), Excel, R/Python
- Statistics: Strong knowledge of statistics
- Should able to do data scraping & Data mining
- Be self-driven, and show ability to deliver on ambiguous projects
- An ability and interest in working in a fast-paced, ambiguous and rapidly-changing environment
- Should have worked on Business Projects for an organization, Ex: customer acquisition, Customer retention.
Should be able to use the transformations components to transform the data
Should possess knowledge on incremental load, full load etc.
Should Design, build and deploy effective packages
Should be able to schedule these packages through task schedulers
Implement stored procedures and effectively query a database
Translate requirements from the business and analyst into technical code
Identify and test for bugs and bottlenecks in the ETL solution
Ensure the best possible performance and quality in the packages
Provide support and fix issues in the packages
Writes advanced SQL including some query tuning
Experience in the identification of data quality
Some database design experience is helpful
Experience designing and building complete ETL/SSIS processes moving and transforming data for
ODS, Staging, and Data Warehousing
ScalaPython|
· Advanced Spark Programming Skills · Advanced Python Skills · Data Engineering ETL and ELT Skills · Expertise on Streaming data · Experience in Hadoop eco system · Basic understanding of Cloud Platforms · Technical Design Skills, Alternative approaches |
· Hands on expertise on writing UDF’s · Hands on expertise on streaming data ingestion · Be able to independently tune spark scripts · Advanced Debugging skills & Large Volume data handling. · Independently breakdown and plan technical Tasks |
Follow CutshortSubodh Popalwar
Software Engineer, Memorres