Software Engineer

Leverage your expertise in Python to design and implement distributed systems for web scraping, ensuring robust data extraction from diverse web sources.

• Develop and optimize scalable, self-healing scraping frameworks, integrated with AI tools for intelligent automation of the data collection process.

• Implement monitoring, logging, and alerting mechanisms to ensure high availability and performance of distributed web scraping systems.

• Work with large-scale NoSQL databases (e.g., MongoDB) to store and query scraped data efficiently.

• Collaborate with cross-functional teams to research and implement innovative AI-driven solutions for data extraction and automation.

• Ensure data integrity and security while interacting with various web sources.

Required Skills: • Extensive experience with Python and web frameworks like Flask, FastAPI, or Django.

• Experience with AI tools and machine learning libraries to enhance and automate scraping processes

. • Solid understanding of building and maintaining distributed systems, with hands-on experience in parallel programming (multithreading, asynchronous, multiprocessing).

• Working knowledge of asynchronous queue systems like Redis, Celery, RabbitMQ, etc., to handle distributed scraping tasks.

• Proven experience with web mining, scraping tools(e.g., Scrapy, BeautifulSoup, Selenium), and handling dynamic content.

• Proficiency in working with NoSQL data storage systems like MongoDB, including querying and handling large datasets.

• Knowledge of working with variousfront-end technologies and how various websites are built

About this roleWe are seeking an experienced MongoDB Developer/DBA who will be

responsible for maintaining MongoDB databases while optimizing performance, security, and

the availability of MongoDB clusters. As a key member of our team, you’ll play a crucial role in

ensuring our data infrastructure runs smoothly.

You'll have the following responsibilities

 Maintain and Configure MongoDB Instances - Responsible for build, design, deploy,

maintain, and lead the MongoDB Atlas infrastructure. Keep clear documentation of the

database setup and architecture.

 Ownership of governance, defining and enforcing policies in MongoDB Atlas.Provide

consultancy in drawing the design and infrastructure (MongoDB Atlas) for use case.

 Service and Governance wrap will be in place to restrict over provisioning for server size,

number of clusters per project and scaling through MongoDB Atlas

 Gathering and documenting detailed business requirements applicable to the data

layer.Responsible for designing, configuring and managing MongoDB on Atlas.

 Design, develop, test, document, and deploy high-quality technical solutions on the

MongoDB Atlas platform based on industry best practices to solve business needs.

Resolves technical issues raised by the team and/or customer and manages escalations as

required.

 Migrate data from on-premise MongoDB and RDBMS to MongoDB AtlasCommunicate

and collaborate with other technical resources and customers in providing timely updates

on status of deliverables, shedding light on technical issues, and obtaining buy-in on

creative solutions.

 Write procedures for backup and disaster recovery.

You'll have the following skills & experience

 Excellent analytical, diagnostic skills, and problem-solving skills

 Should understand the Database concept and develop expertise in designing and

developing NoSQL databases such as MongoDB

 MongoDB query operation, import and export operation in database

 Experience in ETL methodology for performing Data Migration, Extraction,

Transformation, Data Profiling and Loading

 Migrating database by ETL, migrating database by manual process and designing,

development, implementation

 General networking skills, especially in the context of a public cloud (e.g. AWS – VPC,

subnets, routing tables, nat / internet gateways, dns, security groups)

 Experience using Terraform as an IaC tool for setting up infrastructure on AWS

CloudPerforming database backups and recovery

 Competence in at least one of the following languages (in no particular order): Java, C++,

C#, Python, Node.js (JavaScript), Ruby, Perl, Scala, Go

 Excellent communication skills, often being able to compromise but draw out risks and

constraints associated with solutions. Be able to work independently and collaborate with

other teams

 Proficiency in configuring schema and MongoDB data modeling.

 Strong understanding of SQL and NoSQL databases.

 Comfortable with MongoDB syntax.

 Experience with database security management.

 Performance Optimization - Ensure databases achieve maximum performance and

availability. Design effective indexing strategies.

Developer – Golang 

Responsibilities: 

• An individual contributor who will be part of the Software R&D team working on the following areas 

• Backend work in Golang, Micro services architecture 

• Work with internal team and customers to deliver solutions that will meet the customer needs with respect to the performance security, scalability and functionality. 

• Implementation of API and authentication methods using JWT 

• Writing scalable, robust, testable, efficient, and easily maintainable code 

• Translating software requirements into stable, working, high performance software 

Skills 

• 2-5 years of experience in Go programming language 

• Strong experience in working with Goroutine and channel patterns 

• Experience in Go framework for DB, JSON, Sockets, Web Frameworks and tools 

• Knowledge of logging and configuration packages like Zap, Viper 

• Experience in writing unit tests for Golang code 

• Familiarity with code versioning tools such as Git or Azure Repo 

• Should have knowledge of MySQL, PostgreSQL, or any RDBMS 

• Good knowledge of Linux / Ubuntu 

Responsibilities:

• Design and implement backend infrastructure and APIs.

• Write high quality code that is robust, readable and scales.

• Possess the drive to dive deep. thrive and progress even in ambiguous situations.

• Encourage and support knowledge sharing within teams and external groups.

• Troubleshoot & debug applications.

• Adopt problem solving as a way of life - always go to the root cause.

• Collaborate with the team to discuss and implement ideas.

• Build responsive, robust and optimised applications.

Requirements:

• Passionate about building backend systems.

• Desire to explore new ideas, open to other ideas as well.

• Love for writing clean, beautiful, readable and testable code.

• Experience in designing extensible DRY code.

• Our stack is based on Django, Python3 Celery, Angular and Postgres.

• We expect you to have a good understanding of Python. It's even better if you are familiar with some of Git. Django, Celery, Redis and Unix Shell.

JD - Senior Dev with more than 5 years relevant experience

1. Working experience on web development using Python/Django
2. Comfortable with using Django framework.
3. Understanding of web servers
4. Knowledge in PostgreSQL, MySQL, and other database queries
5. Experience in using Redis cache.
6. Understanding of web technologies
7. Experience in REST API development using Django.
8. Strong hold on Apache
9. Integration of data storage solutions
10. Able to integrate multiple data sources and databases into one system
11. Understanding of JavaScript, and front-end JS frameworks like VueJS, React JS and Angular JS
12. Good communication skill
13. .net/.NET core will be an added advantage

Expereince: 3-5 years

Domain: SQL server/SSIS/Cloud technology

Good Knowledge in Creating new tables in database.

Using Triggers, trunket,delete,view,etc

Hands -on experience with the following technologies is needed:

1. Good Knowledge of Python & Django Framework.
2. Experience in Git or any Version control systems.
3. Experience in SQL Databases like MySQL and PostgreSQL.
4. Knowledge of NoSQL Databases like Mongodb.
5. Worked on Django based live projects.
6. To implement microservices using Django rest framework.
7. Handling various technical aspects like coding of modules using the given design specifications
8. Debugging and fixing the defects as well as analyzing its root cause.
9. Deployment(AWS) and monitoring of the live project.
10. Mentoring junior developers.

Knowledge/Experience of AWS will be a big plus

Preferred Qualifications: 
B-Tech-IT/B-Tech -CSE/BCA or any degree in the relevant field

Required Experience: 
3 – 6 Years