
Similar jobs
About the Company
Hypersonix.ai is disrupting the e-commerce space with AI, ML, and advanced decision-making capabilities to drive real-time business insights. Built from the ground up using modern technologies, Hypersonix simplifies data consumption for customers across various industry verticals. We are seeking a well-rounded, hands-on product leader to help manage key capabilities and features in our platform.
Position Overview
We are seeking a highly skilled Web Scraping Architect to join our team. The successful candidate will be responsible for designing, implementing, and maintaining web scraping processes to gather data from various online sources efficiently and accurately. As a Web Scraping Specialist, you will play a crucial role in collecting data for competitor analysis and other business intelligence purposes.
Responsibilities
- Scalability/Performance: Lead and provide expertise in scraping at scale e-commerce marketplaces.
- Data Source Identification: Identify relevant websites and online sources from which data needs to be scraped. Collaborate with the team to understand data requirements and objectives.
- Web Scraping Design: Develop and implement effective web scraping strategies to extract data from targeted websites. This includes selecting appropriate tools, libraries, or frameworks for the task.
- Data Extraction: Create and maintain web scraping scripts or programs to extract the required data. Ensure the code is optimized, reliable, and can handle changes in the website's structure.
- Data Cleansing and Validation: Cleanse and validate the collected data to eliminate errors, inconsistencies, and duplicates. Ensure data integrity and accuracy throughout the process.
- Monitoring and Maintenance: Continuously monitor and maintain the web scraping processes. Address any issues that arise due to website changes, data format modifications, or anti-scraping mechanisms.
- Scalability and Performance: Optimize web scraping procedures for efficiency and scalability, especially when dealing with a large volume of data or multiple data sources.
- Compliance and Legal Considerations: Stay up-to-date with legal and ethical considerations related to web scraping, including website terms of service, copyright, and privacy regulations.
- Documentation: Maintain detailed documentation of web scraping processes, data sources, and methodologies. Create clear and concise instructions for others to follow.
- Collaboration: Collaborate with other teams such as data analysts, developers, and business stakeholders to understand data requirements and deliver insights effectively.
- Security: Implement security measures to ensure the confidentiality and protection of sensitive data throughout the scraping process.
Requirements
- Proven experience of 6+ years as a Web Scraping Specialist or similar role, with a track record of successful web scraping projects
- Expertise in handling dynamic content, user-agent rotation, bypassing CAPTCHAs, rate limits, and use of proxy services
- Knowledge of browser fingerprinting
- Has leadership experience
- Proficiency in programming languages commonly used for web scraping, such as Python, BeautifulSoup, Scrapy, or Selenium
- Strong knowledge of HTML, CSS, XPath, and other web technologies relevant to web scraping and coding
- Knowledge and experience in best-of-class data storage and retrieval for large volumes of scraped data
- Understanding of web scraping best practices, including handling dynamic content, user-agent rotation, and IP address management
- Attention to detail and ability to handle and process large volumes of data accurately
- Familiarity with data cleansing techniques and data validation processes
- Good communication skills and ability to collaborate effectively with cross-functional teams
- Knowledge of web scraping ethics, legal considerations, and compliance with website terms of service
- Strong problem-solving skills and adaptability to changing web environments
Preferred Qualifications
- Bachelor’s degree in Computer Science, Data Science, Information Technology, or related fields
- Experience with cloud-based solutions and distributed web scraping systems
- Familiarity with APIs and data extraction from non-public sources
- Knowledge of machine learning techniques for data extraction and natural language processing is desired but not mandatory
- Prior experience in handling large-scale data projects and working with big data frameworks
- Understanding of various data formats such as JSON, XML, CSV, etc.
- Experience with version control systems like Git
Candidate must be proficient in using oracle database as a powerful
RAD platform. Must possess the following skills
MANDATORY SKILLS
Oracle Application Express
Oracle SQL
Oracle PL/SQL
Oracle Data Modelling
Oracle Database performance and tuning
OPTIONAL SKILLS
HTML5
CSS
Web design
Data security
Web security
Experience of at least 2-3 Years of Business Application development
Position Overview: We are looking for a senior developer who must be capable of building services using modern microservices architecture with NodeJS & serverless frameworks. Should have strong knowledge of ObjectOriented Concepts, and Node. Js frameworks like loopback, RDBMS, and microservice patterns. The candidate should be comfortable with application coding based on design and specifications, and also be comfortable working with JSON and RESTful services. The candidate must be a top-notch developer committed to keeping learning new things and becoming an integral part of and fostering growth within our development team.
Roles & Responsibilities:
- Developing and maintaining the backend systems that support the application
- Designing and implementing APIs and Microservices architecture to handle the application workload
- Developing and optimizing queries for RDBMS (MySQL) to manage data and ensure performance and scalability
- Managing and deploying the application on AWS using Lambda, ECS and other related services
- Collaborating with front-end developers to ensure smooth communication between front-end and back-end systems and back-end systems
- Writing clean, neat and reusable code that follows coding standards and best practices.
- Participating in code reviews and ensuring that coding standards are maintained
- Troubleshooting and debugging issues that arise in production environments
- Optimizing the application for performance, scalability, and security
- Continuously learning and staying up-to-date with the latest technologies and best practices in software development
- Documenting technical specifications, processes, and system design
What We’re Looking For:
Meltwater is a global leader in media intelligence and social analytics. Our mission is to help businesses make more informed decisions by providing them with actionable insights from the vast ocean of online data. With a diverse and talented team spread across the world, we are
committed to driving innovation and pushing the boundaries of what's possible in our field.
Meltwater is seeking a Full Stack Software Engineer to join our new influencer marketing team in our Hyderabad office.. We are looking for an individual who is not only technically proficient but also embodies the values of collaboration, open-mindedness, and a proactive approach to
problem-solving. As a Full Stack Software Engineer, you will play a key role in shaping our technology stack, working with cutting-edge technologies like React, Typescript, Next.js, Node.js & AWS ecosystem.
What You'll Do:
- Collaborate effectively with cross-functional teams to develop and maintain software solutions that meet business requirements.
- Lead the development of high-quality code, ensuring scalability, security, and performance.
- Continuously learn and adapt to new technologies, tools, and best practices.
- Identify and resolve technical challenges.
What You'll Bring:
- Bachelor's or Master's degree in Computer Science or related field.
- At least 3 years of experience as a Full Stack Software Engineer.
- Strong problem-solving skills and the ability to think critically.
- Experience with databases such as MySQL, Elasticsearch, etc.
- Experience with backend technologies, preferably NodeJS & Typescript.
- Experience with frontend technologies, preferably React & Typescript.
- Excellent communication and collaboration skills.
- Self-motivated with a passion for learning and self-improvement.
What We Offer:
- Opportunity to work on cutting-edge technologies and projects.
- A culture that values innovation, collaboration, and personal growth.
- A dynamic and diverse team with a global presence.
- Minimum 3 years of experience in Node JS.
- Minimum 1 years of experience in Mongo DB (optional)
- Good Hands-on experience in Angular or React.
- Experience in designing and building large-scale, high availability, secure web applications and REST-full API using Node.js in an agile methodology.
- Good Hands-on experience in Node Js frameworks like Express, .
- Strong proficiency with JavaScript and Node.JS
- Knowledge and understanding of Event Loop architecture.
- Knowledge and understanding of promises and (async or await)
- Implementation of security like oauth 2.0, JWT and data protection.
- Good Hands-on experience on cache server.
- Knowledge and understanding of Request, Axios, proficiency in REST-ful APIs.
- Knowledge of data structures and Algorithms, Node Global variables and In-built libraries
- Understanding the nature of asynchronous programming and its quirks and Workarounds
- Design and implementation of low-latency, high-availability, and performant applications.
• 3+ Years of experience as a Lead Developer
• Experience in ReactJS (most preferred) or AngularJS similar front end frameworks
• Experience with the Python or/and Golang (preferably both), SQL, and design/architectural
patterns
• Experience in Java or dotnet or other opensource technologies is an added advantage
• Hands-on experience on SQL, Query optimization, DB server migration
• Preferably experience in Postgre SQL or MySQL
• Knowledge of NOSQL databases will be an added advantage
• Experience in Cloud platforms like AWS, Azure with knowledge of containerization, Kubernetes is an
added advantage
• Knowledge of one or more programming languages along with HTML5/CSS3,Bootstrap
• Familiarity with architecture styles/APIs (REST, RPC)
• Understanding of Agile methodologies
• Experience with Threading, Multithreading and pipelines
• Experience in creating RESTful API’s With Golang or Python or Java in JSON, XMLs
• Experience with GitHub, Tortoise SVN Version Control
• Strong attention to detail
• Strong knowledge of asynchronous programming with the latest frameworks
• Excellent troubleshooting and communication skills
• Strong knowledge of unit testing frameworks
• Proven knowledge of ORM techniques
• Skill for writing reusable libraries Understanding of fundamental design principles for building a scalable
application
· Lead from the front, when it comes to delivering high quality work products. Serve as a mentor to the team members
· Create and define performance metrics. Ideate, innovate and hack through the existing systems to improve performance.
· Perform code reviews, set coding practices and guidelines within the team.
· Make sense of the product specs, be nimble in getting the issues resolved. Be a thought leader. · Solid understanding and knowledge of Java/C/C++. You should understand the concepts of multithreading, I/O, messaging, instrumentation etc and their implementation in Java/Any other Object-Oriented Language.
· Proficiency in the implementation of algorithms and the use of advanced data structures to solve problems in computing. A solid foundation in the principles of computer science is desired.
· Ability to write clean, modular, and loosely coupled code by the use of design patterns.
· He/She should be able to break down and architect complex software systems. For example- Build
- Ability to write APIs including Restful APIs.
- Able to consume 3rd party API's
- Familiarity with SQL (MySQL, POSTGRESQL) and NoSQL(MongoDB) databases designing, query optimization techniques and their declarative query languages.
- Creating database schemas that represent and support business processes.
- User authentication and authorization between multiple systems, servers, and environments.
- Experience in Micro-services development will be an added advantage.
- Composer / NPM / Webpack.
- Laravel forge server deployment and management
- Laravel Dusk Testing
- PHP Unit Testing
- Good command of problem-solving and analytical skills.
- Must be technically strong in logical thinking
Addon Front-End Frameworks knowledge:
- Other Front-end frameworks like as Vue.js (Vue CLI, Vuex, Vue Router, and Nuxt.js).
- Proficiency in JavaScript and TypeScript programming.
- Aware about JavaScript basics and Modern JavaScript (ES6 – ES7 – ES8)
- Understanding of server-side rendering and its benefits and use cases
- Working knowledge of npm or Yarn
- Knowledge of Bootstrap, Framework 7
- Working knowledge of HTML5 code template and CSS3 styling (Sass or Less depending on your technology stack)
Addon Server knowledge will be added advantage:
- Proficient understanding of code versioning tools, such as GitHub, GitLab
- Working knowledge of cloud servers (AWS) related services and server administration like setting up
Apache/Nginx, php-fpm etc.. (Nice to have but not mandatory).
- Experience working with Docker (Nice to have but not mandatory)
- Experience with CI/CD tools and processes(Nice to have but not mandatory)







