Web Scraping Jobs in Pune

The Energy Exemplar (EE) data team is looking for an experienced Python Developer (Data Engineer) to join our Pune office. As a dedicated Data Engineer on our Research team, you will apply data engineering expertise, work very closely with the core data team to identify different data sources for specific energy markets and create an automated data pipeline. The pipeline will then incrementally pull the data from its sources and maintain a dataset, which in turn provides tremendous value to hundreds of EE customers.

At EE, you’ll have access to vast amounts of energy-related data from our sources. Our data pipelines are curated and supported by engineering teams. We also offer many company-sponsored classes and conferences that focus on data engineering, data platform. There’s a great growth opportunity for data engineering at EE..

Responsibilities

•  Develop, test and maintain architectures, such as databases and large-scale processing systems using high-performance data pipelines.
•  Recommend and implement ways to improve data reliability, efficiency, and quality.
•  Identify performant features and make them universally accessible to our teams across EE.
•  Work together with data analysts and data scientists to wrangle the data and provide quality datasets and insights to business-critical decisions
• Take end-to-end responsibility for the development, quality, testing, and production readiness of the services you build.
• Define and evangelize Data Engineering best standards and practices to ensure engineering excellence at every stage of a development cycle.
• Act as a resident expert for data engineering, feature engineering, exploratory data analysis.
• Agile methodologies, acting as Scrum Master would be an added plus.

Qualifications

• 6+ years of professional experience in developing data pipelines for large-scale, complex datasets from varieties of data sources.
• Data Engineering expertise with strong experience working with Python, Beautiful Soup, Selenium, Regular Expression, Web Scraping.
• Best practices with Python Development, Doc String, Type Hints, Unit Testing, etc.
• Experience working with Cloud-based data technologies such as Azure Data lake, Azure Data Factory, Azure Data Bricks is optionally desirable.
• Moderate coding skills. SQL or similar required. C# or other languages strongly preferred.
• Outstanding communication and collaboration skills. You can learn from and teach others.
• Strong drive for results. You have a proven record of shepherding experiments to create successful shipping products/services
• A Bachelor or Masters degree in Computer Science or Engineering with coursework in Python, Big Data, Data Engineering is highly desirable.

We are looking for a Node.js Developer responsible for managing the interchange of data between the server and the users. Your primary focus will be the development of all server-side logic, definition and maintenance of the central database, and ensuring high performance and responsiveness to requests from the front-end. You will also be responsible for integrating the front-end elements built by your co-workers into the application. Therefore, a basic understanding of front-end technologies is necessary as well.

Responsibilities

• Integration of user-facing elements developed by front-end developers with server side logic
• Writing reusable, testable, and efficient code
• Design and implementation of low-latency, high-availability, and performant applications
• Implementation of security and data protection
• Creating APIs and backend systems
• Scraping sites and creating backend dashboards for data management
• Experience with React.js and native is a plus but not required
• Backend infrastructure management and deployment
• Experience with MongoDB, Express is a plus.