Web Scraping Jobs in Pune

Want to shape the future of Energy through Data Engineering, sourcing Data from 100s of disparate sources? We all know that without good data there is no Data Science. If you let garbage in, it will emit the garbage out. 60-70% efforts in a data science project are spent in Data Engineering & Feature Engineering. That's where we need your skills to fetch the data from disparate sources, transform it the way business needs (that may also include applying lots of critical business logics per its source and nature) and load it in a data warehouse / big data systems. These critical pieces of works complement the Data Scientist, with a continuous feedback loop based on how a model is performing and what fine tuning is needed in the data.   The Energy Exemplar (EE) data team is looking for an experienced Python Developer (Data Engineer) to join our Pune office. As a dedicated Data Engineer on our Research team, you will apply data engineering  expertise, work very closely with the core data team to identify different data sources for specific energy markets and create an automated data pipeline. The pipeline will then incrementally pull the data from its sources and maintain a dataset, which in turn provides  tremendous value to hundreds of EE customers.   At EE, you’ll have access to vast amounts of energy-related data from our sources. Our data pipelines are curated and supported by engineering teams. We also offer many company-sponsored classes and conferences that focus on data engineering, data platform. There’s great growth opportunity for data engineering at EE.   Responsibilities Develop, test and maintain architectures, such as databases and large-scale processing systems using high-performance data pipeline. Recommend and implement ways to improve data reliability, efficiency, and quality. Identify performant features and make them universally accessible to our teams across EE. Work together with data analysts and data scientists to wrangle the data and provide quality datasets and insights to business critical decisions. Take end-to-end responsibility for the development, quality, testing, and production readiness of the services you build. Define and evangelize Data Engineering best standards and practices to ensure engineering excellence at every stage of development cycle. Act as a resident expert for data engineering, feature engineering, exploratory data analysis.   Qualifications 2+ years of professional experience in developing data-pipelines for large-scale, complex datasets from varieties of data sources. Data Engineering expertise with strong experience working with Python, Beautiful Soup, Web Scraping. Experience working with Cloud based data technologies such as Azure Data lake, Azure Data factory, Azure Data Bricks optionally desirable. Moderate coding skills. SQL or similar required. C# or other languages strongly preferred. Outstanding communication and collaboration skills. You can learn from and teach others. Strong drive for results. You have a proven record of shepherding experiments to create successful shipping products/services. Immediate Joiners will be preferred A Bachelor or Masters degree in Computer Science or Engineering with coursework in Python, BigData, Data Engineering highly desirable.

We are looking for a Node.js Developer responsible for managing the interchange of data between the server and the users. Your primary focus will be the development of all server-side logic, definition and maintenance of the central database, and ensuring high performance and responsiveness to requests from the front-end. You will also be responsible for integrating the front-end elements built by your co-workers into the application. Therefore, a basic understanding of front-end technologies is necessary as well. Responsibilities Integration of user-facing elements developed by front-end developers with server side logic Writing reusable, testable, and efficient code Design and implementation of low-latency, high-availability, and performant applications Implementation of security and data protection Creating APIs and backend systems Scraping sites and creating backend dashboards for data management Experience with React.js and native is a plus but not required Backend infrastructure management and deployment Experience with MongoDB, Express is a plus.