
Data Engineer
Mandatory Requirements
- Experience in AWS Glue
- Experience in Apache Parquet
- Proficient in AWS S3 and data lake
- Knowledge of Snowflake
- Understanding of file-based ingestion best practices.
- Scripting language - Python & pyspark
CORE RESPONSIBILITIES
- Create and manage cloud resources in AWS
- Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies
- Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform
- Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations
- Develop an infrastructure to collect, transform, combine and publish/distribute customer data.
- Define process improvement opportunities to optimize data collection, insights and displays.
- Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible
- Identify and interpret trends and patterns from complex data sets
- Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders.
- Key participant in regular Scrum ceremonies with the agile teams
- Proficient at developing queries, writing reports and presenting findings
- Mentor junior members and bring best industry practices
QUALIFICATIONS
- 5-7+ years’ experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales)
- Strong background in math, statistics, computer science, data science or related discipline
- Advanced knowledge one of language: Java, Scala, Python, C#
- Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake
- Proficient with
- Data mining/programming tools (e.g. SAS, SQL, R, Python)
- Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
- Data visualization (e.g. Tableau, Looker, MicroStrategy)
- Comfortable learning about and deploying new technologies and tools.
- Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines.
- Good written and oral communication skills and ability to present results to non-technical audiences
- Knowledge of business intelligence and analytical tools, technologies and techniques.
Familiarity and experience in the following is a plus:
- AWS certification
- Spark Streaming
- Kafka Streaming / Kafka Connect
- ELK Stack
- Cassandra / MongoDB
- CI/CD: Jenkins, GitLab, Jira, Confluence other related tools

Similar jobs
We are looking for A highly skilled Full Stack .NET Developer with strong hands-on experience in C#, .NET Core, ASP.NET Core, Web API, and Microservices Architecture. Proficient in developing scalable and high-performing applications using SQL Server, NoSQL databases, and Entity Framework (v6+). Recognized for excellent troubleshooting, problem-solving, and communication skills, with the ability to collaborate effectively with cross-functional and international teams, including US counterparts.
- Must have 3+ Years of experience with Hyrbid App Development
- Must have 2+ Years of Experience with Cordova / Ionic
- Priority will be given to candidates having experience in developing Cordova Plugins
- Good Communication Skills (English)
At Egnyte we build and maintain our flagship software: a secure content platform used by companies like Red Bull and Yamaha.
We store, analyze, organize, and secure billions of files and petabytes of data with millions of users. We observe more than 1M API requests per minute on average. To make that possible and to provide the best possible experience, we rely on great engineers. For us, people who own their work from start to finish are integral. Our Engineers are part of the process from design to code, to test, to deployment, and back again for further iterations.
We have 300+ engineers spread across the US, Poland, and India.
You will be part of our DevOps Team working closely with our DBA team in automating, monitoring, and scaling our massive MySQL cluster. Previous MySQL experience is a plus.
Your day-to-day at Egnyte
- Designing, building, and maintaining cloud environments (using Terraform, Puppet or Kubernetes)
- Migrating services to cloud-based environments
- Collaborating with software developers and DBAs to create a reliable and scalable infrastructure for our product.
About you
- 2+ years of proven experience in a DevOps Engineer, System Administrator or Developer role, working on infrastructure or build processes
- Programming prowess (Python, Java, Ruby, Golang, or JavaScript)
- Experience with databases (MySQL or Postgress or RDS/Aurora or others)
- Experience with public cloud services (GCP/AWS/Azure)
- Good understanding of the Linux Operating System on the administration level
- Preferably you have experience with HA solutions: our tools of choice include Orchestrator, Proxysql, HAProxy, Corosync & Pacemaker, etc.
- Experience with metric-based monitoring solutions (Cloud: CloudWatch/Stackdriver, On-prem: InfluxDB/OpenTSDB/Prometheus)
- Drive to grow as a DevOps Engineer (we value open-mindedness and a can-do attitude)
DUTIES INCLUDE:
- Complete content writing projects for digital marketing clients in a timely manner; maintain stated deadlines while also meeting specific content directions, as each individual project dictates
- Revise content submissions according to client requests
- Proofread and rewrite existing content, as needed
- Writing for web page content, blogs, social media posts, infographic content, email marketing campaigns, and other digital marketing categories as assigned
- Work with clients and customers to help them define their content needs
- Offer suggestions and changes to help ensure that content is engaging, consistent, clear, and concise
- Help to create style guides or content templates
- Conduct effective research on any given topic
SKILLS PREFERRED:
- Fluency in both written and conversational English, and mastery of conventional rules of English grammar
- Ability to be mindful of things like flow, readability, and sentence variety
- Ability to embed and leverage SEO-relevant keywords in content production
- Basic understanding of digital marketing and search engine optimization best practices
- Good listening and communication skills for client relationship management
- Working knowledge of Microsoft Office, along with general computer literacy
- Familiarity with major social media platforms
- The ability to cooperate with other writers, editors, and team members to complete projects in a timely manner
- Willing and able to give constructive feedback on content writing projects
Come Dive In
The DevOps Engineer will execute the tools and processes to enable DevOps.
Engage in and improve the whole lifecycle of services from inception and design through deployment, operation, and refinement to efficiently deliver high-quality solutions. The candidate should bridge the gap between Development and Operational teams, working with the development teams to meet acceptance criteria and gather and document the requirements. Candidates should be able to work in fast-paced,
multi-disciplinary environments.
As An DevOps Engineer, You Will
● Work in a dynamic, agile team environment developing excellent new applications.
● Participate in design decisions, including new technology research and prototyping
● Collaborate closely with other AWS engineers and architects, cloud engineers, support teams, and other stakeholders
● Promote great Kubernetes and AWS platform design and quality
● Innovate new ideas to evolve our applications and processes
● Continuously analyzing and evaluating our systems, products, and methods for potential improvements.
Mandatory Skills:
● Experience on Linux based infrastructure
● Experience in ECS - Amazon services*
● Should have hands-on containerized Services
● Must know about AWS CI/CD pipeline.
● Must know DevOps concepts and Agile principles
● Knowledge of Git, Docker, and Jenkins
● Knowledge of Infrastructure as Code.
● Experience in using Automation Tools
● Must have experience in Test Driven Development environment setup.
● Working knowledge of Docker and Kubernetes
We recognize that asking you to give 100% of yourself daily requires us to show you the love.
PERKS: what can we offer you?
● Bi-Yearly performance audits and appraisals
● The flexibility of working days/hours
● 5 working days/week (Mon to Fri) and added payout for working Saturday
● Recognition and Appreciation
● A plethora of industry exposure and self-growth opportunities
Visit our site: www.cedcoss.com
We are looking for a highly capable Node.js developer to optimize our mobile-based application performance. You will be collaborating with our front-end application developers, designing back-end components, and integrating data storage and protection solutions.
To ensure success as a Node.js developer, you should possess extensive knowledge of Node.Js based services and experience in a similar role. An accomplished Node.js developer will be someone whose expertise results in the stable, fast, and secure exchange of data between servers and end-users.
Roles and Responsibilities
- Actively participate in the overall application lifecycle.
- Focus on coding and debugging.
- Collaborate with front-end developers.
- Define and communicate technical and design requirements.
- Build a high-quality reusable code that can be used in the future.
- Create sustainable and functional web applications with clean codes.
- Learn about new technologies and stay up to date with current best practices.
- Conduct UI tests and optimize performance.
- Train, help, and support to other team members.
Requirements
- Hands-on backend mobile app/web development experience
- At least four years' experience as a Node.js developer.
- Hands-on experience of working on MongoDB database
- Knowledgeable of front-end technologies and platforms, such as JavaScript, HTML5, and CSS3
- Good understanding of server-side CSS pre-processors
- Proven experience creating user authentication and authorization between multiple systems, servers, and environments
- Capability to integrate multiple data sources and databases into one system
- Proven experience managing of hosting environment, that includes database administration and scaling an application to support load changes
- Ability to setup, administer of backups, data migration, transformation, and scripting.
- Knowledgeable of OWASP security principles
- Understands “session management” in a server environment
- Tech/M.Tech in Computer Science or a related field
We are a leading payments and API banking solutions company. We provide full-stack payments solutions enabling businesses in India to collect payments and make payouts via all available methods with a simple integration. Our offerings include an advanced and easy way to integrate payment gateway, a split payment solution for marketplaces, bank account verification API and Auto Collect -- a virtual account solution to match inbound payments to customers.
Currently used by over 100,000 businesses for vendor payouts, wage payouts, build refunds, expense reimbursements, loyalty, and rewards. Some of its notable customers include Dunzo, Xiaomi, Tencent, Delhivery, Zomato, Cred, Club Factory, and ExxonMobil.
- Get an excellent grasp on testing complex microservices systems, web and mobile based applications with reference to database and other back-end systems
- Develop and maintain automation frameworks for large scale real time server side systems
- Own the non-functional testing such as Performance, Scale, Longevity, Security, High Availability, Disaster Recovery aspects of a 24x7 system
- Participate in the performance testing, upgrade/migration activities, POC of framework/tools for automation
- Work on end to end testing of feature/products which involve both manual(which will act as a prequel to automation) and automation testing
- Bachelor's degree in Computer Science (or a related discipline)
- 3+ years of experience as SDET/Developer
- Strong analytical and problem solving and communication(both verbal and written)skills
- Have exposure to frameworks like Rest assured, Selenium and Appium
- Exposure to AWS or any other cloud ecosystem
- Good exposure to GIT, Jenkins or similar tools
- You are driven to do all kinds of destructive testing
- You are passionate for Automation First approach
- You are hands-on and stay up to date with the latest technologies
- You love problem solving and attention to detail
- Everyone is empowered to make decisions
- Transparency in communication keeps us aware of where we are heading, and honest feedback helps everyone to grow quickly & in all directions
- You have hybrid working, flexible timings and a great office space to work in
- Meal coupons, Uber Credits, Travel allowance & Covid Programs
- Group Mediclaim and Term Life Insurances
- 24x7 counselling support and a lot more benefits to support you at the time of need
- Exciting Reward and Recognition programs
- Aside all above, you will work with some great peers, colleagues and mentors
Equal Experts is a community of talented experienced software development consultants, specializing in Agile Delivery. A place where you are a Front-End Dev, working alongside the client, enabling them, and adding real value to their products. A world where you are working with great, like-minded people, while you learn new technology on the go and implementing the best practices.
We indulge in all things that would excite you!!
- Work on large-scale, custom distributed software systems using JavaScript,
- Node (ReactJS, VueJs) and the likes
- We work almost exclusively on customer site providing a mix of delivery and
- consulting services, so you'll be flexible about travel
- Employ Agile development including task estimation, test automation, deployment automation, and Continuous Integration to improve overall execution speed and product quality
- Work in a dynamic, collaborative, transparent, non-hierarchical, and ego-free culture where your talent is valued over a designation or title
- Spread the word about best practices in software development inside and outside
- Equal Experts community like Expert Talks
What we look for in you?
- 3 + years of Hands-on development and delivery experience with JavaScript, and frameworks the likes
- HTML, CSS, React Js, Vue Js, Webpack, Grunt, Gulp.
- Testing tools like Jasmine, Jest, Enzyme
- Passion for software engineering and craftsman-like coding prowess
- Great OO skills, including strong design patterns knowledge
- Experience working with Agile, TDD, Lean, and/or Continuous Delivery approaches and best practices, such as Extreme Programming (XP)
- Enthusiastic to work collaboratively with people and sharing your ideas to solve real business problems.
- Join a team of great technical professionals that work hard but have fun and enjoy developing state of the art product-based applications and websites.
- Use best practice coding standards and implement each deliverable.
- Write and manage program and Bug Fixes
- Construct and support integrations that are easy to manage, maintain, customize and configure; ensure overall reliability, availability, and high-performance code.
- Analyze system issues and problems and implement solutions rapidly.
- Manage multiple projects simultaneously varying in sizes and scope
- Work in a fast-paced cross-functional team environment. Effectively handle time-critical issue and appropriately escalate at-risk items.
- Laravel/ Zend/ CakePHP/ CodeIgniter/ Symfony, or equivalent.
- Strong knowledge of OOPs Concepts.
- Computer Science/Software Engineering Bachelor’s degree or higher education.
- Experience in writing high performance and high scalable code using best practice and coding standards.
- Excellent written and spoken communication skills.
- Excellent organizational and attention to detail.
- Good to have familiarity with NoSql DB, web servers (e.g. Apache), caching (Memcache / Redis), Cloud, NodeJs, Microservices architecture Familiar with agile software development methodologies (e.g. SCRUM / Agile / Kanban)
- Must have knowledge of MVC design patterns, XHTML/HTML5, PHP, CSS3, XML, MySQL/MSSQL, JavaScript jQuery, json objects, API, AJAX functionality and client-side validation.
- Understanding accessibility and security compliance.
- Understanding fundamental design principles behind a scalable application
- Knowledge of using version control software such as SVN or GIT will be preferable.
- Experience in either Mobile application or Data Integration with other apps would be preferred.
- 2-4 years of demonstrable hands-on work in web-based application development experiences in a professional environment, including interaction design for dynamic, data-driven applications (not HTML based website) delivered via the web, or the equivalent.











