We’re seeking a skilled Web/Data Scraping Engineer to design, develop, and maintain robust scraping solutions for collecting data from various online sources.
Key Responsibilities:
- Develop efficient, scalable web scraping scripts and tools
- Implement data extraction techniques for diverse web structures
- Ensure data quality, integrity, and compliance with legal/ethical standards
- Optimize scraping processes for performance and reliability
- Troubleshoot and resolve scraping issues
- Collaborate with data science and analytics teams
Requirements:
- Proficiency in Python and related scraping libraries (e.g., BeautifulSoup, Scrapy)
- Experience with HTML, CSS, and JavaScript
- Knowledge of HTTP protocols and web technologies
- Familiarity with anti-scraping techniques and countermeasures
- Understanding of data structures and databases
- Problem-solving skills and attention to detail
Preferred Qualifications:
- Experience with cloud platforms (AWS, GCP) for scraping at scale
- Knowledge of proxy management and IP rotation techniques
- Familiarity with legal and ethical considerations in web scraping
Job Category: Development
Job Type: Full Time
Job Location: Remote Job