Beyondsoft (Malaysia)
Beyondsoft Malaysia is a branch of Beyondsoft, a global IT services and consulting company headquartered in China. Beyondsoft specializes in IT consulting, software development, and outsourcing services. In Malaysia, the company focuses on offering a range of technology solutions including software development, IT consulting, and systems integration.
Job Description:
We are seeking a highly skilled Web Crawler Data Engineer to join our growing team. The
successful candidate will be responsible for designing, developing, and maintaining web scraping systems to collect, process, and manage large datasets from various online sources.
• Design and implement web crawlers to extract data from diverse websites efficiently.
• Develop and maintain data pipelines to process and integrate scraped data into our data
warehouse.
• Write clean, scalable, and maintainable code to ensure robust data extraction.
• Analyse website structures and adapt web crawlers to handle dynamic and complex web pages.
• Ensure data accuracy and integrity through validation and quality checks.
• Troubleshoot and resolve issues related to web scraping and data processing.
• Stay current with the latest web scraping technologies, tools, and methodologies.
• Ensure compliance with legal and ethical standards for web scraping activities.
Qualifications:
• At least one year experience as a Web Crawler Data Engineer or in a similar role.
• Proficiency in programming languages such as Python.
• Strong experience with web scraping frameworks and tools (e.g., Scrapy, Selenium,
BeautifulSoup).
• Solid understanding of web technologies, including HTML, CSS, JavaScript, and HTTP
protocols.
• Experience with data storage solutions, including SQL and NoSQL databases.
• Familiarity with data processing frameworks (e.g., Apache Spark, Hadoop) is a plus.
• Excellent problem-solving skills and attention to detail.
• Strong communication skills and ability to work collaboratively in a team environment.
Preferred Qualifications:
– Experience with cloud platforms (AWS, Azure, GCP) for deploying scalable web scraping solutions.
– Knowledge of data analysis and visualization tools.
– Understanding of machine learning techniques and applications.
– Awareness of ethical and legal considerations in web scraping.
Our office operates from 9 am to 6 pm, and we offer a one-hour flexible lunch break. The salary package includes a basic salary without additional allowances.
As part of our benefits package, employees will receive:
• EPF (Employees Provident Fund)
• SOCSO (Social Security Organization)
• EIS (Employment Insurance System)
• Annual Performance Review
• Festival Celebration
• Medical Claim coverage, which includes in-patient and out-patient treatments
• RM1000 specialist care per year
• In the event that a public holiday falls on a Saturday, employees will be granted a day off in lieu, and this leave will remain valid until the end of the following month.
Additionally, our leave entitlements are as follows:
• 14 days of Annual Leave
• 14 days of Sick Leave
• 60 days of Hospitalization Leave
We also offer a flexible lunch hour for your convenience, and you can enjoy freeflow of premium coffee in our office pantry.
