Applying Web Crawlers to Develop a Sale Management System for Online Malls

Currently, most of the Internet retailers or companies build their online stores at online malls. But before an online store is built, the mall will request retailer/company to open an account for accessing the store online. So a retailer/company who has joined n stores at an online mall will open n accounts. Also these retailers and companies often extend their marketing channels to achieve the "group fighting" effect. Consequently, they need to control the inventory of virtual and physical marketing channels simultaneously so as to reduce their running costs and avoid inventory shortage. On the other hand, when the number of the products approaches hundreds or thousands, they also need to monitor the sale amount of a product at different online malls, and the available amount of the product in warehouse. Tin fact, the sale and inventory must be balanced for all products to maximize profit or reduce the inventory amount allocated to each online mall so as to lower the total cost. Therefore, in this study, we develop a sale management system, named the Web-Crawler-based Sale Management System (WCSMS for short), which collects sale data from different online malls by using web crawlers. Users can then access the data and browse the data on the web pages of the WCSMS. Meanwhile, this sale management system can be used to solve the inventory problem of the online malls and the distribution problem of physical malls. Furthermore, the sale data collected by web crawlers and analyzed by employing data mining techniques can serve as a decision reference for promoting and pricing their products.