A Hybrid Semantic Item Model for Recipe Search by Example

As a necessary part of our daily life, choose what dishes to cook that is a problem and troubles many people every day. In recent years, there has been a proliferation of multimedia recipe data on the Web 2.0 communities. To assist people to navigate and search on from large amounts of recipes, a suitable recipe model is crucial and indispensable. However, recipes have some distinct characteristics that conventional data models are inadequate to represent them for such data. For example, it is unreasonable and insufficient to measure how similar the cooking procedures of two dishes are only through text descriptions and/or their extracted terms. The main reason is that this raw data (or low-level features extracted from the raw data, e.g. term for text, color for image) do not map to the high-level semantics readily. In this paper, we argue that a recipe model should be semantic-based and behavior-oriented, preferably with domain knowledge support. A hybrid semantic item (HSI) model is next presented for addressing this problem. Based on HSI model, we devise a corresponding approach for recipe search by example. The experiment on our multimedia recipe retrieval system demonstrates that our HSI approach outperforms baseline methods.