پديد آورندگان :
حسيني، محمدمهدي دانشگاه آزاد اسلامي واحد شاهرود - دانشكده مهندسي برق و كامپيوتر , زاهدي، مرتضي دانشگاه صنعتي شاهرود - دانشكده مهندسي كامپيوتر و فن آوري اطلاعات , حسن پور، حميد دانشگاه صنعتي شاهرود - دانشكده مهندسي كامپيوتر و فن آوري اطلاعات
كليدواژه :
ارزيابي , سامانه پرسش و پاسخ تعاملي , رگرسيون غيرخطي , استخراج ويژگي
چكيده فارسي :
همانند بسياري از زمينه‌هاي ديگر زبان‌شناسي محاسباتي، ارزيابي نقش مهمي در سامانه‌هاي پرسش و پاسخ تعاملي ايفا مي‌كند. با اين وجود، در زمينه ارزيابي سامانه‌هاي پرسش و پاسخ تعاملي به‌طورتقريبي هيچ روش خاصي وجود ندارد كه به ارزيابي كلي اين سامانه‌ها پرداخته و همواره انسان بايد در فرآيند ارزيابي مشاركت داشته باشد. ارائه مدلي كه بتواند جايگزين انسان در فرآيند ارزيابي شود، يكي از موضوعات مورد توجه در اين حوزه است. در اين مقاله، يك مدل آماري مناسب براي ارزيابي سامانه‌هاي پرسش و پاسخ تعاملي جهت جايگزين‌كردن به جاي انسان توسط مجموعه‌اي از ويژگي‌هاي جديد و رگرسيون ارائه شده است. با استفاده از چهار سامانه تعاملي موجود پايگاه داده‌اي مناسب ايجاد شد. تعداد 540 نمونه به‌عنوان داده مناسب در نظر گرفته شد تا مجموعه آزمون و آموزش بر اساس آن تشكيل شود. ابتدا پيش‌پردازش بر روي مكالمات صورت پذيرفت و بر اساس روابط تعريف‌شده، ويژگي‌هاي آماري از متن مكالمه‌ها استخراج و بر اساس آن ماتريس ويژگي تشكيل و سپس با استفاده از انواع رگرسيون سعي شد تا بهترين مدل استخراج شود كه در‌نهايت رگرسيون غيرخطي سري تواني با RMSE به ميزان 0/13 بهترين مدل را ارائه كرد.
چكيده لاتين :
The development of computer systems and extensive use of information technology in the everyday life of people have just made it more and more important for them to make quick access to information that has received great importance. Increasing the volume of information makes it difficult to manage or control. Thus, some instruments need to be provided to use this information. The QA system is an automated system for obtaining the correct answers to questions posed by the human in the natural language. In these systems, if the response is found, and if it is not the user's expected response or if it needs more information, there is no possibility of exchanging information between the system and the user to ask more questions and get answers related to it. To solve this problem, interactive Question answering (IQA) systems were created. Interactive question answering (IQA) systems are associated with linguistic ambiguous structures, so these systems are more accurate than QA systems. Regarding the probability of ambiguity (ambiguity in the user question or ambiguity in the answer provided by the system), the repetition is possible in these systems to obtain the clarity. No standard methods have been developed on IQA systems evaluation, and the existing evaluation methods have been developed based on the methods used in QA and dialogue systems. In evaluating IQA systems, in addition to quantitative evaluation, a qualitative evaluation is used. It requires users’ participation in the evaluation process to determine the success level of interaction between the system and the user. Evaluation plays an important role in the IQA systems. In the context of evaluating IQA systems, there is partially no specific methodology for evaluating these systems in general. The main problem with designing an assessment method for IQA systems lies in the rare possibility to predict the interaction part. To this end, human needs to be involved in the evaluation process. In this paper, an appropriate model is presented by introducing a set of built-in features for evaluating IQA systems. To conduct the evaluation process, four IQA systems were considered based on the conversation exchanged between users and systems. Moreover, 540 samples were considered as suitable data to create a test and training set. The statistical characteristics of each conversation were extracted after performing the preprocessing on them. Then a feature matrix was formed based on the obtained characteristics. Finally, using linear and nonlinear regression, human thinking was predicted. As a result, the nonlinear power regression with 0.13 Root Mean Square Error (RMSE) was the best model.
