베이지안 SOM 과 붓스트랩을 이용한 문서 군비화에 의한 문서 순위조정

최준혁(Jun Hyeog Choi),전성해(Sung Hae Jun),이정현(Jung Hyun Lee) 한국정보처리학회 2000 정보처리학회논문지 Vol.7 No.7

The conventional Boolean retrieval systems based on vector space model can provide the results of retrieval fast, they can''t reflect exactly user''s retrieval purpose including semantic information. Consequently, the results of retrieval process are very different from those users expected. This fact forces users to waste much time for finding expected documents among retrieved documents. In this paper, we designed a bayesian SOM(Self-Organizing feature Maps) in combination with bayesian statistical method and Kohonen network as a kind of unsupervised learning, then perform classifying documents depending on the semantic similarity to user query in real time. If it is difficult to observe statistical characteristics as there are less than 30 documents for clustering, the number of documents must be increased to at least 50. Also, to give high rank to the documents which is most similar to user query semantically among generalized classifications for generalized clusters, we find the similarity by means of Kohonen centroid of each document classification and adjust the secondary rank depending on the similarity.

확장된 질의 처리를 위해 경로간 의미적 유사도를 고려한 XML 문서 순위화 기법

김현주(Hyun Joo Kim),박소미(Somi Park),박석(Seog Park) 한국정보과학회 2010 정보과학회논문지 : 데이타베이스 Vol.37 No.2

정보기술의 표준으로 사용되고 있는 XML환경에서 방대한 양의 데이터에 대한 사용자의 질의를 효율적이고 정확하게 처리하기 위한 연구가 이슈화되고, 특히 웹 환경에서의 XML문서들은 용어적, 구조적인 측면에서 다양한 형태로 존재하고 있다. 이러한 특성을 갖는 XML 문서들을 대상으로 사용자가 특정한 정보를 얻고자 한다면, 사용자의 질의가 가진 용어 및 구조적 특성과 정확히 일치하지 않는 문서의 정보에 대해서 추가적인 기법이 필요하다. 본 논문은 이와 같은 경우에도 동일한 용어 및 구조를 사용하던 환경에서와 마찬가지로 최상위 순위로 정보를 검색할 수 있는 기법을 제시한다. 또한 정확히 일치하지 않는 문서의 경우에 대해서도 사용자 질의 측과의 경로간 의미적 유사성을 측정하여 사용자 질의와 의미적으로 유사한 경로를 가진 순으로 문서들을 순위화하여 제공한다. 제안된 기법은 실험을 통하여 기존의 기법보다 세밀하고 정확한 검색 결과를 도출함을 보인다. XML is broadly using for data storing and processing. XML is specified its structural characteristic and user can query with XPath when information from data document is needed. XPath query can process when the tern and structure of document and query is matched with each other. However, nowadays there are lots of data documents which are made by using different terminology and structure therefore user can not know the exact idea of target data. In fact, there are many possibilities that target data document has information which user is find or a similar ones. Accordingly user query should be processed when their term usage or structural characteristic is slightly different with data document. In order to do that we suggest a XML document ranking method based on path similarity. The method can measure a semantic similarity between user query and data document using three steps which are position, node and relaxation factors.

인터넷 문서빈도를 통해 본 도시순위규모에 관한 연구

홍일영(Il Young Hong) 한국지역지리학회 2007 한국지역지리학회지 Vol.13 No.3

본 연구는 인터넷 문서상에 나타나는 도시 지명의 문서 빈도를 통계량으로 도시규모에 대한 분석을 실시하였다. 검색어가 갖는 의미상의 차이에 따른 조건과 검색의 범위를 제약하면서 나타나는 유의적인 차이점들에 대해 분석하였고, 도시규모분포의 상관계수에 대한 분석을 통해 인구와 문서빈도와의 차이점을 분석하였다. 각 도시의 인구와 문서빈도와 상관관계 분석에서는 검색어의 종류를 보다 공간적의 의미로 제약할수록 더 높은 상관관계가 나타났고, 문서의 종류는 상용, 네트워크, 기관의 경우에 있어서 높은 상관관계가 나타났다. 그리고 인구와 문서빈도의 통계량을 이용한 군집분석을 통해서, 인구에 비해 더 많은 혹은 낮은 문서빈도를 보이는 도시들을 파악하였다. 이와 같은 분석은 웹 문서라는 정보통신사회 속에서 반영되는 각 도시의 특성을 분석하는 새로운 방안을 제시한다는 점에서 큰 의미를 갖는다고 할 수 있다. In this study, web document frequency of city place name is analyzed and it is used as the dataset for rank-size analysis. The search keywords are compared in the context of spatial meaning and the different domain corpus is applied. The acquired search results are applied for the further analysis. Firstly, the rank-size analysis is applied to compare the result between population and document frequency. Secondly, in case of correlation analysis, the significant changes are revealed when the spatial criteria for search keywords are increased. In case of corpus, COM, NET, and ORG shows the higher coefficient values. Lastly, the cluster analysis is applied to classify the list of cities that shows the similarity and difference. These analyses have a significant role in representing the rank-size distribution of city names that are reflected on the web documents in the information society.

사용자 중심의 블로그 정보 검색 기법

김승종(Kim, Seung-Jong) 한국산학기술학회 2010 한국산학기술학회논문지 Vol.11 No.9

최근 빠른 주기로 많은 양의 새로운 정보가 생성되기 때문에, 사용자 중심의 정보 검색을 위해 RSS라는 신 디케이션 기술이 제공되고 있다. RSS는 새롭게 갱신된 콘텐츠를 자동으로 전달받을 수 있어 신규 정보를 찾기 위해 사이트에 지속적으로 접근하지 않아도 된다. 본 논문에서는 블로그 정보 검색을 위해 RSS 문서의 주소를 수집하는 수집기와 사용자 질의에 따른 RSS 문서의 순위결정 방법을 제안한다. 제안하는 정보 검색 기법을 이용하면 사용자가 RSS 문서를 효과적으로 검색할 수 있다. Due to the recent tremendous growth of internet information, RSS, syndication technology provides internet users with a user-friendly information search. RSS enables you to automatically receive newly updated contents, so users do not need to constantly access web sites to obtain new information. This paper proposes the way of managing the web crawler, which collects the sites of RSS documents and helps the users efficiently use the RSS documents. And it also suggests the proper way of ranking the RSS documents based on the users' popularity. Users can efficiently search out the documents they need by using the proposed information searching methods.

기계학습을 이용한 단문 오피니언 문서의 효율적 검색 기법

장재영 한국인터넷방송통신학회 2013 한국인터넷방송통신학회 논문지 Vol.13 No.4

최근 들어 트위터나 페이스북과 같은 SNS가 대중화되면서, 오피니언 마이닝에 관한 연구가 활발히 진행되고 있다. 그러나 현재의 오피니언 마이닝 연구는 대부분 감성분류나 특징선택 방법에 중점을 두고 있으며, 오피니언 문 서의 검색에 관한 연구는 아직 미진한 실정이다. 본 논문에서는 단문으로 구성된 오피니언 문서로부터 사용자가 원하 는 문서들을 효율적으로 검색하는 기법을 제안한다. 제안된 방법에서는 기존의 감성분류 방법을 활용함과 동시에 문 서의 질적 평가를 위해 여러 가지 특징들을 적용한다. 검색 모델을 생성하기 위해 기계학습 기반 랭킹 기법을 활용하 며, 감성 분류 모델을 기계학습 랭킹 모델에 통합하는 방법을 사용한다. 또한 실험을 통하여 제안된 방법이 오피니언 검색에 효율적으로 적용될 수 있음을 보여준다. Recently, as Social Network Services(SNS), such as Twitter, Facebook, are becoming more popular, much research has been doing on opinion mining. However, current related researches are mostly focused on sentiment classification or feature selection, but there were few studies about opinion document retrieval. In this paper, we propose a new retrieval method of short opinion documents. Proposed method utilizes previous sentiment classification methodology, and applies several features of documents for evaluating the quality of the opinion documents. For generating the retrieval model, we adopt Learning-to-rank technique and integrate sentiment classification model to Learning-to-rank. Experimental results show that proposed method can be applied successfully in opinion search.

An Improved Approach to Ranking Web Documents

Pooja Gupta,Sandeep K. Singh,Divakar Yadav,A. K. Sharma 한국정보처리학회 2013 Journal of information processing systems Vol.9 No.2

Ranking thousands of web documents so that they are matched in response to a user query is really a challenging task. For this purpose, search engines use different ranking mechanisms on apparently related resultant web documents to decide the order in which documents should be displayed. Existing ranking mechanisms decide on the order of a web page based on the amount and popularity of the links pointed to and emerging from it. Sometime search engines result in placing less relevant documents in the top positions in response to a user query. There is a strong need to improve the ranking strategy. In this paper, a novel ranking mechanism is being proposed to rank the web documents that consider both the HTML structure of a page and the contextual senses of keywords that are present within it and its back-links. The approach has been tested on data sets of URLs and on their back-links in relation to different topics. The experimental result shows that the overall search results, in response to user queries, are improved. The ordering of the links that have been obtained is compared with the ordering that has been done by using the page rank score. The results obtained thereafter shows that the proposed mechanism contextually puts more related web pages in the top order, as compared to the page rank score

An Improved Approach to Ranking Web Documents

Gupta, Pooja,Singh, Sandeep K.,Yadav, Divakar,Sharma, A.K. Korea Information Processing Society 2013 Journal of information processing systems Vol.9 No.2

Ranking thousands of web documents so that they are matched in response to a user query is really a challenging task. For this purpose, search engines use different ranking mechanisms on apparently related resultant web documents to decide the order in which documents should be displayed. Existing ranking mechanisms decide on the order of a web page based on the amount and popularity of the links pointed to and emerging from it. Sometime search engines result in placing less relevant documents in the top positions in response to a user query. There is a strong need to improve the ranking strategy. In this paper, a novel ranking mechanism is being proposed to rank the web documents that consider both the HTML structure of a page and the contextual senses of keywords that are present within it and its back-links. The approach has been tested on data sets of URLs and on their back-links in relation to different topics. The experimental result shows that the overall search results, in response to user queries, are improved. The ordering of the links that have been obtained is compared with the ordering that has been done by using the page rank score. The results obtained thereafter shows that the proposed mechanism contextually puts more related web pages in the top order, as compared to the page rank score.

텍스트 분류를 위한 자질 순위화 기법에 관한 연구

김판준 한국정보관리학회 2023 정보관리학회지 Vol.40 No.1

This study specifically reviewed the performance of the ranking schemes as an efficient feature selection method for text classification. Until now, feature ranking schemes are mostly based on document frequency, and relatively few cases have used the term frequency. Therefore, the performance of single ranking metrics using term frequency and document frequency individually was examined as a feature selection method for text classification, and then the performance of combination ranking schemes using both was reviewed. Specifically, a classification experiment was conducted in an environment using two data sets (Reuters-21578, 20NG) and five classifiers (SVM, NB, ROC, TRA, RNN), and to secure the reliability of the results, 5-Fold cross-validation and t-test were applied. As a result, as a single ranking scheme, the document frequency-based single ranking metric (chi) showed good performance overall. In addition, it was found that there was no significant difference between the highest-performance single ranking and the combination ranking schemes. Therefore, in an environment where sufficient learning documents can be secured in text classification, it is more efficient to use a single ranking metric (chi) based on document frequency as a feature selection method. 본 연구는 텍스트 분류를 위한 효율적인 자질선정 방법으로 자질 순위화 기법의 성능을 구체적으로 검토하였다. 지금까지 자질 순위화 기법은 주로 문헌빈도에 기초한 경우가 대부분이며, 상대적으로 용어빈도를 사용한 경우는 많지 않았다. 따라서 텍스트 분류를 위한 자질선정 방법으로 용어빈도와 문헌빈도를 개별적으로 적용한 단일 순위화 기법들의 성능을 살펴본 다음, 양자를 함께 사용하는 조합 순위화 기법의 성능을 검토하였다. 구체적으로 두 개의 실험 문헌집단(Reuters-21578, 20NG)과 5개 분류기(SVM, NB, ROC, TRA, RNN)를 사용하는 환경에서 분류 실험을 진행하였고, 결과의 신뢰성 확보를 위해 5-fold cross validation과 t-test를 적용하였다. 결과적으로, 단일 순위화 기법으로는 문헌빈도 기반의 단일 순위화 기법(chi)이 전반적으로 좋은 성능을 보였다. 또한, 최고 성능의 단일 순위화 기법과 조합 순위화 기법 간에는 유의한 성능 차이가 없는 것으로 나타났다. 따라서 충분한 학습문헌을 확보할 수 있는 환경에서는 텍스트 분류의 자질선정 방법으로 문헌빈도 기반의 단일 순위화 기법(chi)을 사용하는 것이 보다 효율적이라 할 수 있다.

불리언 검색 시스템에서의 새로운 문서 순위결정 알고리즘

이복기 경원전문대학 1999 論文集 Vol.21 No.-

This paper suggests a new document ranking algorithm which evaluates documents retrieved in boolean information retrieval system. Such existing document ranking algorithms as Relevance, R-Distance, K-Distance, and E-Relevance evaluates similarity between a query and documents using hierarchical relation of thesaurus terms. In thesaurus, other relations as well as hierarchical relation is included. Using this important knowledge, similarity between a query and documents is evaluated more correct. A new document ranking algorithm suggested in this paper evaluates similarity between a query and documents using all relation in thesaurus.

An Efficient Density Based Ant Colony Approach on Web Document Clustering

M. Reka 한국전산응용수학회 2023 Journal of applied mathematics & informatics Vol.41 No.6

World Wide Web (WWW) use has been increasing recently due to users needing more information. Lately, there has been a growing trend in the document information available to end users through the internet. The web's document search process is essential to find relevant documents for user queries.As the number of general web pages increases, it becomes increasingly challenging for users to find records that are appropriate to their interests. However, using existing Document Information Retrieval (DIR) approaches is time-consuming for large document collections. To alleviate the problem, this novel presents Spatial Clustering Ranking Pattern (SCRP) based Density Ant Colony Information Retrieval (DACIR) for user queries based DIR. The proposed first stage is the Term Frequency Weight (TFW) technique to identify the query weightage-based frequency. Based on the weight score, they are grouped and ranked using the proposed Spatial Clustering Ranking Pattern (SCRP) technique. Finally, based on ranking, select the most relevant information retrieves the document using DACIR algorithm.The proposed method outperforms traditional information retrieval methods regarding the quality of returned objects while performing significantly better in run time.