基于一類SVM的網(wǎng)絡不良信息過濾算法研究
發(fā)布時間：2019-05-10 07:32
【摘要】：互聯(lián)網(wǎng)的高速發(fā)展使得通過網(wǎng)絡傳輸?shù)奈募O(jiān)控和過濾成為一個熱門課題。這些文件中可能包含了不良信息。網(wǎng)絡流量中的信息包含著各種網(wǎng)絡協(xié)議，可能被分片，編碼。機器無法直接識別其中的需要監(jiān)控的內(nèi)容。而對于內(nèi)容過濾，使用傳統(tǒng)的基于字符串匹配的算法顯然無法滿足呈幾何爆炸級別的信息增長的監(jiān)管需求。雖然使用SVM確實可以提高分類效率，但依然存在維數(shù)過大，導致存儲資源和計算能力浪費的現(xiàn)象。 本文首先分析如何在眾多網(wǎng)絡協(xié)議中，根據(jù)協(xié)議本身的特點和協(xié)議狀態(tài)機，對協(xié)議中包含的傳輸內(nèi)容進行自動識別匹配，然后對數(shù)據(jù)流部分進行重組還原，并且進行必要的解碼操作，以獲得需要過濾的文本信息。本文重點研究了主流的應用層HTTP協(xié)議，F(xiàn)TP協(xié)議，SMTP協(xié)議和POP3協(xié)議，，以及主流的私有應用飛信協(xié)議，QQ協(xié)議和MSN協(xié)議。然后本文提出了一種針對如何有效減少SVM的維數(shù)的改進算法，提出通過使用三種特征簡約對向量機的維數(shù)進行約束。這種算法的改進達到加快運算速度，節(jié)省存儲空間、提高準確率的作用。 實驗表明在選用相同數(shù)量的特征詞的前提下，基于文檔頻率，基于信息增益和開方擬合算法取舍向量機的特征值各有優(yōu)缺點。在僅僅選取500個特征值的情況下，改進算法使得不良信息分類和過濾的正確率達到了80%以上。在選取超過1000個特征值的情況下，DF算法的正確率超過了90%。
[Abstract]:With the rapid development of the Internet, file monitoring and filtering through the network has become a hot topic. These files may contain bad information. The information in the network traffic contains a variety of network protocols, which may be sliced and encoded. The machine cannot directly identify what needs to be monitored. For content filtering, the traditional string matching algorithm can not meet the regulatory needs of geometric explosion level information growth. Although the use of SVM can improve the classification efficiency, there is still a phenomenon that the dimension is too large, which leads to the waste of storage resources and computing power. This paper first analyzes how to automatically identify and match the transmission content contained in the protocol according to the characteristics of the protocol itself and the protocol state machine in many network protocols, and then reorganize and restore the data flow part. And carry out the necessary decoding operation to obtain the text information that needs to be filtered. This paper focuses on the mainstream application layer HTTP protocol, FTP protocol, SMTP protocol and POP3 protocol, as well as the mainstream private applications such as Fetion protocol, QQ protocol and MSN protocol. Then this paper proposes an improved algorithm to reduce the dimension of SVM effectively, and proposes to use three kinds of feature reduction to constrain the dimension of vector machine. The improvement of this algorithm can accelerate the operation speed, save the storage space and improve the accuracy. Experiments show that on the premise of choosing the same number of feature words, based on document frequency, based on information gain and square fitting algorithm, the eigenvalues of vector machines have their own advantages and disadvantages. When only 500 eigenvalues are selected, the correct rate of classification and filtering of bad information is more than 80%. When more than 1000 eigenvalues are selected, the correct rate of DF algorithm is more than 90%.
【學位授予單位】：上海交通大學
【學位級別】：碩士
【學位授予年份】：2014
【分類號】：TP393.08
本文編號：2473458
[Abstract]:With the rapid development of the Internet, file monitoring and filtering through the network has become a hot topic. These files may contain bad information. The information in the network traffic contains a variety of network protocols, which may be sliced and encoded. The machine cannot directly identify what needs to be monitored. For content filtering, the traditional string matching algorithm can not meet the regulatory needs of geometric explosion level information growth. Although the use of SVM can improve the classification efficiency, there is still a phenomenon that the dimension is too large, which leads to the waste of storage resources and computing power. This paper first analyzes how to automatically identify and match the transmission content contained in the protocol according to the characteristics of the protocol itself and the protocol state machine in many network protocols, and then reorganize and restore the data flow part. And carry out the necessary decoding operation to obtain the text information that needs to be filtered. This paper focuses on the mainstream application layer HTTP protocol, FTP protocol, SMTP protocol and POP3 protocol, as well as the mainstream private applications such as Fetion protocol, QQ protocol and MSN protocol. Then this paper proposes an improved algorithm to reduce the dimension of SVM effectively, and proposes to use three kinds of feature reduction to constrain the dimension of vector machine. The improvement of this algorithm can accelerate the operation speed, save the storage space and improve the accuracy. Experiments show that on the premise of choosing the same number of feature words, based on document frequency, based on information gain and square fitting algorithm, the eigenvalues of vector machines have their own advantages and disadvantages. When only 500 eigenvalues are selected, the correct rate of classification and filtering of bad information is more than 80%. When more than 1000 eigenvalues are selected, the correct rate of DF algorithm is more than 90%.
【學位授予單位】：上海交通大學
【學位級別】：碩士
【學位授予年份】：2014
【分類號】：TP393.08
【參考文獻】
相關期刊論文 前10條
1 閉樂鵬;徐偉;宋瀚濤;;基于一類SVM的貝葉斯分類算法[J];北京理工大學學報;2006年02期
2 詹毅;;樸素貝葉斯算法和SVM算法在Web文本分類中的效率分析[J];成都大學學報(自然科學版);2013年01期
3 藍惠賓;;基于包過濾控制在局域網(wǎng)的應用[J];電子制作;2013年09期
4 杜隆胤;;嵌入式調(diào)試中的數(shù)據(jù)獲取和錯誤定位[J];計算機安全;2012年04期
5 唐續(xù);劉心松;楊峰;;Linux網(wǎng)絡協(xié)議棧分析及協(xié)議添加的實現(xiàn)[J];計算機科學;2003年02期
6 楊凱峰;張毅坤;李燕;;基于文檔頻率的特征選擇方法[J];計算機工程;2010年17期
7 郭傳雄,鄭少仁;對Linux操作系統(tǒng)中TCP/IP網(wǎng)絡協(xié)議的IP層排隊分析[J];計算機學報;2001年08期
8 申紅;呂寶糧;內(nèi)山將夫;井佐原均;;文本分類的特征提取方法比較與改進[J];計算機仿真;2006年03期
9 曹建芳;王鴻斌;;一種新的基于SVM-KNN的Web文本分類算法[J];計算機與數(shù)字工程;2010年04期
10 馮長遠,普杰信;Web文本特征選擇算法的研究[J];計算機應用研究;2005年07期
本文編號：2473458
本文鏈接：http://www.wukwdryxk.cn/guanlilunwen/ydhl/2473458.html
最近更新
教材專著
