[免費]【NLP輿情分析】基于python微博輿情分析可視化系統(flask+pandas+echarts)【論文+源碼+SQL腳本】

大家好，我是python222_小鋒老師，看到一個不錯的【NLP輿情分析】基于python微博輿情分析可視化系統(flask+pandas+echarts)，分享下哈。

項目視頻演示

【免費】【NLP輿情分析】基于python微博輿情分析可視化系統(flask+pandas+echarts+爬蟲) Python畢業設計_嗶哩嗶哩_bilibili

項目介紹

隨著社交媒體的快速發展，微博作為一種重要的公共信息平臺，承載著大量的社會輿論和民眾情感。如何有效地從微博平臺獲取并分析海量的輿情數據，成為了當前信息社會中的重要課題。本論文基于自然語言處理（NLP）技術，結合Flask、Pandas、ECharts和爬蟲技術，設計并實現了一套微博輿情分析可視化系統。

該系統首先利用Python中的爬蟲技術從微博平臺實時抓取相關輿情數據，包括微博內容、評論、轉發等信息。通過對抓取數據的清洗和預處理，使用Pandas進行數據分析，提取關鍵詞、情感傾向和輿情熱度等信息。接著，通過NLP技術對微博內容進行情感分析，識別出用戶的情感態度（如正面、負面、中立）。最后，系統使用ECharts進行數據的可視化展示，為用戶提供直觀、動態的輿情分析圖表，幫助決策者實時掌握輿情走向。

系統不僅能夠有效地獲取和分析微博上的實時輿情信息，還提供了良好的交互性和可操作性，用戶可以根據需要進行輿情趨勢分析、情感分布展示及關鍵詞云等操作。通過對實際案例的分析驗證，系統具有較高的準確性和實用性，能夠幫助相關部門及時識別網絡輿論的動態變化，并做出相應的應對措施。

本論文的研究成果為基于Python的輿情分析系統提供了一種新的實現思路，具有廣泛的應用前景，尤其在公共事務管理、品牌輿情監控以及社會熱點事件的監測中具有重要的實際意義。

系統展示

部分代碼

import pandas as pd
from flask import Blueprint, render_template, jsonify, request
from snownlp import SnowNLPfrom dao import articleDao, commentDao
from util import wordcloudUtil, mapUtilpb = Blueprint('page', __name__, url_prefix='/page', template_folder='templates')@pb.route('/home')
def home():"""進入主頁面，獲取相應的數據，帶到頁面去:return:"""articleData = articleDao.get7DayArticle()xAxis7ArticleData = []yAxis7ArticleData = []for article in articleData:xAxis7ArticleData.append(article[0])yAxis7ArticleData.append(article[1])# 獲取帖子類別數量arcTypeData = []articleTypeAmountList = articleDao.getArticleTypeAmount()for arcType in articleTypeAmountList:arcTypeData.append({'value': arcType[1], 'name': arcType[0]})# 獲取top50評論用戶名top50CommentUserList = commentDao.getTopCommentUser()top50CommentUserNameList = [cu[0] for cu in top50CommentUserList]str = ' '.join(top50CommentUserNameList)wordcloudUtil.genWordCloudPic(str, 'comment_mask.jpg', 'comment_user_cloud.jpg')# 獲取7天評論數量commentData = []commentAmountList = commentDao.getCommentAmount()for comment in commentAmountList:commentData.append({'value': comment[1], 'name': comment[0]})return render_template('index.html',xAxis7ArticleData=xAxis7ArticleData,yAxis7ArticleData=yAxis7ArticleData,arcTypeData=arcTypeData,commentData=commentData)@pb.route('homePageData')
def getHomePageData():"""獲取主頁數據 ajax異步交互 前端每隔5分鐘請求一次 實時數據:return:"""totalArticle = articleDao.getTotalArticle()topAuthor = articleDao.getTopAuthor()topRegion = articleDao.getTopRegion()topArticles = articleDao.getArticleTopZan()return jsonify(totalArticle=totalArticle, topAuthor=topAuthor, topRegion=topRegion, topArticles=topArticles)@pb.route('hotWord')
def hotWord():"""熱詞分析統計:return:"""hotwordList = []# 只讀取前100條df = pd.read_csv('./fenci/comment_fre.csv', nrows=100)for value in df.values:hotwordList.append(value[0])# 獲取請求參數，如果沒有獲取到，給個默認值 第一個列表數據defaultHotWord = request.args.get('word', default=hotwordList[0])hotwordNum = 0  # 出現次數for value in df.values:if defaultHotWord == value[0]:hotwordNum = value[1]# 情感分析sentiments = ''stc = SnowNLP(defaultHotWord).sentimentsif stc > 0.6:sentiments = '正面'elif stc < 0.2:sentiments = '負面'else:sentiments = '中性'commentHotWordData = commentDao.getCommentHotWordAmount(defaultHotWord)xAxisHotWordData = []yAxisHotWordData = []for comment in commentHotWordData:xAxisHotWordData.append(comment[0])yAxisHotWordData.append(comment[1])commentList = commentDao.getCommentByHotWord(defaultHotWord)return render_template('hotWord.html',hotwordList=hotwordList,defaultHotWord=defaultHotWord,hotwordNum=hotwordNum,sentiments=sentiments,xAxisHotWordData=xAxisHotWordData,yAxisHotWordData=yAxisHotWordData,commentList=commentList)@pb.route('articleData')
def articleData():"""微博輿情分析:return:"""articleOldList = articleDao.getAllArticle()articleNewList = []for article in articleOldList:article = list(article)# 情感分析sentiments = ''stc = SnowNLP(article[1]).sentimentsif stc > 0.6:sentiments = '正面'elif stc < 0.2:sentiments = '負面'else:sentiments = '中性'article.append(sentiments)articleNewList.append(article)return render_template('articleData.html', articleList=articleNewList)@pb.route('articleDataAnalysis')
def articleDataAnalysis():"""微博數據分析:return:"""arcTypeList = []df = pd.read_csv('./spider/arcType_data.csv')for value in df.values:arcTypeList.append(value[0])# 獲取請求參數，如果沒有獲取到，給個默認值 第一個列表數據defaultArcType = request.args.get('arcType', default=arcTypeList[0])articleList = articleDao.getArticleByArcType(defaultArcType)xDzData = []  # 點贊x軸數據xPlData = []  # 評論x軸數據xZfData = []  # 轉發x軸數據rangeNum = 1000rangeNum2 = 100for item in range(0, 10):xDzData.append(str(rangeNum * item) + '-' + str(rangeNum * (item + 1)))xPlData.append(str(rangeNum * item) + '-' + str(rangeNum * (item + 1)))for item in range(0, 20):xZfData.append(str(rangeNum2 * item) + '-' + str(rangeNum2 * (item + 1)))xDzData.append('1萬+')xPlData.append('1萬+')xZfData.append('2千+')yDzData = [0 for x in range(len(xDzData))]  # 點贊y軸數據yPlData = [0 for x in range(len(xPlData))]  # 評論y軸數據yZfData = [0 for x in range(len(xZfData))]  # 轉發y軸數據for article in articleList:for item in range(len(xDzData)):if int(article[4]) < rangeNum * (item + 1):yDzData[item] += 1breakelif int(article[4]) > 10000:yDzData[len(xDzData) - 1] += 1breakif int(article[3]) < rangeNum * (item + 1):yPlData[item] += 1breakelif int(article[3]) > 10000:yPlData[len(xDzData) - 1] += 1breakfor article in articleList:for item in range(len(xZfData)):if int(article[2]) < rangeNum2 * (item + 1):yZfData[item] += 1breakelif int(article[2]) > 2000:yZfData[len(xZfData) - 1] += 1breakreturn render_template('articleDataAnalysis.html',arcTypeList=arcTypeList,defaultArcType=defaultArcType,xDzData=xDzData,yDzData=yDzData,xPlData=xPlData,yPlData=yPlData,xZfData=xZfData,yZfData=yZfData)@pb.route('commentDataAnalysis')
def commentDataAnalysis():"""微博評論數據分析:return:"""commentList = commentDao.getAllComment()xDzData = []  # 點贊X軸數據rangeNum = 5for item in range(0, 20):xDzData.append(str(rangeNum * item) + '-' + str(rangeNum * (item + 1)))xDzData.append('1百+')yDzData = [0 for x in range(len(xDzData))]  # 點贊y軸數據genderDic = {'男': 0, '女': 0}for comment in commentList:for item in range(len(xDzData)):if int(comment[4]) < rangeNum * (item + 1):yDzData[item] += 1breakelif int(comment[4]) > 100:yDzData[len(xDzData) - 1] += 1breakif genderDic.get(comment[8], -1) != -1:genderDic[comment[8]] += 1genderData = [{'name': x[0], 'value': x[1]} for x in genderDic.items()]# 只讀取前50條數據df = pd.read_csv('./fenci/comment_fre.csv', nrows=50)hotCommentwordList = [x[0] for x in df.values]str2 = ' '.join(hotCommentwordList)wordcloudUtil.genWordCloudPic(str2, 'comment_mask.jpg', 'comment_cloud.jpg')return render_template('commentDataAnalysis.html',xDzData=xDzData,yDzData=yDzData,genderData=genderData)@pb.route('articleCloud')
def articleCloud():"""微博內容詞云圖:return:"""# 只讀取前50條數據df = pd.read_csv('./fenci/article_fre.csv', nrows=50)hotArticlewordList = [x[0] for x in df.values]str2 = ' '.join(hotArticlewordList)wordcloudUtil.genWordCloudPic(str2, 'article_mask.jpg', 'article_cloud.jpg')return render_template('articleCloud.html')@pb.route('commentCloud')
def commentCloud():"""微博評論詞云圖:return:"""# 只讀取前50條數據df = pd.read_csv('./fenci/comment_fre.csv', nrows=50)hotCommentwordList = [x[0] for x in df.values]str2 = ' '.join(hotCommentwordList)wordcloudUtil.genWordCloudPic(str2, 'comment_mask.jpg', 'comment_cloud.jpg')return render_template('commentCloud.html')@pb.route('commentUserCloud')
def commentUserCloud():"""微博評論用戶詞云圖:return:"""# 獲取top50評論用戶名top50CommentUserList = commentDao.getTopCommentUser()top50CommentUserNameList = [cu[0] for cu in top50CommentUserList]str = ' '.join(top50CommentUserNameList)wordcloudUtil.genWordCloudPic(str, 'comment_mask.jpg', 'comment_user_cloud.jpg')return render_template('commentUserCloud.html')@pb.route('ipDataAnalysis')
def ipDataAnalysis():"""IP地址數據分析:return:"""cityDic = {}  # 微博文章作者IPcityList = mapUtil.cityListarticleList = articleDao.getAllArticle()for article in articleList:if article[5]:for city in cityList:if city['province'].find(article[5]) != -1:if cityDic.get(city['province'], -1) == -1:cityDic[city['province']] = 1else:cityDic[city['province']] += 1articleCityDicList = [{'name': x[0], 'value': x[1]} for x in cityDic.items()]cityDic2 = {}  # 微博評論作者IPcommentList = commentDao.getAllComment()for comment in commentList:if comment[3]:for city in cityList:if city['province'].find(comment[3]) != -1:if cityDic2.get(city['province'], -1) == -1:cityDic2[city['province']] = 1else:cityDic2[city['province']] += 1commentCityDicList = [{'name': x[0], 'value': x[1]} for x in cityDic2.items()]return render_template('ipDataAnalysis.html',articleCityDicList=articleCityDicList,commentCityDicList=commentCityDicList)@pb.route('sentimentAnalysis')
def sentimentAnalysis():"""輿情數據分析:return:"""xHotBarData = ['正面', '中性', '負面']yHotBarData = [0, 0, 0]# 只讀取前100條df = pd.read_csv('./fenci/comment_fre.csv', nrows=100)for value in df.values:# 情感分析stc = SnowNLP(value[0]).sentimentsif stc > 0.6:yHotBarData[0] += 1elif stc < 0.2:yHotBarData[2] += 1else:yHotBarData[1] += 1hotTreeMapData = [{'name': xHotBarData[0],'value': yHotBarData[0]}, {'name': xHotBarData[1],'value': yHotBarData[1]}, {'name': xHotBarData[2],'value': yHotBarData[2]}]commentPieData = [{'name': '正面','value': 0}, {'name': '中性','value': 0}, {'name': '負面','value': 0}]articlePieData = [{'name': '正面','value': 0}, {'name': '中性','value': 0}, {'name': '負面','value': 0}]commentList = commentDao.getAllComment()for comment in commentList:# 情感分析stc = SnowNLP(comment[1]).sentimentsif stc > 0.6:commentPieData[0]['value'] += 1elif stc < 0.2:commentPieData[2]['value'] += 1else:commentPieData[1]['value'] += 1articleList = articleDao.getAllArticle()for article in articleList:# 情感分析stc = SnowNLP(article[1]).sentimentsif stc > 0.6:articlePieData[0]['value'] += 1elif stc < 0.2:articlePieData[2]['value'] += 1else:articlePieData[1]['value'] += 1df2 = pd.read_csv('./fenci/comment_fre.csv', nrows=15)xhotData15 = [x[0] for x in df2.values][::-1]yhotData15 = [x[1] for x in df2.values][::-1]return render_template('sentimentAnalysis.html',xHotBarData=xHotBarData,yHotBarData=yHotBarData,hotTreeMapData=hotTreeMapData,commentPieData=commentPieData,articlePieData=articlePieData,xhotData15=xhotData15,yhotData15=yhotData15)

<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"><title>用戶登錄</title><link rel="shortcut icon" href="{{ url_for('static', filename='favicon.ico') }}" type="image/x-icon"><link rel="stylesheet" href="/static/css/backend-plugin.min.css"><link rel="stylesheet" href="/static/css/backend.css">
</head>
<body class=" ">
<!-- loader Start -->
<div id="loading"><div id="loading-center"></div>
</div>
<!-- loader END --><div class="wrapper"><section class="login-content"><div class="container h-100"><div class="row align-items-center justify-content-center h-100"><div class="col-md-5"><div class="card p-3"><div class="card-body"><div class="auth-logo"><img src="/static/picture/logo.png" class="img-fluid  rounded-normal  darkmode-logo"alt="logo"><img src="/static/picture/logo-dark.png" class="img-fluid rounded-normal light-logo"alt="logo"></div><h3 class="mb-3 font-weight-bold text-center">用戶登錄</h3><div class="mb-5"><p class="line-around text-secondary mb-0"><spanclass="line-around-1">微博輿情分析管理系統</span></p></div><form><div class="row"><div class="col-lg-12"><div class="form-group"><label class="text-secondary">用戶名：</label><input class="form-control" id="username" name="username" type="text"placeholder="請輸入用戶名..."></div></div><div class="col-lg-12 mt-2"><div class="form-group"><div class="d-flex justify-content-between align-items-center"><label class="text-secondary">密碼：</label></div><input class="form-control" id="password" name="password" type="password"placeholder="請輸入密碼..."></div></div></div><button type="button" class="btn btn-primary btn-block mt-2" onclick="submitForm()">登錄</button><div class=" col-lg-12 mt-3"><p class="mb-0 text-center"><font id="info" color="red"></font>&nbsp;&nbsp;還沒有賬號?<ahref="/user/register">用戶注冊</a></p>  </div></form></div></div></div></div></div></section>
</div>
<script>function submitForm() {let username = $("#username").val()let password = $("#password").val()if (username == "") {$("#info").text("用戶名不能為空!")return false;}if (password == "") {$("#info").text("密碼不能為空!")return false;}$.post('/user/login', {'username': username,'password': password}, function (result) {if (result.error) {$("#info").text(result.info)} else {window.location.href = '/page/home';}})}
</script>
<!-- Backend Bundle JavaScript -->
<script src="/static/js/backend-bundle.min.js"></script>
<!-- Chart Custom JavaScript -->
<script src="/static/js/customizer.js"></script><script src="/static/js/sidebar.js"></script><!-- Flextree Javascript-->
<script src="/static/js/flex-tree.min.js"></script>
<script src="/static/js/tree.js"></script><!-- Table Treeview JavaScript -->
<script src="/static/js/table-treeview.js"></script><!-- SweetAlert JavaScript -->
<script src="/static/js/sweetalert.js"></script><!-- Vectoe Map JavaScript -->
<script src="/static/js/vector-map-custom.js"></script><!-- Chart Custom JavaScript -->
<script src="/static/js/chart-custom.js"></script>
<script src="/static/js/01.js"></script>
<script src="/static/js/02.js"></script><!-- slider JavaScript -->
<script src="/static/js/slider.js"></script><!-- Emoji picker -->
<script src="/static/js/index.js" type="module"></script><!-- app JavaScript -->
<script src="/static/js/app.js"></script>
</body>
</html>