A Three-Stage Rocket of BERT-BiLSTM-Attention: Progressive Sentiment Computing for Noisy Chinese Reviews
Author Names:
Zhihang Tang, Jinyang Shi, Qiwen Yang
Author Affiliation:
Author Email:
zhtang@hnie.edu.cn
Publication Date:
June 12, 2026
Page numbers:
DOI Number:
https://doi.org/10.66113/jcmse.26585
Abstract:
To overcome the bottleneck of existing models in capturing intricate semantics for Chinese review sentiment analysis, we propose a progressive hybrid architecture that operates in three cascading stages: context encoding, sequential refinement, and salience focusing. First, Bidirectional Encoder Representations from Transformers (BERT) pretrained weights are exploited to generate noise-resilient contextual embeddings. Bidirectional Long Short-Term Memory (BiLSTM) then reinforces local phrasal dependencies while pruning redundant long-range information. Finally, a lightweight attention head dynamically reweights sentiment-critical tokens, effectively suppressing noise from misspellings, jargon, and code-mixing. Evaluated on both public benchmarks and our self-collected high-noise corpus, the model attains 91.77 % accuracy, 92.22 % F1-score, and 96.48 % ROC-AUC, outperforming vanilla BERT by 2.43 %, 2.51 %, and 1.91 % absolute gains with stronger cross-domain robustness. The study delivers an off-the-shelf baseline for sentiment analysis in noisy Chinese user-generated content scenarios such as ecommerce and government opinion mining. https://mc.manuscriptcentral.com/jcmse Journal of Computational Methods in Science and Engineering For Peer Review A Three-Stage Rocket of BERT-BiLSTM-Attention: Progressive Sentiment Computing for Noisy Chinese Reviews Zhihang Tang*, Jinyang Shi,Qiwen Yang School of Information Science and Engineering, Hunan University of Engineering, Xiangtan 411104, China zhtang@hnie.edu.cn Abstract: To overcome the bottleneck of existing models in capturing intricate semantics for Chinese review sentiment analysis, we propose a progressive hybrid architecture that operates in three cascading stages: context encoding, sequential refinement, and salience focusing. First, Bidirectional Encoder Representations from Transformers (BERT) pretrained weights are exploited to generate noise-resilient contextual embeddings. Bidirectional Long Short-Term Memory (BiLSTM) then reinforces local phrasal dependencies while pruning redundant long-range information. Finally, a lightweight attention head dynamically reweights sentiment-critical tokens, effectively suppressing noise from misspellings, jargon, and code-mixing. Evaluated on both public benchmarks and our self-collected high-noise corpus, the model attains 91.77 % accuracy, 92.22 % F1-score, and 96.48 % ROC-AUC, outperforming vanilla BERT by 2.43 %, 2.51 %, and 1.91 % absolute gains with stronger cross-domain robustness. The study delivers an off-the-shelf baseline for sentiment analysis in noisy Chinese user-generated content scenarios such as e-commerce and government opinion mining.
Keywords:
BERT, BiLSTM, attention mechanism, sentiment analysis, deep learning Abstract: To overcome the bottleneck of existing models in capturing intricate semantics for Chinese review sentiment analysis, we propose a progressive hybrid architecture that operates in three cascading stages: context encoding, sequential refinement, and salience focusing. First, Bidirectional Encoder Representations from Transformers (BERT) pretrained weights are exploited to generate noise-resilient contextual embeddings. Bidirectional Long Short-Term Memory (BiLSTM) then reinforces local phrasal dependencies while pruning redundant long-range information. Finally, a lightweight attention head dynamically reweights sentiment-critical tokens, effectively suppressing noise from misspellings, jargon, and code-mixing. Evaluated on both public benchmarks and our self-collected high-noise corpus, the model attains 91.77 % accuracy, 92.22 % F1-score, and 96.48 % ROC-AUC, outperforming vanilla BERT by 2.43 %, 2.51 %, and 1.91 % absolute gains with stronger cross-domain robustness. The study delivers an off-the-shelf baseline for sentiment analysis in noisy Chinese user-generated content scenarios such as ecommerce and government opinion mining. https://mc.manuscriptcentral.com/jcmse Journal of Computational Methods in Science and Engineering For Peer Review A Three-Stage Rocket of BERT-BiLSTM-Attention: Progressive Sentiment Computing for Noisy Chinese Reviews Zhihang Tang*, Jinyang Shi,Qiwen Yang School of Information Science and Engineering, Hunan University of Engineering, Xiangtan 411104, China zhtang@hnie.edu.cn Abstract: To overcome the bottleneck of existing models in capturing intricate semantics for Chinese review sentiment analysis, we propose a progressive hybrid architecture that operates in three cascading stages: context encoding, sequential refinement, and salience focusing. First, Bidirectional Encoder Representations from Transformers (BERT) pretrained weights are exploited to generate noise-resilient contextual embeddings. Bidirectional Long Short-Term Memory (BiLSTM) then reinforces local phrasal dependencies while pruning redundant long-range information. Finally, a lightweight attention head dynamically reweights sentiment-critical tokens, effectively suppressing noise from misspellings, jargon, and code-mixing. Evaluated on both public benchmarks and our self-collected high-noise corpus, the model attains 91.77 % accuracy, 92.22 % F1-score, and 96.48 % ROC-AUC, outperforming vanilla BERT by 2.43 %, 2.51 %, and 1.91 % absolute gains with stronger cross-domain robustness. The study delivers an off-the-shelf baseline for sentiment analysis in noisy Chinese user-generated content scenarios such as e-commerce and government opinion mining. Keywords: BERT; BiLSTM; attention mechanism; sentiment analysis; deep learning 1.
You need to register before accessing this content.