BanglaMamba: Exploring State Space Models for Bangla Fake News Detection

M. K. Khalidi Siam Affiliation: Department of Computer Science and Engineering Affiliation: BRAC University Affiliation: Dhaka, Bangladesh Email: mk.khalidi.siam@g.bracu.ac.bd

Abstract

Fake news detection has become an important Natural Language Processing (NLP) task due to the rapid spread of misinformation through online news platforms and social media. While transformer-based models such as BanglaBERT achieve strong performance for Bangla text classification, their quadratic computational complexity makes them less suitable for long-document processing in resource-constrained environments. This paper investigates Mamba-based State Space Models (SSMs) as an efficient alternative for Bangla fake news detection. We propose BanglaMamba and compare it with pre-trained BanglaBERT and a similarly configured BERT model trained from scratch. Experimental results show that BanglaBERT achieves the highest Macro-F1 score (0.9260), while BanglaMamba (0.9029) achieves performance comparable to the from-scratch CustomBERT (0.9057) despite using a different architecture. Meanwhile, BanglaMamba achieves approximately $2.2\times$ higher inference throughput and 49% lower inference peak GPU memory usage than the BERT-based models. Cross-dataset evaluation shows that BanglaBERT generalizes better to an external dataset, highlighting the importance of large-scale pretraining. These findings demonstrate that Mamba-based SSMs can provide a competitive and computationally efficient alternative to Transformer-based architectures for Bangla fake news detection, particularly in resource-constrained settings.

1 Introduction

Fake news has become a major challenge in the digital information era, where misleading information spreads rapidly through online news portals and social media. The widespread dissemination of false information can influence public opinion and undermine trust in credible news sources, making automatic fake news detection an important research problem in Natural Language Processing (NLP).

In recent years, transformer-based models such as BERT have significantly improved fake news detection by learning contextual representations that outperform traditional machine learning and conventional deep learning approaches (Devlin et al., 2019; Dell’Oglio et al., 2026). For Bangla NLP, BanglaBERT has emerged as a strong pretrained language model for text classification tasks (Bhattacharjee et al., 2022).

Despite their effectiveness, transformer-based models have important limitations. The self-attention mechanism has quadratic time and memory complexity with respect to input length, resulting in increased computational cost and GPU memory consumption for longer sequences (Vaswani et al., 2017).

Recent advances in State Space Models (SSMs), particularly the Mamba architecture, provide an alternative approach for efficient long-sequence modeling through linear scaling with sequence length while maintaining competitive sequence modeling performance (Gu and Dao, 2024). However, the application of SSMs to Bangla fake news detection remains largely unexplored. Furthermore, existing studies have not systematically compared transformer-based and SSM-based models in terms of classification performance and computational efficiency.

To address these gaps, this paper investigates a Mamba-based State Space Model, referred to as BanglaMamba, for Bangla fake news detection and compares it with a pre-trained BanglaBERT and a similarly configured Transformer model trained from scratch (CustomBERT). The main contributions of this work are as follows:

•

To the best of our knowledge, this is the first study to investigate a Mamba-based State Space Model (SSM) for Bangla fake news detection.
•

We provide a systematic comparison of BanglaMamba with BanglaBERT and a similarly configured CustomBERT in terms of classification performance and computational efficiency, including GPU memory usage, inference latency, and throughput, while using the from-scratch Transformer baseline to distinguish architectural effects from the benefits of language pretraining.
•

We evaluate the out-of-domain generalization of all three models on an external Bangla fake news dataset to examine their robustness under a shifted data distribution.

2 Related Work

Fake news detection has been extensively studied using traditional Machine Learning (ML), Deep Learning (DL), and, more recently, Transformer-based architectures across different languages. Early studies primarily relied on statistical machine learning models before gradually transitioning to contextual language models.

For Bangla fake news detection, Khatun and Khan (2024) constructed a dataset of fake and authentic Bangla news headlines and evaluated several machine learning algorithms alongside transformer-based models. Their results showed that BanglaBERT consistently outperformed all evaluated machine learning methods, demonstrating the effectiveness of contextual language representations.

Subsequent studies explored deep learning architectures. Habiba et al. (2024) compared Bangla ELECTRA with a custom Deep CNN and reported that the CNN slightly outperformed the transformer model. Similarly, Rasel et al. (2022) evaluated machine learning, deep learning, and transformer-based approaches for Bangla fake news detection and found that CNN achieved the highest classification performance, exceeding BanglaBERT by approximately 1%. Nevertheless, BanglaBERT remained highly competitive, indicating the effectiveness of transformer-based models.

More recent studies have highlighted the superior generalization capability of transformers. Dell’Oglio et al. (2026) compared machine learning, deep learning, transformer-based models, and large language models across ten English fake news datasets. While Logistic Regression and Support Vector Machine (SVM) performed competitively on smaller datasets, CNN and BiLSTM showed inconsistent performance and were more susceptible to overfitting. Fine-tuned transformer models achieved the best balance between classification accuracy and cross-domain generalization. Similar findings were reported by Tabassum et al. (2024) for Malayalam fake news detection. Additionally, Chowdhury et al. (2025) showed that combining data augmentation with transformer fine-tuning significantly improved Bangla fake news detection performance.

Earlier Bangla fake news detection studies largely relied on handcrafted or shallow feature extraction techniques. Traditional machine learning methods commonly used Bag-of-Words (BoW), TF-IDF, and stemming, whereas deep learning models typically employed static Word2Vec embeddings (Khatun and Khan, 2024; Habiba et al., 2024; Rasel et al., 2022). Although computationally efficient, these approaches cannot effectively capture contextual meaning, long-range semantic dependencies, or word order (Coenen et al., 2019). In contrast, transformer-based models automatically learn contextual embeddings using subword tokenization and self-attention (Devlin et al., 2019; Clark et al., 2020; Bhattacharjee et al., 2022). Their hierarchical representations capture lexical, syntactic, and semantic information across different layers (Jawahar et al., 2019; Coenen et al., 2019), making transformers the dominant paradigm for modern Bangla NLP.

However, self-attention of transformer has quadratic time and memory complexity with respect to sequence length, making transformers computationally expensive for long-document processing (Vaswani et al., 2017). State Space Models (SSMs) have emerged as an alternative for efficient long-sequence modeling, with Structured State Spaces (S4) demonstrating the ability to capture long-range dependencies while maintaining favorable computational efficiency (Gu et al., 2022). Building on this line of work, Mamba introduces a selective state-space mechanism that enables input-dependent information propagation and achieves linear scaling in sequence length, while maintaining competitive performance on long sequences (Gu and Dao, 2024). To the best of our knowledge, no published study has investigated SSMs for Bangla fake news detection. Moreover, existing studies have not systematically compared SSMs with BanglaBERT in terms of classification performance and computational efficiency.

3 Methodology

3.1 Problem Formulation

This study addresses Bangla fake news detection as a binary text classification problem. Given a news article consisting of a headline and body text, the objective is to classify the article as Authentic (label = 1) or Fake (label = 0). Formally, the classifier learns a mapping function

f\colon X\rightarrow\{0,1\},

where $X$ denotes the space of tokenized Bangla text sequences.

Two neural architectures and one reference baseline are evaluated:

•

BanglaBERT, a pre-trained Transformer encoder used as the primary baseline.
•

CustomBERT, a Transformer model with a configuration comparable to BanglaBERT but trained entirely from scratch, providing a fair architecture-level comparison without large-scale pretraining.
•

BanglaMamba, a Mamba-based State Space Model (SSM) trained entirely from scratch.

This study investigates the capability of the Mamba State Space Model architecture for Bangla fake news detection by comparing its classification performance with both a pre-trained Transformer baseline (BanglaBERT) and an architecturally comparable Transformer trained from scratch (CustomBERT). This comparison enables the analysis of the impact of model architecture while reducing the influence of large-scale language pretraining.

3.2 Dataset and Preprocessing

3.2.1 Dataset

All experiments are conducted using the BanFakeNews-2.0 dataset (Shibu et al., 2025), a publicly available Bangla fake news corpus containing news articles annotated as Fake (0) or Real (1). Each sample consists of a news headline and its corresponding article body. To evaluate out-of-domain generalization, we additionally use the BanglaFakeNews2025 dataset (Maliha et al., 2026) as an external evaluation dataset. Both datasets undergo the same preprocessing procedure to ensure consistency. After preprocessing, BanFakeNews-2.0 contains 58,001 samples, with 16.5% labeled as Fake and 83.5% as Real, while the external BanglaFakeNews2025 test set contains 3,979 samples, with 49.9% Fake and 50.1% Real.

3.2.2 Preprocessing

A preprocessing pipeline was applied to standardize the textual data before training. First, the headline and article body were concatenated using the [SEP] token to enable joint contextual learning. The preprocessing steps included: (i) removal of HTML tags and URLs using regular expressions, (ii) Unicode NFC normalization for consistent Bangla encoding, (iii) whitespace normalization by collapsing multiple spaces and trimming leading/trailing spaces, (iv) removal of articles with fewer than 20 or more than 2,000 words, (v) duplicate removal, and (vi) elimination of empty samples after preprocessing. Punctuation marks were preserved, as they may contain stylistic cues relevant to fake news detection. Approximately 5.5% of the articles were removed as duplicates, while a further 0.3% were removed because they contained fewer than 20 or more than 2,000 words.

The processed dataset was split into 80% training, 10% validation, and 10% testing sets using stratified sampling to preserve the class distribution. Since the dataset was imbalanced, with approximately 83.5% Real and 16.5% Fake samples, inverse-frequency class weights were computed from the training set using Scikit-learn’s balanced class-weighting strategy and incorporated into the weighted cross-entropy loss during training.

3.2.3 Context-Length Evaluation Sets

Since all three models are evaluated with a maximum input length of 512 tokens, the held-out test set is divided into two subsets according to the BanglaBERT tokenizer to examine the effect of truncation:

•

Short articles ( $\leq 512$ tokens): BanglaBERT, CustomBERT, and BanglaMamba process the complete article without truncation.
•

Long articles ( $>512$ tokens): BanglaBERT, CustomBERT, and BanglaMamba process only the first 512 tokens, with the remaining content truncated.

This evaluation examines whether the 512-token input constraint and the resulting truncation for longer articles substantially affect fake news detection performance across the three models.

3.3 Tokenization

All models use the BanglaBERT’s SentencePiece tokenizer with a vocabulary size of 32,000 subword tokens. Using a common tokenizer ensures that differences in classification performance arise from the model architectures rather than differences in tokenization. Sequences shorter than the maximum sequence length are padded, while longer sequences are truncated according to each model’s context limit.

3.4 Model Architecture

3.4.1 BanglaBERT and CustomBERT

BanglaBERT serves as the pre-trained Transformer baseline. It is a BERT-style Transformer encoder pre-trained on a large Bangla corpus. We use the smaller version of the BanglaBERT base model, referred to as BanglaBERT-small; throughout this paper, we simply refer to it as BanglaBERT (Bhattacharjee et al., 2022). The exact pretrained checkpoint used in our experiments is publicly available on the Hugging Face Model Hub.¹¹ 1 https://huggingface.co/csebuetnlp/banglabert_small A linear classification layer is attached to the [CLS] token representation, and all parameters are fine-tuned end-to-end on the training set.

CustomBERT uses the same architecture and configuration as BanglaBERT but is trained entirely from scratch on the training data, without any large-scale language pretraining. This model is introduced to provide a more direct comparison with BanglaMamba by controlling for the effect of pretraining. Thus, the comparison between CustomBERT and BanglaMamba primarily reflects differences in the underlying Transformer and State Space Model architectures, while the comparison between BanglaBERT and CustomBERT illustrates the effect of pretraining under the same Transformer configuration.

Property	BanglaBERT	CustomBERT
Architecture	Transformer Encoder	Transformer Encoder
Parameters	13.7M	13.7M
Hidden Dimension	256	256
Hidden Layers	12	12
Maximum Context Length	512 Tokens	512 Tokens
Pretraining	Yes	No

Table 1: Configuration of BanglaBERT and CustomBERT.

3.4.2 BanglaMamba

The proposed BanglaMamba model is built upon the Mamba State Space Model architecture and is trained entirely from scratch without using any pre-trained language representations. Since no publicly available pre-trained Mamba model is available within the parameter range of BanglaBERT (110M) and BanglaBERT-small (13.7M), we train BanglaMamba from scratch. To ensure a fair comparison with BanglaBERT, the model is configured with an almost similar parameter count to BanglaBERT.

Property	BanglaMamba
Architecture	Mamba
Parameters	12.61M
Hidden Dimension (d_model)	256
Hidden Layer (n_layer)	10
Maximum Context Length	512 Tokens

Table 2: Configuration of the BanglaMamba.

The architecture consists of three components, illustrated in Figure 1:

1.

Embedding Layer and Mamba Backbone, which generate contextual token representations.
2.

Masked Mean Pooling, which computes a document-level representation by averaging hidden states corresponding only to valid tokens.
3.

Classification Head, consisting of:

$\text{LayerNorm}\rightarrow\text{Linear}\rightarrow\text{GELU}\rightarrow\text{Dropout}(0.2)\rightarrow\text{Linear}$

to perform binary classification.

Refer to caption — Figure 1: BanglaMamba model architecture.

3.5 Training Procedure

All three models are trained using the weighted cross-entropy loss function and optimized with the AdamW optimizer. Since the three models use different architectures and initialization strategies, different learning rates are employed for each model. Training is conducted for 5 epochs using BF16 (bfloat16) mixed-precision computation to improve training efficiency while maintaining numerical stability. All experiments are performed on a single NVIDIA A100 GPU with 40 GB of memory.

At the end of each epoch, the models are evaluated on the validation set. The checkpoint achieving the highest Macro-F1 score on the validation set is selected as the final model and subsequently used for evaluation on the test set.

3.6 Evaluation Methodology

Due to the substantial class imbalance in BanFakeNews-2.0 (83.5% authentic, 16.5% fake), overall accuracy is not a reliable evaluation metric. Therefore, model performance is evaluated using multiple complementary metrics covering overall classification performance, class-wise discrimination, prediction reliability, and inference efficiency. In addition to predictive performance, inference efficiency is evaluated under a consistent configuration using a batch size of 64, torch.no_grad() to disable gradient computation, and a maximum sequence length of 512 tokens for both models. All experiments are repeated using three random seeds (22, 33, and 42) to ensure reproducibility and reduce the influence of run-specific randomness. The results are averaged across the three runs to provide a more stable and reliable estimate of model performance, with the corresponding standard deviations also reported to quantify the variability across runs.

3.6.1 Evaluation Metrics

Primary metric. Macro-F1 is used as the primary evaluation metric because it gives equal weight to both classes, preventing the performance on the majority authentic class from masking performance on the minority fake class.

Classification metrics. F1-score are reported separately for the fake (label 0) and authentic (label 1) classes. AUC-ROC measures the model’s threshold-independent discriminative ability, while PR-AUC provides a complementary evaluation that is particularly informative under class imbalance by emphasizing performance on the positive class.

Reliability metrics. Matthews Correlation Coefficient (MCC) is used as a robust measure of classification performance under class imbalance. Expected Calibration Error (ECE) is used to assess the alignment between predicted confidence and observed accuracy, computed using 10 equal-width bins over the confidence interval $[0,1]$ :

\mathrm{ECE}=\sum_{b=1}^{B}\frac{|S_{b}|}{N}\left|\mathrm{acc}(S_{b})-\mathrm{conf}(S_{b})\right|,

(1)

where $B=10$ , $S_{b}$ denotes the samples in bin $b$ , and $\mathrm{acc}(S_{b})$ and $\mathrm{conf}(S_{b})$ represent the empirical accuracy and mean confidence, respectively.

3.6.2 Efficiency Metrics

Peak GPU VRAM. Measures the maximum GPU memory allocated during inference.

P50 and P95 Latency. P50 (median) latency represents typical inference latency per batch, while P95 latency captures tail inference latency per batch.

Throughput (samples/s). Measures the number of news articles processed per second during inference.

Complete details of the experimental setup, training, hyperparameter configurations, and detailed results are provided in the Appendix.

4 Result Analysis

4.1 Classification Performance Comparison

Metric	BanglaBERT	BanglaMamba	CustomBERT
Macro F1	0.9260 $\pm$ 0.0008	0.9029 $\pm$ 0.0034	0.9057 $\pm$ 0.0078
AUC-ROC	0.9632 $\pm$ 0.0028	0.9420 $\pm$ 0.0103	0.9407 $\pm$ 0.0235
MCC	0.8534 $\pm$ 0.0024	0.8086 $\pm$ 0.0085	0.8137 $\pm$ 0.0144
PR-AUC	0.9896 $\pm$ 0.0013	0.9820 $\pm$ 0.0049	0.9810 $\pm$ 0.0097
ECE	0.0507 $\pm$ 0.0371	0.0457 $\pm$ 0.0134	0.0537 $\pm$ 0.0176
Fake F1	0.8753 $\pm$ 0.0011	0.8361 $\pm$ 0.0057	0.8408 $\pm$ 0.0136
Real F1	0.9766 $\pm$ 0.0005	0.9697 $\pm$ 0.0017	0.9706 $\pm$ 0.0021

Table 3: Classification performance comparison among BanglaBERT, BanglaMamba, and CustomBERT.

Table 3 shows that BanglaBERT consistently achieves the highest classification performance across the evaluated metrics. In contrast, BanglaMamba and CustomBERT, both trained entirely from scratch with comparable model sizes, exhibit very similar performance. Their Macro-F1 scores are 0.9029 and 0.9057, respectively, while their Fake F1 scores are 0.8361 and 0.8408. The close results across the other metrics further indicate that neither model has a clear overall advantage when both are trained without large-scale language pretraining.

The performance gap between the two scratch-trained models and BanglaBERT suggests that pre-trained linguistic representations provide an important advantage for this classification task, particularly for the more challenging Fake class. However, the similar performance of BanglaMamba and CustomBERT indicates that, under comparable training conditions, the Mamba architecture can achieve classification performance comparable to a Transformer architecture of similar scale. This provides a more controlled comparison of the two architectures while separating the effect of large-scale pretraining from the effect of the underlying model architecture.

4.2 Computational Efficiency Comparison

Metric	BanglaBERT	BanglaMamba	CustomBERT
P50 Latency (ms)	37.0767 $\pm$ 0.2701	14.8433 $\pm$ 2.2382	37.0333 $\pm$ 0.2845
P95 Latency (ms)	39.03 $\pm$ 3.0967	17.4267 $\pm$ 3.1683	39.05 $\pm$ 3.12
Throughput (sample/s)	1675.03 $\pm$ 24.8858	3623.2333 $\pm$ 125.4971	1680.13 $\pm$ 23.3548
Inference Peak VRAM (MB)	1011.9 $\pm$ 0	514.5 $\pm$ 0	1011.9 $\pm$ 0
Training Peak VRAM (MB)	5141.6 $\pm$ 0	1145.4 $\pm$ 0	5141.6 $\pm$ 0

Table 4: Efficiency comparison among BanglaBERT, BanglaMamba, and CustomBERT.

Table 4 compares the efficiency of BanglaBERT, BanglaMamba, and CustomBERT across inference latency, throughput, and memory consumption. BanglaMamba demonstrates substantial improvements in inference speed and processing capacity, achieving approximately $2.5\times$ lower P50 latency and $2.2\times$ lower P95 latency than BanglaBERT. At the same time, it achieves approximately $2.2\times$ higher throughput, indicating that it can process substantially more samples per second under the same evaluation configuration. Although BanglaMamba exhibits higher variability in throughput across runs, its average throughput remains considerably higher than that of BanglaBERT.

Since CustomBERT uses exactly the same architecture and configuration as BanglaBERT, its efficiency metrics are effectively identical. Overall, BanglaMamba demonstrates a clear efficiency advantage, requiring approximately $49\%$ less peak VRAM during inference and nearly $78\%$ less during training. The lower latency, higher throughput, and reduced memory consumption are consistent with the computational advantages of its linear-time sequence processing, highlighting its potential for resource-constrained deployment.

4.3 Input Length Robustness Analysis

Metric	BanglaBERT		BanglaMamba		CustomBERT
	Short	Long	Short	Long	Short	Long
Macro F1	0.9309 $\pm$ 0.0047	0.9054 $\pm$ 0.0175	0.9030 $\pm$ 0.0012	0.9012 $\pm$ 0.0217	0.9049 $\pm$ 0.0063	0.9080 $\pm$ 0.0149
AUC-ROC	0.9662 $\pm$ 0.0062	0.9466 $\pm$ 0.0193	0.9435 $\pm$ 0.0088	0.9351 $\pm$ 0.0201	0.9424 $\pm$ 0.0213	0.9336 $\pm$ 0.0363
MCC	0.8633 $\pm$ 0.0099	0.8127 $\pm$ 0.0342	0.8092 $\pm$ 0.0046	0.8042 $\pm$ 0.0427	0.8120 $\pm$ 0.0122	0.8196 $\pm$ 0.0253
Fake F1	0.8831 $\pm$ 0.0082	0.8427 $\pm$ 0.0323	0.8357 $\pm$ 0.0024	0.8351 $\pm$ 0.0391	0.8390 $\pm$ 0.0106	0.8462 $\pm$ 0.0284

Table 5: Performance comparison of BanglaBERT, BanglaMamba, and CustomBERT on short and long articles.

Table 5 compares model performance on short and long articles when all models are restricted to a maximum input length of 512 tokens. Overall, the performance differences are relatively small, indicating that truncation of longer articles does not substantially degrade classification performance. BanglaBERT shows a somewhat larger decrease across the evaluated metrics, while BanglaMamba and CustomBERT exhibit smaller changes, with CustomBERT even showing a slight improvement in Macro-F1 and Fake F1 on long articles. These results suggest that the 512-token input constraint does not significantly affect the overall classification behavior of the models on longer articles.

4.4 Cross-Dataset Generalization

Metric	BanglaBERT	BanglaMamba	CustomBERT
Macro F1	0.7914 $\pm$ 0.0348	0.5753 $\pm$ 0.0703	0.7227 $\pm$ 0.0892
AUC-ROC	0.8693 $\pm$ 0.0185	0.7598 $\pm$ 0.0164	0.7777 $\pm$ 0.1218
MCC	0.5913 $\pm$ 0.0708	0.2590 $\pm$ 0.0767	0.4746 $\pm$ 0.1696
PR-AUC	0.8622 $\pm$ 0.0076	0.7845 $\pm$ 0.0172	0.7197 $\pm$ 0.1502
ECE	0.0929 $\pm$ 0.0423	0.2564 $\pm$ 0.0472	0.1448 $\pm$ 0.0605
Fake F1	0.7763 $\pm$ 0.0365	0.4564 $\pm$ 0.1253	0.6824 $\pm$ 0.1081
Real F1	0.8064 $\pm$ 0.0337	0.6942 $\pm$ 0.0154	0.7630 $\pm$ 0.0706

Table 6: Classification performance among BanglaBERT, BanglaMamba, and CustomBERT on the cross-dataset.

To assess out-of-domain generalization, all three models were evaluated on an external Bangla fake news dataset that was not used during training. Out-of-distribution (OOD) generalization remains a known challenge in NLP when the evaluation distribution differs from the training distribution (Yang et al., 2023). As shown in Table 6, BanglaBERT achieves the strongest overall OOD performance, followed by CustomBERT and BanglaMamba. In particular, BanglaBERT achieves substantially higher Macro-F1, MCC, and Fake F1 than BanglaMamba, while CustomBERT also outperforms BanglaMamba across these classification metrics. However, CustomBERT exhibits noticeably larger standard deviations across several metrics, indicating greater variation in performance across runs. In contrast, BanglaMamba shows lower variability for several metrics, although its overall classification performance remains substantially weaker.

The weaker OOD performance of the two models trained from scratch, particularly BanglaMamba, suggests that the absence of large-scale language pretraining may limit generalization beyond the downstream training distribution. This interpretation is consistent with prior work showing that effective language-model training depends strongly on an appropriate balance between model capacity and the amount of training data (Hoffmann et al., 2022). However, the observed performance gap cannot be attributed solely to the absence of pretraining, as architectural and optimization differences may also contribute. These results motivate large-scale pretraining of BanglaMamba on diverse Bangla text as an important direction for future work.

4.5 Ablation Study

Metric	BanglaBERT (Mean Pooling)
Macro F1	0.9144 $\pm$ 0.0081
AUC-ROC	0.9565 $\pm$ 0.0077
MCC	0.8300 $\pm$ 0.0172
PR-AUC	0.9886 $\pm$ 0.0019
ECE	0.0866 $\pm$ 0.0182
Fake F1	0.8565 $\pm$ 0.0124
Real F1	0.9723 $\pm$ 0.0038

Table 7: Classification performance of BanglaBERT using mean pooling instead of the [CLS] representation.

Our main experiments use masked mean pooling for BanglaMamba and the [CLS] representation for BanglaBERT. To examine whether the pooling strategy substantially affects the BanglaBERT results, we additionally evaluate BanglaBERT using masked mean pooling, with the results shown in Table 7. The performance remains broadly comparable to the original [CLS]-based approach, although [CLS] pooling achieves slightly better overall results. This is consistent with the fact that BanglaBERT was pre-trained with the [CLS] representation as the sequence-level representation.

We do not perform a corresponding [CLS] ablation for BanglaMamba because the two architectures use fundamentally different sequence-processing mechanisms. The [CLS] representation is specifically designed for BERT-style bidirectional self-attention, whereas BanglaMamba processes sequences causally. Masked mean pooling therefore provides a more architecture-independent sequence representation for our comparison.

5 Limitations and Future Work

Although both BanglaBERT and BanglaMamba achieve strong in-domain performance, the class imbalance in the BanFakeNews-2.0 dataset remains a limitation. Both models exhibit lower performance on the minority (Fake) class than on the majority (Real) class, indicating that identifying fake news remains the more challenging aspect of the task. A key limitation of this study is that BanglaMamba is trained entirely from scratch using only the downstream fake news dataset. Due to computational resource constraints, we were unable to perform large-scale language pretraining. The cross-dataset evaluation further shows that BanglaMamba performs substantially worse than BanglaBERT on an external dataset, suggesting that the absence of large-scale pretraining may limit its out-of-domain generalization. However, this performance gap cannot be attributed solely to the lack of pretraining, as architectural and optimization differences may also contribute.

Future work should investigate large-scale pretraining of BanglaMamba on diverse Bangla text and evaluate the resulting model across multiple Bangla fake news datasets. Such experiments would provide a stronger assessment of the generalization capability and potential of pretrained Mamba-based models for Bangla fake news detection.

6 Conclusion

This study investigated Mamba-based State Space Models (SSMs) for Bangla fake news detection by comparing BanglaMamba with BanglaBERT and a similarly configured CustomBERT trained from scratch. Although BanglaBERT achieves the highest in-domain classification performance, BanglaMamba achieves competitive results despite being trained entirely from scratch, while providing substantially lower inference latency, higher throughput, and lower GPU memory usage. The similar in-domain performance of BanglaMamba and CustomBERT also suggests that training from scratch can produce competitive classification performance despite the absence of large-scale language pretraining.

Cross-dataset evaluation reveals weaker out-of-domain generalization for both models trained from scratch, with BanglaMamba showing the largest performance degradation. These results highlight the potential importance of large-scale language pretraining for improving generalization, while indicating that the observed gap cannot be attributed solely to model architecture. Overall, the findings demonstrate that Mamba-based SSMs are a promising and computationally efficient alternative to Transformer-based models for Bangla fake news detection. Future work should investigate large-scale pretraining of BanglaMamba and evaluate it across multiple Bangla fake news datasets.

References

Bhattacharjee et al. (2022) A. Bhattacharjee, T. Hasan, W. Ahmad, K. S. Mubasshir, M. S. Islam, A. Iqbal, M. S. Rahman, and R. Shahriyar BanglaBERT: language model pretraining and benchmarks for low-resource language understanding evaluation in Bangla. In Findings of the Association for Computational Linguistics: NAACL 2022, M. Carpuat, M. de Marneffe, and I. V. Meza Ruiz (Eds.), Seattle, United States, pp. 1318–1327. External Links: Link, Document Cited by: §1, §2, §3.4.1.
Chowdhury et al. (2025) A. S. Chowdhury, G. M. Shahariar, A. T. Aziz, S. M. Alam, Md. A. Sheikh, and T. A. Belal Tackling fake news in bengali: unraveling the impact of summarization vs. augmentation on pre-trained language models. SN Computer Science 6 (8). External Links: ISSN 2661-8907, Link, Document Cited by: §2.
Clark et al. (2020) K. Clark, M. Luong, Q. V. Le, and C. D. Manning ELECTRA: pre-training text encoders as discriminators rather than generators. External Links: 2003.10555, Link Cited by: §2.
Coenen et al. (2019) A. Coenen, E. Reif, A. Yuan, B. Kim, A. Pearce, F. Viégas, and M. Wattenberg Visualizing and measuring the geometry of bert. External Links: 1906.02715, Link Cited by: §2.
Dell’Oglio et al. (2026) P. Dell’Oglio, A. Bondielli, F. Marcelloni, and L. C. Passaro An experimental comparison of the most popular approaches to fake news detection. Information Sciences 745, pp. 123407. External Links: ISSN 0020-0255, Link, Document Cited by: §1, §2.
Devlin et al. (2019) J. Devlin, M. Chang, K. Lee, and K. Toutanova BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio (Eds.), Minneapolis, Minnesota, pp. 4171–4186. External Links: Link, Document Cited by: §1, §2.
Gu and Dao (2024) A. Gu and T. Dao Mamba: linear-time sequence modeling with selective state spaces. External Links: 2312.00752, Link Cited by: §1, §2.
Gu et al. (2022) A. Gu, K. Goel, and C. Ré Efficiently modeling long sequences with structured state spaces. External Links: 2111.00396, Link Cited by: §2.
Habiba et al. (2024) S. U. Habiba, F. T. Johora, T. Mahmud, T. Tasnim, F. Tasnim, S. Kabir, M. S. Hossain, and K. Andersson Enhancing low-resource bangla fake news detection through deep convolutional neural networks. In Intelligent Computing and Optimization, P. Vasant, V. Panchenko, E. Munapo, G. Weber, J. J. Thomas, R. Intan, and M. Shamsul Arefin (Eds.), Cham, pp. 104–114. External Links: ISBN 978-3-031-73318-5 Cited by: §2, §2.
Hoffmann et al. (2022) J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, and L. Sifre Training compute-optimal large language models. External Links: 2203.15556, Link Cited by: §4.4.
Jawahar et al. (2019) G. Jawahar, B. Sagot, and D. Seddah What does BERT learn about the structure of language?. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, A. Korhonen, D. Traum, and L. Màrquez (Eds.), Florence, Italy, pp. 3651–3657. External Links: Link, Document Cited by: §2.
Khatun and Khan (2024) Mst. S. Khatun and I. Khan Bangla counterfeit news identification: using the power of bert. In 2024 IEEE International Conference on Power, Electrical, Electronics and Industrial Applications (PEEIACON), Vol. , pp. 518–522. External Links: Document Cited by: §2, §2.
Maliha et al. (2026) S. Maliha, Md. S. Hossain, Md. M. Anwar, and S. J. Seema BanglaFakeNews2025: a high-quality benchmark dataset for bangla fake news detection. Mendeley Data. External Links: Document Cited by: §3.2.1.
Rasel et al. (2022) R. I. Rasel, A. H. Zihad, N. Sultana, and M. M. Hoque Bangla fake news detection using machine learning, deep learning and transformer models. In 2022 25th International Conference on Computer and Information Technology (ICCIT), Vol. , pp. 959–964. External Links: Document Cited by: §2, §2.
Shibu et al. (2025) H. M. Shibu, S. Datta, Md. S. Miah, N. Sami, M. S. Chowdhury, and M. S. Islam From scarcity to capability: empowering fake news detection in low-resource languages with LLMs. In Proceedings of the First Workshop on Natural Language Processing for Indo-Aryan and Dravidian Languages, R. Weerasinghe, I. Anuradha, and D. Sumanathilaka (Eds.), Abu Dhabi, pp. 100–107. External Links: Link Cited by: §3.2.1.
Tabassum et al. (2024) N. Tabassum, S. Aodhora, R. Akter, J. Hossain, S. Ahsan, and M. M. Hoque Punny_Punctuators@DravidianLangTech-EACL2024: transformer-based approach for detection and classification of fake news in Malayalam social media text. In Proceedings of the Fourth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages, B. R. Chakravarthi, R. Priyadharshini, A. K. Madasamy, S. Thavareesan, E. Sherly, R. Nadarajan, and M. Ravikiran (Eds.), St. Julian’s, Malta, pp. 180–186. External Links: Link, Document Cited by: §2.
Vaswani et al. (2017) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin Attention is all you need. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30, pp. . External Links: Link Cited by: §1, §2.
Yang et al. (2023) L. Yang, Y. Song, X. Ren, C. Lyu, Y. Wang, J. Zhuo, L. Liu, J. Wang, J. Foster, and Y. Zhang Out-of-distribution generalization in natural language processing: past, present, and future. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali (Eds.), Singapore, pp. 4533–4559. External Links: Link, Document Cited by: §4.4.

AI Usage Statement

We used AI-assisted tools to improve the quality of our writing, including grammatical correction and language polishing. We also used AI tools to assist with code development. All experimental data were generated through our own experiments and pipelines. No results, claims, or empirical findings were generated by AI tools. All interpretations, analyses, and conclusions were produced by the authors, with AI tools used solely to polish the presentation of the analysis.

Appendix A Experimental Setup

Component	Specification
Programming Language	Python 3.11
Deep Learning Framework	PyTorch 2.4.0 + CUDA 12.4
Transformer Library	Transformers 4.40.0
State Space Model Library	Mamba-SSM 2.2.2
GPU	NVIDIA A100 (40 GB)

Table 8: Experimental setup.

Appendix B BanglaMamba Model Configuration

Configuration	Value
$d_{\mathrm{model}}$	256
$head\_hidden\_dim$	128
$n_{\mathrm{layer}}$	10
Dropout	0.2
Parameters	12.61M

Table 9: Configurations of the BanglaMamba model.

The Mamba-specific parameters were kept unchanged from the default configuration of the official mamba-ssm package. Specifically, the state dimension ( $d_{\mathrm{state}}$ ) was set to 16, the convolutional kernel size ( $d_{\mathrm{conv}}$ ) was set to 4, and the expansion factor (expand) was set to 2. The $\Delta$ rank (dt_rank) was set to auto, calculated as $\lceil d_{\mathrm{model}}/16\rceil$ . In addition, use_bias was set to False and conv_bias was set to True, following the default settings of the official mamba-ssm implementation.

Appendix C Training Configuration

Hyperparameter	BanglaBERT	BanglaMamba	CustomBERT
Vocabulary Size	32,000	32,000	32,000
Maximum Sequence Length	512	512	512
Number of Labels	2	2	2
Epochs	5	5	5
Batch Size	32	32	32
Gradient Accumulation	2	2	2
Effective Batch Size	64	64	64
Learning Rate	$2\times 10^{-5}$	$1\times 10^{-3}$	$2\times 10^{-4}$
Weight Decay	0.01	0.01	0.01
Warm-up Ratio	0.10	0.10	0.10
Precision	BF16	BF16	BF16

Table 10: Training configurations for BanglaBERT, BanglaMamba, and CustomBERT.

C.1 Training Dynamics

Figure 2 presents the validation Macro-F1 across the five training epochs for BanglaBERT, BanglaMamba, and CustomBERT over three random seeds. The models were trained using the learning rates reported in Table 10. We also evaluated alternative learning rates for each model, but none produced better validation performance than the selected configurations. For the pretrained BanglaBERT, the lower learning rate of $2\times 10^{-5}$ provided the most stable and effective fine-tuning, whereas larger learning rates did not improve performance. For BanglaMamba, trained entirely from scratch, $10^{-3}$ provided the best performance. In contrast, applying $10^{-3}$ to CustomBERT resulted in ineffective optimization, with the model collapsing to the majority class and obtaining a validation Macro-F1 of only 0.455 across all five epochs. Reducing the learning rate to $2\times 10^{-4}$ substantially improved CustomBERT’s performance.

Under the selected learning rates, all three models generally reached their best validation performance within the first few epochs, with limited or inconsistent improvement afterward. BanglaBERT improved more gradually, while BanglaMamba and CustomBERT generally reached their best performance earlier. These training dynamics, together with the lack of meaningful improvement in later epochs, support the use of five training epochs for the reported experiments.

Appendix D Additional Experimental Results

D.1 BanglaBERT Results

Metric	Seed 22	Seed 33	Seed 42
Macro F1	0.9255	0.9255	0.9269
AUC-ROC	0.9607	0.9626	0.9662
MCC	0.8523	0.8517	0.8561
PR-AUC	0.9885	0.9892	0.9910
ECE	0.0313	0.0934	0.0273
Fake F1	0.8746	0.8747	0.8766
Real F1	0.9765	0.9762	0.9772

Table 11: BanglaBERT classification performance across three random seeds.

Metric	Seed 22	Seed 33	Seed 42
P50 Latency (ms)	37.35	36.81	37.07
P95 Latency (ms)	42.60	37.07	37.42
Throughput (samples/s)	1649.46	1699.17	1676.46
Inference Peak VRAM (MB)	1011.9	1011.9	1011.9
Training Peak VRAM (MB)	5141.6	5141.6	5141.6

Table 12: BanglaBERT efficiency metrics across three random seeds.

D.2 BanglaMamba Results

Metric	Seed 22	Seed 33	Seed 42
Macro F1	0.9036	0.9060	0.8992
AUC-ROC	0.9539	0.9352	0.9369
MCC	0.8075	0.8176	0.8007
PR-AUC	0.9873	0.9776	0.9810
ECE	0.0313	0.0479	0.0578
Fake F1	0.8384	0.8403	0.8297
Real F1	0.9688	0.9717	0.9687

Table 13: BanglaMamba classification performance across three random seeds.

Metric	Seed 22	Seed 33	Seed 42
P50 Latency (ms)	16.07	16.20	12.26
P95 Latency (ms)	17.87	20.35	14.06
Throughput (samples/s)	3744.20	3631.85	3493.65
Inference Peak VRAM (MB)	514.5	514.5	514.5
Training Peak VRAM (MB)	1145.4	1145.4	1145.4

Table 14: BanglaMamba efficiency metrics across three random seeds.

D.3 CustomBERT Results

Metric	Seed 22	Seed 33	Seed 42
Macro F1	0.9051	0.9137	0.8982
AUC-ROC	0.9504	0.9579	0.9139
MCC	0.8108	0.8294	0.8010
PR-AUC	0.9845	0.9884	0.9700
ECE	0.0406	0.0467	0.0737
Fake F1	0.8405	0.8545	0.8273
Real F1	0.9696	0.9730	0.9691

Table 15: CustomBERT classification performance across three random seeds.

Metric	Seed 22	Seed 33	Seed 42
P50 Latency (ms)	36.90	36.84	37.36
P95 Latency (ms)	37.37	37.13	42.65
Throughput (samples/s)	1690.40	1696.59	1653.40
Inference Peak VRAM (MB)	1011.9	1011.9	1011.9
Training Peak VRAM (MB)	5141.6	5141.6	5141.6

Table 16: CustomBERT efficiency metrics across three random seeds.

Appendix E Data Integrity and Leakage Prevention

To ensure that the reported results are not affected by data leakage, we performed a systematic check of the data preprocessing, partitioning, and evaluation pipeline. All corpus-level cleaning and deduplication operations were completed before dataset partitioning. The cleaned corpus was then divided using a stratified 80/10/10 train–validation–test split with a fixed random seed.

E.1 Deduplication and Data Preprocessing

Each article was represented by the concatenation of its headline and body. Before deduplication, the text was normalized by removing HTML artifacts and URLs, applying Unicode NFC normalization, and collapsing redundant whitespace. Exact duplicates were then removed based on the complete normalized headline–body representation, retaining only the first occurrence. No approximate or semantic similarity matching was used.

After deduplication, articles containing fewer than 20 or more than 2,000 words were removed. Only after these preprocessing steps was the dataset partitioned into training, validation, and test sets. This ordering prevents identical normalized articles from occurring across different partitions.

E.2 Leakage Prevention During Model Training and Evaluation

The BanglaBERT tokenizer was used in its pretrained form and was not fitted or adapted to the study dataset. Class weights were computed exclusively from the training partition. During training, model checkpoints were selected solely according to validation Macro-F1, while the test set remained completely excluded from model selection and parameter updates. The final reported test results were obtained from the selected checkpoint in a separate evaluation pass.

These procedures provide safeguards against duplicate contamination, tokenizer fitting leakage, label-statistic leakage, and test-set-driven model selection.