Sparse Attention Combined with RAG Technology for Financial Data Analysis

Authors

  • Zhaoyan Zhang

    Zhongke Zhidao (Beijing) Technology Co., Ltd., Beijing 102627, China
  • Kaixian Xu

    Risk & Quant Analytics, BlackRock, 50 Hudson Yards, New York, NY 10001, USA
  • Yu Qiao

    Meta Platforms, Inc., Bellevue, WA 98005, USA
  • Alan Wilson

    Intact Financial Corporation, Toronto, ON M5H 1H1, Canada

DOI:

https://doi.org/10.30564/jcsr.v7i2.8933
Received: 3 March 2025 | Revised: 13 March 2025 | Accepted: 13 March 2025 | Published Online: 26 March 2025

Abstract

In response to the challenges of multimodal data integration, real-time information retrieval, model hallucination, and lack of interpretability in financial stock analysis, this paper proposes an innovative financial analysis framework—FSframe. It aims to address multiple challenges in stock analysis within the financial sector. The framework integrates various technological modules to provide comprehensive and efficient solutions for stock trend prediction and financial question answering tasks. First, FSframe optimizes large language models (LLMs), enhancing their adaptability to financial tasks, and incorporates prompt engineering to mitigate potential hallucination issues during the generation process, thereby improving the accuracy and reliability of the analysis. Secondly, the framework introduces Retrieval-Augmented Generation (RAG) technology, creating a dynamically updated financial knowledge base that enables the model to retrieve and integrate the latest market data, providing real-time external knowledge support for tasks. Furthermore, FSframe adopts a sparse attention mechanism, optimizing the processing efficiency of time-series data by filtering irrelevant information and focusing on key points, while also achieving efficient integration of time-series and textual data. Finally, through its modular design, FSframe organically combines the aforementioned advanced technologies, forming an innovative solution that blends multimodal data processing with real-time analysis, offering strong technical support for intelligent analysis in the financial sector. Validation on large-scale financial datasets (including historical stock prices, financial news, and market announcements) shows that FSframe significantly improves prediction accuracy and real-time responsiveness in stock trend forecasting and financial question answering tasks. Experimental results indicate that FSframe offers significant advantages in multimodal data integration, real-time performance, and interpretability, demonstrating excellent task adaptability and addressing the shortcomings of traditional methods. The FSframe framework not only provides an innovative solution for stock analysis in the financial sector but also opens new pathways for the development of intelligent financial technologies.

Keywords:

Financial Analysis; Financial Question Answering; Large Language Models; Retrieval-Augmented Generation; Sparse Attention Mechanism

References

[1] Mishra, A., Ranjan, N., Sharma, K.M., et al., 2023. Advancement of Digital Transformation to Boost Financial Services Industry. Journal of Informatics Education. 3(2), 1–10.

[2] Thakkar, A., Chaudhari, K., 2021. Fusion in stock market prediction: a decade survey on the necessity, recent developments, and potential future directions. Information Fusion. 65, 95–107.

[3] Thakkar, A., Chaudhari, K., 2021. A comprehensive survey on deep neural networks for stock market: The need, challenges, and future directions. Expert Systems with Applications. 177, 114800.

[4] Linardatos, P., Papastefanopoulos, V., Kotsiantis, S., 2020. Explainable ai: A review of machine learning interpretability methods. Entropy. 23(1), 18.

[5] Deekshith, A., 2020. AI-Enhanced Data Science: Techniques for Improved Data Visualization and Interpretation. International Journal of Creative Research In Computer Technology Design. 2(2), 22–29.

[6] Vidushi, Z., Zubair, M., Agrawal, S., et al., 2023. Machine Learning Framework for Detecting Fake News over Social Media Platforms. Proceedings of the Second International Conference on Intelligence Science; Ho Chi Minh City, Vietnam; 29–30 September 2023. Springer: Singapore. pp. 243–259.

[7] Karanikolas, N., Manga, E., Samaridi, N., et al., 2023. Large language models versus natural language understanding and generation. Proceedings of the 27th Pan-Hellenic Conference on Progress in Computing and Informatics. pp. 278–290.

[8] Huang, L., Yu, W., Ma, W., et al., 2023. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems. 41(1), 1–32.

[9] Pan, J.Z., Razniewski, S., Kalo, J.C., et al., 2023. Large language models and knowledge graphs: Opportunities and challenges. arXiv preprint. arXiv:2301.06374.

[10] Goldstein, I., 2023. Information in financial markets and its real effects. Review of Finance. 27(1), 1–32.

[11] Jiang, W., 2021. Applications of deep learning in stock market prediction: recent progress. Expert Systems with Applications. 184, 115537.

[12] Kontopoulou, V.I., Panagopoulos, A.D., Kakkos, I., et al., 2023. A review of ARIMA vs. machine learning approaches for time series forecasting in data driven networks. Future Internet. 15(8), 255.

[13] Singh, A., Markande, L., 2023. Stock Market Forecasting Using LSTM Neural Network. International Journal of Scientific Research in Computer Science, Engineering and Information Technology. 544–554.

[14] Naithani, K., Raiwani, Y.P., 2023. Realization of natural language processing and machine learning approaches for text‐based sentiment analysis. Expert Systems. 40(5), e13114.

[15] Krimberg, S., Vanetik, N., Litvak, M., 2021. Summarization of financial documents with TF-IDF weighting of multi-word terms. Proceedings of the 3rd Financial Narrative Processing Workshop. pp. 75–80.

[16] Kim, S., Park, H., Lee, J., 2020. Word2vec-based latent semantic analysis (W2V-LSA) for topic modeling: A study on blockchain technology trend analysis. Expert Systems with Applications. 152, 113401.

[17] Fataliyev, K., Chivukula, A., Prasad, M., et al., 2021. Stock market analysis with text data: A review. arXiv preprint. arXiv:2101.12985.

[18] Kusal, S., Patil, S., Choudrie, J., et al., 2023. A systematic review of applications of natural language processing and future challenges with special emphasis in text-based emotion detection. Artificial Intelligence Review. 56(12), 15129–15215.

[19] Luk, M., 2023. Generative AI: Overview, economic impact, and applications in asset management. Economic Impact, Applications in Asset Management.

[20] Sivakumar, S., Rajalakshmi, R., 2022. Context-aware sentiment analysis with attention-enhanced features from bidirectional transformers. Social Network Analysis. 12(1), 104.

[21] Pittaras, N., Giannakopoulos, G., Papadakis, G., et al., 2021. Text classification with semantically enriched word embeddings. Natural Language Engineering. 27(4), 391–425.

[22] Zhang, Q., Qin, C., Zhang, Y., et al., 2022. Transformer-based attention network for stock movement prediction. Expert Systems with Applications. 202, 117239.

[23] Bilokon, P., Qiu, Y., 2023. Transformers versus LSTMs for electronic trading. arXiv preprint. arXiv:2301.11400.

[24] Feng, Z., Ma, W., Yu, W., et al., 2023. Trends in integration of knowledge and large language models: A survey and taxonomy of methods, benchmarks, and applications. arXiv preprint. arXiv:2301.05876.

[25] Huang, A.H., Wang, H., Yang, Y., 2023. FinBERT: A large language model for extracting information from financial text. Contemporary Accounting Research. 40(2), 806–841.

[26] Gaddala, V.S., 2023. Unleashing the Power of Generative AI and RAG Agents in Supply Chain Management: A Futuristic Perspective.

[27] Liu, L., Qu, Z., Chen, Z., et al., 2022. Dynamic sparse attention for scalable transformer acceleration. IEEE Transactions on Computers. 71(12), 3165–3178.

[28] Baltussen, G., van Vliet, B., Van Vliet, P., 2023. The Cross-Section of Stock Returns before CRSP. Available at SSRN 3969743.

[29] Fazlija, B., Harder, P., 2022. Using financial news sentiment for stock price direction prediction. Mathematics. 10(13), 2156.

[30] Velez-Calle, A., Robledo-Ardila, C., 2020. Exploring the US Securities and Exchange Commission’s Edgar database by sampling joint venture contracts. International Journal of Disclosure Governance. 17, 73–85.

[31] Abdel-Basset, M., Ding, W., Mohamed, R., et al., 2020. An integrated plithogenic MCDM approach for financial performance evaluation of manufacturing industries. Risk Management. 22, 192–218.

[32] Zhao, J., Zeng, D., Liang, S., et al., 2021. Prediction model for stock price trend based on recurrent neural network. Journal of Ambient Intelligence Humanized Computing. 12, 745–753.

[33] Chen, W., Jiang, M., Zhang, W.-G., et al., 2021. A novel graph convolutional feature based convolutional neural network for stock trend prediction. Information Sciences. 556, 67–94.

[34] Qiu, Y., Song, Z., Chen, Z., 2022. Short-term stock trends prediction based on sentiment analysis and machine learning. Soft Computing. 26(5), 2209–2224.

[35] Chaudhari, K., Thakkar, A., 2023. Neural network systems with an integrated coefficient of variation-based feature selection for stock price and trend prediction. Expert Systems with Applications. 219, 119527.

[36] Teng, X., Zhang, X., Luo, Z., 2022. Multi-scale local cues and hierarchical attention-based LSTM for stock price trend prediction. Neurocomputing. 505, 92–100.

[37] Verma, J.P., Tanwar, S., Garg, S., et al., 2022. Evaluation of pattern based customized approach for stock market trend prediction with big data and machine learning techniques. In: Research Anthology on Machine Learning Techniques, Methods, and Applications, IGI Global. pp. 1255–1270.

Downloads

How to Cite

Zhaoyan Zhang, Kaixian Xu, Yu Qiao, & Alan Wilson. (2025). Sparse Attention Combined with RAG Technology for Financial Data Analysis. Journal of Computer Science Research, 7(2), 1–16. https://doi.org/10.30564/jcsr.v7i2.8933

Issue

Vol. 7 , Iss. 2 (April 2025)

Article Type

Article

License

Copyright © 2025 Author(s)

Creative Commons License

This is an open access article under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License.

Crossref
Scopus
Google Scholar
Europe PMC