Open-FinLLMs: Democratizing Financial Intelligence with Open-Source Multimodal AI

In global finance, where split-second decisions hinge on parsing earnings calls, regulatory filings, and market trends, access to advanced AI tools has long been a privilege of Wall Street giants. Open-FinLLMs, a new suite of open-source financial large language models, aims to disrupt this status quo by delivering institutional-grade analysis to startups, researchers, and emerging markets.  

Built on Meta’s Llama 3 architecture and trained on a groundbreaking 52-billion-token financial corpus, Open-FinLLMs introduces three specialized models:  

- FinLLaMA: A foundational model pretrained on diverse financial data, including SEC filings, earnings calls, and market indicators.  

- FinLLaMA-Instruct: Instruction-tuned with 573,000 financial task examples for precise sentiment analysis, risk assessment, and numerical reasoning.  

- FinLLaVA: The first open-source multimodal financial LLM, capable of interpreting charts, tables, and text simultaneously.  

Developed through an academic collaboration via The Fin AI community, the project addresses critical gaps in financial AI: existing models like BloombergGPT rely on limited datasets, lack multimodal support, and remain closed-source. Open-FinLLMs outperforms BloombergGPT by 12% on zero-shot financial tasks and surpasses GPT-4 in tabular data analysis, despite using just 8B parameters.  

Breaking Down Financial Complexity

The models’ strength lies in their training strategy:  

1. Domain-Specific Pretraining: FinLLaMA’s corpus spans 7 financial domains—from technical indicators (12B tokens) to historical market data (13B tokens)—mixed with general data at a 3:1 ratio to prevent "catastrophic forgetting."  

2. Multimodal Grounding: FinLLaVA processes financial charts and tables using 1.43M image-text pairs, including synthetic tabular data from SynthTabNet to reduce hallucinations.  

3. Cost Efficiency: Training leveraged LoRA adapters and 64 A100 GPUs.  

“Most financial LLMs are either closed-box or narrowly focused on text,” noted by Dr. Qianqian Xie, the lead researcher in the paper. “Open-FinLLMs isn’t just about scale—it’s about holistic financial understanding, from a CEO’s earnings call remark to the fine print in a 10-K filing.”  

Benchmarks to Boardrooms

In rigorous testing across 30 datasets, Open-FinLLMs achieved:  

- 82.1 F1 score in financial named entity recognition (NER), outperforming BloombergGPT by 21 points.  

- 72.4% accuracy on TableBench (tabular data), exceeding GPT-4 and Gemini-1.5 Pro.  

- 1.41 Sharpe ratio in simulated stock trading, demonstrating profitable decision-making under volatility.  

Open Source as a Force Multiplier

By releasing models, codes and evaluation datasets under OSI-approved licenses, the team aims to democratize financial AI.

“Proprietary models create an AI divide,” argues the paper. “Our benchmarks show a community credit union can now access tools rivaling hedge fund systems.”  

Current limitations include English-only support and a 8B parameter ceiling—future work will explore multilingual training and scaled variants. The researchers emphasize ethical safeguards: all data is anonymized, and models should include guardrails against providing investment advice.  

Explore the paper: https://huggingface.co/papers/2408.11878 | Join the evaluation initiative: https://thefin.ai