Resources for Greek Financial LLMs

Plutus

Plutus is a benchmark for evaluating LLMs in Greek financial applications, covering tasks like named entity recognition, question answering, and summarization. It addresses low-resource challenges with domain-specific datasets for systematic assessment.

Plutus is the first comprehensive benchmark designed to evaluate large language models (LLMs) in Greek financial applications. Developed by The Fin AI, it addresses the challenges of low-resource Greek financial NLP by defining five core tasks: numeric and textual named entity recognition, question answering, abstractive summarization, and topic classification.

To ensure fairness and prevent overfitting, Plutus incorporates diverse, high-quality Greek financial datasets, rigorously annotated by expert native speakers. It enables systematic and reproducible model evaluations, helping developers refine their models while providing financial professionals with clear insights into AI performance.

Plutus also introduces Plutus-8B, the first Greek financial LLM, fine-tuned on domain-specific Greek data. Evaluations on 22 LLMs highlight the limitations of cross-lingual transfer and the need for financial expertise in Greek-trained models. By assessing strengths and weaknesses, Plutus offers actionable insights to build more accurate and trustworthy Greek financial AI systems, fostering multilingual inclusivity in financial NLP.


Benchmark Scope

Plutus provides a comprehensive evaluation framework for Greek financial AI, addressing the challenges of low-resource financial NLP. It includes:


Open Greek Financial LLM Leaderboard

The Plutus Living Leaderboard continuously tracks and updates performance results across key Greek financial NLP tasks, providing real-time, transparent evaluation of LLMs. It enables systematic benchmarking of models on five core tasks using rigorously curated Greek financial datasets, ensuring fair assessment and progress tracking for Greek financial AI development.

Supported Tasks

Plutus assesses large language models across key financial NLP tasks tailored to the Greek financial domain. Each task is designed to evaluate specific capabilities, ensuring a comprehensive assessment of model performance in real-world financial applications.

Numeric NER

Identify and classify numerical financial entities, such as monetary values, percentages, dates, and quantities.

Textual NER

Detect and categorize textual financial entities, including companies, organizations, people, and locations in Greek financial reports and regulatory filings.

Question Answering

Test a model’s comprehension and reasoning over Greek financial texts through multiple-choice  QA tasks.


Abstractive Summarization

Generate concise, coherent, and informative summaries of Greek financial documents, such as earnings reports, market analyses, and regulatory filings.


Topic Classification

Categorize Greek financial news articles and reports into predefined topics, such as banking, investments, taxation, and regulatory affairs.


Current Best Models and Surprising Results

Try it now!

Partnerships

We would like to thank our partners, including the NACTEM and Archemedes RC, for their generous support in making the Plutus possible.

Greek Working Group Contributors