📊 Full opportunity report: Minerva. The opposite path. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.

Italy’s Minerva-3B, a sovereign language model trained from scratch on 2.5 trillion tokens with approximately 50% Italian content, scored just 4.9% on the INVALSI Italian school-exam benchmark, raising questions about the effectiveness of large-scale native-language training.

The Minerva project, led by Sapienza University of Rome and funded through Italy’s national AI strategy, built a model family ranging from 350 million to 7 billion parameters. Despite impressive technical achievements and open publication of weights and data, the 3B model’s performance on Italian academic content was near chance, indicating that scale alone may not suffice for deep language understanding. Researchers concluded that larger datasets and more parameters are necessary for complex language tasks, challenging assumptions that native-language models can be effective at current scales without even greater investment. Italy’s approach contrasts with other European projects like Portugal’s AMÁLIA, which layered specialization onto multilingual models, whereas Minerva trained from scratch, emphasizing the structural debate over optimal strategies for sovereign language models.

Implications for European Sovereign-Language Model Strategies

The results suggest that simply increasing dataset size and parameters may not guarantee deep, country-specific language understanding. This challenges European policymakers and AI strategists to reconsider the scale and investment levels necessary to produce truly effective sovereign-language models, highlighting that technical success in training does not automatically translate into functional language comprehension at academic or practical levels. The findings also raise questions about the cost-effectiveness of current approaches and whether alternative architectures or targeted data investments are required to meet national AI goals.

Background of Italy’s Minerva Sovereign-Language Initiative

Italy’s Minerva project emerged as a prominent example of a European sovereign-language model, built from scratch with significant national funding, infrastructure, and open data. The project aimed to demonstrate that a large-scale, native-language model could outperform multilingual counterparts on Italian benchmarks. Despite this, Minerva-3B’s poor performance on the INVALSI exam—just 4.9%—has surfaced as a key empirical data point, challenging the assumption that scale alone can produce country-knowledge depth. The project involved 15 researchers and was supported by Italy’s national supercomputing infrastructure and strategic funding, representing a major institutional effort in European AI development.

Unresolved Questions About Scaling and Effectiveness

It remains unclear whether increasing the number of parameters or dataset size further will significantly improve Minerva’s performance on complex language tasks. The ongoing research may explore alternative architectures, data strategies, or training methodologies to address these shortcomings. Additionally, the broader applicability of these findings to other European sovereign models is still under investigation.

Next Steps for Minerva and European Sovereign Models

Researchers plan to iterate on Minerva’s training methodology, potentially increasing scale or refining data curation, to improve performance on complex language understanding tasks. The project team also intends to publish further empirical results and explore alternative architectures. Policymakers and AI strategists will reassess investment levels and strategic approaches based on these findings, considering whether to prioritize scale or targeted data and architecture innovations for future sovereign-language models.

Key Questions

Why did Minerva perform poorly on the Italian INVALSI exam?

Despite large-scale training on extensive Italian data, Minerva-3B’s architecture and dataset composition may not be sufficient for deep academic language understanding. The results suggest that scale alone is not enough to achieve country-specific knowledge depth.

What does this mean for European AI sovereignty efforts?

The findings indicate that European projects may need to reconsider their investment strategies, potentially requiring larger models, more targeted data, or different architectures to meet their language and knowledge goals effectively.

Is the Minerva project considered a failure?

Not necessarily. Minerva has demonstrated impressive technical achievements and provides valuable empirical data. The poor exam performance highlights the complexity of language understanding and the need for further research, not a total failure.

Will increasing model size improve performance?

It is not yet certain. While larger models generally perform better, the Minerva results suggest that beyond a certain point, simply scaling up may not yield proportional gains without improvements in architecture or data quality.

What are the broader implications for AI development in Europe?

This case underscores the importance of strategic investments in data quality and model architecture, and it may influence future policy and funding decisions around sovereign-language AI initiatives across Europe.

Source: ThorstenMeyerAI.com