The silent dependency

An analysis of how generative artificial intelligence is reconfiguring decision-making capacity in the Barcelona business ecosystem — and what tech-ethical risks this transformation implies.

In November 2023, the digitalisation report by ACCIÓ —the business competitiveness agency of the Generalitat de Catalunya— revealed a significant datum: 28% of Catalan SMEs were using some artificial intelligence tool. The figure, drawn from a survey of 1,200 companies with a margin of error of ±2.8%, marked an inflection point. For the first time, AI adoption stopped being anecdotal and became a structural trend in the Catalan productive fabric.

This percentage is contextualised in a Barcelona ecosystem that has positioned itself as southern Europe’s technological hub. According to the Barcelona Tech Ecosystem 2024 report, the 22@ district concentrates more than 10,000 companies and 150,000 workers, with artificial intelligence as one of its key verticals. Barcelona Tech City, in turn, groups more than 1,300 member companies and explicitly notes that “Barcelona has become a hub for AI startups in southern Europe.”

The numbers tell a story of accelerated adoption. The global State of AI 2024 report by Air Street Capital documents that 44% of organisations globally are experimenting with generative AI. In Barcelona, this trend materialises through initiatives like those of Mobile World Capital, which in 2023 involved more than 200 companies in AI programmes.

But behind these adoption figures hides a deeper transformation — and less visible. It is not just that companies are using new tools, but that they are outsourcing fundamental cognitive processes to systems whose internal workings they do not know.

The technical anatomy of dependency

To understand this dependency, we need to distinguish between two types of AI systems. On one hand, traditional machine learning systems, trained for specific tasks with labelled data. On the other, large language models (LLMs) like GPT-4, Claude 3 or the open models the Barcelona Supercomputing Center helps develop within European digital sovereignty initiatives.

These LLMs represent a qualitative leap. As documented in the paper “Enterprise Adoption of LLMs: Opportunities and Risks” (arXiv:2403.15712), based on the analysis of 150 enterprise implementations, most organisations lack the technical capacity to audit or understand the decisions of these models. The authors —researchers from Stanford and MIT— identify a recurring pattern: companies adopt LLMs for their versatility, but without developing in parallel the competencies necessary to critically evaluate their outputs.

The case of the Barcelona Supercomputing Center is illustrative of the technical complexity. In its research on “Large Language Models for Scientific Discovery” (arXiv:2401.03946), BSC scientists document how models with billions of parameters can generate plausible scientific hypotheses —but also how the opacity of these systems makes validating their reasoning difficult. If this happens in one of Europe’s most advanced supercomputing centres, what happens in a Barcelona SME that implements AI solutions without a specialised team?

The paradox is evident: the most powerful tools are also the least understandable. And Barcelona, as an ecosystem, is massively adopting technologies that very few really understand.

So far, the technical. But…

The data show growing adoption. Reports document efficiencies. Use cases multiply in sectors ranging from 22@’s HealthTech to Barcelona Tech City’s FoodTech. Public narrative, amplified in events like 4YFN or Barcelona Digital Week, celebrates this transformation as inevitable and desirable.

But there is something deeper than process optimisation. What is at stake is not productivity, but the collective cognitive immunity of our business ecosystem: that shared capacity to understand complex problems, to question assumptions, to collectively learn from mistakes.

The paper arXiv:2403.15712 puts it clearly: “Most enterprises lack the capability to audit or understand LLM decisions”. Translated to the Barcelona context: most companies adopting generative AI cannot critically evaluate what these systems tell them. They delegate not just tasks, but judgement. They outsource not just processes, but thinking.

This outsourcing has a geopolitical dimension the Barcelona Supercomputing Center knows well. In its efforts to develop sovereign AI capabilities in Europe, BSC researchers face an uncomfortable reality: most advanced models are developed by US or Chinese corporations. When a Barcelona company implements solutions based on these models, it is depending not just on foreign technology, but on cognitive infrastructures designed in different cultural and regulatory contexts.

Not tools, but oracles

We are not talking about tools that amplify human intelligence, but about oracles that replace it. The difference is fundamental, and determines the type of dependency we are creating.

A tool —from accounting software to spreadsheets— demands understanding of the problem and mastery of the instrument. Its logic is transparent, or at least comprehensible with adequate training. An oracle, in contrast, offers answers without demanding understanding. Its logic is opaque, its internal processes inscrutable even to many experts.

This leap from tool to oracle has consequences at three levels:

At the organisational level, companies lose institutional learning capacity. When decision processes are not transparent, there is no way to improve collectively. Errors repeat without anyone understanding their deep causes.

At the ecosystem level, Barcelona loses cognitive sovereignty. If critical business-thinking infrastructures are owned by corporations headquartered in Silicon Valley or Shenzhen, our ability to define business models adapted to our cultural and economic context weakens.

At the regulatory level, the recently approved European AI Regulation (AI Act) establishes specific obligations for high-risk systems. But as Stanford and MIT researchers point out, most companies do not have the capacity to determine if their implementations fall in this category —nor to comply with the transparency requirements the regulation demands.

The risk is not hypothetical. It is the reality facing today hundreds of Barcelona companies that have adopted AI solutions without having developed in parallel the competencies necessary to use them critically.

The questions no one is asking

• How does a Barcelona SME —with limited resources— develop the critical capacity to evaluate AI systems whose creators acknowledge that not even they fully understand them?
• What responsibility do local institutions —from Barcelona Tech City to universities— have in building understanding infrastructures that accompany the adoption of technological infrastructures?
• How does the Barcelona ecosystem balance the need to compete globally with the need to maintain cognitive autonomy over what it adopts?
• What role should local research —like that done by the Barcelona Supercomputing Center on sovereign AI— play in reducing dependency on external technological oracles?

A responsibility that belongs to us

The challenge Barcelona faces is not technical, but tech-ethical. It is not solved with better algorithms, but with better questions. Not with more adoption, but with more understanding.

The ACCIÓ report tells us that 28% of Catalan SMEs use AI. What it does not tell us —because no report can— is what percentage of these companies really understand what they are using. That question does not appear in surveys, but it is the fundamental question.

It is not about rejecting technology, but about reclaiming our right to understand it. Not about returning to a pre-digital past, but about building a future where intelligence —artificial and human— dialogues instead of supplanting.

Because what defines Barcelona as an innovative ecosystem is not its algorithms, but its collective capacity to face complex problems with creativity, ethical rigour and community sense.

And that responsibility —that of understanding what we adopt, of questioning what we delegate, of keeping critical thinking alive in the era of algorithmic oracles— belongs to us.

Cited sources

1. ACCIÓ (Generalitat de Catalunya) — Digitalització de les PIMES a Catalunya 2023. Datum: 28% Catalan SMEs use AI. Survey of 1,200 companies, margin ±2.8%. accio.gencat.cat
2. Barcelona Tech City — Barcelona Tech Ecosystem 2024. barcelonatechcity.com
3. 22@ Barcelona — 2023 activity report. Data: 10,000 companies, 150,000 workers. 22barcelona.com
4. Mobile World Capital Barcelona — Annual report 2023-2024. mobileworldcapital.com
5. State of AI Report 2024 (Air Street Capital) — Datum: 44% organisations experimenting with generative AI. stateof.ai
6. Enterprise Adoption of LLMs: Opportunities and Risks — arXiv:2403.15712. Stanford/MIT.
7. Large Language Models for Scientific Discovery — arXiv:2401.03946. Barcelona Supercomputing Center.
8. European AI Regulation (AI Act, 2024) — artificialintelligenceact.eu