5. February 20252 min reading time

Open source, infrastructure, compliance: what it takes to implement AI

What IT resources are required for sophisticated AI applications? What security measures and legal requirements must companies observe in order to use AI safely and in compliance with the law? What role does open source play? These and other questions are the focus of the new episode of "No Hype AI".

What does it take to implement AI in a company?

In the latest episode of "No Hype AI" at brutkasten, Florian Böttcher (Solution Architect, CANCOM Austria), Stephan Kraft (Community Advocate & Business Development OpenShift & Application Services, Red Hat), Natalie Ségur-Cabanac (Policy Lead, Women in AI) and Patrick Ratheiser (Founder & CEO, Leftshift.One) discussed what it takes to implement AI in the enterprise.

Open source as a driver for AI development

Open source technologies offer companies far-reaching opportunities in AI development. In the discussion, Stephan Kraft emphasized that open source is not just a question of ethics, but is essential for start-ups and SMEs. Transparency and collaboration facilitate innovation and lower barriers to entry.

Patrick Ratheiser emphasizes that open source enables an exponential increase in AI models. From 5,000 models in 2021, there are now over one million open source models that are freely available. in other words, growth of over 10,000 %. However, open source is not always the best choice, explains Patrick Ratheiser. Depending on the use case, companies have to weigh up between open source, cloud or closed source.

Open source also plays an important role in the infrastructure, says Florian Böttcher. It sets the requirements for the infrastructure. Without open source, the whole AI topic would not exist, explains Böttcher. Natalie Ségur-Cabanac also sees open source as a "key technology" for reducing discrimination and promoting diversity in AI research.

Data structure and compliance: challenges of open source

A clean database is essential, explains Natalie Ségur-Cabanac. With open source, it is particularly important that if models are purchased or used, these models have been trained with clean data. She warns that open source systems often create ambiguity and the challenge for compliance is "Who is responsible for what?". Companies therefore offer enterprise support for quality assurance and also in a legal sense. Patrick Ratheiser sees compliance as a key success factor. It is crucial to be able to guarantee the customer that all systems comply with regulations and are regularly maintained.

If companies want to use AI, they face different infrastructural challenges that depend on the respective use case. Florian Böttcher emphasizes that training your own large language models (LLMs) in particular entails considerable infrastructural requirements. A key challenge in Austria is the high energy consumption and heat generated during AI training. For this reason, such training often takes place in the Nordic countries or in the USA. "Especially if you work a lot with open source or are active in research, you still need the right infrastructure to understand which solutions can be used by the customer," explains Böttcher. Companies should therefore first define their specific use case and strategy. Only then can the appropriate software solution be determined, which in turn specifies the requirements for the infrastructure.

Conclusion Open source offers great opportunities in AI development, but also poses challenges in terms of compliance, infrastructure and data quality. Companies should pursue a clear AI strategy, consider different solutions and focus on building up expertise. Find out more in the brutkasten episode of "No Hype AI".