5. February 20252 min reading time

Open source and AI: opportunities, challenges and best practices

What is needed for AI? What IT resources are required for sophisticated AI applications? What security measures and legal requirements must companies observe in order to use AI safely and in compliance with the law? What role does open source play? These and other questions are the focus of the new episode of "No Hype AI".

What does it take to implement AI in a company?

In the latest episode of "No Hype AI" at brutkasten, Florian Böttcher (Solution Architect, CANCOM Austria), Stephan Kraft (Community Advocate & Business Development OpenShift & Application Services, Red Hat), Natalie Ségur-Cabanac (Policy Lead, Women in AI) and Patrick Ratheiser (Founder & CEO, Leftshift.One) discussed what it takes to implement AI in the enterprise.

Open source as a driver for AI development

Open source technologies offer companies far-reaching opportunities in AI development. In the discussion, Stephan Kraft emphasized that open source is not just a question of ethics, but is essential for start-ups and SMEs. Transparency and collaboration facilitate innovation and lower barriers to entry.

Patrick Ratheiser emphasizes that open source enables an exponential increase in AI models. From 5,000 models in 2021, there are now over one million open source models that are freely available. in other words, growth of over 10,000%. However, open source is not always the best choice, explains Patrick Ratheiser. Depending on the use case, companies have to weigh up between open source, cloud or closed source.

Open source also plays an important role in the infrastructure, says Florian Böttcher. It sets the requirements for the infrastructure. Without open source, the whole AI topic would not exist. Natalie Ségur-Cabanac also sees open source as a "key technology" for reducing discrimination and promoting diversity in AI research.

Data structure and compliance: challenges of open source

A clean database is essential, explains Natalie Ségur-Cabanac. With open source, it is particularly important that if models are purchased or used, that these models have been trained with clean data. She warns that open source systems often create ambiguity in this regard and pose the challenge of "Who is responsible for what?" for compliance. Companies therefore offer enterprise support for quality assurance and are also responsible in a legal sense. Patrick Ratheiser also focuses on compliance as a key factor, because you have to guarantee the customer that the whole thing is compliant and these systems have to be maintained.

The choice between in-house AI infrastructure or cloud solutions depends on the use case. Florian Böttcher also emphasized that training your own LLMs makes a big difference in terms of infrastructure. The big challenge here is that in Austria, AIs consume a lot of electricity and produce a lot of heat. This is why AI training usually takes place in the Nordics or in the USA. "Especially if you want to work with open source or do research, you still need the infrastructure so that you know what you can use with the customer," says Böttcher.

expects more efficient models with lower energy requirements in the future. Hybrid approaches are particularly popular in Europe, where data protection (GDPR) plays a major role.

Conclusion Open source offers great opportunities in AI development, but also poses challenges in terms of compliance, infrastructure and data quality. Companies should pursue a clear AI strategy, consider hybrid solutions and focus on building up expertise. Find out more in the brutkasten episode of "No Hype AI".