Systems | Development | Analytics | API | Testing

As LLMs become central to AI-driven products like copilots and customer support chatbots, data science teams need to ensure the LLM performs well for the use case. The process of LLM evaluation ensures reliability, safety and performance in production AI systems. In this guide, we explore how to approach evaluations across development and production lifecycles, what frameworks to use, and how the integration between open-source MLRun and Evidently AI enables more scalable, structured testing.

Customer service chatbots and co-pilots and smart call center analysis applications are prime use cases for AI and generative AI. These AI systems and agents can provide real-time recommendations, support customer service at scale, generate insights that can be used in downstream applications to reduce churn and increase revenue, and more. How can customer service organizations grow and optimize their use of data and AI?

Chatbots are quickly becoming more sophisticated and integrated into business workflows, enhancing productivity and scalability. However, they also expand the attack surface for organizations. This new exploitation vector requires data engineers and security teams to incorporate various security guardrails when building their gen AI architecture. In this blog post, we discuss the risk of data leakage through AI chatbots.

We are proud to announce a new integration between MLRun, the open-source AI orchestration framework, and NVIDIA NeMo microservices, by extending NVIDIA Data Flywheel Blueprint. This integration streamlines training, evaluation, fine-tuning and monitoring of AI models at scale, ensuring high-performance, low latency and lowering costs while significantly reducing the manual effort required through intelligent automation.

In less than three years, gen AI has become a staple technology in the business world. In November of 2022, OpenAI launched ChatGPT, with explosive growth of over 1 million users in just five days, galvanizing the widespread use of gen AI. Over the course of 2023 enterprises entered the experimentation stage and kicked off POCs with API services and open models including Llama 2, Mistral, NVIDIA and others.

We’re proud to announce that the next version of MLRun has been released to community users. On the heels of MLRun v1.7’s focus on monitoring, MLRun v1.8 adds features to make LLM and ML evaluation and monitoring more accessible, practical and resource-efficient. New Highlights: MLRun is an open-source AI orchestration tool that provides AI practitioners with capabilities to accelerate and streamline the development, deployment and management of gen AI and ML applications.

Generative AI models are not just tools for producing text, audio or video—they're systems that learn patterns, improvise, and generate unexpected outcomes. When we look at LLMs, we're struck by their capacity to generate surprisingly creative and context-aware results. They can weave coherent narratives, propose novel solutions, mimic human conversation, and even offer nuanced insights across a wide range of topics. While this creativity is their strength, it also introduces variability and risk.