Systems | Development | Analytics | API | Testing

Serverless

Best LLM Inference Engines and Servers to Deploy LLMs in Production

AI applications that produce human-like text, such as chatbots, virtual assistants, language translation, text generation, and more, are built on top of Large Language Models (LLMs). If you are deploying LLMs in production-grade applications, you might have faced some of the performance challenges with running these models. You might have also considered optimizing your deployment with an LLM inference engine or server.

A Software Engineer's Tips and Tricks #4: Collaborating on Visual Studio Code with Live Share

Hey there! We're back for our third edition of Tips and Tricks, our new mini series where we share some helpful insights and cool tech that we've stumbled upon while working on technical stuff. Catch up on the previous posts: All of our posts are super short reads, just a couple of minutes tops. If you don’t like one of the posts, no problem! Just skip it and check out the next one. If you enjoy any of the topics, I encourage you to check out the "further reading" links.

The engineering behind autoscaling with HashiCorp's Nomad on a global serverless platform

There are several ways to handle load spikes on a service. However, these methods are not cost-effective: you either pay for resources you don't use, or you risk not having enough resources to handle the load. Fortunately, there is a third way: horizontal autoscaling. Horizontal autoscaling is the process of dynamically adjusting the number of instances of a service based on the current load. This way, you only pay for the resources you use, and you can handle load spikes without any manual intervention.

A Software Engineer's Tips and Tricks #3: CPU Utilization Is Not Always What It Seems

Hey there! We're back for our third edition of Tips and Tricks. As we said in our first posts on Drizzle ORM and Template Databases in PostgreSQL, our new Tips and Tricks mini blog series is going to share some helpful insights and cool tech that we've stumbled upon while working on technical stuff. Today's topic is short and sweet. It'll be on CPU utilization and what that metric indicates. If you enjoy it and want to learn more, I encourage you to check out the "further reading" links.

Serverless GPUs in Private Preview: L4, L40S, V100, and more

Today, we’re excited to share that Serverless GPUs are available for all your AI inference needs directly through the Koyeb platform! We're starting with GPU Instances designed to support AI inference workloads including both heavy generative AI models and lighter computer vision models. These GPUs provide up to 48GB of vRAM, 733 TFLOPS and 900GB/s of memory bandwidth to support large models including LLMs and text-to-image models.

A Software Engineer's Tips and Tricks #2: Template Databases in PostgreSQL

Hey there! We're back for our second edition of Tips and Tricks. As we said in our first post on Drizzle ORM, our new Tips and Tricks mini blog series is going to share some helpful insights and cool tech that we've stumbled upon while working on technical stuff. Today, we're going to talk about the template databases of PostgreSQL. Remember, these posts will be super short reads. If you don’t like the topic of one of the posts, no problem! Just skip it and check out the next one.

What are LLMs? An intro into AI, models, tokens, parameters, weights, quantization and more

To keep up with everything happening in the world of artificial intelligence, it helps to understand and grasp key terms and concepts behind the technology. In this introduction, we are going to dive into what is generative AI, looking at the technology and models they are built on. We'll discuss how these models are built, trained, and deployed into the world.

A Software Engineer's Tips and Tricks #1: Drizzle

Hey there! At Koyeb, we really like diving into technical stuff. But here’s the thing: not every cool thing we stumble upon or think about needs a massive blog post. And honestly, not everything we’re into is directly related to what Koyeb does or about infrastructure in general. So, we’ve got an idea: what if we start sharing these bits and pieces with you in a series of really short blog posts?

toddl.co: Spain's Leading Platform for Extra-Curricular Activities Deploys 10x Faster with Koyeb

toddl.co is an all-in-one booking platform for kids' activities in Spain. Offering more than 2,000 classes, camps, and events to over 17,000 monthly visitors, toddl.co is on a mission to help parents navigate the complex world of extra-curricular activities and to help activity organizers manage and grow their businesses. For businesses, toddl.co streamlines bookings, payments, and client management. This year, toddl.co plans to serve over 15,000 businesses and 50,000 families.

What is RAG? Retrieval-Augmented Generation for AI

Retrieval-augmented generation (RAG) is an AI framework and powerful approach in NLP (Natural Language Processing) where generative AI models are enhanced with external knowledge sources and retrieval-based mechanisms. These appended pieces of outside knowledge provide the model with accurate, up-to-date information that supplements the LLM’s existing internal representation of information. As the name suggests, RAG models have a retrieval component and a generation component.