profiq Video - MLflow: serving LLMs and prompt engineering by Miloš Švaňa

If you want to improve your product by developing new AI features built on top of large language models (LLMs), you have many options to choose from. GPT models from Open AI are often considered the go-to solution for most use cases. But the competition in this space is heating up. Other proprietary solutions such as Gemini from Google or Claude from Anthropic are catching up in terms of quality, features, and pricing. There are also many high-quality open-weight models such as Llama-3.1 from Meta or the Mistral family from Mistral AI.
This abundance of choice introduces a new challenge — choosing the best solution for your specific use case. MLflow is a great tool for tackling this issue. This MLOps tool was originally developed to handle the management, evaluation, or deployment of machine learning solutions in general. However, recent updates added a few LLM-specific features that can help you quickly compare different LLM solutions.
To understand the capabilities of MLflow when evaluating LLMs, check out this video byprofiq’s Miloš Švaňa.Milošis a PhD in Systems Engineering, ML engineer, and a profiq researcher specializing in large language models, deep learning, classical ML, and statistics.
Miloš begins by exploring the LLM deployment server, demonstrating how MLflow’s deployment capabilities can seamlessly integrate with various platforms. The video covers MLflow’s prompt engineering interface, which is designed to facilitate the customization of LLMs for specific tasks. The hands-on experiment demo provides a practical example of how these tools come together, offering a clear view of MLflow’s potential to optimize the deployment and management of large language models.

Here are some useful time stamps in the video:

• 0:00 - 0:47 Intro
• 0:50 LLM Deployment Server
• 1:59 Creating a config file
• 5:18 Experiment
• 7:27 Adding new rows
• 9:58 Running the prompt
• 11:26 Conclusion

(Note:This video builds on our previous episode about using MLflow to evaluate LLMs. For the best experience, we recommend reading and watching profiq Video Blog: Evaluating LLMs with MLflow by Miloš Švaňafirst.)
You May Also Like:
profiq Video: Tech demo - Autonomous Agents using LLMs by Viktor Nawrath
profiq Video: Training your own speech-to-speech AI model
MLflow
MLflow: MLflow’s LLM Tracking Capabilities
LangChain: MLflow Deployments for LLMs
Databricks: Evaluate large language models with MLflow

If you’d like to meet our team and talk about how to bring your innovative software projects to life using AI, please contact us.