AI post-training: Finetuning using PEFT and DPO on Cloudera AMP
Post-training is rapidly becoming a critical phase of enterprise AI development. To get reliable output from an AI model, organizations must align its terminology (e.g., abbreviation) to fit their specific use cases.
But getting started shouldn't require heavy computing resources—you can quickly train an open-source model right on your local device.
In this tutorial, we sit down with the ASAP_DPO_Finetuning Cloudera AMP to demonstrate exactly how to align a language model to specific industry standards—in this case, Oil & Gas abbreviations.
By leveraging Direct Preference Optimization and Quantized Low-Rank Adaptation, we show you how to effectively post-train a model on a standard desktop or laptop. This highly efficient local workflow serves as the perfect starting point: you can build and validate your private AI quickly, then easily scale up later with larger datasets, bigger models, and additional remote GPUs.
If you’re looking to get hands-on with AI post-training quickly and securely, this guide provides the exact starting point you need.
Want to learn more about Cloudera AMPs? Check out Cloudera’s official website. https://cloudera.github.io/Applied-ML-Prototypes/#/community
Stay in touch with Mason:
Mason Jung on Linkedin: https://www.linkedin.com/in/minseok-jung/
Stay up to date on the latest episodes by liking and subscribing to Cloudera's YouTube channel.
Links & Resources
Check out the ASAP_DPO_Finetuning Repository: https://github.com/masonjung/ASAP_DPO_Finetuning
Explore Cloudera AMPs: https://cloudera.github.io/Applied-ML-Prototypes/#/community
Other AI-post training AMPs
Supervised Finetuning (SFT): https://github.com/cloudera/CML_AMP_Finetune_Foundation_Model_Multiple_Tasks
Group Relative Policy Optimization (GRPO): https://github.com/cloudera/CML_AMP_Med_Reasoning
Connect with Cloudera
Subscribe to stay ahead of the curve with the latest in data strategy, open architectures, and enterprise AI innovations. https://www.cloudera.com
LinkedIn ► https://www.linkedin.com/company/cloudera
Facebook ► https://www.facebook.com/cloudera/
X ► https://www.facebook.com/cloudera/
Podcast on Spotify ► https://open.spotify.com/show/102S8zoZR6nmZV0HxZlxZu