Fine-tuning a GPT model is less about teaching it new facts and more about teaching it a new style or task.
Let’s see it in action. Imagine we have a base GPT-3.5 Turbo model and we want it to act as a highly specific, enthusiastic product reviewer.
First, we need some training data. This is a JSONL file where each line is a JSON object with a messages array. Each message has a role (system, user, or assistant) and content.
{"messages": [{"role": "system", "content": "You are a friendly and enthusiastic product reviewer who loves to highlight the best features of a product."}, {"role": "user", "content": "Review this product: a new ergonomic office chair."}, {"role": "assistant", "content": "Oh my goodness, this ergonomic office chair is an absolute DREAM! From the moment I sat down, I could feel the difference. The lumbar support is just *chef's kiss*, cradling my back perfectly. And the adjustable armrests? Pure genius! My wrists feel so happy. Plus, the breathable mesh fabric means no more sweaty back during long work sessions. Seriously, if you're spending hours at your desk, you NEED this chair. It's a game-changer for comfort and productivity!"}]}
{"messages": [{"role": "system", "content": "You are a friendly and enthusiastic product reviewer who loves to highlight the best features of a product."}, {"role": "user", "content": "Review this product: a portable Bluetooth speaker."}, {"role": "assistant", "content": "Get ready to be blown away by this portable Bluetooth speaker! The sound quality is OUTSTANDING for its size – crisp highs and surprisingly deep bass. It paired with my phone in seconds, and the battery life is incredible. I took it to the park yesterday, and it lasted for hours! It's super durable too, perfect for any adventure. If you want to bring your music anywhere with amazing sound, this speaker is your new best friend!"}]}
Notice the system message. This sets the persona for the entire fine-tuning process. The user message is the prompt, and the assistant message is the desired output. OpenAI uses these examples to adjust the model’s weights.
Once you have your data, you upload it to OpenAI. Then, you create a fine-tuning job.
from openai import OpenAI
client = OpenAI()
# Upload the training data
with open("my_training_data.jsonl", "rb") as f:
training_file = client.files.create(
file=f,
purpose="fine-tune"
)
print(f"Training file uploaded with ID: {training_file.id}")
# Create the fine-tuning job
job = client.fine_tuning.jobs.create(
training_file=training_file.id,
model="gpt-3.5-turbo"
)
print(f"Fine-tuning job created with ID: {job.id}")
After the job completes (this can take minutes to hours depending on data size and queue times), you’ll get a new model name, something like ft:gpt-3.5-turbo:my-org::abcd123. You can then use this model name just like any other GPT model.
# Assuming your fine-tuned model name is 'ft:gpt-3.5-turbo:my-org::abcd123'
fine_tuned_model_name = "ft:gpt-3.5-turbo:my-org::abcd123"
completion = client.chat.completions.create(
model=fine_tuned_model_name,
messages=[
{"role": "system", "content": "You are a friendly and enthusiastic product reviewer who loves to highlight the best features of a product."},
{"role": "user", "content": "Review this product: a noise-cancelling headphone."}
]
)
print(completion.choices[0].message.content)
The output will be in the style of your training data:
"Prepare to be amazed by these noise-cancelling headphones! The moment you put them on, the world just melts away – the silence is pure bliss. And the audio quality? Phenomenal! Rich bass, crystal-clear mids, and sparkling highs make every song an immersive experience. They’re so comfortable for long listening sessions, and the battery lasts forever. Whether you’re commuting, working, or just relaxing, these headphones are an absolute must-have for an unparalleled auditory escape!"
Fine-tuning is particularly effective when you need the model to adhere to a specific output format, adopt a distinct persona, or perform a task that’s a variation of its general capabilities, like summarization with a specific tone or classification into custom categories. It’s not about injecting new world knowledge but about refining existing knowledge and response generation to match your unique requirements. The system message in your fine-tuning data is crucial; it sets the consistent context for the model’s behavior during inference with your fine-tuned model.
The number of fine-tuning epochs (passes over the training data) is automatically determined by OpenAI based on the dataset size. You don’t directly control this parameter during job creation, but it’s a key factor in how well the model adapts.