Prompt engineering for open-source LLMs like Llama is less about "telling" the model what to do and more about "guiding" its latent capabilities through carefully crafted input sequences.
Let’s see this in action. Imagine we want Llama to act as a travel agent, suggesting a personalized itinerary.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "meta-llama/Llama-2-7b-chat-hf" # Or your chosen Llama variant
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
prompt = """
<s>[INST] <<SYS>>
You are a helpful, respectful, and honest AI assistant. Always answer in a way that is helpful and harmless.
<</SYS>>
User: I want to plan a 3-day trip to Kyoto, Japan, for a solo traveler interested in history and food. My budget is moderate, and I prefer a relaxed pace. Suggest an itinerary. [/INST]
"""
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=500, num_return_sequences=1)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
The output might look something like this (simplified for brevity):
<s>[INST] <<SYS>>
You are a helpful, respectful, and honest AI assistant. Always answer in a way that is helpful and harmless.
<</SYS>>
User: I want to plan a 3-day trip to Kyoto, Japan, for a solo traveler interested in history and food. My budget is moderate, and I prefer a relaxed pace. Suggest an itinerary. [/INST]
Here is a possible 3-day itinerary for your solo trip to Kyoto, focusing on history and food with a moderate budget and relaxed pace:
**Day 1: Ancient Temples and Gion Charm**
* **Morning (9:00 AM - 12:00 PM):** Start at **Kiyomizu-dera Temple**. This iconic wooden temple offers stunning views of the city and has a rich history. Afterward, wander through the charming **Higashiyama District** with its traditional wooden buildings, shops, and cafes.
* **Lunch (12:00 PM - 1:30 PM):** Enjoy a simple, delicious meal in Higashiyama. Look for a local Udon or Soba noodle shop.
* **Afternoon (1:30 PM - 4:30 PM):** Visit **Kodai-ji Temple**, known for its beautiful gardens and historical significance. Then, explore the **Gion district**, Kyoto's famous geisha quarter. While spotting a geiko or maiko is a matter of luck, the atmosphere is historical and unique.
* **Evening (6:00 PM onwards):** For dinner, try some **Kyoto-style Obanzai**, a type of home-style cooking featuring seasonal vegetables and local ingredients. Many small restaurants in Gion or Pontocho alley offer this.
**Day 2: Golden Pavilions and Zen Gardens**
* **Morning (9:30 AM - 12:30 PM):** Head to **Kinkaku-ji (Golden Pavilion)**. The shimmering gold-leaf-covered temple is a must-see. Then, visit **Ryoan-ji Temple**, famous for its enigmatic Zen rock garden.
* **Lunch (12:30 PM - 2:00 PM):** Find a restaurant near Ryoan-ji. Perhaps try some **Yudofu** (boiled tofu), a Kyoto specialty.
* **Afternoon (2:00 PM - 5:00 PM):** Explore **Nijo Castle**, the former residence of the Tokugawa shogunate, known for its "nightingale floors" and impressive architecture.
* **Evening (6:30 PM onwards):** Consider a **Kaiseki meal** if your budget allows for a splurge, or explore the food stalls and restaurants around **Nishiki Market** (though many stalls close by late afternoon, some restaurants remain open).
**Day 3: Bamboo Forests and Serene Paths**
* **Morning (9:00 AM - 12:00 PM):** Travel to the **Arashiyama district**. Walk through the breathtaking **Arashiyama Bamboo Grove**. Visit **Tenryu-ji Temple**, a UNESCO World Heritage site with a stunning landscape garden.
* **Lunch (12:00 PM - 1:30 PM):** Enjoy lunch in Arashiyama. Many restaurants offer views of the river.
* **Afternoon (1:30 PM - 4:00 PM):** Take a leisurely stroll along the **Togetsukyo Bridge** and explore the smaller temples and shops in Arashiyama. If you enjoy sweets, try some **matcha-flavored treats**.
* **Evening (Departure):** Depending on your travel plans, you can have a final Kyoto meal or head to the station.
**Budgeting Tips:**
* Utilize Kyoto's efficient public transport (buses and subways).
* Look for set lunch menus (teishoku) which are often more affordable.
* Enjoy street food and casual eateries for some meals.
* Consider staying in a guesthouse or a business hotel for moderate accommodation.
This itinerary balances iconic historical sites with opportunities to savor Kyoto's culinary delights, all at a pace that allows for enjoyment and reflection.
The core of Llama’s ability here lies in its training on vast amounts of text, allowing it to understand the relationships between concepts like "Kyoto," "history," "food," "solo traveler," and "itinerary." The prompt acts as a context window, activating the relevant parts of its learned knowledge.
The system works by taking your input prompt, tokenizing it (breaking it into numerical representations), feeding these tokens into the model’s neural network, and then generating a sequence of output tokens. The model’s internal weights, learned during pre-training, determine how likely certain tokens are to follow others, effectively "writing" the response.
The exact levers you control are:
- System Message (
<<SYS>>...<</SYS>>): This sets the persona and overarching rules for the model. A clear, concise system message significantly impacts behavior. For instance, changing "helpful assistant" to "cynical tour guide" would yield a different output. - User Message (
User: ...): This is where you provide the specific task, constraints, and desired output format. The more detailed and unambiguous, the better. - Few-Shot Examples (not shown above, but crucial): Providing a few examples of desired input-output pairs within the prompt can dramatically improve performance for specific tasks. E.g.,
User: [Example Input] Assistant: [Example Output] User: [Actual Input] Assistant: - Generation Parameters:
max_new_tokenscontrols the length,temperature(e.g., 0.7) influences randomness (higher = more creative, lower = more focused), andtop_p(e.g., 0.9) also controls creativity by sampling from a cumulative probability mass.
The specific framing of the prompt is paramount. Instead of asking "What are historical sites in Kyoto?", the prompt "Suggest a 3-day itinerary for a solo traveler interested in history and food in Kyoto, with a moderate budget and relaxed pace" is far more effective because it provides context and constraints. This allows Llama to synthesize information rather than just retrieving facts. The [INST] and [/INST] tags are specific to the Llama chat format, signaling conversational turns.
What often trips people up is assuming the model "understands" in a human sense. It doesn’t. It’s a highly sophisticated pattern-matching machine. The "understanding" emerges from the statistical relationships in its training data. Therefore, when a prompt doesn’t yield the desired result, the first instinct should be to re-examine the prompt’s structure, clarity, and the explicit constraints provided, rather than looking for a bug in the model itself. The model is a tool, and the prompt is the instruction manual; if the instructions are ambiguous, the tool will misbehave.
The next step in mastering Llama prompt engineering is exploring techniques for controlling factual accuracy and mitigating hallucination.