🤖 GPT Models

GPT-3, GPT-4, and ChatGPT Explained

The GPT Family

GPT (Generative Pre-trained Transformer) is a family of large language models developed by OpenAI that have revolutionized natural language processing and AI applications.

📈 Evolution of GPT

Model Release Parameters Context Length Key Features
GPT-1 2018 117M 512 tokens Proof of concept
GPT-2 2019 1.5B 1024 tokens Coherent text generation
GPT-3 2020 175B 2048-4096 tokens Few-shot learning
ChatGPT Nov 2022 ~175B 4096 tokens RLHF, conversations
GPT-4 Mar 2023 ~1.76T 8k-32k tokens Multimodal, reasoning
GPT-4 Turbo Nov 2023 ~1.76T 128k tokens Longer context, cheaper

🔍 GPT-3: The Breakthrough

What Made GPT-3 Special?

  • Scale: 175 billion parameters (100x larger than GPT-2)
  • Few-shot Learning: Can learn tasks from just a few examples
  • Versatility: Works across many tasks without fine-tuning
  • API Access: First GPT model widely available via API

Using GPT-3 API

import openai

openai.api_key = "your-api-key-here"

# Simple completion
response = openai.Completion.create(
    model="text-davinci-003",  # GPT-3.5
    prompt="Write a tagline for an ice cream shop:",
    max_tokens=50,
    temperature=0.7
)

print(response.choices[0].text)
# Output: "Where every scoop is a moment of pure joy!"

# Few-shot learning example
prompt = """
Translate English to French:
English: Hello
French: Bonjour

English: Goodbye
French: Au revoir

English: Thank you
French:"""

response = openai.Completion.create(
    model="text-davinci-003",
    prompt=prompt,
    max_tokens=10
)

print(response.choices[0].text)  # "Merci"

💬 ChatGPT: Conversational AI

Key Innovations

  • RLHF: Reinforcement Learning from Human Feedback
  • Instruction Following: Better at following user intent
  • Safety: Reduced harmful/biased outputs
  • Conversation Memory: Maintains context within conversation

How ChatGPT Was Trained

Step 1: Supervised Fine-tuning (SFT)

Human AI trainers have conversations, providing ideal responses

Step 2: Reward Model Training

Trainers rank multiple model outputs from best to worst

Step 3: Proximal Policy Optimization (PPO)

Model learns to maximize reward from human preferences

Using ChatGPT API

import openai

# Chat completion
response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in simple terms"}
    ],
    temperature=0.7,
    max_tokens=150
)

print(response.choices[0].message.content)

# Multi-turn conversation
conversation = [
    {"role": "system", "content": "You are a Python tutor."},
    {"role": "user", "content": "How do I create a list in Python?"},
]

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=conversation
)

# Add assistant's response to conversation
conversation.append({
    "role": "assistant",
    "content": response.choices[0].message.content
})

# Continue conversation
conversation.append({
    "role": "user",
    "content": "Can you show me an example?"
})

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=conversation
)

print(response.choices[0].message.content)

🚀 GPT-4: Multimodal Reasoning

What's New in GPT-4?

  • ✅ Multimodal: Understands both text and images
  • ✅ Better Reasoning: Improved logic and math
  • ✅ Longer Context: Up to 32k tokens (vs 4k)
  • ✅ More Accurate: Fewer hallucinations
  • ✅ Safer: Better at refusing harmful requests
  • ✅ Creative: Better at creative writing and code

Performance Improvements

90%

Bar Exam Percentile
(vs 10% for GPT-3.5)

97%

Biology Olympiad
(vs 31% for GPT-3.5)

40%

Less Harmful
Content Generated

32k

Token Context
(~50 pages)

Using GPT-4 with Vision

import openai
import base64

# Encode image
with open("diagram.jpg", "rb") as image_file:
    image_data = base64.b64encode(image_file.read()).decode('utf-8')

response = openai.ChatCompletion.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What's in this image? Explain in detail."
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{image_data}"
                    }
                }
            ]
        }
    ],
    max_tokens=500
)

print(response.choices[0].message.content)

🎛️ Key Parameters

temperature

Range: 0 - 2

Default: 0.7

  • 0: Deterministic, focused
  • 0.7: Balanced
  • 1.5+: Creative, random
temperature=0.2  # Factual answers
temperature=0.9  # Creative writing

max_tokens

Range: 1 - 4096+

Maximum length of generated response

  • Short: 50-100 tokens
  • Medium: 500-1000 tokens
  • Long: 2000+ tokens
max_tokens=100  # Brief response
max_tokens=1000  # Detailed essay

top_p

Range: 0 - 1

Default: 1

Nucleus sampling - consider only top P% probability mass

  • 0.1: Very focused
  • 0.9: Diverse
top_p=0.9  # Use with temp=1

frequency_penalty

Range: -2.0 - 2.0

Reduce repetition of frequent tokens

  • 0: No penalty
  • 0.5: Moderate
  • 1+: Strong diversity
frequency_penalty=0.5

💰 Pricing & Models

Model Input (per 1k tokens) Output (per 1k tokens) Best For
GPT-3.5 Turbo $0.0005 $0.0015 Fast, affordable
GPT-4 $0.03 $0.06 Complex tasks
GPT-4 Turbo $0.01 $0.03 Long context

🔧 Practical Examples

Code Generation

messages = [
    {"role": "system", "content": "You are an expert Python programmer."},
    {"role": "user", "content": "Write a function to find prime numbers up to n using Sieve of Eratosthenes"}
]

response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=messages,
    temperature=0.3  # Lower for code
)

Content Summarization

long_article = "..."  # Your long text

messages = [
    {"role": "user", "content": f"Summarize this in 3 bullet points:\n\n{long_article}"}
]

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo-16k",  # For long inputs
    messages=messages
)

Data Extraction

email_text = "Dear John, Your order #12345 will arrive on March 15..."

messages = [
    {"role": "system", "content": "Extract structured data from emails in JSON format."},
    {"role": "user", "content": email_text}
]

response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=messages,
    temperature=0  # Deterministic for data extraction
)

# Output: {"recipient": "John", "order_number": "12345", "delivery_date": "2024-03-15"}

⚠️ Limitations

Current Limitations

  • ❌ Hallucinations: Can generate false information confidently
  • ❌ Knowledge Cutoff: Training data ends at specific date
  • ❌ No Internet: Can't access real-time information
  • ❌ Context Window: Limited memory of conversation
  • ❌ Math: Can make calculation errors

Best Practices

  • ✅ Verify Facts: Check important information
  • ✅ Be Specific: Clear prompts get better results
  • ✅ Use System Messages: Set behavior expectations
  • ✅ Iterate: Refine prompts based on outputs
  • ✅ Use Tools: Combine with calculators, search, databases

🎯 Key Takeaways