Add complete local Ollama setup with OpenELM - includes setup script, API server, test scripts, and documentation

Dockerfile.api

@@ -0,0 +1,18 @@
+# Dockerfile for the API server that uses Ollama
+
+FROM python:3.10-slim
+
+WORKDIR /app
+
+# Install dependencies
+COPY requirements_local.txt .
+RUN pip install --no-cache-dir -r requirements_local.txt
+
+# Copy application
+COPY app_ollama.py .
+
+# Expose port
+EXPOSE 8000
+
+# Run the API server
+CMD ["python", "app_ollama.py"]

README_LOCAL.md

@@ -0,0 +1,393 @@
+# Complete Ollama OpenELM Setup Guide
+
+This guide provides complete instructions to set up a local Ollama instance with Apple's OpenELM model and use it via OpenAI/Anthropic compatible APIs.
+
+## Table of Contents
+
+1. [Prerequisites](#prerequisites)
+2. [Quick Start (One Command)](#quick-start-one-command)
+3. [Manual Setup](#manual-setup)
+4. [Testing](#testing)
+5. [API Usage](#api-usage)
+6. [Docker Compose Setup](#docker-compose-setup)
+7. [Troubleshooting](#troubleshooting)
+
+---
+
+## Prerequisites
+
+### Required Software
+
+- **Docker**: [Install Docker](https://docs.docker.com/get-docker/)
+- **NVIDIA Driver**: For GPU support (check with `nvidia-smi`)
+- **NVIDIA Container Toolkit**: [Install Guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
+
+### Verify GPU Access
+
+```bash
+# Check NVIDIA driver
+nvidia-smi
+
+# Verify Docker can see GPU
+docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
+```
+
+---
+
+## Quick Start (One Command)
+
+### For Linux/macOS
+
+```bash
+# Download and run the complete setup script
+curl -O https://raw.githubusercontent.com/your-repo/setup_ollama_openelm.sh
+chmod +x setup_ollama_openelm.sh
+./setup_ollama_openelm.sh
+```
+
+### For Windows (PowerShell)
+
+```powershell
+# Run each command manually (see Manual Setup below)
+```
+
+---
+
+## Manual Setup
+
+### Step 1: Start Ollama Container
+
+```bash
+# Start Ollama with GPU support
+docker run -d \
+    --name ollama \
+    -v ollama:/root/.ollama \
+    -p 127.0.0.1:11434:11434 \
+    --gpus all \
+    ollama/ollama
+
+# Verify it's running
+docker ps | grep ollama
+```
+
+### Step 2: Pull OpenELM Model
+
+```bash
+# Pull the 3B parameter model (2.1 GB)
+docker exec -it ollama ollama pull apple/OpenELM-3B-Instruct
+
+# Verify installation
+docker exec ollama ollama list
+```
+
+Expected output:
+```
+NAME                             	ID          	SIZE  	MODIFIED
+apple/OpenELM-3B-Instruct:latest	abc123...	2.1 GB	About a minute ago
+```
+
+### Step 3: Install Python Dependencies
+
+```bash
+# Create virtual environment (optional but recommended)
+python3 -m venv ollama_env
+source ollama_env/bin/activate  # Linux/macOS
+# or: .\ollama_env\Scripts\activate  # Windows
+
+# Install dependencies
+pip install -r requirements_local.txt
+```
+
+### Step 4: Run the API Server
+
+```bash
+# Start the FastAPI server (runs on port 8001)
+python app_ollama.py
+```
+
+Or using uvicorn directly:
+```bash
+uvicorn app_ollama:app --host 0.0.0.0 --port 8001
+```
+
+---
+
+## Testing
+
+### Test 1: Verify Ollama is Running
+
+```bash
+# Check Ollama status
+curl http://127.0.0.1:11434/api/tags
+
+# Should return something like:
+# {"models":[{"name":"apple/OpenELM-3B-Instruct"...}]}
+```
+
+### Test 2: Quick Generation Test
+
+```bash
+# Test basic generation
+curl http://127.0.0.1:11434/api/generate \
+    -d '{"model": "apple/OpenELM-3B-Instruct", "prompt": "Say hello!", "stream": false}'
+```
+
+### Test 3: Run Test Scripts
+
+```bash
+# Make test scripts executable
+chmod +x test_curl.sh
+
+# Run curl tests
+./test_curl.sh
+
+# Run Python tests (requires openai package)
+python test_python.py
+```
+
+### Test 4: Test the Full API Server
+
+```bash
+# Test OpenAI format endpoint
+curl -X POST http://127.0.0.1:8001/v1/chat/completions \
+    -H "Content-Type: application/json" \
+    -d '{
+        "model": "apple/OpenELM-3B-Instruct",
+        "messages": [{"role": "user", "content": "Hello!"}],
+        "max_tokens": 100
+    }'
+
+# Test Anthropic format endpoint
+curl -X POST http://127.0.0.1:8001/v1/messages \
+    -H "Content-Type: application/json" \
+    -d '{
+        "model": "apple/OpenELM-3B-Instruct",
+        "messages": [{"role": "user", "content": "Hello!"}],
+        "max_tokens": 100
+    }'
+```
+
+---
+
+## API Usage
+
+### Using OpenAI SDK (Python)
+
+```python
+from openai import OpenAI
+
+# Connect to local Ollama
+client = OpenAI(
+    base_url="http://127.0.0.1:11434/v1",
+    api_key="ollama",  # Any string works
+)
+
+# Basic usage
+response = client.chat.completions.create(
+    model="apple/OpenELM-3B-Instruct",
+    messages=[
+        {"role": "system", "content": "You are a helpful assistant."},
+        {"role": "user", "content": "Explain quantum computing simply."}
+    ],
+    max_tokens=200,
+    temperature=0.7
+)
+
+print(response.choices[0].message.content)
+```
+
+### Using Anthropic SDK (Python)
+
+```python
+import anthropic
+
+# Connect to local Ollama (via API server)
+client = anthropic.Anthropic(
+    base_url="http://127.0.0.1:8001/v1",
+    api_key="ollama",  # Any string works
+)
+
+# Basic usage
+message = client.messages.create(
+    model="apple/OpenELM-3B-Instruct",
+    messages=[{"role": "user", "content": "Hello!"}],
+    max_tokens=100
+)
+
+print(message.content[0].text)
+```
+
+### Using cURL
+
+```bash
+# Basic generation
+curl http://127.0.0.1:11434/api/generate \
+    -d '{"model": "apple/OpenELM-3B-Instruct", "prompt": "Your prompt here"}'
+
+# Chat completion (OpenAI format)
+curl http://127.0.0.1:8001/v1/chat/completions \
+    -H "Content-Type: application/json" \
+    -d '{
+        "model": "apple/OpenELM-3B-Instruct",
+        "messages": [{"role": "user", "content": "Your prompt here"}],
+        "max_tokens": 100
+    }'
+```
+
+---
+
+## Docker Compose Setup
+
+For easier deployment, use Docker Compose:
+
+### Step 1: Start All Services
+
+```bash
+# Start Ollama and API server together
+docker-compose up -d
+
+# View logs
+docker-compose logs -f
+```
+
+### Step 2: Access the Services
+
+- **Ollama API**: http://localhost:11434
+- **FastAPI Server**: http://localhost:8001
+
+### Step 3: Stop Services
+
+```bash
+docker-compose down
+```
+
+---
+
+## Troubleshooting
+
+### Issue: GPU Not Detected
+
+**Error**: `Error response from daemon: could not select device driver "" with capabilities: [[gpu]]`
+
+**Solution**:
+```bash
+# Install NVIDIA Container Toolkit
+distribution=$(. /etc/ossa;echo $ID$VERSION_ID)
+curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | sudo apt-key add -
+curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
+    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
+
+sudo apt-get update
+sudo apt-get install -y nvidia-container-toolkit
+sudo systemctl restart docker
+```
+
+### Issue: Model Download Fails
+
+**Error**: `Error: pull model manifest`
+
+**Solution**:
+```bash
+# Check network connection
+curl -I https://huggingface.co
+
+# Retry with verbose output
+docker exec -it ollama ollama pull apple/OpenELM-3B-Instruct --verbose
+```
+
+### Issue: API Server Can't Connect to Ollama
+
+**Error**: `Connection refused` or `Ollama not responding`
+
+**Solution**:
+```bash
+# Check if Ollama is running
+docker ps | grep ollama
+
+# Check Ollama logs
+docker logs ollama
+
+# Restart Ollama
+docker restart ollama
+```
+
+### Issue: Out of Memory
+
+**Error**: `CUDA out of memory`
+
+**Solution**:
+- Reduce `max_tokens` parameter
+- Use smaller batch sizes
+- Restart the Ollama container to free memory
+
+### Issue: Port Already in Use
+
+**Error**: `Address already in use`
+
+**Solution**:
+```bash
+# Find the process using the port
+lsof -i :11434  # Linux/macOS
+netstat -ano | findstr :11434  # Windows
+
+# Kill the process or use a different port
+```
+
+---
+
+## File Structure
+
+```
+ollama-openelm/
+├── setup_ollama_openelm.sh    # Complete setup script
+├── app_ollama.py              # FastAPI server
+├── requirements_local.txt     # Python dependencies
+├── docker-compose.yml         # Docker Compose configuration
+├── Dockerfile.api             # API server Docker image
+├── test_python.py             # Python test script
+├── test_curl.sh               # cURL test script
+└── README.md                  # This file
+```
+
+---
+
+## Environment Variables
+
+### For API Server
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `OLLAMA_BASE_URL` | `http://127.0.0.1:11434` | Ollama server URL |
+| `OLLAMA_MODEL` | `apple/OpenELM-3B-Instruct` | Model name |
+| `PORT` | `8001` | API server port |
+
+### For Ollama Container
+
+| Variable | Description |
+|----------|-------------|
+| `OLLAMA_HOST` | Override the Ollama server URL |
+| `OLLAMA_MODELS` | Path to model storage |
+
+---
+
+## Performance Tips
+
+1. **GPU Memory**: The 3B model uses ~6GB GPU memory
+2. **CPU Inference**: Falls back to CPU if no GPU available (slower)
+3. **Batch Size**: Use `num_predict` to control output length
+4. **Temperature**: Lower values (0.0-0.5) for more deterministic output
+
+---
+
+## Additional Resources
+
+- [Ollama Documentation](https://ollama.com/)
+- [OpenELM Model Card](https://huggingface.co/apple/OpenELM-3B-Instruct)
+- [OpenAI API Compatibility](https://platform.openai.com/docs/api-reference)
+- [FastAPI Documentation](https://fastapi.tiangolo.com/)
+
+---
+
+## License
+
+This setup is provided for educational and research purposes. The OpenELM models from Apple are released under their respective licenses.

app_ollama.py

@@ -0,0 +1,391 @@
+"""
+OpenELM API Server using Local Ollama
+
+This version uses a local Ollama instance instead of Hugging Face,
+providing much faster inference with GPU acceleration.
+
+Requirements:
+- Ollama running locally (docker run ollama/ollama)
+- OpenELM model pulled (docker exec ollama ollama pull apple/OpenELM-3B-Instruct)
+- Python packages: pip install -r requirements_local.txt
+"""
+
+import uuid
+from typing import List, Optional, Dict, Any
+import requests
+from fastapi import FastAPI, HTTPException
+from fastapi.middleware.cors import CORSMiddleware
+from pydantic import BaseModel, Field
+import os
+
+# Configuration for local Ollama
+OLLAMA_BASE_URL = os.environ.get("OLLAMA_BASE_URL", "http://127.0.0.1:11434")
+OLLAMA_MODEL = os.environ.get("OLLAMA_MODEL", "apple/OpenELM-3B-Instruct")
+
+# Create FastAPI app
+app = FastAPI(
+    title="OpenELM API (Ollama)",
+    description="OpenAI & Anthropic compatible API using local Ollama instance",
+    version="3.0.0"
+)
+
+# Add CORS
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+
+# ==================== Pydantic Models ====================
+
+class ChatMessage(BaseModel):
+    role: str
+    content: str
+    name: Optional[str] = None
+
+
+class ChatCompletionRequest(BaseModel):
+    model: str = OLLAMA_MODEL
+    messages: List[ChatMessage]
+    temperature: Optional[float] = Field(default=None, ge=0.0, le=2.0)
+    top_p: Optional[float] = Field(default=None, ge=0.0, le=1.0)
+    max_tokens: Optional[int] = Field(default=None, ge=1, le=4096)
+    stream: Optional[bool] = False
+
+
+class ChatCompletionChoice(BaseModel):
+    index: int
+    message: ChatMessage
+    finish_reason: Optional[str] = None
+
+
+class ChatCompletionUsage(BaseModel):
+    prompt_tokens: int
+    completion_tokens: int
+    total_tokens: int
+
+
+class ChatCompletionResponse(BaseModel):
+    id: str
+    object: str = "chat.completion"
+    created: int
+    model: str
+    choices: List[ChatCompletionChoice]
+    usage: ChatCompletionUsage
+
+
+class MessageContent(BaseModel):
+    type: str = "text"
+    text: str
+
+
+class Message(BaseModel):
+    role: str
+    content: str | List[MessageContent]
+    name: Optional[str] = None
+
+
+class Usage(BaseModel):
+    input_tokens: int = 0
+    output_tokens: int = 0
+    total_tokens: int = 0
+
+
+class ContentBlock(BaseModel):
+    type: str = "text"
+    text: str
+
+
+class MessageResponse(BaseModel):
+    id: str
+    type: str = "message"
+    role: str = "assistant"
+    content: List[ContentBlock]
+    model: str
+    stop_reason: Optional[str] = None
+    usage: Usage
+
+
+class MessageCreateParams(BaseModel):
+    model: str = OLLAMA_MODEL
+    messages: List[Message]
+    system: Optional[str] = None
+    max_tokens: int = Field(default=1024, ge=1, le=4096)
+    temperature: Optional[float] = Field(default=None, ge=0.0, le=1.0)
+    stream: Optional[bool] = False
+
+
+# ==================== Ollama Helper Functions ====================
+
+def generate_with_ollama(
+    prompt: str,
+    system: Optional[str] = None,
+    max_tokens: int = 1024,
+    temperature: Optional[float] = None,
+    stream: bool = False
+) -> Dict[str, Any]:
+    """Generate text using local Ollama instance."""
+    
+    # Build the prompt in chat format
+    full_prompt = ""
+    
+    if system:
+        full_prompt += f"[System: {system}]\n\n"
+    
+    # Extract messages from prompt
+    lines = prompt.split("\n\n")
+    for line in lines:
+        if line.startswith("User:"):
+            full_prompt += f"User: {line[5:].strip()}\n"
+        elif line.startswith("Assistant:"):
+            full_prompt += f"Assistant: {line[10:].strip()}\n"
+        elif line.startswith("User:"):
+            full_prompt += f"User: {line[5:].strip()}\n"
+    
+    # Add final assistant prefix
+    full_prompt += "Assistant:"
+    
+    # Prepare options
+    options = {
+        "num_predict": max_tokens,
+    }
+    
+    if temperature is not None:
+        options["temperature"] = temperature
+    
+    # Make request to Ollama
+    response = requests.post(
+        f"{OLLAMA_BASE_URL}/api/generate",
+        json={
+            "model": OLLAMA_MODEL,
+            "prompt": full_prompt,
+            "stream": stream,
+            "options": options
+        }
+    )
+    
+    if response.status_code != 200:
+        raise HTTPException(
+            status_code=500,
+            detail=f"Ollama request failed: {response.text}"
+        )
+    
+    return response.json()
+
+
+def chat_with_ollama(
+    messages: List[ChatMessage],
+    max_tokens: int = 1024,
+    temperature: Optional[float] = None,
+    stream: bool = False
+) -> Dict[str, Any]:
+    """Chat completion using Ollama's chat API."""
+    
+    # Convert messages to Ollama format
+    ollama_messages = []
+    for msg in messages:
+        ollama_messages.append({
+            "role": msg.role,
+            "content": msg.content
+        })
+    
+    # Prepare options
+    options = {
+        "num_predict": max_tokens,
+    }
+    
+    if temperature is not None:
+        options["temperature"] = temperature
+    
+    # Make request to Ollama chat API
+    response = requests.post(
+        f"{OLLAMA_BASE_URL}/v1/chat/completions",
+        json={
+            "model": OLLAMA_MODEL,
+            "messages": ollama_messages,
+            "stream": stream,
+            "options": options
+        }
+    )
+    
+    if response.status_code != 200:
+        raise HTTPException(
+            status_code=500,
+            detail=f"Ollama chat request failed: {response.text}"
+        )
+    
+    return response.json()
+
+
+# ==================== API Endpoints ====================
+
+@app.get("/", tags=["Root"])
+async def root():
+    """Root endpoint with API information."""
+    return {
+        "name": "OpenELM API (Ollama Local)",
+        "version": "3.0.0",
+        "model": OLLAMA_MODEL,
+        "ollama_url": OLLAMA_BASE_URL,
+        "endpoints": {
+            "chat": "POST /v1/chat/completions",
+            "messages": "POST /v1/messages",
+            "health": "GET /health"
+        }
+    }
+
+
+@app.get("/health", tags=["Health"])
+async def health_check():
+    """Health check endpoint."""
+    try:
+        response = requests.get(f"{OLLAMA_BASE_URL}/api/tags", timeout=5)
+        if response.status_code == 200:
+            return {
+                "status": "healthy",
+                "ollama_connected": True,
+                "model": OLLAMA_MODEL
+            }
+        else:
+            return {
+                "status": "unhealthy",
+                "ollama_connected": False,
+                "error": "Ollama not responding"
+            }
+    except Exception as e:
+        return {
+            "status": "unhealthy",
+            "ollama_connected": False,
+            "error": str(e)
+        }
+
+
+@app.post("/v1/chat/completions", response_model=ChatCompletionResponse, tags=["OpenAI"])
+async def create_chat_completion(request: ChatCompletionRequest):
+    """Create chat completion (OpenAI API format)."""
+    
+    try:
+        # Use Ollama chat API
+        result = chat_with_ollama(
+            messages=request.messages,
+            max_tokens=request.max_tokens or 1024,
+            temperature=request.temperature,
+            stream=request.stream
+        )
+        
+        # Convert to OpenAI format
+        choice = result["choices"][0]
+        message = choice["message"]
+        
+        response_id = f"chatcmpl-{uuid.uuid4().hex[:12]}"
+        timestamp = int(uuid.uuid1().time)
+        
+        return ChatCompletionResponse(
+            id=response_id,
+            created=timestamp,
+            model=OLLAMA_MODEL,
+            choices=[
+                ChatCompletionChoice(
+                    index=0,
+                    message=ChatMessage(role=message["role"], content=message["content"]),
+                    finish_reason=choice.get("finish_reason", "stop")
+                )
+            ],
+            usage=ChatCompletionUsage(
+                prompt_tokens=result["usage"]["prompt_tokens"],
+                completion_tokens=result["usage"]["completion_tokens"],
+                total_tokens=result["usage"]["total_tokens"]
+            )
+        )
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Generation failed: {str(e)}")
+
+
+@app.post("/v1/messages", response_model=MessageResponse, tags=["Anthropic"])
+async def create_message(params: MessageCreateParams):
+    """Create message (Anthropic API format)."""
+    
+    try:
+        # Convert Anthropic messages to prompt
+        prompt_parts = []
+        
+        if params.system:
+            prompt_parts.append(f"[System: {params.system}]")
+        
+        for msg in params.messages:
+            content = msg.content
+            if isinstance(content, list):
+                content = "".join(b.text for b in content if hasattr(b, 'text'))
+            
+            if msg.role == "user":
+                prompt_parts.append(f"User: {content}")
+            elif msg.role == "assistant":
+                prompt_parts.append(f"Assistant: {content}")
+        
+        prompt_parts.append("Assistant:")
+        prompt = "\n\n".join(prompt_parts)
+        
+        # Generate with Ollama
+        result = generate_with_ollama(
+            prompt=prompt,
+            system=params.system,
+            max_tokens=params.max_tokens,
+            temperature=params.temperature
+        )
+        
+        # Extract response
+        response_text = result.get("response", "")
+        
+        # Count tokens (approximate)
+        input_tokens = len(prompt.split())
+        output_tokens = len(response_text.split())
+        
+        return MessageResponse(
+            id=f"msg_{uuid.uuid4().hex[:8]}",
+            role="assistant",
+            content=[ContentBlock(type="text", text=response_text)],
+            model=OLLAMA_MODEL,
+            stop_reason="end_turn",
+            usage=Usage(
+                input_tokens=input_tokens,
+                output_tokens=output_tokens,
+                total_tokens=input_tokens + output_tokens
+            )
+        )
+        
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Generation failed: {str(e)}")
+
+
+# ==================== Main Entry Point ====================
+
+if __name__ == "__main__":
+    import uvicorn
+    
+    port = int(os.environ.get("PORT", 8001))  # Different port than Hugging Face Space
+    
+    print(f"""
+========================================
+OpenELM API Server (Ollama Local)
+========================================
+Model: {OLLAMA_MODEL}
+Ollama URL: {OLLAMA_BASE_URL}
+Server: http://127.0.0.1:{port}
+
+Endpoints:
+  OpenAI: POST http://127.0.0.1:{port}/v1/chat/completions
+  Anthropic: POST http://127.0.0.1:{port}/v1/messages
+  Health: GET http://127.0.0.1:{port}/health
+========================================
+    """)
+    
+    uvicorn.run(
+        "app_ollama:app",
+        host="0.0.0.0",
+        port=port,
+        reload=False
+    )

docker-compose.yml

@@ -0,0 +1,41 @@
+version: '3.8'
+
+services:
+  ollama:
+    image: ollama/ollama:latest
+    container_name: ollama
+    ports:
+      - "127.0.0.1:11434:11434"
+    volumes:
+      - ollama_data:/root/.ollama
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: all
+              capabilities: [gpu]
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+
+  api:
+    build:
+      context: .
+      dockerfile: Dockerfile.api
+    container_name: openelm-api
+    ports:
+      - "8001:8000"
+    environment:
+      - OLLAMA_BASE_URL=http://ollama:11434
+      - OLLAMA_MODEL=apple/OpenELM-3B-Instruct
+    depends_on:
+      ollama:
+        condition: service_healthy
+    restart: unless-stopped
+
+volumes:
+  ollama_data:

requirements_local.txt

@@ -0,0 +1,15 @@
+# Requirements for Local Ollama Setup
+# Install with: pip install -r requirements_local.txt
+
+# FastAPI and server
+fastapi>=0.100.0
+uvicorn[standard]>=0.23.0
+
+# HTTP client
+requests>=2.31.0
+
+# OpenAI SDK (for testing)
+openai>=1.0.0
+
+# For JSON pretty printing (optional)
+pygments>=2.15.0

setup_ollama_openelm.sh

@@ -0,0 +1,314 @@
+#!/bin/bash
+
+# ============================================
+# Complete Ollama + OpenELM Setup Script
+# ============================================
+
+set -e  # Exit on error
+
+echo "========================================"
+echo "Ollama OpenELM Setup Script"
+echo "========================================"
+echo ""
+
+# Color codes for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+# Function to print colored output
+print_status() {
+    echo -e "${GREEN}[✓]${NC} $1"
+}
+
+print_warning() {
+    echo -e "${YELLOW}[!]${NC} $1"
+}
+
+print_error() {
+    echo -e "${RED}[✗]${NC} $1"
+}
+
+# ============================================
+# Step 1: Check Prerequisites
+# ============================================
+echo "Step 1: Checking prerequisites..."
+echo "-----------------------------------"
+
+# Check Docker
+if ! command -v docker &> /dev/null; then
+    print_error "Docker is not installed!"
+    echo ""
+    echo "Please install Docker first:"
+    echo "  Ubuntu/Debian: sudo apt-get install docker.io"
+    echo "  macOS: brew install --cask docker"
+    echo "  Windows: Download from https://docker.com/products/docker-desktop"
+    exit 1
+fi
+print_status "Docker is installed"
+
+# Check Docker daemon is running
+if ! docker info &> /dev/null; then
+    print_error "Docker daemon is not running!"
+    echo ""
+    echo "Please start Docker:"
+    echo "  Ubuntu/Debian: sudo systemctl start docker"
+    echo "  macOS/Windows: Start Docker Desktop application"
+    exit 1
+fi
+print_status "Docker daemon is running"
+
+# Check NVIDIA driver
+if ! command -v nvidia-smi &> /dev/null; then
+    print_warning "NVIDIA driver not found. GPU support may not work!"
+else
+    print_status "NVIDIA driver is installed"
+    nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null || echo "  GPU detected"
+fi
+
+echo ""
+
+# ============================================
+# Step 2: Stop and Remove Existing Ollama Container
+# ============================================
+echo "Step 2: Cleaning up existing containers..."
+echo "-----------------------------------"
+
+if docker ps -a | grep -q "ollama"; then
+    print_warning "Removing existing ollama container..."
+    docker stop ollama 2>/dev/null || true
+    docker rm ollama 2>/dev/null || true
+    print_status "Existing container removed"
+else
+    print_status "No existing ollama container found"
+fi
+
+echo ""
+
+# ============================================
+# Step 3: Start Ollama Container
+# ============================================
+echo "Step 3: Starting Ollama container..."
+echo "-----------------------------------"
+
+docker run -d \
+    --name ollama \
+    -v ollama:/root/.ollama \
+    -p 127.0.0.1:11434:11434 \
+    --gpus all \
+    ollama/ollama
+
+print_status "Ollama container started"
+echo "Container ID: $(docker ps -q --filter ancestor=ollama/ollama)"
+
+echo ""
+
+# ============================================
+# Step 4: Wait for Ollama to be ready
+# ============================================
+echo "Step 4: Waiting for Ollama to initialize..."
+echo "-----------------------------------"
+
+max_attempts=30
+attempt=0
+while [ $attempt -lt $max_attempts ]; do
+    if docker exec ollama curl -s http://localhost:11434/api/tags > /dev/null 2>&1; then
+        print_status "Ollama is ready!"
+        break
+    fi
+    
+    attempt=$((attempt + 1))
+    echo "  Attempt $attempt/$max_attempts (waiting 2 seconds)..."
+    sleep 2
+done
+
+if [ $attempt -eq $max_attempts ]; then
+    print_error "Ollama failed to start within 60 seconds"
+    echo ""
+    echo "Checking logs..."
+    docker logs ollama
+    exit 1
+fi
+
+echo ""
+
+# ============================================
+# Step 5: Pull OpenELM Model
+# ============================================
+echo "Step 5: Pulling OpenELM 3B model..."
+echo "-----------------------------------"
+echo "This may take 5-15 minutes depending on your internet connection."
+echo "Model size: approximately 2.1 GB"
+echo ""
+
+# Pull the model
+if docker exec -it ollama ollama pull apple/OpenELM-3B-Instruct; then
+    print_status "OpenELM model pulled successfully!"
+else
+    print_error "Failed to pull OpenELM model"
+    exit 1
+fi
+
+echo ""
+
+# ============================================
+# Step 6: Verify Installation
+# ============================================
+echo "Step 6: Verifying installation..."
+echo "-----------------------------------"
+
+echo "Installed models:"
+docker exec ollama ollama list
+
+echo ""
+echo "Testing model with a simple prompt..."
+test_response=$(docker exec ollama curl -s http://localhost:11434/api/generate \
+    -d '{"model": "apple/OpenELM-3B-Instruct", "prompt": "Hello", "stream": false}')
+
+if echo "$test_response" | grep -q "response"; then
+    print_status "Model is working!"
+    echo ""
+    echo "Test response preview:"
+    echo "$test_response" | python3 -m json.tool 2>/dev/null | head -20 || echo "$test_response"
+else
+    print_warning "Model test returned unexpected response"
+    echo "$test_response"
+fi
+
+echo ""
+
+# ============================================
+# Step 7: Create Test Scripts
+# ============================================
+echo "Step 7: Creating test scripts..."
+echo "-----------------------------------"
+
+# Create test_python.py
+cat > test_python.py << 'PYTHON_TEST'
+#!/usr/bin/env python3
+"""
+Test script for Ollama OpenELM API
+"""
+
+from openai import OpenAI
+
+def test_ollama():
+    """Test Ollama with OpenAI SDK."""
+    print("Testing Ollama OpenELM with OpenAI SDK...")
+    print("-" * 50)
+    
+    client = OpenAI(
+        base_url="http://127.0.0.1:11434/v1",
+        api_key="ollama",
+    )
+    
+    # Test 1: Basic generation
+    print("\n[Test 1] Basic generation:")
+    response = client.chat.completions.create(
+        model="apple/OpenELM-3B-Instruct",
+        messages=[{"role": "user", "content": "Say hello!"}],
+        max_tokens=100,
+        temperature=0.7
+    )
+    print(f"Response: {response.choices[0].message.content}")
+    print(f"Tokens: {response.usage.total_tokens}")
+    
+    # Test 2: Multi-turn conversation
+    print("\n[Test 2] Multi-turn conversation:")
+    response = client.chat.completions.create(
+        model="apple/OpenELM-3B-Instruct",
+        messages=[
+            {"role": "user", "content": "What is AI?"},
+            {"role": "assistant", "content": "AI is Artificial Intelligence."},
+            {"role": "user", "content": "What are examples?"}
+        ],
+        max_tokens=150,
+        temperature=0.5
+    )
+    print(f"Response: {response.choices[0].message.content}")
+    
+    # Test 3: Creative writing
+    print("\n[Test 3] Creative writing:")
+    response = client.chat.completions.create(
+        model="apple/OpenELM-3B-Instruct",
+        messages=[{"role": "user", "content": "Write a short poem about technology."}],
+        max_tokens=200,
+        temperature=0.8
+    )
+    print(f"Response: {response.choices[0].message.content}")
+    
+    print("\n" + "=" * 50)
+    print("All tests completed successfully!")
+
+if __name__ == "__main__":
+    test_ollama()
+PYTHON_TEST
+
+# Create test_curl.sh
+cat > test_curl.sh << 'CURL_TEST'
+#!/bin/bash
+
+# Test curl commands for Ollama OpenELM
+
+echo "========================================"
+echo "Testing Ollama OpenELM with curl"
+echo "========================================"
+echo ""
+
+# Test 1: Basic generation
+echo "[Test 1] Basic Generation"
+echo "-------------------------"
+curl -s http://127.0.0.1:11434/api/generate \
+    -d '{"model": "apple/OpenELM-3B-Instruct", "prompt": "Say hello!", "stream": false}' | \
+    python3 -m json.tool 2>/dev/null || echo "Raw output:"
+echo ""
+
+# Test 2: With temperature
+echo "[Test 2] Creative Generation (temperature=0.9)"
+echo "------------------------------------------------"
+curl -s http://127.0.0.1:11434/api/generate \
+    -d '{"model": "apple/OpenELM-3B-Instruct", "prompt": "Write a short story about AI", "stream": false, "options": {"temperature": 0.9}}' | \
+    python3 -m json.tool 2>/dev/null | head -30 || echo "Raw output:"
+echo ""
+
+# Test 3: List models
+echo "[Test 3] List Installed Models"
+echo "------------------------------"
+curl -s http://127.0.0.1:11434/api/tags | python3 -m json.tool 2>/dev/null || echo "Raw output:"
+echo ""
+
+echo "========================================"
+echo "Tests completed!"
+echo "========================================"
+CURL_TEST
+
+chmod +x test_python.py test_curl.sh
+print_status "Test scripts created: test_python.py, test_curl.sh"
+
+echo ""
+
+# ============================================
+# Summary
+# ============================================
+echo "========================================"
+echo "Setup Complete!"
+echo "========================================"
+echo ""
+echo "Ollama is now running at: http://127.0.0.1:11434"
+echo ""
+echo "Quick Test Commands:"
+echo ""
+echo "# Test with curl:"
+echo "./test_curl.sh"
+echo ""
+echo "# Test with Python OpenAI SDK:"
+echo "pip install openai"
+echo "python test_python.py"
+echo ""
+echo "# Stop Ollama:"
+echo "docker stop ollama"
+echo ""
+echo "# Start Ollama again (model persists):"
+echo "docker start ollama"
+echo ""

test_curl.sh

@@ -0,0 +1,47 @@
+#!/bin/bash
+
+# Test curl commands for Ollama OpenELM
+
+echo "========================================"
+echo "Testing Ollama OpenELM with curl"
+echo "========================================"
+echo ""
+
+# Test 1: Basic generation
+echo "[Test 1] Basic Generation"
+echo "-------------------------"
+curl -s http://127.0.0.1:11434/api/generate \
+    -d '{"model": "apple/OpenELM-3B-Instruct", "prompt": "Say hello!", "stream": false}' | \
+    python3 -m json.tool 2>/dev/null || echo "Raw output:"
+echo ""
+
+# Test 2: With temperature
+echo "[Test 2] Creative Generation (temperature=0.9)"
+echo "------------------------------------------------"
+curl -s http://127.0.0.1:11434/api/generate \
+    -d '{"model": "apple/OpenELM-3B-Instruct", "prompt": "Write a short story about AI", "stream": false, "options": {"temperature": 0.9}}' | \
+    python3 -m json.tool 2>/dev/null | head -30 || echo "Raw output:"
+echo ""
+
+# Test 3: List models
+echo "[Test 3] List Installed Models"
+echo "------------------------------"
+curl -s http://127.0.0.1:11434/api/tags | python3 -m json.tool 2>/dev/null || echo "Raw output:"
+echo ""
+
+# Test 4: Chat completions format
+echo "[Test 4] Chat Completions Format"
+echo "---------------------------------"
+curl -s http://127.0.0.1:11434/v1/chat/completions \
+    -H "Content-Type: application/json" \
+    -d '{
+        "model": "apple/OpenELM-3B-Instruct",
+        "messages": [{"role": "user", "content": "What is 2+2?"}],
+        "max_tokens": 50,
+        "temperature": 0.0
+    }' | python3 -m json.tool 2>/dev/null || echo "Raw output:"
+echo ""
+
+echo "========================================"
+echo "Tests completed!"
+echo "========================================"

test_python.py

@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+"""
+Test script for Ollama OpenELM API
+"""
+
+from openai import OpenClient
+
+def test_ollama():
+    """Test Ollama with OpenAI SDK."""
+    print("Testing Ollama OpenELM with OpenAI SDK...")
+    print("-" * 50)
+    
+    client = OpenAI(
+        base_url="http://127.0.0.1:11434/v1",
+        api_key="ollama",
+    )
+    
+    # Test 1: Basic generation
+    print("\n[Test 1] Basic generation:")
+    response = client.chat.completions.create(
+        model="apple/OpenELM-3B-Instruct",
+        messages=[{"role": "user", "content": "Say hello!"}],
+        max_tokens=100,
+        temperature=0.7
+    )
+    print(f"Response: {response.choices[0].message.content}")
+    print(f"Tokens: {response.usage.total_tokens}")
+    
+    # Test 2: Multi-turn conversation
+    print("\n[Test 2] Multi-turn conversation:")
+    response = client.chat.completions.create(
+        model="apple/OpenELM-3B-Instruct",
+        messages=[
+            {"role": "user", "content": "What is AI?"},
+            {"role": "assistant", "content": "AI is Artificial Intelligence."},
+            {"role": "user", "content": "What are examples?"}
+        ],
+        max_tokens=150,
+        temperature=0.5
+    )
+    print(f"Response: {response.choices[0].message.content}")
+    
+    # Test 3: Creative writing
+    print("\n[Test 3] Creative writing:")
+    response = client.chat.completions.create(
+        model="apple/OpenELM-3B-Instruct",
+        messages=[{"role": "user", "content": "Write a short poem about technology."}],
+        max_tokens=200,
+        temperature=0.8
+    )
+    print(f"Response: {response.choices[0].message.content}")
+    
+    # Test 4: Question answering
+    print("\n[Test 4] Question answering:")
+    response = client.chat.completions.create(
+        model="apple/OpenELM-3B-Instruct",
+        messages=[{"role": "user", "content": "Explain what is machine learning in simple terms."}],
+        max_tokens=250,
+        temperature=0.6
+    )
+    print(f"Response: {response.choices[0].message.content}")
+    
+    print("\n" + "=" * 50)
+    print("All tests completed successfully!")
+    print("=" * 50)
+
+
+if __name__ == "__main__":
+    test_ollama()