docs/spec_mcp_progressive_disclosure_v2_0.md

# MCP Extension: Progressive Disclosure for Tool Descriptions

**Version:** 2.1  
**Status:** Stable  
**Compatibility:** MCP Specification v1.0+  
**Authors:** Michael Martin  
**Last Updated:** 2025-11-30

---

## 1. Overview

This specification defines an extension to the Model Context Protocol (MCP) that enables token-efficient tool description delivery through progressive disclosure. It introduces a two-stage tool discovery process where minimal descriptions are provided initially, and full tool definitions are retrieved on-demand via MCP resources.

### 1.1 Purpose

Large language models (LLMs) benefit from complete tool documentation to ensure reliable invocation. However, providing full descriptions for all available tools at connection time consumes significant context window space. This extension solves this problem by:

1. Exposing minimal tool descriptions sufficient for tool selection
2. Providing full descriptions on-demand through a standardized resource
3. Enforcing proper tool description retrieval through session-based authorization

### 1.2 Benefits

- **Token Efficiency**: Reduces initial context footprint by 80-90% compared to full tool descriptions
- **Scalability**: Supports servers with many tools without overwhelming LLM context
- **Flexibility**: Allows rich tool documentation without baseline context penalty
- **Reliability**: Ensures LLMs receive complete information before tool invocation

---

## 2. Conformance

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119).

Implementations that conform to this specification:
- MUST implement the two-stage tool description pattern
- MUST provide a `tool_descriptions` resource
- MUST enforce session-based authorization
- SHOULD provide system prompt guidance for optimal LLM behavior

---

## 3. Two-Stage Tool Discovery

### 3.1 Stage 1: Minimal Tool Descriptions

Servers implementing this extension SHALL return minimal tool descriptions via the standard MCP `tools/list` endpoint.

**Requirements:**
- Each tool description MUST be semantically complete enough for tool selection
- Descriptions SHOULD convey the tool's purpose and primary use cases
- Descriptions MUST NOT include detailed parameter schemas, examples, or usage instructions
- Input schemas MAY be omitted or simplified to `{"type": "object", "additionalProperties": true}`

**Purpose:** Enable LLMs to identify which tool is appropriate for a task without consuming excess tokens.

### 3.2 Stage 2: Full Tool Descriptions

Servers SHALL provide full tool descriptions through a resource named `tool_descriptions` exposed via `resources/list`.

**Requirements:**
- Full descriptions MUST include complete input schemas
- Full descriptions SHOULD include usage examples
- Full descriptions SHOULD include error handling guidance
- The resource MUST support query parameters to specify which tools to retrieve

---

## 4. Resource Definition

### 4.1 Resource Identification

The `tool_descriptions` resource SHALL be exposed via the MCP `resources/list` endpoint.

**Required Properties:**
- **uri**: `resource:///tool_descriptions` (or server-specific URI scheme)
- **name**: MUST clearly indicate this resource contains tool descriptions
- **description**: MUST explain the workflow for using this resource (see Section 4.2)
- **mimeType**: SHOULD be `application/json`

### 4.2 Resource Description Content

The resource description MUST provide sufficient guidance for LLMs to understand:

1. The relationship between `tools/list` and this resource
2. The requirement to fetch descriptions before tool invocation
3. The query parameter syntax for specifying tools
4. The consequences of calling tools without fetching descriptions

The description SHOULD:
- Use clear sequential workflow steps
- Distinguish between tool selection (using `tools/list`) and parameter learning (using this resource)
- Provide concrete URI examples with query parameters
- Explain that fetching authorizes tool use for the session

### 4.3 Resource Query Parameters

The resource URI MUST accept a `tools` query parameter specifying one or more tool names.

**Format:**
```
resource:///tool_descriptions?tools=TOOL_NAME
resource:///tool_descriptions?tools=TOOL1,TOOL2,TOOL3
```

**Requirements:**
- Multiple tool names MUST be comma-separated
- Tool names MUST match exactly as shown in `tools/list`
- Requests without the `tools` parameter MUST return an error (see Section 6.1)

### 4.4 Resource Response Format

When fetched with valid tool names, the resource SHALL return a JSON object mapping tool names to their full descriptions.

**Schema:**
```json
{
  "tool_name": {
    "name": "tool_name",
    "description": "Full detailed description",
    "inputSchema": { /* complete JSON schema */ },
    "examples": [ /* usage examples */ ],
    "usage_guidance": { /* optional usage notes */ },
    "error_guidance": { /* optional error documentation */ }
  }
}
```

The response:
- MUST contain only the tools specified in the query parameter
- MUST NOT include instructional preamble (guidance goes in resource description)
- SHOULD be dynamically generated at request time

---

## 5. Session-Based Authorization

### 5.1 Authorization Model

Servers implementing this extension SHALL enforce session-based authorization to ensure tool descriptions are fetched before tool invocation.

**Mechanism:**
1. When a tool description is fetched via the resource, the server authorizes that tool for use in the current MCP session
2. Tools called without prior authorization MUST fail with error code `TOOL_DESCRIPTION_REQUIRED`
3. Authorization persists for the lifetime of the MCP session
4. Authorization state MUST be isolated per session (no cross-session sharing)

### 5.2 Session Identification

Servers MUST use the MCP protocol's session identification mechanism to track authorization state.

**Implementation Requirements:**
- Session state MUST be scoped to individual MCP sessions
- Session lifecycle MUST conform to MCP session management patterns
- Servers MAY implement cleanup of stale sessions after a timeout period

### 5.3 Authorization Layering

The authorization mechanism defined in this specification is ADDITIVE to any existing authorization layers (e.g., OAuth, user permissions, API keys).

**Authorization Flow:**
1. **Primary Authorization**: Controls which tools appear in `tools/list`
2. **Tool Description Authorization** (this specification): Ensures descriptions are fetched before use
3. **Runtime Authorization**: Additional validation during tool execution (rate limits, scopes, etc.)

All authorization layers MUST pass for tool invocation to succeed.

---

## 6. Error Handling

### 6.1 Missing Tool Selection Error

When the `tool_descriptions` resource is fetched without the `tools` parameter, the server MUST return:

```json
{
  "error": {
    "code": "MISSING_TOOL_SELECTION",
    "message": "You must specify one or more tool names in the 'tools' parameter.",
    "examples": [
      "resource:///tool_descriptions?tools=tool_name",
      "resource:///tool_descriptions?tools=tool1,tool2"
    ]
  }
}
```

### 6.2 Tool Description Required Error

When a tool is called before its description has been fetched, the server MUST return:

```json
{
  "error": {
    "code": "TOOL_DESCRIPTION_REQUIRED",
    "message": "Tool 'tool_name' requires fetching its description before use.",
    "resource_uri": "resource:///tool_descriptions?tools=tool_name"
  }
}
```

### 6.3 Tool Not Found Error

When the `tool_descriptions` resource is fetched with an unknown tool name, the server SHOULD include an error entry for that tool:

```json
{
  "unknown_tool": {
    "error": "Tool 'unknown_tool' not found",
    "available_tools": ["tool1", "tool2", "tool3"]
  }
}
```

---

## 7. Storage and Implementation

### 7.1 Recommended Storage Structure

Servers SHOULD store each tool's full description in a separate file for modularity:

```
/tool_descriptions/
  /tool1.json
  /tool2.json
  /tool3.json
```

At runtime, the server dynamically assembles the requested descriptions into the resource response.

### 7.2 Caching Considerations

**Server-Side:**
- Servers MAY cache parsed tool descriptions in memory
- Servers SHOULD reload descriptions when source files change (development)

**Client-Side:**
- Clients MAY cache fetched descriptions for the session duration
- Servers MAY include version information to enable cache invalidation

---

## 8. System Prompt Guidance

### 8.1 Rationale

While the `tool_descriptions` resource description provides workflow guidance, LLMs benefit from reinforcement in the initial system prompt. This is particularly important because:

1. System prompts are processed before resource discovery
2. LLMs may misinterpret resource-level guidance without upfront context
3. Agent implementations vary in how they present resources to LLMs

### 8.2 Requirements

Agents implementing this extension SHOULD include system prompt guidance that:

1. Explains the two-stage workflow (select from `tools/list`, then fetch details)
2. Clarifies that minimal descriptions are sufficient for tool selection
3. Prohibits fetching the resource without specifying tools
4. Provides example syntax for fetching tool descriptions

Servers CANNOT enforce this requirement but SHOULD document recommended system prompt patterns in implementation guides.

---

## 9. Conformance Criteria

An implementation conforms to this specification if:

1. ✅ Minimal tool descriptions are provided via `tools/list`
2. ✅ A `tool_descriptions` resource is exposed via `resources/list`
3. ✅ The resource accepts a `tools` query parameter
4. ✅ The resource returns full tool descriptions for specified tools
5. ✅ Fetching without `tools` parameter returns `MISSING_TOOL_SELECTION` error
6. ✅ Session-based authorization is enforced
7. ✅ Calling tools without fetching returns `TOOL_DESCRIPTION_REQUIRED` error
8. ✅ Authorization state is isolated per MCP session

---

## 10. Security Considerations

### 10.1 Authorization Bypass

Tool description authorization is NOT a security boundary. It is a workflow enforcement mechanism to ensure LLMs have complete information before tool use.

**Security Implications:**
- Primary authorization (user permissions, API keys) MUST be enforced independently
- Tool description authorization does NOT grant access to tools
- Fetching descriptions does NOT bypass other authorization layers

### 10.2 Information Disclosure

Tool descriptions may contain sensitive information about server capabilities, parameter formats, or internal operations.

**Recommendations:**
- Apply the same authorization to `tool_descriptions` resource as to the tools themselves
- Do not expose tools in `tools/list` that the user lacks permission to use
- Sanitize descriptions to avoid exposing internal implementation details

### 10.3 Denial of Service

Clients could potentially cause resource exhaustion by repeatedly fetching tool descriptions.

**Mitigations:**
- Implement rate limiting on resource fetches
- Cache parsed descriptions in memory
- Set reasonable session timeouts
- Monitor for unusual fetch patterns

---

## 11. Example Implementation

### 11.1 Minimal Tool Description

```json
{
  "tools": [
    {
      "name": "get_data",
      "description": "Retrieves data from the system with filtering and pagination options.",
      "inputSchema": {
        "type": "object",
        "additionalProperties": true
      }
    }
  ]
}
```

### 11.2 Resource Listing

```json
{
  "resources": [
    {
      "uri": "resource:///tool_descriptions",
      "name": "Tool Descriptions - Required for tool use",
      "description": "WORKFLOW: Step 1: Pick tool from tools/list based on what it does. Step 2: Fetch full description: resource:///tool_descriptions?tools=TOOL_NAME. Step 3: Call tool with parameters learned. Must specify ?tools= parameter when fetching.",
      "mimeType": "application/json"
    }
  ]
}
```

### 11.3 Full Tool Description

Request: `resource:///tool_descriptions?tools=get_data`

Response:
```json
{
  "get_data": {
    "name": "get_data",
    "description": "Retrieves data from the system with comprehensive filtering, sorting, and pagination capabilities.",
    "inputSchema": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "Search query to filter results"
        },
        "limit": {
          "type": "integer",
          "description": "Maximum number of results (default: 50, max: 500)",
          "default": 50
        },
        "sort": {
          "type": "string",
          "enum": ["asc", "desc"],
          "description": "Sort order for results"
        }
      },
      "required": []
    },
    "examples": [
      {
        "description": "Basic query",
        "input": {"query": "search term", "limit": 10}
      },
      {
        "description": "Sorted results",
        "input": {"query": "search term", "sort": "desc"}
      }
    ]
  }
}
```

---

## 12. References

- [MCP Specification](https://spec.modelcontextprotocol.io/)
- [RFC 2119: Key words for use in RFCs](https://www.rfc-editor.org/rfc/rfc2119)
- [JSON-RPC 2.0 Specification](https://www.jsonrpc.org/specification)
- [JSON Schema Specification](https://json-schema.org/)

---

## Appendix A: Version History

| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2025-10-24 | Initial specification |
| 1.5 | 2025-10-29 | Added implementation guidance for LLM behavior |
| 2.0 | 2025-11-10 | Refined specification format, separated implementation guidance into companion guide, clarified normative requirements, added conformance criteria |
| 2.1 | 2025-11-30 | Production implementation for MCP 1st Birthday Hackathon with demo server and reference agent |

---

## Appendix B: Relationship to Base MCP Specification

This extension:
- ✅ Uses only standard MCP primitives (`tools/list`, `resources/list`, `resources/read`)
- ✅ Introduces no new protocol methods
- ✅ Is backward compatible (servers can implement progressively)
- ✅ Follows MCP session management patterns
- ✅ Adheres to JSON-RPC 2.0 transport layer

Servers implementing this extension remain fully compliant with the base MCP specification.

---

**End of Specification**