Skip to content

Conversation

@cxxxr
Copy link
Member

@cxxxr cxxxr commented Dec 17, 2025

Summary

Add multi-language call graph visualization to Living Canvas using the LSP Call Hierarchy API.

  • New: call-graph-lsp package - Lem-independent LSP to call-graph conversion
  • New: lsp-provider.lisp - Async analysis with idle timers (UI-responsive)
  • New: lsp-call-hierarchy.lisp - LSP request wrappers
  • New: json-format.lisp - JSON serialization for call-graph

Architecture

┌─────────────────────────────────────┐
│  User Commands                      │
│  M-x living-canvas-lsp              │
│  M-x living-canvas-lsp-full         │
│  M-x living-canvas-lsp-cancel       │
└──────────────┬──────────────────────┘
               │
               ▼
┌─────────────────────────────────────┐
│  lem-living-canvas                  │
│  lsp-provider.lisp                  │
│  (idle timer-based async analysis)  │
└──────────────┬──────────────────────┘
               │
               ▼
┌─────────────────────────────────────┐
│  call-graph-lsp (Lem-independent)   │
│  LSP → call-graph conversion        │
└──────────────┬──────────────────────┘
               │
               ▼
┌─────────────────────────────────────┐
│  Language Server (gopls, etc.)      │
│  textDocument/prepareCallHierarchy  │
│  callHierarchy/incomingCalls        │
│  callHierarchy/outgoingCalls        │
└─────────────────────────────────────┘

Motivation

  1. Multi-language support: Works with any LSP server supporting Call Hierarchy (Go, TypeScript, Rust, etc.)
  2. UI-responsive: Idle timer-based processing prevents editor freeze during analysis
  3. Clean separation: call-graph-lsp has no Lem dependencies (only LSP protocol definitions)

Changes

File Description
extensions/call-graph-lsp/ New package: LSP→call-graph conversion
extensions/call-graph/json-format.lisp New: JSON serialization
extensions/living-canvas/lsp-call-hierarchy.lisp New: LSP request wrappers
extensions/living-canvas/lsp-provider.lisp New: Async analysis provider
extensions/living-canvas/living-canvas.lisp New commands: living-canvas-lsp, etc.

Test Plan

  • Unit tests pass (call-graph/tests, call-graph-lsp/tests, lem-living-canvas/tests)
  • M-x living-canvas-lsp works with Go files (gopls)
  • M-x living-canvas-lsp-full shows incoming + outgoing calls
  • M-x living-canvas-lsp-cancel aborts running analysis
  • Progress messages update during analysis
  • UI remains responsive during large file analysis

AI Usage Disclosure

Did you use LLMs/AI tools for this PR?

  • No, this is 100% human-authored.
  • Yes, AI-assisted (Human-led logic, AI-assisted implementation).
  • Yes, AI-generated (Logic primarily derived from prompt/vibe).

Tooling Used: Claude Code (Claude Opus 4.5)


The "Human-in-the-Loop" Verification

1. Logic Walkthrough

The LSP Call Hierarchy integration follows a three-layer architecture:

  1. call-graph-lsp layer: Pure conversion functions from LSP protocol objects to call-graph structures. No Lem dependencies, only uses lem-lsp-base/protocol-3-17 for type definitions.

  2. lsp-call-hierarchy layer: Lem-specific LSP request wrappers. Handles buffer→workspace lookup, capability checking, and LSP method invocation.

  3. lsp-provider layer: Orchestrates async analysis using idle timers. Key design decisions:

    • LSP requests must run on main thread (not thread-safe in Lem)
    • Idle timer processes symbols in batches (*lsp-batch-size* = 5)
    • Yields between batches to keep UI responsive
    • Exponential backoff retry for failed requests

2. Reviewer's Guide

High-risk areas:

  • lsp-provider.lisp:261-266: Error handling in async batch processing
  • lsp-provider.lisp:110-123: Retry with backoff logic
  • Uses internal Lem LSP mode APIs (lem-lsp-mode/lsp-mode::buffer-workspace)

Suggested focus:

  • Verify idle timer cleanup on cancellation (lsp-async-cleanup)
  • Check edge cases: empty buffer, LSP server disconnect mid-analysis
  • Validate node ID uniqueness for overloaded functions

@cxxxr cxxxr force-pushed the feature/living-canvas-lsp-call-hierarchy branch from f10f9bc to 54f7a71 Compare December 18, 2025 03:34
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
@lem-project lem-project deleted a comment from code-contractor-app bot Dec 20, 2025
cxxxr and others added 9 commits December 20, 2025 17:31
Introduce bidirectional conversion between call-graph structures and JSON
format for testing, debugging, and data exchange with external tools.

Key additions:
- graph-node-to-alist, graph-edge-to-alist: Structure to alist conversion
- call-graph-to-json: Full graph to JSON string serialization
- json-to-call-graph: JSON parsing back to call-graph structures
- Comprehensive test suite with roundtrip verification
Integrate LSP Call Hierarchy protocol with Living Canvas for language-agnostic
call graph visualization.

Key additions:
- LSP request wrappers: prepare-call-hierarchy, get-incoming-calls, get-outgoing-calls
- Capability checking: buffer-has-call-hierarchy-p
- Conversion functions: call-hierarchy-item-to-node-alist, incoming/outgoing-call-to-edge-alist
- Document symbol utilities for position conversion and flattening
Implement call-graph-provider using LSP Call Hierarchy API for language-agnostic
call graph analysis.

Key additions:
- lsp-call-hierarchy-provider class with priority 20 (higher than micros)
- collect-file-call-hierarchy: analyze all callables in a buffer
- Automatic node/edge deduplication for complete file coverage
- Export document symbol utilities from lsp-call-hierarchy
Move LSP Call Hierarchy conversion logic to Lem-independent package for
better modularity and testability.

Key changes:
- New call-graph-lsp package with pure conversion functions
- build-call-graph-from-hierarchy with callback abstraction for LSP requests
- Progress reporting support via progress-fn parameter
- Update living-canvas to use call-graph-lsp
- Comprehensive unit tests with mock callbacks
Replaced lem-lsp-base/protocol-3-17: with lsp: local nickname
for better readability and consistency.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add lem:redraw-display calls during LSP request processing to keep
the UI responsive. Show progress during both symbol preparation and
call hierarchy analysis phases.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add *max-symbols-to-analyze* (default 30) to prevent LSP overload
- Abort processing after 3 consecutive errors (likely connection lost)
- Show partial results when connection is lost during analysis
- Display informative message when symbol limit is applied

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add background thread processing via bt2:make-thread to avoid UI freeze
- Add request throttling (100ms delay) to prevent LSP server overload
- Add exponential backoff retry for transient LSP errors
- Add cancellation support via living-canvas-lsp-cancel command
- Remove symbol limit (*max-symbols-to-analyze* now defaults to nil)
- Update progress display via send-event for non-blocking UI updates

This allows analyzing large files (200+ symbols) without freezing
the editor or crashing the LSP server.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
LSP requests are not thread-safe (they use a shared socket connection).
Changed from bt2:make-thread to idle-timer based processing:

- Process symbols in batches of 5 (configurable via *lsp-batch-size*)
- Yield between batches to keep UI responsive
- All LSP requests now run on the main thread
- Added lsp-async-state struct to track incremental progress

This fixes the "Couldn't write to FD-STREAM" socket error that occurred
when making LSP requests from a background thread.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@cxxxr cxxxr force-pushed the feature/living-canvas-lsp-call-hierarchy branch from e23cdbf to 1d2544d Compare December 20, 2025 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants