Skip to content

A high-performance, in-memory vector database written in Rust, designed for semantic search and top-k nearest neighbor queries in AI-driven applications, with binary file persistence for durability.

License

Notifications You must be signed in to change notification settings

hivellm/vectorizer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Vectorizer

Rust Rust Edition License Crates.io GitHub release Tests Coverage Production Ready

A high-performance vector database and search engine built in Rust, designed for semantic search, document indexing, and AI-powered applications.

✨ Key Features

  • πŸ” Semantic Search: Advanced vector similarity with multiple distance metrics (Cosine, Euclidean, Dot Product)
  • ⚑ SIMD Acceleration: AVX2-optimized vector operations (5-10x faster) with automatic CPU feature detection
  • πŸ’Ύ Memory-Mapped Storage: MMap support for datasets larger than RAM with efficient OS paging
  • πŸš€ GPU Acceleration: Metal GPU support for macOS (Apple Silicon) with cross-platform compatibility
  • πŸ“¦ Product Quantization: PQ compression for 64x memory reduction with minimal accuracy loss
  • πŸ’Ύ Compact Storage: Unified .vecdb format with 20-30% space savings and automatic snapshots
  • πŸ”„ Master-Replica Replication: High availability with automatic failover and SDK routing support
  • πŸ”— Distributed Sharding: Horizontal scaling across multiple servers with automatic shard routing
  • ☁️ HiveHub Cluster Mode: Multi-tenant cluster deployment with HiveHub.Cloud
    • Tenant isolation with user-scoped collections
    • Quota enforcement (collections, vectors, storage)
    • Usage tracking and reporting
    • Memory limits and MMap storage enforcement
    • Operation logging with cloud integration
    • Comprehensive audit trail and analytics
    • Tenant migration API (export, transfer, clone, cleanup)
  • πŸ“„ Document Conversion: Automatic conversion of PDF, DOCX, XLSX, PPTX, HTML, XML, and images
  • πŸ”„ Qdrant Migration: Complete migration tools and full Qdrant 1.14.x API compatibility
    • Snapshots API (create, list, delete, recover)
    • Sharding API (create shard keys, distribute data)
    • Cluster Management API (status, recovery, peer management, metadata)
    • Query API (query, batch query, grouped queries with prefetch)
    • Search Groups and Matrix API (grouped results, similarity matrices)
    • Named Vectors support (partial)
    • Quantization configuration (PQ and Binary)
  • 🎯 MCP Integration: 26 focused individual tools for AI model integration
  • πŸ”„ UMICP Protocol: Native JSON types + Tool Discovery endpoint
  • πŸ“Š GraphQL API: Full GraphQL API with async-graphql
    • Complete REST API parity with flexible queries
    • GraphiQL playground for interactive exploration
    • Mutations for collections, vectors, and search
  • πŸ–₯️ Web Dashboard: Modern React + TypeScript dashboard with complete graph management interface
    • JWT-based authentication with login page and session management
    • Create/delete edges with relationship types and weights
    • Explore node neighbors and related nodes
    • Find shortest paths between nodes
    • Node-specific edge discovery with configurable parameters
    • Real-time graph visualization with vis-network
  • πŸ–₯️ Desktop GUI: Electron-based desktop application with vis-network graph visualization for visual database management
  • ⚑ High Performance: Sub-3ms search times with HNSW indexing
  • 🧠 Multiple Embeddings: TF-IDF, BM25, FastEmbed (production), BERT/MiniLM (real or placeholder), and custom models
  • πŸ”€ Hybrid Search: Dense + Sparse search with Reciprocal Rank Fusion (RRF)
  • πŸ“ Smart Summarization: Extractive, keyword, sentence, and abstractive (OpenAI GPT) methods
  • πŸ” TLS/SSL Security: Full TLS 1.2/1.3 support with mTLS, configurable cipher suites, and ALPN
  • ⚑ Rate Limiting: Per-API-key rate limiting with configurable tiers and overrides
  • πŸ“Š Quantization Cache: Cache hit ratio tracking with comprehensive metrics
  • πŸ•ΈοΈ Graph Relationships: Automatic relationship discovery and graph traversal with full GUI support for edge management, node exploration, and path finding
  • πŸ”— n8n Integration: Official n8n community node for no-code workflow automation (400+ node integrations)
  • 🎨 Langflow Integration: LangChain-compatible components for visual LLM app building
  • πŸ”’ Security: JWT + API Key authentication with RBAC

πŸš€ Quick Start

Install Script (Linux/macOS)

Installs Vectorizer CLI and configures it as a system service that starts automatically on boot:

curl -fsSL https://raw.githubusercontent.com/hivellm/vectorizer/main/scripts/install.sh | bash

After installation:

  • βœ… CLI available: vectorizer --help
  • βœ… Service running: sudo systemctl status vectorizer
  • βœ… Auto-starts on boot
  • βœ… Service commands:
    • sudo systemctl restart vectorizer - Restart service
    • sudo systemctl stop vectorizer - Stop service
    • sudo journalctl -u vectorizer -f - View logs

Install Script (Windows PowerShell)

Installs Vectorizer CLI and configures it as a Windows Service that starts automatically on boot:

powershell -c "irm https://raw.githubusercontent.com/hivellm/vectorizer/main/scripts/install.ps1 | iex"

Note: Service installation requires Administrator privileges. If not running as admin, the script will provide instructions.

After installation:

  • βœ… CLI available: vectorizer --help
  • βœ… Service running: Get-Service Vectorizer
  • βœ… Auto-starts on boot
  • βœ… Service commands:
    • Restart-Service Vectorizer - Restart service
    • Stop-Service Vectorizer - Stop service
    • Start-Service Vectorizer - Start service

Using Docker (Recommended)

Basic Docker Run (with authentication):

docker run -d \
  --name vectorizer \
  -p 15002:15002 \
  -v $(pwd)/vectorizer-data:/vectorizer/data \
  -e VECTORIZER_AUTH_ENABLED=true \
  -e VECTORIZER_ADMIN_USERNAME=admin \
  -e VECTORIZER_ADMIN_PASSWORD=admin \
  -e VECTORIZER_JWT_SECRET=change-this-secret-in-production \
  --restart unless-stopped \
  hivehub/vectorizer:latest

Production Docker Run (with custom credentials):

docker run -d \
  --name vectorizer \
  -p 15002:15002 \
  -v $(pwd)/vectorizer-data:/vectorizer/data \
  -e VECTORIZER_AUTH_ENABLED=true \
  -e VECTORIZER_ADMIN_USERNAME=admin \
  -e VECTORIZER_ADMIN_PASSWORD=your-secure-password \
  -e VECTORIZER_JWT_SECRET=your-jwt-secret-key \
  --restart unless-stopped \
  hivehub/vectorizer:latest

Using Docker Compose:

# Copy .env.example to .env and customize
cp .env.example .env
# Edit .env with your credentials

# Start with docker-compose
docker-compose up -d

Default Credentials (CHANGE IN PRODUCTION!):

  • Username: admin
  • Password: admin
  • JWT Secret: change-this-secret-in-production

Available at:

Building from Source

git clone https://github.com/hivellm/vectorizer.git
cd vectorizer

# Basic build
cargo build --release
./target/release/vectorizer

# With GPU acceleration (macOS Metal)
cargo build --release --features hive-gpu

# With all features
cargo build --release --features full

Access Points

πŸ”’ Authentication (Docker)

When using Docker, authentication is enabled by default:

Default Credentials (⚠️ CHANGE IN PRODUCTION!):

  • Username: admin
  • Password: admin
  • Login Endpoint: POST http://localhost:15002/auth/login

Authentication Example:

# Login to get JWT token
curl -X POST http://localhost:15002/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"admin"}'

# Use JWT token in requests
curl -X GET http://localhost:15002/collections \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

API Key Authentication:

API Keys can be created in the dashboard (/api-keys) or via REST API for programmatic access.

⚠️ IMPORTANT: API Keys do NOT use the Bearer prefix. Use them directly in the Authorization header:

# Create API key (requires JWT authentication)
curl -X POST http://localhost:15002/auth/keys \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -d '{
    "name": "Production API Key",
    "permissions": ["read", "write"],
    "expires_in_days": 90
  }'

# Use API key in requests (NO Bearer prefix!)
curl -X GET http://localhost:15002/collections \
  -H "Authorization: YOUR_API_KEY"

# MCP Configuration (mcp.json)
{
  "mcpServers": {
    "vectorizer": {
      "command": "npx",
      "args": ["-y", "@hivellm/mcp-vectorizer"],
      "env": {
        "VECTORIZER_API_URL": "http://localhost:15002",
        "VECTORIZER_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

Authentication Methods Comparison:

Method Header Format Use Case
JWT Token Authorization: Bearer YOUR_JWT_TOKEN Dashboard, short-lived sessions
API Key Authorization: YOUR_API_KEY MCP, CLI, long-lived integrations

Production Security:

  • Change default credentials using environment variables
  • Use strong passwords (minimum 32 characters)
  • Generate secure JWT secret (minimum 48 characters)
  • See Docker Authentication Guide for details
  • Review Security Policy for best practices

πŸ“Š Performance

Metric Value
Search Speed < 3ms (CPU), < 1ms (Metal GPU)
Storage Reduction 30-50% with normalization
Test Coverage 95%+ coverage
Test Suite 1514 passing, 101 ignored
MCP Tools 26 focused individual tools
Document Formats 14 formats supported

Benchmark Results (vs Qdrant)

Comprehensive benchmark comparing Vectorizer with Qdrant across multiple scenarios:

  • Search Performance: Vectorizer is 4-5x faster than Qdrant in all test scenarios
    • Average latency: 0.16-0.23ms (Vectorizer) vs 0.80-0.87ms (Qdrant)
    • Throughput: 4,400-6,000 queries/sec (Vectorizer) vs 1,100-1,300 queries/sec (Qdrant)
  • Insert Performance: Optimized with fire-and-forget pattern for non-blocking operations
    • Configurable batch sizes and request body limits
    • Background processing prevents API blocking
  • Test Scenarios: 5 comprehensive scenarios tested
    • Small (1K vectors), Medium (5K vectors), Large (10K vectors) datasets
    • Multiple dimensions: 384, 512, 768
    • Full benchmark reports available in docs/ directory

See Benchmark Documentation for detailed performance metrics and how to run benchmarks.

πŸ”„ Feature Comparison

Comprehensive feature comparison with major vector database solutions:

Feature Vectorizer Qdrant pgvector Pinecone Weaviate Milvus Chroma
Core
Language Rust Rust C (PostgreSQL) C++/Go Go C++/Go Python
License Apache 2.0 Apache 2.0 PostgreSQL Proprietary BSD 3-Clause Apache 2.0 Apache 2.0
Deployment Standalone/Embedded Standalone PostgreSQL Extension Cloud/Self-hosted Standalone Standalone Standalone
APIs & Integration
REST API βœ… Full βœ… Full ❌ (via PostgreSQL) βœ… Full βœ… Full βœ… Full βœ… Full
gRPC API βœ… Qdrant-compatible βœ… ❌ βœ… βœ… βœ… ❌
GraphQL API βœ… Full with GraphiQL ❌ ❌ ❌ βœ… ❌ ❌
MCP Integration βœ… 26 tools ❌ ❌ ❌ ❌ ❌ ❌
n8n Integration βœ… Official node ❌ ❌ ❌ ❌ ❌ ❌
Langflow Integration βœ… LangChain components ❌ ❌ ❌ ❌ ❌ ❌
Python SDK βœ… βœ… βœ… βœ… βœ… βœ… βœ…
TypeScript SDK βœ… βœ… βœ… βœ… βœ… βœ… βœ…
JavaScript SDK βœ… βœ… βœ… βœ… βœ… βœ… βœ…
Rust SDK βœ… βœ… βœ… ❌ ❌ βœ… ❌
C# SDK βœ… βœ… βœ… βœ… βœ… ❌ ❌
Go SDK βœ… βœ… βœ… βœ… βœ… βœ… ❌
Performance
Search Latency < 3ms (CPU)
< 1ms (GPU)
~1-5ms ~5-50ms ~50-100ms ~10-50ms ~5-20ms ~10-100ms
SIMD Acceleration βœ… AVX2 βœ… βœ… βœ… ❌ βœ… ❌
GPU Support βœ… Metal (macOS) βœ… CUDA ❌ βœ… Cloud GPU ❌ βœ… CUDA ❌
Storage & Indexing
HNSW Index βœ… βœ… βœ… βœ… βœ… βœ… βœ…
Product Quantization βœ… 64x compression βœ… ❌ βœ… ❌ βœ… ❌
Scalar Quantization βœ… βœ… ❌ βœ… ❌ βœ… ❌
Memory-Mapped Storage βœ… βœ… βœ… ❌ βœ… βœ… ❌
Persistent Storage βœ… .vecdb format βœ… βœ… βœ… Cloud βœ… βœ… βœ…
Distance Metrics
Cosine Similarity βœ… βœ… βœ… βœ… βœ… βœ… βœ…
Euclidean Distance βœ… βœ… βœ… βœ… βœ… βœ… βœ…
Dot Product βœ… βœ… βœ… βœ… βœ… βœ… βœ…
Advanced Features
Graph Relationships βœ… Auto-discovery ❌ ❌ ❌ βœ… ❌ ❌
Document Processing βœ… 14 formats ❌ ❌ ❌ βœ… ❌ βœ…
Multi-Collection Search βœ… βœ… βœ… βœ… βœ… βœ… ❌
Hybrid Search βœ… Dense + Sparse βœ… βœ… βœ… βœ… βœ… ❌
Semantic Reranking βœ… βœ… ❌ βœ… βœ… βœ… ❌
Query Expansion βœ… ❌ ❌ ❌ ❌ ❌ ❌
Embedding Providers
Built-in Embeddings βœ… TF-IDF, BM25, BERT, MiniLM ❌ ❌ ❌ βœ… ❌ βœ…
Custom Models βœ… βœ… βœ… βœ… βœ… βœ… βœ…
Scalability
Horizontal Sharding βœ… (BETA) βœ… βœ… (PostgreSQL) βœ… Cloud βœ… βœ… ❌
Replication βœ… Master-Replica (BETA) βœ… βœ… (PostgreSQL) βœ… Cloud βœ… βœ… ❌
Auto-scaling ❌ ❌ ❌ βœ… Cloud ❌ βœ… ❌
Management & UI
Web Dashboard βœ… React + Full Graph UI βœ… Basic ❌ (pgAdmin) βœ… Cloud βœ… βœ… βœ… Basic
Desktop GUI βœ… Electron + vis-network ❌ ❌ ❌ ❌ ❌ ❌
Graph Visualization βœ… vis-network + Full Controls ❌ ❌ ❌ βœ… ❌ ❌
Graph Management βœ… Create/Delete Edges, Path Finding ❌ ❌ ❌ βœ… Basic ❌ ❌
CLI Tools βœ… βœ… βœ… ❌ βœ… βœ… βœ…
Migration & Compatibility
Qdrant Compatibility βœ… Full API βœ… ❌ ❌ ❌ ❌ ❌
Migration Tools βœ… Qdrant β†’ Vectorizer ❌ ❌ ❌ ❌ ❌ ❌
Security
Authentication βœ… JWT + API Keys βœ… βœ… (PostgreSQL) βœ… Cloud βœ… βœ… βœ…
RBAC βœ… βœ… βœ… (PostgreSQL) βœ… Cloud βœ… βœ… ❌
Encryption at Rest βœ… βœ… βœ… (PostgreSQL) βœ… Cloud βœ… βœ… ❌
Cost & Licensing
Open Source βœ… βœ… βœ… ❌ βœ… βœ… βœ…
Self-Hosted βœ… βœ… βœ… βœ… (Enterprise) βœ… βœ… βœ…
Cloud Hosted βœ… (HiveHub.Cloud) βœ… (Qdrant Cloud) βœ… (Various) βœ… βœ… (Weaviate Cloud) βœ… (Zilliz Cloud) βœ…
Free Tier βœ… Unlimited βœ… βœ… βœ… Limited βœ… βœ… βœ…

Key Differentiators

Vectorizer Advantages:

  • βœ… MCP Integration: Native Model Context Protocol support with 20 focused tools
  • βœ… Graph Relationships: Automatic relationship discovery with complete GUI management (create/delete edges, path finding, neighbor exploration)
  • βœ… No-Code Integrations: Official n8n node and Langflow components for visual workflow/LLM app building
  • βœ… GraphQL API: Full GraphQL API with GraphiQL playground and complete REST parity
  • βœ… Document Processing: Built-in support for 14 document formats (PDF, Office, images)
  • βœ… Desktop GUI: Electron-based desktop application with vis-network graph visualization
  • βœ… Qdrant Compatibility: Full API compatibility + migration tools + gRPC support
  • βœ… Performance: 4-5x faster search than Qdrant in benchmarks
  • βœ… Unified Storage: Compact .vecdb format with 20-30% space savings
  • βœ… Complete SDK Coverage: 6 official SDKs (Python, TypeScript, JavaScript, Rust, C#, Go)

Best Use Cases:

  • Vectorizer: AI applications requiring MCP integration, no-code workflows, document processing, graph relationships, and high-performance search
  • Qdrant: Production-ready vector search with good performance and cloud options
  • pgvector: PostgreSQL-based applications needing vector search alongside relational data
  • Pinecone: Managed cloud solution with minimal infrastructure management
  • Weaviate: Applications requiring GraphQL and built-in ML models
  • Milvus: Large-scale deployments requiring advanced scalability features
  • Chroma: Python-first applications with simple setup requirements

πŸ”§ Recent Improvements (v1.8.x)

New Features (v1.8.3+)

  • βœ… Dashboard Authentication: Complete authentication system for the dashboard
    • Login page with username/password form and modern UI
    • JWT token-based authentication via /auth/login endpoint
    • Session persistence with localStorage and automatic route protection
  • βœ… HiveHub Cluster Integration: Multi-tenant cluster mode support
    • HubManager for HiveHub API integration with tenant isolation
    • API key validation, quota enforcement, and usage tracking
    • Request signing and IP whitelist support for security
  • βœ… Cluster Memory Limits: Enforce predictable memory usage in cluster mode
    • Global cache memory limit (default: 1GB)
    • MMap storage enforcement and file watcher auto-disable
    • Comprehensive configuration validator at startup
  • βœ… MMap Storage Deadlock Fix: Fixed deadlock during concurrent vector insertions
    • Removed internal Arc<RwLock<>> wrapper for proper lock management
    • Stable concurrent insert operations without blocking

Quality Improvements (v1.8.1+)

  • βœ… Dashboard SPA Routing Fix: Browser refresh now works on all dashboard routes
  • βœ… File Watcher Improvements: Uses default collection instead of creating empty collections
  • βœ… Empty Collection Management: New endpoints to list and cleanup empty collections
  • βœ… Dashboard Cache Headers: Proper caching for faster dashboard loading

Previous Features (v1.6.0 - v1.7.0)

  • βœ… Graph Dashboard Enhancements: Complete graph management interface
  • βœ… n8n Integration: Official community node for workflow automation
  • βœ… Langflow Integration: LangChain-compatible components for RAG pipelines
  • βœ… GraphQL API: Full GraphQL API with async-graphql and GraphiQL playground
  • βœ… SDK Master/Replica Routing: Automatic read/write routing for high availability
  • βœ… All core tests passing: 1514+ tests with comprehensive coverage

🎯 Use Cases

  • RAG Systems: Semantic search for AI applications with automatic document conversion
  • Document Search: Intelligent indexing and retrieval of PDFs, Office files, and web content
  • Code Analysis: Semantic code search and navigation
  • Knowledge Bases: Enterprise knowledge management with multi-format support

πŸ”§ MCP Integration

Cursor IDE configuration:

{
  "mcpServers": {
    "vectorizer": {
      "url": "http://localhost:15002/mcp",
      "type": "streamablehttp"
    }
  }
}

Available MCP Tools (26 tools):

Core Operations (9 tools)

  • list_collections, create_collection, get_collection_info
  • insert_text, get_vector, update_vector, delete_vector
  • search, multi_collection_search

Advanced Search (4 tools)

  • search_intelligent - AI-powered search with query expansion
  • search_semantic - Semantic search with reranking
  • search_extra - Combined search using multiple strategies
  • search_hybrid - Hybrid dense + sparse vector search

Discovery & Files (7 tools)

  • filter_collections, expand_queries
  • get_file_content, list_files, get_file_chunks
  • get_project_outline, get_related_files

Graph Operations (6 tools)

  • graph_list_nodes, graph_list_edges, graph_find_related
  • graph_create_edge, graph_delete_edge
  • graph_discover_edges, graph_discover_status

Note: Cluster management operations are available via REST API only for security reasons.

πŸ“¦ Client SDKs

All SDKs are synchronized with server version 1.8.5:

  • Python: pip install vectorizer-sdk (v1.8.5)
  • TypeScript: npm install @hivellm/vectorizer-sdk (v1.8.5)
  • Rust: cargo add vectorizer-sdk (v1.8.5)
  • JavaScript: npm install @hivellm/vectorizer-sdk-js (v1.8.5)
  • C#: dotnet add package Vectorizer.SDK (v1.8.5)
  • Go: go get github.com/hivellm/vectorizer/sdks/go (v1.8.5)

πŸ”— Workflow & LLM Integrations

n8n Integration

Official n8n community node for no-code workflow automation.

Installation:

npm install @vectorizer/n8n-nodes-vectorizer

Features:

  • Collection management (create, delete, get, list)
  • Vector operations (insert, batch insert, delete, get)
  • Search operations (vector, semantic, hybrid)
  • 400+ n8n node integrations available
  • Visual workflow builder

Example Workflow:

Document Loader β†’ Vectorizer (Insert) β†’ Trigger β†’ Vectorizer (Search) β†’ Response

See n8n Integration Guide for detailed usage.

Langflow Integration

LangChain-compatible components for visual LLM application building.

Installation:

pip install vectorizer-langflow

Components:

  • VectorizerVectorStore - Full LangChain VectorStore implementation
  • VectorizerRetriever - RAG pipeline retriever
  • VectorizerLoader - Document loader for existing vectors

Example:

from vectorizer_langflow import VectorizerVectorStore
from langchain.embeddings import OpenAIEmbeddings

vectorstore = VectorizerVectorStore(
    host="http://localhost:15002",
    collection_name="docs",
    embedding=OpenAIEmbeddings()
)

# Add documents
vectorstore.add_texts(["Document 1", "Document 2"])

# Search
results = vectorstore.similarity_search("query", k=5)

See Langflow Integration Guide for detailed usage.

πŸ”„ Qdrant Migration

Vectorizer provides comprehensive migration tools to help you migrate from Qdrant:

  • Configuration Migration: Parse and convert Qdrant config files (YAML/JSON) to Vectorizer format
  • Data Migration: Export collections from Qdrant and import into Vectorizer
  • Validation: Validate exported data, check compatibility, and verify integrity after migration
  • REST API Compatibility: Full Qdrant REST API compatibility at /qdrant/* endpoints

Quick Migration Example:

use vectorizer::migration::qdrant::{QdrantDataExporter, QdrantDataImporter};

// Export from Qdrant
let exported = QdrantDataExporter::export_collection(
    "http://localhost:6333",
    "my_collection"
).await?;

// Import into Vectorizer
let result = QdrantDataImporter::import_collection(&store, &exported).await?;

See Qdrant Migration Guide for detailed instructions.

☁️ HiveHub Cloud Integration

Vectorizer supports multi-tenant cluster mode integration with HiveHub.Cloud for managed deployment:

Features

  • Multi-Tenant Isolation: Each user's collections are isolated with owner-based filtering
  • Quota Management: Collection count, vector count, and storage quotas enforced per tenant
  • Usage Tracking: Automatic tracking and reporting of resource usage
  • User-Scoped Backups: Create, download, and restore backups per user

Configuration

Enable HiveHub integration in config.yml:

hub:
  enabled: true
  api_url: "https://api.hivehub.cloud"
  tenant_isolation: "collection"
  usage_report_interval: 300

Set the service API key:

export HIVEHUB_SERVICE_API_KEY="your-service-api-key"

Internal Request Headers

For internal HiveHub requests:

# Bypass authentication
curl -H "x-hivehub-service: true" \
     http://localhost:15002/collections

# With user context (tenant scoping)
curl -H "x-hivehub-service: true" \
     -H "x-hivehub-user-id: <user-uuid>" \
     http://localhost:15002/collections

See HiveHub Integration Guide for detailed documentation.

Cluster Mode Requirements

When running Vectorizer in cluster mode, the following requirements are enforced:

Requirement Description Default
MMap Storage Memory storage is not allowed; MMap is required Enforced
Cache Limit Maximum cache memory across all caches 1GB
File Watcher Automatically disabled in cluster mode Disabled
Strict Validation Server fails to start on config violations Enabled

Example cluster configuration:

cluster:
  enabled: true
  node_id: "node-1"
  memory:
    max_cache_memory_bytes: 1073741824  # 1GB
    enforce_mmap_storage: true
    disable_file_watcher: true
    strict_validation: true

See Cluster Memory Limits for detailed configuration and troubleshooting.

πŸ“š Documentation

πŸ“„ License

Apache License 2.0 - See LICENSE for details

🀝 Contributing

Contributions are welcome! Please see our Contributing Guide for details.

About

A high-performance, in-memory vector database written in Rust, designed for semantic search and top-k nearest neighbor queries in AI-driven applications, with binary file persistence for durability.

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •