标签 · Database — Glean

Recent picks

9picks · chronological

09-17

Your Database Skills Are Not 'Good to Have'

A MySQL war story from 2006: a three-person team builds faceted search for New York Magazine's Fashion Week portal, with exact per-tag counts, before Solr facets or Endeca existed. The author tunes MySQL 4 by timing queries and reading EXPLAIN output. Twenty years later he sees the opposite trend: engineers reach for "planet-scale" databases while barely knowing the relational engine they already run. He recounts an e-commerce incident where a product listing page took over 10 seconds even with no traffic, caused by three mistakes at once: no index, ORM loops firing 200-500 queries per page, and SELECT-ing every column. The argument: a modern RDBMS is innocent until proven guilty, and the burden of proof is on you. Includes a troubleshooting runbook and anti-patterns (exotic databases, unnecessary caching, data landfill). For backend and data engineers.

renegadeotter.com · 14 min · Database · MySQL · Performance

09-14

My Principles for Building Software

A practitioner's list of principles for building software, most aimed at making systems simpler: make invalid states unrepresentable, enforce data consistency, design data before code, measure before trading away simplicity. The appendix shows what inconsistency costs — split two Boolean variables x and y that must stay equal into separate databases and the data gains two more states, leaving the toggle function with no correct answer. The author argues consistency is the most undervalued property in software engineering and that most bugs are data failing an expectation. Other principles: avoid trading local simplicity for global complexity (smaller services often do this), don't optimize without measurement, keep code consistent even when the consistent thing isn't the "correct" thing, and learn concepts — the relational model, algebraic data types, borrow checking — rather than surface details of React or Kubernetes. Aimed at backend and data engineers weighing service splits and schema design.

kevinmahoney.co.uk · 9 min · Database · Programming Languages · Software Engineering

08-27

PostgreSQL for Everything

Drawing on years as CTO and interim manager, the author argues that PostgreSQL can replace most of your stack: full-text search, document storage, queues, time-series, vector search, cache, even graph databases. He cites Contentful, Instacart and The Guardian as real-world examples, then walks through the mechanisms: tsvector/tsquery, GIN indexes, SELECT ... SKIP LOCKED for queues, UNLOGGED tables for cache-like speed, TimescaleDB for analytics, pgvector for AI retrieval, and Apache AGE for openCypher. The post is a practical starting point for engineers who want to simplify operations, but it is an opinion essay rather than a benchmark-driven study, so the performance claims need validation before adoption.

www.raphaelbauer.com · 13 min · Database · Full-Text Search · PostgreSQL

08-27

SQLite for Everything: Cut Your Stack Down to One File

In this reply to 'PostgreSQL for Everything', the author makes the case that SQLite can collapse an entire stack into one file and one function call. It covers reusable patterns: using BEGIN IMMEDIATE plus RETURNING to turn a table into a durable work queue; indexing JSON via generated columns so schemaless writes get indexed reads; treating FTS5 as a same-transaction search index; and using sqlite-vec so embeddings, documents, metadata and full-text live in the same file, making hybrid search a JOIN. The article cites SQLite's own '35% Faster Than The Filesystem' benchmark, and claims warm-cache point lookups land around 1 microsecond versus roughly 100 microseconds for a localhost Redis GET. It also names real limits: a single writer, serialized concurrent consumers, and the need to move to PostgreSQL or specialized systems once you hit them. Useful for backend and infrastructure engineers who want concrete arguments for cutting components out of their stack before adding new ones.

joecode.com · 16 min · Cache · Database · Full-Text Search

07-19

CodeGraph: A Pre-indexed Semantic Knowledge Graph for AI Coding Agents with Dramatic Token & Tool Call Reduction

CodeGraph is a local knowledge graph tool that pre-indexes source code into a SQLite database for AI coding agents (Claude Code, Cursor, etc.). By providing agents with instant access to symbol relationships and call graphs via tree-sitter, it replaces traditional grep-file-scan loops. Benchmarking across 7 real projects shows an average of 25% cost reduction and 62% fewer tool calls. It features framework-aware route detection and cross-language bridging for mixed iOS/React Native codebases. Completely local with no API keys required, it targets engineers looking to cut token usage and accelerate AI-driven coding.

github.com · 35 min · AI · Database

05-29

ClickHouse 10 Best Practices

A ClickHouse solution architect shares 10 field-tested best practices derived from customer engagements, covering schema design, data types, partitioning, skipping indexes, JSON type, data ingestion, materialized views, system tables, ReplacingMergeTree, and JOIN optimization. Benchmarks on a 150M-row Amazon reviews dataset quantify the impact: proper ORDER BY reduces rows scanned by 347×, unnecessary partitioning slows queries by 46×, correct data types cut storage by 12% and double query speed, skipping indexes reduce scans by 80%, and dictionary lookups beat regular JOINs by nearly 3×. The article emphasizes understanding ClickHouse internals to achieve orders-of-magnitude improvements without hardware changes.

www.infoq.cn · 15 min · Database · Performance

05-29

ClickStack Observability: MCP Server, AI Notebooks, and ClickStack Cloud

At Open House, ClickHouse announced three major observability updates: ClickStack Cloud (serverless, managed, private preview), AI Notebooks (beta), and an open-source ClickStack MCP server. AI Notebooks replace linear chat with persistent, branchable investigation workspaces, exposing every query and step. The MCP server provides semantic investigative tools to external agents; internal benchmarks show 25% fewer tool calls, 2.5× consistency improvement, and 20% higher evaluation scores vs. raw SQL MCP. The server also supports bi‑directional orchestration: agents can create dashboards and persist results. The design philosophy is “bring your own agents,” with SQL as an escape hatch when pre‑built tools fall short. The post includes setup instructions and a demo. For infrastructure/SRE engineers evaluating ClickHouse-based observability.

clickhouse.com · 15 min · Agents · AI · Database

05-29

From OTel to Rotel: 4x Throughput Increase in PB-Scale Tracing

This article benchmarks OpenTelemetry data planes for writing trace spans to ClickHouse. On the same 8‑core host, Rotel achieves 3.7 million spans/sec (462k spans/core/sec), a >4× improvement over the OTel Collector. Gains come from three optimizations: binary encoding of JSON columns in RowBinary, moving deserialization to a shared thread pool to avoid tokio blocking and glibc allocator lock contention, and enabling fast LZ4 compression. The test also exposes silent data loss in the OTel Collector under backpressure. For engineers scaling large telemetry pipelines.

www.infoq.cn · 18 min · Database · Performance

05-29

Introducing ClickHouse Agent Skills

ClickHouse has released official Agent Skills: an open-source set of 28 prioritized best-practice rules covering schema design, query optimization, and data ingestion, packaged using Anthropic's Agent Skills specification. Users can add them locally with `npx skills add clickhouse/agent-skills`. AI agents (e.g., Claude Code) automatically invoke these rules when appropriate, helping avoid common pitfalls like wrong ORDER BY, non-scalable JOINs, or missing materialized views. The Apache 2.0-licensed repo welcomes community contributions.

clickhouse.com · 3 min · Agents · AI · Database