Database stories from Lobsters

SELF (Structured Executable & Linkable Format) is a prototype that replaces ELF with SQLite databases that can be chmod +x and executed. It represents segments, symbols, relocations, and dependencies as queryable tables, enabling SQL-based tooling, dynamic linking, closures, and transactional modifications. Built with Nix, it is lossless but currently has startup, memory-mapping, and size overheads.

A major security vulnerability in the meeting recording platform tl;dv allows any authenticated user to access metadata for all meetings across the entire platform due to a lack of Firestore tenant isolation. Additionally, an internal employee application leaks sensitive personnel information without authentication. Despite disclosures starting in January 2026, the issues remain unaddressed by the company.

The author argues that SQLite should adopt a versioned 'editions' system similar to Rust. Currently, SQLite’s default settings—such as disabled foreign key constraints, loose type affinity, and lack of busy-timeouts—often lead to bugs and poor performance. An edition-based approach would allow for modernizing global defaults while maintaining backward compatibility.

Tailscale experienced significant uptime instability due to a 16-year-old 'WAL-Reset' bug in SQLite. Triggered by Tailscale's non-standard, aggressive manual checkpointing, the bug caused silent data corruption. After months of forensic investigation and collaboration with SQLite maintainers, the issue was identified, patched, and verified, restoring reliability and hardening Tailscale's database infrastructure.

The creator of Hister, a self-hosted web history management tool, details how they reduced their dependence on Google Search by 50% in just 1.5 months. The core insight is the distinction between Discovery Search (finding new content) and Recall Search (refinding previously visited content). While search engines like Google are plagued by excessive advertising, SEO manipulation, and privacy concerns, Hister allows users to index the full text of their own browsing history locally. This includes content behind authentication and private networks that global search engines cannot access. By prioritizing speed, privacy, and local full-text indexing, Hister serves as a personal knowledge base that improves over time. The project is open-source under the AGPLv3 license and aims to provide a faster, ad-free alternative for professionals who frequently revisit known documentation or resources.

A developer shares a cautionary tale about losing their PostgreSQL database twice due to exploitable configurations. The root causes were weak default credentials and Docker modifying host firewall rules to expose the database port publicly. The incident highlights the importance of securing docker-compose ports, using strong passwords, and properly configuring UFW on a VPS.

Gitgres is an experimental system that uses PostgreSQL as a storage backend for Git repositories via libgit2. By treating Git objects and refs as database rows, it simplifies DevOps by unifying application data and repository storage, enabling complex SQL queries across code and metadata while streamlining backups and scalability.

Jepsen tested MariaDB Galera Cluster by simulating network faults and crashes. Findings reveal significant data persistence issues, including the loss of committed writes during node recovery, as well as consistency anomalies like Lost Updates and Stale Reads in healthy clusters. The study concludes that the system's actual performance fails to meet documented consistency and availability claims.

Observability often struggles with the high volume and inconsistent nature of logs, which become increasingly difficult to manage as data grows. Most logging stacks mutate in complexity or cost at scale. ClickHouse stands out by offering a consistent, columnar architecture that scales linearly through sharding, providing significant performance and efficiency advantages as data volumes increase.

SQLite 3.53.0 introduces the Query Result Formatter (QRF) library to enhance CLI output, adds support for modifying constraints in ALTER TABLE, improves the query planner, and introduces new JSON functions. It also provides new C APIs and optimizations while addressing critical reliability fixes like the WAL-reset corruption bug.

SQLite provides limited support for the ALTER TABLE command, primarily allowing table and column renames, and the addition or deletion of columns. While simple renames and additions are fast because they only modify schema text, certain operations like adding constraints or dropping columns require rewriting table data. For complex schema changes not directly supported, a 12-step manual procedure involving temporary tables is recommended to maintain database integrity and cross-version compatibility.

PostgreSQL 18 introduces pg_restore_relation_stats and pg_restore_attribute_stats, allowing users to export and inject optimizer statistics as portable SQL artifacts. This feature enables developers to replicate production query plans in CI/CD or local environments without needing full production datasets, significantly improving performance debugging and preventing query regressions.