RSS Bilingual Reader

Manufacturing as Maintenance The maintenance spectrum has two ends. At one end, an object is lovingly maintained with effort and care through the years. A cherished blade sharpened on a whetstone every few months. At the other end, we have manufacturing: just toss the dull knife into the smelter at one end of a factory, and out the other end comes a beautiful, perfect, factory-sharp replacement. The former garners respect. And it's easy to see why: frequent maintenance was requisite for civiliza...

LLM Memory I've been thinking about LLM memory since GPT3 came out. Back then, my LLM side project was story generation (i.e. fiction). Context windows for LLMs were tiny back then: 4K tokens, input + output. So just a few pages of text. How to write a novella if your entire knowledge of the text is only a few pages, as if you were an amnesiac author? Suppose you're writing a scene. Where does the scene take place? Your description must match any previous description of the same place and be con...

Claude Code A few months ago, Anthropic released Claude Code . It's basically Claude in a little agentic CLI loop. Out of the box, it's good, but not great. But it can be made great with a few tips. This post describes all the tips and tricks I have for using it effectively. N.B. I've been in the habit of writing it as claude-code instead of Claude Code to really clarify that I'm talking about a tool and not, like, just code written by Claude. It's a really confusing name for people not already ...

Solution-space taste There's a visual analogy I've been drawing on whiteboards for years to explain a key element of taste in software design. Imagine the problem domain as a 2D field of obstacles you're trying to get across: In this analogy, a path represents a solution. Suppose you're trying to get from the center of the bottom to the center of the top. A common failure mode of junior engineers is only considering what obstacle is directly front of them. They'll tend to produce solutions like ...

#controls { display: flex; justify-content: center; align-items: center; margin: 10px auto; gap: 10px; } #controls button, #controls span { margin: 2px; white-space: nowrap; } #timerDisplay, #moveDisplay { display: inline-block; padding: 0 4px; } canvas { display: block; margin: 0 auto; } New Game timer: 0 moves: 0 const COLS = 16; const TILE_SIZE = 50; // larger tiles const LABEL_MARGIN = 6; const LABEL_FONT_SIZE = 21; const TILE_FONT = 18; let canvas, ctx, tiles = []; let selectedIndex = -1; l...

Typo Minimizing Keyboard I'm always frustrated when I make a typo, but the typo is of another valid word, so a spellchecker can't easily catch it. For instance, "love" and "live" are easy to typo — 'o' and 'i' are adjacent on a QWERTY keyboard. There's a ton of these words. Precious and previous. Funding and Finding. All these words differ in 1 letter and those letters are adjacent on the keyboard. A little question I'd always wanted to answer but never got around to until now: what is the keybo...

The curve is bending We've finally crossed an inflection point of model utility where the output is actually worth more than the cost for real dev work. It's starting to make sense to throw lots of money towards inference. Some devs will likely get 5-digit AI inference budgets in 2026. There will probably be some disruption of the junior dev job market, or at least disruption of what it means to be a junior dev. This prediction is based on the last few months of using the newest generation of mo...

#controls { display: flex; justify-content: center; align-items: center; margin: 10px auto; gap: 10px; } #controls button { margin: 2px; } canvas { display: block; margin: 0 auto; background: #fafafa; border: 1px solid #ccc; } New Game Cheat remaining: 10 moves: 0 (optimal: 0) timer: 0 Mouseover a puck to select it Click in a direction to slide Pucks stop at walls or other pucks Get the matching puck to the colored X This game is obviously inspired by Ricochet Robots which is the best nerd party...

Rohit Chess @krishnanrohit tweeted this tonight: 7yo created a new game - The front lines move like King in chess - Large circles in the back like bishop, diagonally - The spiral round thing is the King to get checkmated, though it moves like Queen in chess Surprisingly fun! pic.twitter.com/cBtkfX7XPB — rohit (@krishnanrohit) March 10, 2025 I described the game to o1-pro and had it spit out this webgame. Really fun that this is possible now. The game is pretty draw-ish since all the pieces...

AI software I'm surprised doesn't exist yet State of play I'm writing this on 20 February 2025. It's been: 4.5 years since GPT3 2.5 years since DALLE-2 and StableDiffusion 2 years since ElevenLabs voice cloning 1.5 years since Llama 2 1 year since Suno AI music 3 months since OpenAI o1 The good I don't like to be overly pessimistic, so I'll start by listing a few applications of AI that do exist that I'm impressed by: Chat This goes without saying. The chat models continue to get better. I use t...

Technocapital In this essay, I posit: A technocapital entity will exist That entity will consume an ever-growing percentage of the universe Humanity will be contained to a bounded slice of the universe Within that slice, the only labor that will have value is that in which humanity is inseparable from the value (i.e. art) Definition Let's recursively define technocapital as the abstract entity whose goal is increasing the amount of technocapital. This recursive definition is true if you think ab...

How to write a good design document This essay is a version of the response I gave to my friend Vik's call for suggestions: had a lot of people ask how they can learn to write design docs, and i didn’t have a good answer besides “work at a place with writing culture and smart senior engineers”. anyone have more useful advice? https://t.co/7F9BV31shv — vik (@vikhyatk) August 31, 2024 Definition A design document is a technical report that outlines the implementation strategy of a system in ...

Design Patterns A software design pattern is just what the name suggests: a pattern that comes up repeatedly in software design. Just like an architect may design two different houses, there are a lot of similar problems faced. For example, where to position the door to a room? The middle of a wall? The corner? Does it open in or out? A skilled architect may say "this is a small bedroom, so I'll apply the corner-door-that-swings-outwards pattern". The architect has used his experience to match a...

How to write complex software Suppose you are implementing some complex piece of software — a database, a word processor, a filesystem, a web browser, whatever. How do you get started? How do you actually organize the code? This post attempts to lay out a general method for approaching this problem with some specific techniques and guiding heuristics. Overview The 10,000 foot view of what we'll discuss: Write some toy driver programs to find out the physical constraints of the solution-space you...

Bureaulogy I'm fascinated by the formation, evolution, and perpetuation of bureaucracies. If I were to ever lose my mind and decide to go do a sociology PhD, I'd absolutely do it about bureaucracy. I'd want to establish a new subdiscipline of bureaulogy — the study of bureaucracies. I think this subdiscipline is just as real and interesting as something like anthropology or psychology. Some principal questions of the field would be: Why do bureaucracies inevitably form in every organization of h...

A peasant's plight I recently read the excellent book The Inheritance of Rome . It covers the period between 400 and 1000 AD, i.e. the Dark Ages. Until reading it, I hadn't really considered what it was like to be a serf in the feudal era. You learn about the period in World History class in high school, but mostly just mechanistically — how the feudal system worked, etc. It's hard for the modern reader to understand just how oppressive the situation was. Suppose you're a peasant and you want to...

Every Man his own API We're witnessing the spread of social trend I've been calling API-ification . This isn't a new trend, but has previously only been available to the few. By API-ification, I mean a person exposes well-defined processes for interfacing with that person. For most of history, only high-demand people such as politicians, CEOs, celebrities had access to this ability, by having "people" who act as the API. Want to get dinner with the CEO of XYZ Corp? Call his assistant and they'll...

Culture is a set of social Schelling points Imagine if I picked 1 person from 1000 disparate tribes from around the planet and put them in a town together. The town would have 1000 citizens, but no culture. It may develop a culture over time, but immediately post-transplant, there is none. Culture is a shared experience. It's high-context. And, to persist as a coherent entity, must have traits that encourage cohesion, it must have homeorhesis . What are a culture's traits that promote social coh...

Portals are Undertheorized I'm not talking about fictional teleportation portals. I'm talking about portals in sense of passages between places. This post discusses a few types of portals and their psychological effects. Threshold Let's call all door-like things threshold portals. Gates, archways, doors, etc. A threshold portal has two primary purposes: to divide spaces, and to frame spaces. A good threshold should induce movement . It should beckon the viewer to come closer, view the other spac...

Binary IQ I'm defining a term I refer to as Binary IQ , or sometimes Yes/No IQ . The basic idea is suppose you have a prompt where everything that could possibly be relevant is in the context window. Nothing irrelevant is in there. You ask a simple yes or no question. How well does the model do? My hypothesis is that performance on this benchmark is the main bottleneck to getting LLM-based agents to work in the near term. A related thought is that the best LLMs (Claude 3.5, GPT-4o, Llama 3 405B)...