Matt Rogish

Disposable Software - How to Stop Worrying and Love the AI Code

Fri, 03 Apr 2026 11:19:34 -0400

It’s pretty clear we’re in the “disposable software era”. Plenty of blog posts chatting about it:

For the most part, though, the examples referenced are about small, bespoke solutions to particular problems, often short-lived and/or non-production coded stuff, or things like “LLM needs to do a thing so it spits out some python, executes it, and deletes it.”

Thinking out Loud

Mon, 30 Mar 2026 15:42:01 -0400

LLMs & Agents: How do they Work?

Fri, 20 Mar 2026 10:23:26 -0400


# CONCEPTUAL PRE-TRAINING LOOP
internet_documents = crawl_internet()

internet_documents.each do |doc|
 # 1. Tokenize the entire document into an array of integers
 # In reality, this is done in chunks since the docs are probably too big. 
 # ATFLLM (Ask The F.. LLM) to learn about different chunk sizes.
 tokens = tokenizer.encode(doc) 

 # Inputs are tokens 0 to N-1
 # ["The", "capital", "of", "France", "is", "Paris", "and", "its", "population", "is", "about", "2", ...] 
 inputs = tokens[0...-1] 

 # Labels are tokens 1 to N (The "Answer" is just the next token)
 # ["capital", "of", "France", "is", "Paris", "and", "its", "population", "is", "about", "2", "million", ....]
 labels = tokens[1..-1] 

 # 2. THE FORWARD PASS
 # The model predicts the probability for EVERY token in the sequence at once 
 # (this is magic that GPUs provide since they are embarrassingly parallel) 
 predictions = model.predict(inputs)

 # 3. THE LOSS (The "Grade")
 # We compare the predicted probability of the 'correct' next token 
 # to the actual token that was in the text. 
 # Lower number is better.
 loss = calculate_loss(predictions, labels)

 # 4. THE BACKWARDS UPDATE
 # Nudge the weights prior to this so that (for example) 
 # 'is' predicts 'about' more strongly next time.
 model.adjust_weights(loss)
end


def generate_thoughts(prompt, number_of_samples=10)
 trajectories = []
 n_samples.times do
 thought_trace = model.generate(prompt,
 max_tokens: 8192,
 temperature: 0.8)
 trajectories << { thoughts: thought_trace.thoughts, answer: thought_trace.answer }
 end
 return trajectories
end


# Process-Supervised Training (Thinking)
def train_thinking_step(prompt, ground_truth_answer)
 # 1. Generate a "Thought Trace" (Chain of Thought)
 thought_trace = model.generate_thoughts(prompt)
 
 # 2. EVALUATE THE PROCESS (The "Secret Sauce")
 # We use a Process Reward Model (PRM) to grade each step
 step_rewards = thought_trace.map do |step|
 process_reward_model.grade(step) # Is this step logical?
 end
 
 # 3. EVALUATE THE OUTCOME
 final_answer = model.generate_final_answer(prompt, thought_trace)
 outcome_reward = (final_answer == ground_truth_answer ? 1.0 : 0.0)

 # 4. REINFORCEMENT LEARNING
 # We update the weights based on the SUM of the step rewards
 total_reward = step_rewards.sum + outcome_reward
 model.update_weights_via_ppo(total_reward)
end


def generate_with_cot(prompt, number_of_samples=10)
 trajectories = []
 
 number_of_samples.times do
 # We sample with Temperature > 0 to encourage exploration
 # Higher values create more hallucinations as it starts to 
 # drift further and further from the training data.
 thought_trace = model.generate( prompt, 
 max_tokens=8192,
 temperature: 0.8 
 )
 
 trajectories << { thoughts: thought_trace.thoughts, answer: thought_trace.answer }
 end
 
 return trajectories
end


def calculate_reward(thought_trace, final_answer, correct_answer, thought_complexity)
 # Step 1: RL VR 
 # If it's wrong we need to negatively reinforce this path
 return -1.0 unless final_answer == correct_answer
 
 # Base reward for being right
 reward = 1.0 
 
 # Step 2: Evaluation via Process Reward Model
 helpfulness, correctness, coherence, complexity, verbosity = process_reward_model.evaluate(thought_trace)
 # Did it actually 'think' or just guess?
 # Various models might tune these all differently
 reward += 0.2 if helpfulness > 0.5 && coherence > 0.5
 reward -= 0.2 if correctness < 0.5 || complexity > 0.5
 # And so on
 
 # Token efficiency check: 
 # If the thought trace is 10,000 words for a 2+2 problem, penalize it.
 reward -= 0.2 if verbosity > 0.5
 
 return reward
end


def calculate_loss_for_tool_syntax(response, ideal_response)
 reward = 0.0

 # This is handwavy to keep the code simple
 # We needed a tool, they didn't get it
 # Could still be helpful so don't kill it entirely
 reward -= 0.5 if response.didnt_use_tool?

 # Picked the wrong tool - big problem
 reward -= 1.0 if response.tool != ideal_response.tool
 
 # We know we needed to use the right tool
 if response.tool == ideal_response.tool
 reward += 0.5 

 if is_valid_tool_call(response.tool_call)
 reward += 0.3
 else
 # A broken tool call is bad, we don't want to learn wrong syntax
 reward -= 1.0 
 end
 end

 reward
end

For as much as AI has dramatically changed all of our lives, mine included, I was a bit uneasy that I didn’t really know how they are implemented and how they work. What is thinking and chain-of-thought reasoning? How does it choose a tool? What are parameters, exactly, and why are more of them better? What’s an Agent? What’s MCP?

Why Doesn't Apple Get AI?

Wed, 18 Mar 2026 13:01:20 -0400

A Few Single-line Stories in Everyday Frustration

“Hey Siri, what’s the weather tomorrow?”

Looks like thunderstorms tomorrow. The high temperature will be 66 degrees, and low will be 33.

So you go outside with an umbrella and a t-shirt only to realize that it’s nearly snowing when you’re out. Why? Because of a chart that looks like this:

“Hey Siri, play Toxic”. And then “Toxic” by Britney Spears plays instead of the one in my library that I already listened to today (BoyWithUke)!

The Wanting Comes in Waves

Tue, 17 Mar 2026 15:28:13 -0400

Drowning in a Sea of Tokens

It’s no secret that AI-produced slop is filling up social networks, websites, blogs, job search sites, homework. B2B marketers have completely clogged the internets here and - perhaps because they lack the confidence, or just value the convenience - I see clearly AI-produced “personal” content in Slack groups, newsletters, mailing lists and Discords of all shapes and sizes.

And I get it. The pull is undeniable. The magical autocomplete super tempting. I keep trying to resist; when I’m typing out a post I will have Sonnet critique my work and I have to fight taking the suggestions verbatim. To quote the famous Jim Gaffigan: “Hey, that’s something I’d say!”

Agents and the Era of Overproduction

Wed, 11 Mar 2026 11:14:45 -0400

Memory Lane

It seems somewhat fitting that now, March 11, 2026, almost three years ago to the day that OpenAI’s GPT-4 was released (and almost three years after I asked GPT-4 my first recorded questionI distinctly remember asking GPT3.5 to create a song about something in the style of Nine Inch Nails, but my ChatGPT.com history only goes back to March 27, 2023.

Compare March 2023 with March 2026. (I have no idea why I was interested in this.) ), I’m writing my first thoughts on it.

Writing - All Posts

Tue, 10 Mar 2026 10:00:00 -0500

How I Write Software With Agents

Mon, 09 Mar 2026 15:00:58 -0400

There’s a lot of churn in the AI/LLM/Agent space, but I’ve landed on an approach that reliably produces production-quality code for non-trivial work. The core insight is simple: use multiple LLMs to develop a solid architecture, then implement it through a phased crawl/walk/run approach.

Getting the plan right before writing code is where most of the value is. Here’s how that plays out in practice.

Process Overview

Set up your environment
Arrive at an architecture thru multiple LLMs
Adversarially review the result
Generate a crawl/walk/run implementation plan
Implement phase by phase, reviewing as I/you go

Environment Prep

Dev Containers

The agent needs to run your tests, catch its own errors, and iterate without you babysitting it. Without that feedback loop, you’re stuck manually pasting error messages back in or clicking “yes, run bin/rails test” after every step.

Choking: How Systems We Create at Work Sabotage Employees and Job Applicants

Thu, 15 Nov 2018 00:00:00 +0000

Preamble

I recently attended a small networking meetup of entrepreneurs and startup founders. About halfway through, the organizers invited people to give a small elevator pitch about what they’re working on and how they can help the group, and what help they need. As an ENFP I relish the opportunity to get up in front of people and share something and hopefully learn something in return.

So, naturally, I raised my hand. Instead of giving the smooth, polished pitch I normally give about what I’m working on, I stammered. I had trouble clearly articulating my vision and opportunity, and how I could help everyone there. In short, I choked.

iPad Only?

Thu, 15 Nov 2018 00:00:00 +0000

I’ve never been completely satisfied with my computer setup. Sure, the 2015 13” MacBook Pro I have is fine, it does what I need it to do. But as someone that travels a lot, it’s bulky and the battery life isn’t that great, and outside of this blog and some side projects, I don’t really do any software development. Nothing that remotely taxes the machine, aside from occasionally compiling new versions of Ruby.

How to Setup Eero With Multiple Wired Eero

Wed, 29 Aug 2018 00:00:00 +0000

After upgrading to Gigabit FiOS, I recently setup a series of eero devices in my house, replacing a couple of non-meshed Netgear routers. It wasn’t incredibly obvious how to setup a wired backhaul and so I thought y’all could use some quick pointers, as I got it wrong a few times.

Why I Work Remotely

Mon, 16 Jul 2018 00:00:00 +0000

Remote work is in the news again. Oh, boy.

How to Come Up With Profitable Business Ideas

Thu, 24 May 2018 00:00:00 +0000

A few days ago I came across this article from Indiehackers (a really cool site!) called “How to Come Up with Profitable Online Business Ideas”. It’s basically a giant list of folks that built something describing their process, which I guess is helpful to some degree, but ultimately feels like it misses the mark.

It opens with:

Interested in building a startup, or starting your own small business to make money on the side? Learn by example from dozens of entrepreneurs who share how they came up with great business ideas.

Lessons From Theranos Disaster

Wed, 23 May 2018 00:00:00 +0000

I recently had some time to myself (a rarity nowadays!) as I was doing a few transcon flights, so I picked up the new book on Theranos (Bad Blood: Secrets and Lies in a Silicon Valley Startup). I just finished the book - it’s a great read with a lot of interesting, painful lessons.

Silicon Valley’s VC “Cult of Personality” venerates the unstoppable, delusional young founder that builds the next Facebook when all the data shows us that - generally speaking - entrepreneurial success comes from knowledge, skills, and experience. Repeated studies¹/data² show that the average age of a successful startup founder is closer to 40³ than 20.

A Modest Proposal How to Replace Semver

Thu, 01 Jan 2015 00:00:00 +0000

I was browsing the Twitterscape and saw this tweet:

Ultimately, SemVer is a false promise that appeals to many developers — the promise of pain-free, don’t-have-to-think-about-it, updates to dependencies. But it simply isn’t true.

**jashkenas**, Why Semantic Versioning Isn't - https://gist.github.com/jashkenas/cbd2b088e20279ae2c8e

Mercifully I’ve been able to ignore Semver for a long time, but all this bru-ha-ha got me thinking about how we solved it in a previous open-source project (years ago).

How to Make Remote Teams Work

Thu, 01 May 2014 00:00:00 +0000

274 years

The population of Manhattan is roughly 1.5 million people. On the weekend. On a weekday, the population doubles to 3 million people. Almost two million people commute from outside the city. Each work day.

Their commute time is, on average, 48 minutes each way.

For 1.5 million people, who spend 96 minutes a day commuting, that’s 144,000,000 total minutes. 2,400,000 hours. 100,000 days. 274 years.

274 years spent commuting to/from Manhattan each work day

Metro.

How to Interview Programmers Part 2

Tue, 18 Feb 2014 00:00:00 +0000

🚩

This was written before AI changed everything.

In a previous post, I outlined a general strategy to interview programmers. Today, I’ll be more specific, including examples and rationales, on how to hire Ruby/Rails developers.

Automatically Maintaining and Improving Code Quality

Thu, 06 Feb 2014 00:00:00 +0000

New software projects start the same way: clean, well-tested, and fresh in everyone’s minds. Over time, however, the knowledge fades as little-visited parts of the code aren’t touched, folks move to other projects, and new developers come on board. Test coverage drops. Mass hysteria.

Without automated, enforced quality and test coverage metrics, entropy wins and you get stuck in a vicious cycle.

Slop feedback loop.

We have some tools in our arsenal to help prevent code rust: we can institute code reviews, we can pair program, or folks can “lunch-n-learn” to demo code. All these are well and good, but require active effort to maintain quality. What if there were automated methods to help ensure code quality that didn’t require manual intervention?

30 Years of Mac

Sun, 26 Jan 2014 00:00:00 +0000

30 years ago, Steve Jobs and Apple unveiled the Macintosh. Most people have seen that video, but I bet most haven’t seen the one he filmed eight years later, demoing the latest version of his new operating system, NeXTSTEP3.0.

Fascinating to see the Mac heritage in NeXTSTEP carry through the acquisition and transformation into OS X. It’s hard to remember, but in 1992 Microsoft had just released Windows 3.1 - NeXT was light-years ahead of Microsoft.

Remember Your Target Audience

Fri, 24 Jan 2014 00:00:00 +0000

In the software business, most folks work in an office. Maybe a quiet, private one, or one of the dastardly open plan ones. But not usually, say, a sports arena. Or in a moving vehicle.

Sometimes, this fact is lost on the people making the software. They design buttons that are washed out when the surveyor on a phone or tablet views it outside in the sun. Or they use too many external libraries in the HTML and end up swamping users who are on high-latency connections.

How Winners Win: John Boyd and the Four Qualities of Victorious Organizations

Sun, 19 Jan 2014 00:00:00 +0000

How winners win: John Boyd and the four qualities of victorious organizations

I’ve read “Boyd: The Fighter Pilot Who Changed the Art of War” and it has changed how I look at problems. Finding my own OODA loop has been instrumental at improving my success in my work. His four qualities might help improve yours.

Is This What Its Supposed to Feel Like

Sun, 19 Jan 2014 00:00:00 +0000

Is this what it’s supposed to feel like?

I just got an email from an engineering prospect who has decided to turn down our offer. We put so much time and effort and passion into recruiting him. He decided he wants to go off and start his own company right now.

I think more and more startup CEOs (and hiring managers) are figuring this out. At least, I’m telling every one I meet - for technical talent, you are not competing against the other startups in your neighborhood. Or even ones doing the same thing. Or startups in general. You’re competing against the awesome developer doing their own thing.

The Death of Expertise

Sun, 19 Jan 2014 00:00:00 +0000

The Death of Expertise

Having equal rights does not mean having equal talents, equal abilities, or equal knowledge. It assuredly does not mean that “everyone’s opinion about anything is as good as anyone else’s.” And yet, this is now enshrined as the credo of a fair number of people despite being obvious nonsense.

Tom Nichols makes a point that we in technology have known / identified for a long time. As the cost of knowledge transfer approaches zero, everyone thinks they know everything and can make expert level decisions. “I know about JSON and I heard Google uses MapReduce, let’s use MongoDB!”

How to Interview Programmers

Wed, 26 Jun 2013 00:00:00 +0000

🚩

This was written before AI changed everything.

I’ve posted a follow up with more specifics on “How to Interview Programmers Part 2”

Google has finally come clean that they figured out what many of us have been repeating for years: brain teasers in interviews don’t lead to measurably better employee outcomes.

I Judge a Company by Its Bathrooms

Wed, 12 Dec 2012 00:00:00 +0000

I travel quite a bit. As such, I have a membership to my preferred airline’s lounge. It provides a nice quiet space away from the blaring security announcements and inane airport CNN. The airline has recently spent a lot of money renovating the lounge network to give it a “fresh” look-and-feel. I think it looks sort of tacky, but the one thing they got right happened in the bathrooms.

They’re noisy. That is to say, there’s an exhaust fan that is just the right amount of white noise.
The urinals have an appropriately sized divider between them
The stalls are actually little rooms. Floor to ceiling walls, a door that closes.
The sinks are fashionable, have towels, and hand lotion.

It’s really quite nice. So, why do most employee bathrooms look like a prison?

Don't Build Big Software: A Small Software Manifesto

Fri, 14 Sep 2012 00:00:00 +0000

Facts do not cease to exist because they are ignored.

Aldous Huxley, "Proper Studies", 1927

In my last post, I discussed why software effort estimation was harmful. I mentioned that you can’t add up all the individual tasks to get a completion date at some arbitrary point in the future, and you can’t (with some caveats) use estimation to “staff up” a project from the beginning.

Where does this leave us? Should we not estimate at all? These folks think so:

Software Effort Estimation Considered Harmful

Thu, 16 Aug 2012 00:00:00 +0000

Dal73.

Delta Air Lines tells me it will take two hours and twenty-five minutes to fly from New York City to Atlanta, GA. American says two thirty. United? Two thirty-four. Yet, the captain just told me it was “one-hour forty-five” minutes “wheels-up to wheels-down”. What gives?

It turns out that the US Department of Transportation (US DOT) rates the airlines by “on-time performance”: whether or not the aircraft reaches the gate by the quoted time. So, like any rational actors, the airlines pad the flight time to account for ground delays (waiting by the gate for the tug driver to appear), re-routes due to bad weather, etc. - but also to goose the ratings.

Vagrant a Repeatable Dev Project Setup

Thu, 17 May 2012 00:00:00 +0000

A few weeks ago I wrote a post lamenting the lack of standardized project bootstrapping. Several of the commenters suggested Vagrant, so I have spent the last week or so giving it a try.

Challenge accepted.

Here is what’s awesome about Vagrant:

Uses Chef or Puppet to add packages to a base VM
Easily maintain and rebuild VMs
Pretty straightforward install process on OS X, Linux, and Windows
Mounts your source project internally so you can work on your code in the VM

Here’s what I don’t like about Vagrant:

On Repeatable Dev Project Setup

Thu, 03 May 2012 00:00:00 +0000

Last night I had the following exchange on Twitter:

Devsetuptwitter.

Having just jumped on board to a newish Django project here at work, I feel their pain. Although I don’t know the Django ecosystem as well as Rails, I do know that projects should contain a readme, and I shouldn’t have to:

execute server runner
install missing lib or package that it complained about
goto: 1 until working

Every development project, no matter how complex, should have a repeatable, predictable (preferably scripted) new developer project bootstrap.

Working Too Much Is Stupid

Tue, 17 Apr 2012 00:00:00 +0000

There is a big brouhaha because Facebook COO Sheryl Sandberg “admits … walk[ing] out of this office every day at 5:30”.

Oh, the shock and horror. The wailing and gnashing of teeth.

Why is this news? Why do folks think that working gobs and gobs of hours, as software developers, leads to better software, faster?

Sure, for something repetitive and methodological like bricklaying, you can crank up the hours worked dial and see the wall getting built faster. And, sure, there’ll be small bugs here and there (sloppiness as the worker gets tired, etc.) but generally nothing that will materially affect the structural integrity of the wall.

Expense Reports Are Bullshit

Mon, 16 Apr 2012 00:00:00 +0000

Or: Every Developer Gets an Amex

How often does this happen where you work:

Developer begs, pleads with boss to go to technical conference
Boss negotiates with developer on getting a cheaper hotel. (It’s only $69 a night! Clean sheets optional. 30 minute bus ride from conference. What a steal!)
Boss orders developer to get cheaper, three stop flight instead of more expensive non-stop
Developer attends conference
Developer returns from conference with a pile of low value receipts ($20 for lunch, $40 for cab to airport)
Developer spends two hours painstakingly scanning each receipt and constructs a spreadsheet with itemized list of charges
Boss approves charges without even looking at them
goto 1

How brain-dead is this? Here you have a highly-paid software developer doing super-low value data entry. It probably cost you $200 to have them enter all this junk.

Open Plan Offices Must Die

Sat, 17 Mar 2012 00:00:00 +0000

There are a million ways to lose a work day, but not even a single way to get one back.

DeMarco & Lister, Peopleware

How does a large software project get to be one year late? Answer: One day at a time!

Brooks, The Mythical Man-Month

Apple Store Upper West Side

It’s trendy right now to produce office space that looks like an Apple store. Bright, airy, cheerful, lots of open space, big ceilings, blonde wood, attractive and friendly employees. I can’t blame the urge - I love seeing the latest architectural masterpieces the geniuses at Apple routinely cook up. Living in New York I have the opportunity of being a short hop away from no less than five of them.

About