IT HootClub — AI Community Newsletter
Hands-on. Career-focused. Future-ready.
Open weights are free to download and expensive to power — so this week is about compute: the laptop wall, the full menu of places a model can run, and the hands-on reality of renting a GPU on RunPod for the showdown ahead.
I went to Summerfest Tech AI expecting one kind of event and found a better one, and two things caught me off guard. The first was how easy it was to talk to people. I'd braced for the usual conference stiffness — lanyards, polite nods, everyone half-looking past you for someone more important. Instead the hallways were genuinely warm. Conversations started on their own, people were curious and generous, and I left with a stack of new connections and a few that felt real. It was more inviting than I expected going in. The second surprise was the programming itself — and that's the bigger story.
It was fresh in a way I didn't see coming. This was not a conference about AI bells and whistles. No parade of features, no thinly veiled tool demos dressed up as talks. The presentations were about the tech industry that surrounds AI — data and security, education, software development, entrepreneurship, and the specific industry use cases where the ideas actually meet the ground. It was, in a word, informing. I came away knowing more about the real backbone of this field than I did walking in, and that is not something I can say about most events I attend.
And sitting through all of it, one idea kept surfacing and wouldn't leave: AI isn't just another vertical in the tech industry. It's the connective tissue between all of them.
Data, security, development, networking, a dozen industry verticals — these have always been separate rooms speaking separate languages. What I watched at Summerfest is AI becoming the glue that pulls them into the same conversation. And you could hear it in how each one now frames its own work around AI. I'm simplifying, and not every talk fit a neat box — I'm not trying to pigeonhole anyone — but as a rough read: cybersecurity was largely about how to implement AI safely; data, about how to structure it for quality AI; software development, about how to wire AI in; even networking and hardware, about how to accommodate the GPUs all of this runs on. Different rooms, the same organizing question. Every one of these verticals has taken a shot in the arm from AI — not AI as a feature bolted on, but AI as the current now running under the whole field.
You could feel it in the energy. So many projects, so many founders bursting with ideas, so many people building things they couldn't wait to show you. It's creating a real buzz — and I want to resist the reflexive move of filing that under "hype." Some of it is, sure. But most of what I saw was people genuinely excited to build, and that deserves better than a skeptic's shrug. So let's call it what it is: exciting, fun, and challenging. Embrace it. The buzz is earned.
So yes — Milwaukee showed up, and this is a vibrant, busy moment to be standing in this space. I'm new enough that I can't honestly tell you how it stacks up against two or three years ago; I don't have the receipts yet. What I can tell you is that I left more certain than ever that the career pivot I made was the right one. I'm glad I'm here, I'm glad I went, and I fully endorse jumping in. If Summerfest was any indication, the water's warm.
Last week I picked the fighters: six open-weight models — models whose internals you can download and run on your own hardware — chosen on the right criteria for one job, answering Milwaukee questions from my curated tech inventory. The plan was to put them on rented graphics cards and score them against a frontier model. Then I went to actually do that and hit the question every one of these projects eventually trips over, the one nobody puts in the demo: where is this thing going to run?
It sounds like a small logistics detail. It is not. It's the difference between an idea and a working system, and it's the single biggest reason "I'll just run my own model" stays a fantasy for most people who say it. So before any scoring happens, this edition is about compute — the raw horsepower a model needs to think — and the surprisingly long menu of places you can go to get it.
Here's the wall everyone hits first. You hear that open models are "free to run," you have a perfectly good laptop, so you try to run one — and either it refuses to load or it answers at the speed of a fax machine. That's not your laptop being old. It's a hardware mismatch that no amount of patience fixes.
Language models don't run on the ordinary processor that handles your email and spreadsheets. They run on a GPU — a graphics processing unit, the same kind of chip built for video games — because a GPU does thousands of small calculations at once, which is exactly the shape of the math a model does to produce each word. And a GPU answers from its own dedicated memory, called VRAM (video RAM), which is separate from and far smaller than your computer's regular memory. That VRAM is the bottleneck. The model's "weights" — the billions of numbers that are the trained model — have to fit inside it.
Put rough numbers on it and the wall appears fast. A model carries roughly two bytes of weight per parameter, so a 20-billion-parameter model needs on the order of 40 gigabytes of VRAM just to hold still — before it does a single second of actual work. A typical laptop GPU has 8, maybe 16. The model simply doesn't fit. You can shrink it with a trick called quantization (storing each number at lower precision — think of it as rounding the weights to save space), and that buys you real ground, but it doesn't move a mid-sized model onto a laptop without paying for it in quality or speed. The honest summary: consumer laptops are wonderful for using models through a website or an app, and the wrong tool for hosting one yourself.
This is the quiet asymmetry behind the whole "run your own AI" pitch. Downloading the model is free and takes a minute. Powering it is the expensive part — and the part the marketing skips. Open weights remove the license fee, not the electricity bill.
Once you accept that you need a real GPU, the question becomes whose. There are more answers than most people realize, and they line up on a spectrum from "own it" to "rent it by the second":
Your own machine. A desktop with a serious gaming card (an NVIDIA RTX 4090, say) will run small and some mid-sized models comfortably. Apple's recent Macs are a sleeper option here too — their unified memory lets the GPU reach a large shared pool, so a well-equipped Mac can hold models a Windows laptop can't. Best case for privacy and for tinkering; you pay once, up front, in hardware.
Free cloud tiers. Google Colab and a few others hand you a borrowed GPU in your browser at no cost. Genuinely great for a first experiment — and genuinely frustrating the moment you need something reliable, because the free tier can evict you mid-run and the good cards are rationed.
The hyperscalers. Amazon, Google, and Microsoft will of course rent you GPUs. They're the default for big companies and, for a project like mine, the wrong fit: built for enterprise scale, wrapped in setup, and priced like it.
The specialist GPU clouds. A whole class of providers — RunPod, Vast.ai, Lambda, CoreWeave, and others — exists to do one thing: rent you a GPU by the hour or the second, with the AI tooling already in place and a fraction of the ceremony. This is the tier built for exactly what I'm doing.
For this showdown I went with RunPod, and the reasoning is worth seeing because it's the same shape as last week's model pick: choose the cheapest thing that genuinely fits the job, not the most powerful thing available.
Three features decided it. First, per-second billing — you pay only for the seconds a GPU is actually on, with no monthly minimum. My experiment is bursty: spin a card up, run the Milwaukee questions, shut it down. Paying by the second instead of the month means an afternoon of testing costs the price of an afternoon. Second, a choice between two grades of capacity: a pricier Secure Cloud running in vetted datacenters, and a cheaper Community Cloud that pools capacity from independent hosts. For a benchmark with no sensitive data, the cheaper pool is the obvious call — and being able to make that trade-off knowingly is the point. Third, the AI tooling is already there, so I spend my time on the experiment, not on assembling a machine from parts.
To keep the figures honest: as of mid-2026, RunPod's published rates put a high-end card like an H100 in the neighborhood of a few dollars an hour, with smaller cards well under a dollar — but GPU prices move constantly, so treat that as the shape of it, not a quote. The durable point isn't the number. It's that renting frontier-grade compute for an afternoon now costs less than lunch, which is genuinely new, and which is the entire reason a student in Milwaukee can run the same experiment a lab could.
None of this is an endorsement — RunPod is what fit my job this month, and Vast.ai or Lambda might fit yours better. The transferable skill isn't "use RunPod." It's knowing the menu exists, knowing which tier matches your job, and refusing to pay hyperscaler prices for a hobby-scale experiment.
So the contenders have a ring to fight in. Choosing it took the same discipline as choosing them: name the real job, find the cheapest option that honestly covers it, and skip the expensive overkill that only looks like rigor. If you want the rest of this story — what happened when I actually put RunPod to work, and the wall I walked straight into the moment I tried to point a model at my own data — that's where Under the Hood picks up this week. The scoring is finally next: same April rubric, same Milwaukee questions, every model grounded against the inventory. The fighters are picked. The ring is booked. Next time, somebody wins.
Building Intelligence made the case for renting a GPU and explained why I chose RunPod. This is the part where I actually use it — and I'll be honest up front about what I'm not going to do, which is walk you through every click. There isn't room in a newsletter for the full setup, and I've made my peace with that, because the step-by-step was never the barrier anyway. The barrier is purely picking one platform and starting. So here's the most useful nudge I can give you: this is cheap. I started with $25 on the account, and that was genuinely enough to rent real compute and work with several models — a few dollars buys real time on a real GPU. The intention to "learn to run my own models" almost always dies in the gap between intending and starting, and the price was never what guarded that gap. Twenty-five dollars and one afternoon clears it. Right-size the card, and with a serverless setup you don't even have to remember to switch anything off — the workers scale themselves down to zero when you stop calling them. The rest you learn by doing.
Once I had compute on tap, I built the part I actually cared about. I vibe-coded a small control panel into my admin portal at hardais.com — a GUI (graphical user interface, the clickable screen instead of a command line) that connects to RunPod with an API key (application programming interface key — a credential that lets my app talk to RunPod's servers) and lets me flip between the open-weight models I've stood up as endpoints. Compared to the old days of wiring each model up by hand, this was startlingly easy. And that is exactly where I thought I was finished. I wasn't — because the next question I asked turned out to be built on a wrong picture of how any of this works.
This is the part of every tech project nobody warns you about: the real obstacle is almost never the one you planned for. It ambushes you. With models a click away, I went to do the actual job — have one answer questions about my Milwaukee data — and asked what I assumed was simple plumbing. I have network volume storage on the RunPod server. So: how do I copy my database onto it, so the model sitting right there can reach it? It felt obvious. The model lives on RunPod's hardware; put the data next to the model. That question has no answer, and the reason it has no answer is the single most important thing I learned all month.
A model endpoint is stateless: text in, text out, and nothing else. It cannot open a connection, run a query, or reach across the network to "access" anything. The GPU loads the model and does math on the text you send it — full stop. The model never touches your database on its own. Not on the network volume, not anywhere, not in any design. That volume is for the model's own files (its weights), not your business data.
So the question I was asking was the wrong question. It isn't "how does the model reach my database?" It's "how do I get the relevant knowledge into the text I hand the model, before I call it?" All the data work happens on my server, in front of the call. RunPod only ever sees a block of text and answers it. Once that clicked, the design got simpler, not harder — and I had a choice to make about how simple.
My inventory is small, and it's a structured table. There's no needle-in-a-haystack problem to solve, because there's barely a haystack — the entire database fits inside the model's context window (the amount of text it can read in one pass). So I export the table as a CSV (comma-separated values — a plain spreadsheet file), paste it straight into the prompt, and say: here's the data, answer from it. No retrieval machinery, no embeddings, no vector search. People reach for RAG (retrieval-augmented generation, the embeddings-and-search pattern) by reflex here — but RAG exists to find what you need in data too big to send whole. For small data, building it is over-engineering. Skipping it was the right call.
I'll admit how this felt: like a cop-out. After renting GPUs and standing up model endpoints and building a control panel, the answer turned out to be "paste a spreadsheet into the prompt." It's the least exciting solution on the board, and some part of me wanted the problem to be harder so the fix could be more impressive. But that instinct is a trap, and naming it is half the lesson. Getting an answer that beats a frontier model does not have to be glamorous or complicated. The unimpressive solution that works beats the elaborate one that flatters me. I chose the spreadsheet.
One real warning, because this failure is silent — a confident wrong answer, not an error. Models are genuinely bad at arithmetic across a table. Ask one to count rows or total a column over a few hundred rows and it'll hand you a clean, confident number that's simply wrong, because it pattern-matches rather than tabulates. It's excellent at lookups and single-row reasoning, and weak at aggregation. So I keep my questions to what it's good at and verify any number that required counting. (And if your data spans several related tables, don't dump them all and hope — pre-join them in the database into one flat file, so the join is done exactly, not approximately.)
There is a more powerful option, and I want you to know it exists so the simple choice reads as deliberate rather than naive. When CSV-dump runs out — the table outgrows the context window, or I get tired of double-checking sums — the next step for structured data isn't RAG. It's text-to-SQL. SQL (structured query language) is what databases speak, and the loop is: my app hands the model the database's shape (just the table definitions, which are tiny) plus the question; the model writes a query as text; my server runs it; the exact result comes back and the model phrases it. And the catch that looks like a contradiction — doesn't that finally give the model access to my database? — resolves into the very same lesson: no. The model only writes a string of SQL, the way it writes a sentence. My server runs it; the model never participates. The stateless wall never moves. That's why it stays exact at any scale (the database does the math, not the model) and never fills the context window (you send the schema, never the data). It would also change which models I'd want in the ring: writing correct SQL is a coding task, so that path rewards coding models — a different class than the knowledge-Q&A models I picked for this showdown, which I chose precisely because the CSV approach only asks a model to read and answer, never to write code. It's more to build, it would need a read-only database account so a model-written query physically can't change anything, and it would send me back to re-pick the whole field — so I'm not building it yet. I'm noting the door and walking past it on purpose.
And I'll name the irony in walking past it, because it's the honest tension of using my own small project as the teaching example: this experiment is both a blessing and a curse for a real-world demonstration. The blessing is that I don't have a complex database to have to navigate and structure a fancy solution around. The curse is that I don't have a complex database to have to navigate and structure a fancy solution around. The same fact, read two ways. The clean, simple case is exactly what makes the lesson legible — and exactly what spares me the more impressive tool I'd otherwise get to show you.
Here's the thing I keep circling back to, and it's bigger than any of this plumbing: the work was never about the model. It was about compiling the data — gathering it, cleaning it, structuring it so a question has an answer. No model made that part easy. The model was the last and smallest step. The data was the whole job.
And there's a second half to that lesson I was slow to see, because it's the one that finally explains how a small model could ever win. The frontier model I'm racing — Claude with its web search — isn't a fundamentally different machine that gets to reach out and grab facts while mine sits blind. It's the same kind of thing mine is: a text-in, text-out engine that can't actually do anything on its own. When it "searches the web," it doesn't touch the internet either — it writes out a request, and the program wrapped around it runs the search and feeds the results back as text. That wrapper is the part worth naming: the harness, the code that turns a passive text engine into something that appears to act. Which means this contest was never model versus model. It's harness versus harness — their general-purpose web search against the narrow one I hand-wired to my own curated data.
That is the most pro-human conclusion I've reached in a year of this. If both sides run the same kind of engine, the model itself is close to a commodity, and the entire edge lives in the harness and the data feeding it — both of which are built, deliberately, by a person who knows the territory. A small open-weight model can beat a frontier giant on Milwaukee questions for exactly one reason: I cornered it better — a tighter harness and SME-grounded data (SME being subject-matter expert, someone who actually knows the field) aimed at the questions that matter. That combination wins, and it doesn't fall out of a model download. Someone has to build it, and that someone is precisely the part AI isn't replacing.
Now the concession that, oddly, is the whole reason any of this matters — and it sharpens the point rather than denting it. Earlier I said my small model can beat a frontier giant on Milwaukee questions, and that's true when the giant is leaning on a general web search and mine is working from my curated data. But level the field — hand that same curated data to Claude, or Gemini, or OpenAI — and the frontier model probably pulls ahead, because a frontier model fed the same narrowed facts is still a frontier model. So I'll say it plainly: this was never about proving an open-weight model is smarter. It isn't. Once I've done the hard part and curated the data, the easiest path of all is to just hand it to a frontier name and likely get a better answer. The point is that those names aren't always viable, or even preferable, for the person who actually needs the answer — sometimes the data legally can't leave the building, sometimes the per-call cost doesn't pencil out at scale, sometimes you can't afford to depend on a vendor who can change the price, the terms, or the model's very availability out from under you (ask anyone whose preferred model got pulled by a government order this month). For those users — and there are many — the question was never "is Claude good?" It plainly is. The question is whether you can get a similar, or better, answer from a model that isn't Claude, Gemini, or OpenAI — one you can own, run, and trust on your own terms. That is the entire reason to harness a lesser model well: not because the giants fail, but because for the user in front of me, the giants often aren't an option.
My own example, I'll grant, is small enough to deflate the drama a little: the data is so clean and compact that the harness barely had to break a sweat, and a tougher problem would lean on it far harder. But the lesson holds at any size, and the build was real labor even at this one. So once the showdown is scored, the interesting question won't be the victory — it'll be the invoice. Did the cost savings outweigh the effort? Did the result? Would there be any savings at all for someone who had to hire this out instead of build it themselves? My example is about as simple as these get and it still took the work — which is exactly where I think enterprise keeps stalling: no vendor's slide deck makes your data ready for you. That reckoning is its own edition, maybe more than one, once I've earned the answer.
One last bit of honesty, because everything above is conclusion ahead of evidence: while these realizations have been coming to light, I still haven't actually run the results. So the real question will be the one this whole build was secretly about — how effectively did I build the harness that makes a lesser model, smaller and less trained, beat an already adequately harnessed frontier model? That's the showdown. That's next.
“The art of being wise is the art of knowing what to overlook.”
— William James — William James was a 19th-century American philosopher and psychologist widely regarded as the father of functional psychology. He was the first educator to offer a psychology course in the United States and spent decades on the faculty at Harvard University. His writings on pragmatism and the nature of consciousness remain foundational texts in both philosophy and cognitive science.