Robert Ritz

Computer Usage Flattery

Sat, 22 Aug 2026 10:50:12 +0800

For the past week I have enabled Computer History from Codex on my main computer. This utility records interactions on my computer including where I click, what I type, and in general what I’m doing. Today it prompted me to summarize my week. It gave a pretty decent summary of my week, with this as the kicker at the end:

Unscientific final diagnosis: high-agency operator, unusually low tolerance for vague interfaces, and dangerously susceptible to “while Codex is thinking…”

I’ll take the “high agency operator” flattery, and I’ll admit I am dangerously susceptible to doing about 4-5 things at at time with Codex. I’ll keep it on for now, although I don’t really yet see any use for it.

Neural network experiments are easy, I think

Fri, 14 Aug 2026 16:26:32 +0800

Heard recently on a podcast about how an ML model was created to predict how the neurons in a fruit fly were connected. The main finding was that the neurons connections could only be predicted reliably with a hyperbolic geometry. I thought it would be interesting to test, and I asked ChatGPT to research if this already existed. It turned up some similar things but not the exact idea.

My core question was:

Could hyperbolic proximity help a learning network choose which dormant connections to activate?

I’m not a deep learning research at all, but I do know how they work and have trained models from scratch and fine tuned them before. So I was familiar enough to know what to ask for.

It seemed like an easy enough thing to test, especially with the help of AI. The end result wasn’t promising (which is probably why it wasn’t published anywhere). I still feel there is some merit in the concept of rewiring, and I know researchers are looking into it. In any event it was a fun side quest to think about.

I’m almost certain someone has tried this idea before. But in a pretty wide search I couldn’t find anything exactly like what I wanted. If more people posted their bad results, it would probably save people (and AI agents) a lot of time!

I’ll let Codex/GPT5.6 Sol explain the experiment and results in its own words.

What We Tested

This summary was written by OpenAI’s Codex. Robert proposed the idea; I built, ran, and audited the experiment.

The idea was to let a sparse neural network change its wiring while it learned. Instead of keeping the same connections throughout training, the model periodically removed weak connections and replaced them with promising inactive ones.

The question was how to choose those replacements. A standard method uses gradients, which estimate which connections would most reduce the model’s error. Robert wondered whether Hyperbolic geometry might do better. Hyperbolic space represents hierarchical structures efficiently, and language contains plenty of hierarchy. Perhaps learned Hyperbolic positions could reveal useful connections that gradients alone missed.

The experiment

I trained a 6.90-million-parameter language model on TinyStories. Only 25% of the connection blocks in its feed-forward layers were active at once. The main comparison included:

Gradient-only rewiring

Gradient plus Euclidean geometry

Gradient plus Hyperbolic geometry

Within each seed, the models shared identical starting weights, sparse masks, training data, and evaluation schedules. Each model trained for 100,663,296 tokens. The full confirmation included five treatments across three seeds, or 15 trained models.

Lower validation loss means better prediction on text the model did not train on.

Method Mean validation loss

Gradient rewiring 2.293695

Euclidean geometry 2.293754

Hyperbolic geometry 2.293781

The three methods were effectively tied. Hyperbolic was 0.00114% worse than Euclidean on average and took 50.7% longer than gradient-only rewiring. Gradient rewiring did beat static sparsity in all three seeds, although only by about 0.082%.

One larger test

To see whether scale changed the result, I increased the model to 24.54 million parameters and repeated the three main treatments with the same data, token budget, sparse density, and seed.

Method Validation loss

Hyperbolic geometry 2.046006

Euclidean geometry 2.046079

Gradient rewiring 2.046139

Hyperbolic technically finished first, but its improvement over Euclidean was only 0.0036%. Before training, we required at least 0.5% to count as a meaningful result. The observed advantage was about 140 times smaller. Hyperbolic was also worse on two of the final five evaluations, while Euclidean had the slightly better complete learning curve.

Scale did improve efficiency. Hyperbolic’s runtime overhead fell from 50.7% to 15.9%. It became less expensive relative to the other methods, but it did not learn meaningfully better.

Conclusion

This does not prove Hyperbolic rewiring can never work. It tested one architecture, dataset, sparse structure, and rewiring schedule. It does show that this particular method failed to produce a useful advantage at both 6.90 million and 24.54 million parameters.

The entire project cost $2.96 and produced 18 trained models with verified checkpoints. My conclusion is that dynamic rewiring may help slightly, but adding learned Euclidean or Hyperbolic geometry did not improve it. Making the current design larger does not appear to be money well spent.

Method	Mean validation loss
Gradient rewiring	2.293695
Euclidean geometry	2.293754
Hyperbolic geometry	2.293781

Method	Validation loss
Hyperbolic geometry	2.046006
Euclidean geometry	2.046079
Gradient rewiring	2.046139

Daybreak Red scares me, Daybreak Blue to the rescue

Thu, 13 Aug 2026 15:08:19 +0800

Now that we have superhuman AI that can hack nearly anything, I welcome the announcement of Daybreak by the OpenAI team. Presumably these advanced cyber attack capabilities are still in the hands of state or other larger organizations, which means little fish like me probably aren’t the immediate target of these actors. That window is closing though, as OpenAI so delicately put it.

Once I saw the announcement I immediately signed up for the Trusted Access program. It took a few days for Daybreak Blue to show up, but today it was there! Daybreak Blue is a GPT5.6 Sol based model seemingly with cyber restrictions loosened (for defensive purposes, hence the blue name). In cybersecurity a “blue” team defends and a “red” team attacks. The red model is reserved for institutions and cybersecurity researchers.

The first thing I did was ask Daybreak Blue for a security audit of the various apps and services I have deployed on my main public server. It’s used for some business services, school services, and some personal projects (like my weather app). In one week these were the attack stats on this server:

31,050 rejected password guesses (Peak: 8,549 guesses in one day)
510 distinct hostile source IPs.
Most-targeted usernames:
- root: 12,075
- admin: 1,598
- ubuntu: 1,487
- user: 959
- aum: 772

Notably password authentication only existed with a silly long password up until last week. This was an oversight and I really should have turned it off earlier. Now login is much more difficult but there are still about 2,000 attempts per day.

It also checked our main web proxy for a one hour snapshot of access attempts:

7,786 requests total.
109 recognizable vulnerability probes targeting .env, WordPress, .git, AWS credentials, PHPMyAdmin, and CGI paths.
None received a successful response: 87 returned 404, 12 redirects, and 10 returned 502.

These are bots trying to find services on the server with exposed credentials, vulnerable paths, and common exploits in popular software like WordPress (which we don’t run on this server).

The funny thing is that this is the least surprising thing it found. Anyone who runs a server connected to the internet knows about these issues (vibe coders beware, there be dragons). Back in 2010 if I checked my server logs I would see very similar stats.

In general I feel like I run a solid server, and I’ve only knowingly had servers compromised a few times, and I’ve never lost any data. I do a monthly Common Vulnerabilities and Exposures (CVE) sweep on all my deployed apps and fix any CVEs that are known. I harden my servers and ensure the firewall only allows those ports that are necessary. Password login is always off (oops except this one, it’s the things you forget that can bite you).

For the most part, up until this year, the steps I’ve taken without Daybreak’s help would be enough to protect most small business or individual servers from most attacks. It wouldn’t be enough to protect against an “advanced persistent threat” like the NSA, Unit 8200, or any of the various Chinese or Russian APT groups. But I slept reasonably well knowing that I’m probably not the target of any of those groups. I simply run some business applications and a learning management system on this server.

Then all the news about offensive grade hacking models came out. I no longer feel any type of security. It doesn’t help that I’m reading Pandora’s Star, a book that features a super-intelligent AI that can break through insane levels of encryption and has advanced cyber capabilities. Sci-fi aside, the risk seems to be real and coming fast. So I’m doing what I can to reduce my attack surface. That is hard when most of the information in our lives today is digital and internet accessible.

Back when I briefly worked on staff for a member of Congress (circa 2010), I attended cybersecurity briefings aimed to help staffers protect themselves and their offices from attacks. These briefings were run by the NSA blue team. Back then it was mostly focused on mobile security. Being technical myself I knew all of the attack vectors they were talking about, but I could see the disinterest/confusion/apathy in the eyes of the audience. Most of these staffers were made to go by their offices, and the NSA team did a valiant job trying to make it understandable. Ever since I’ve had little hope that average people have any hope to secure their lives, and the real responsibility lies with our technology vendors and under appreciated systems administrators (buy yours a blueberry muffin next time you see them).

It is only a matter of time before open weight models obtain cyber capabilities, and then my server logs might start changing. If that happens, a single vulnerability or chain of small vulnerabilities can open your entire professional and personal digital existence. These days I’m mostly a CEO/president/professor, but I still do my sys admin duties on my cloud server, the rack at my university, and my home systems. I have given only passing thoughts to security for the past 6 or 7 years as my usual processes work great.

Daybreak Blue gave me some recommendations on further hardening my system, which I will promptly do. I’ll probably start a nightly or weekly CVE sweep as well. I only hope other sysadmins around the world are doing the same.

Google's Weather API doesn't want me to use it for weather.

Mon, 10 Aug 2026 12:12:58 +0800

Personally I’ve never wanted to make a weather app, but then I got an email from Google saying that their new AI based weather forecast model, which is supposed to be cutting edge, is now available for unlimited free experimental preview.

I’ve always found the precipitation forecasts to be quite poor in Mongolia. Since 2011 Mongolia’s National Agency for Meteorology and Environmental Monitoring (NAMEM) have used a [Cray](National Agency for Meteorology and Environmental Monitoring) super computer for forecasting. It works, but gets hourly precipitation wrong quite often. They are directionally correct, but they usually don’t help me in deciding to take an umbrella or not on a given day.

So when I got the email, I thought, why don’t I see how good it is? The best way to do that is to throw up a simple web app and test it over the next few weeks. So I go over to the console, enable the API, get a key, then ask Codex to whip it up.

It refuses….

The rationale is that 21.1 of the Google Maps Platform Service Specific Terms specifically prohibits this…

Weather API 21.1 Restrictions. Customers may not use Google Maps Content retrieved from Weather API to recreate a Google service or product (e.g. use data retrieved from Weather API in a weather app or weather model whose primary purpose is to provide weather information).

Codex helpfully nudged me into something that would tell me if I should take an umbrella or not. An app that tells me to take an umbrella or not is a weather app, albeit one that gives a binary signal rather than a precipitation percentage.

So if I make an app where the primary purpose is giving the news but I want to throw weather at the top that is fine. But if I want to make an app to test out this new weather API that is not, according to their own conditions. So to test out a weather API I need to make an app where the purpose isn’t weather. Lovely…

Update: I explained the silliness of this refusal and said that I’m just testing the API. It went ahead and built it. weather.robertritz.com is live and I’ll see if this API is any good.

Serious fuel shortage in Mongolia

Thu, 06 Aug 2026 12:34:21 +0800

This shortage is showing the strength of third neighbors and the weakness of direct ones

For the past few weeks, there has been a shortage of fuel in Mongolia. This is mostly due to the Russian invasion of Ukraine and Ukraine’s serious hammering of Russian refineries and pipelines. An additional challenge coming up will be the annual winter maintenance at the refinery in Irkutsk, where most of Mongolia’s fuel comes from. That usually shuts the refinery down for two to three weeks and causes shortages each year during the back-to-school rush.

Here are Mongolia’s fuel imports from 2021 to 2026. I wrote about a shortage back in 2021, and that one seemed to show an emerging pattern. At that time, a brief reduction in imports in early summer caused a cascade that resulted in a shortage a few months later. This happens for two reasons:

Mongolia has a limited amount of fuel storage capacity (about 45 days at most) for AI-92, the most common fuel for passenger vehicles.
Supply from Russia is limited, even before accounting for the war in Ukraine.

Mongolia is building more storage tanks, but this will only put a Band-Aid on the real problem: supply. About 95% of all fuel in Mongolia is imported from Russia, mostly from one refinery. If that isn’t a supply risk, I don’t know what is.

Source: [Data.mn](https://data.mn/en/data/mrpam-petroleum-imports)

Supply shocks from the Russian side are a yearly occurrence, and the war in Ukraine has only exacerbated them. When I wrote about the shortage in 2021, I noted that the government should have had a pretty good idea, months ahead of time, that there would be a shortage. Yet there was seemingly little public activity by the government to address the issue.

Third-neighbor strength

2026 is no different. New fuel supplies from South Korea and China have been secured, but they will have to arrive by southern rail. This route uses a different rail gauge, adding cost and time to the import process. In addition, fuel from South Korea must first be transported by sea, offloaded at a port, and then taken by train to Mongolia.

South Korea has pledged to supply 50,000 tons of fuel products monthly. China pledged an emergency supply of 6,000 tons, presumably as a one-time shipment.

Notably, Chinese oil imports have gone down significantly during the Iranian conflict. This is something of a mystery, given that China’s fuel storage, much of which is outdoors and visible from the sky, has not drawn down. Some experts in this space believe China may have a “secret stash” of fuel created in preparation for a potential conflict over Taiwan.

Benzin.mn

I’m astounded by the number of direct-name domains in Mongolia: oboi.mn, meaning wallpaper; unegui.mn, meaning free, although it is used for classified ads; data.mn, my own domain, meaning data; and medee.mn, meaning news. There are a few others that I can’t remember right now.

Well, now there is benzin.mn. Benzin is the Mongolian word commonly used for gasoline, although technically it means benzene, which is not gasoline. The site is a crowdsourced map showing which stations have gasoline.

This morning, I checked and only four stations in Ulaanbaatar were reported to have fuel. This is on top of an odd-even license plate restriction and a maximum purchase of 50,000 MNT.

Oil, oil everywhere, and not a drop for my Prius

Ironically, Mongolia has a lot of oil. It exports basically everything it produces. Production appears to be declining, something I didn’t realize until I looked at the chart this morning.

Source: [Data.mn](https://data.mn/en/data/mrpam-petroleum-production)

Nevertheless, in 2016, India decided to invest $1 billion in an oil refinery in Mongolia. At the time, this was the estimate of the fuel it would be able to produce:

…the oil refinery will have a processing capacity of 1.5 million tons of oil. As estimated, the oil refinery will produce 560 thousand tons of gasoline, 670 thousand tons of diesel fuel, and 107 thousand tons of liquefied gas per year.

By comparison, Mongolia imported 828,000 tons of AI-92 in 2025. This would therefore be a significant step toward reducing Mongolia’s dependence on foreign fuel.

There are a few problems though. Oil production declined from 4.6 million barrels in 2021 to 3.7 million barrels in 2025. Doing the math, 1.5 million tons is roughly equivalent to 30,000 barrels per day. If the refinery operates year-round, which is fairly standard apart from short maintenance shutdowns. That amounts to 10.95 million barrels per year. This means Mongolia’s current oil production would utilize only about 34% of the refinery’s expected capacity and therefore produce less gasoline that it is capable of.

It’s possible that there is a lot of oil-production capacity that could quickly be brought online and that I’m not aware of. The existing numbers don’t look good, though. Part of me wonders whether India’s investment in Mongolia was more of a snub at China than an effort to actually help Mongolia.

Winter prospects

It’s hard to say whether this fuel shortage will continue through the winter. The promised emergency supply from China will do little to help. However, the 20,000 tons of AI-92 (out of the 50,000 tons total) promised by South Korea will put a meaningful dent in the average 72,000 tons of AI-92 imported each month.

In the short term, at least through September, I expect the odd-even rule to continue. The real risk in all of this is diesel fuel. If that runs low, food trucks can’t get to market, and trains can’t transport food or coal.

With COP17 coming up, desertification looks like a small problem.

Lecture 2: The GPU Economy

Wed, 05 Aug 2026 22:32:02 +0800

In this second lecture, held on April 9, 2026, Brad Gerstner of Altimeter Capital and Sunny Madra of Nvidia (previously Groq) discuss the economics of GPUs and inference. Anything that is my personal opinion will be denoted with italics.

Course Page
Watch the Lecture

The main question for the lecture is fairly simple: how does the cost of inference keep falling when both model size and demand keep going up?

The GPU economy

Gerstner starts with a long-term view of technology. His slides show GDP per capita roughly doubling every 25 years, while technology’s share of GDP grows from about 5% in 1998 to an estimated 15% in 2030.

He also compares investment returns over the previous 15 years:

Technology companies: about 15% per year
Non-technology companies: about 6% per year

His point is that compute sits at the root of these changes. Most of the gap between tech and non-tech appeared after 2020. I have a hard time seeing this as a compute story and not a market run up that has little to do with value….but whatevs

Gerstner then introduces Madra, describing him as a founder who keeps getting acquired by larger companies. Madra was president of Groq before joining Nvidia.

How Groq works

Groq founder Jonathan Ross previously worked at Google, where he helped develop the company’s Tensor Processing Unit (TPU).

Madra describes Groq’s chip as a deterministic dataflow system. A compiler decides where the calculations will happen before the program runs. This is useful for large language models because generating tokens requires quite a bit of math.

The amount of work needed to generate one token depends partly on the number of parameters in the model and the amount of context it must process.

Prefill and decode are different problems

Madra separates inference into two parts:

Prefill processes the prompt and its context.
Decode generates the response one token at a time.

These operations put different demands on the hardware. Madra says it could make sense to use different machines for each one.

Groq’s design has a large amount of SRAM, which is fast memory located directly on the chip. Nvidia has a wider collection of chips and systems designed to work together. Madra also discusses NVLink Fusion, which allows custom processors to connect to Nvidia’s systems.

In the example he gives, combining the two approaches can generate 2.5 times as many tokens using the same amount of electricity.

Will inference keep getting cheaper?

Gerstner asks what will push the unit cost of inference down. Madra gives three answers:

Improvements in the supply chain
Work by hardware and systems engineers
Power

Lithography, the process used to manufacture increasingly small features on chips, is beginning to reach physical limits. One response is to make the chips larger. Cerebras produces wafer-scale chips that Madra compares in size to pizza boxes.

Hardware development is also happening alongside model development. Better hardware makes larger models possible, while larger models create demand for more hardware.

Madra says models with as many as 10 trillion parameters are coming. Even a 50-fold increase in capacity might not be enough over five years if models continue to get bigger.

There are already tens of trillions of tokens being generated each year. This was in April. Appetite for tokens has increased like they predicted, if not more so.

Cost goes down, usage goes up

Gerstner says OpenAI initially had negative gross margins because inference was expensive. As the cost of inference came down, the models became more useful and people became more willing to pay for them.

The models are also beginning to take actions rather than only answering questions. That increases the number of tokens used for each task.

Madra says there are two things that people outside the industry may not see yet. First, today’s public models were not trained on the newest hardware. Second, companies are still figuring out how to get more work from the models they already have.

A system that can take a problem and work on it continuously will use far more tokens than a chatbot waiting for someone to ask a question. Both alluded to the fact that there isn’t enough compute, and probably won’t be for some time. Cost goes down….great, but I want 1,000X the number of tokens.

Is AI a bubble?

Gerstner points to Anthropic’s rapid revenue growth as evidence that willingness to pay is increasing along with model capability.

Madra argues that the current models and chips do not show where demand will settle. Newer models will be trained on newer hardware (Blackwell at the time of this lecture), and companies will get better at putting those models to work.

This does not answer whether every company or infrastructure investment will earn a good return. It does explain why falling inference costs do not necessarily mean lower total spending.

Q&A

How should people prepare for this?

Gerstner says the Industrial Revolution disrupted many existing jobs, but people found new ways to create value.

He gives a more immediate example from Altimeter: he would not hire someone who does not use Excel. His argument is that some forms of IQ may become easier to obtain, making EQ and problem-solving more valuable.

Madra uses mathematics as another example. If AI systems begin making new discoveries, the people and organizations that know how to use those discoveries could have an advantage.

They both gave pretty crappy answers. But I think the answer is….we don’t know. And that’s ok to give as an answer, but they don’t do that.

What is going on with Apple?

Gerstner says Apple’s AI strategy is risky because language models are not yet good enough to do everything locally on a device. Privacy is also a major consideration for the company.

The case for Apple is its existing base of devices. If smaller models improve enough, products such as Siri could become much more useful without sending every request to a data center.

Power remains a problem. Madra says an 8-billion-parameter model running on a phone can drain the battery in about 30 minutes.

I personally feel that Apple is doing a great job holding off. Apple has always succeeded by thoughtfully doing something, not slapping new crap everywhere…ahem Samsung.

How should AI CEOs talk about AGI?

Gerstner criticizes fear-based arguments intended to produce regulatory capture (cough…Dario….cough). At the same time, he says people should not put their heads in the sand.

He takes Dario Amodei and Sam Altman seriously when they describe how quickly the models are improving. The rate of change, in his view, is becoming fairly parabolic.

What is Nvidia’s long-term business model?

A student presents two possibilities. Nvidia could protect its margins and accept slower revenue growth, or lower its margins and maintain a larger share of the market.

Gerstner says Nvidia has a strong product schedule and much of its capacity is already booked for the next eight quarters. He also points out that people once said Nvidia could never become a $1 trillion company.

Competitors such as Groq, Cerebras, and Google’s TPU can succeed without Nvidia failing. Gerstner describes this as the good part of capitalism.

This student can’t see the forest for the trees. He sees the GPU market as a zero sum game. Nvidia has never worked like that. Silly premise.

Course notes

That is it for this week. I will keep posting my course notes and comments on this page.

My 10 year old coded his first Minecraft mod....sort of

Wed, 05 Aug 2026 19:01:16 +0800

My son used Claude Code for a while to help him fix mods and do some other small things, mostly gaming related, on his home computer. Since I switched to Codex a few weeks ago I convinced him to switch as well. It was easy since he uses perhaps once a day to a few times a week. He is 10 so he can type reasonably, but he prefers Wispr Flow to dictate (I set him up with my unused account since I use a local app on my Mac now).

Last night he was looking for a mod for Minecraft, which is focused on quickly switching between keybind profiles by hitting a hotkey. He found one, but it was buggy and didn’t work well. So I said….just make it yourself. He was super skeptical at first, but I said Codex with Terra could easily do it.

I sat him down with and helped him with the first prompt and showed him how to phrase it so it would actually make it and not just brainstorm endlessly. He set it to work and it took two turns to get it to launch and work properly, and another 30 minutes or so of work to polish it into something he really really likes. He is over the moon at what he did and he thinks it is amazing. He specifically directed the model to ensure it complies with server rules and doesn’t give him an unfair advantage (this impressed me).

As a 10 year old he is somewhat exasperated at the huge amount of AI generated content out there and he generally considers it a menace. But this got to him and showed him the real power. Now I’m convincing him to polish it a bit more then release it on Curse Forge, a popular modding platform, so others can benefit from it.

If he would have had to learn enough Java to make this work on his own, it would have taken him two weeks of work, and he would have given up after a few hours of learning. Part of me feels like learning how to code would have been better, because it would have taught him how to learn a new skill. Another part of me thinks that is stupid, and that I don’t even code anymore. What matters is the process, asking the right questions, and ensuring you make something that is quality.

So my son “coded” his first Minecraft mod.

US visa bond program is now permanent

Tue, 04 Aug 2026 13:54:21 +0800

The visa bond program for the United States started impacting Mongolia in April 2026. This program requires B1/2 visa applicants to post a bond of $10,000-$20,000 dollars and limits visa terms from three months to a year. This was a pilot program, and it was just made permanent.

It’s annoying for us, to say the least. At least for now, when my wife’s 10 year visa expires, she will need to go through this process. Since she was 18 or 19 she has gone to the US many, many times. In fact she lived there for 8 years, which is when I met her. We were even married there.

We go to the US to visit family, and to do business, every year. One of the largest jewelry trade shows in the US is in Las Vegas each year and she attends. In fact most of the purchases for her jewelry business in Mongolia come from US suppliers.

I understand the State Department’s reasoning for this decision. I’d argue that this program, as it is currently structured, will probably cost the State Department more money. For all those thousands of Mongolians who travel to the US regularly, and who don’t break the rules, they will now have to go to the US Embassy in Ulaanbaatar once a year (in the best case) and do a visa interview. It is going to overload the already stressed system they have.

It would make much more sense to be able to waive this program for those with a long and obviously strong visa history.

Back in 2015 when my wife and I were married, she had to get a visa to go to the US. We made no secret of the plan to get married in Texas, and we planned to go on our honeymoon and then come right back to Mongolia. The consular officer basically said that she wasn’t allowed to get married in the US, which isn’t correct for obvious reasons. But the reason they said this is because once you are married to a US citizen and in the US you can file for an adjustment of status and then stay. That wasn’t our intention of course, but for a consular officer it’s certainly a bit of a weird thing to hear from a visa applicant in a country where most people would love to immigrate to the US.

Anyways, I tell this story to point out that my wife has had ample opportunity over the past 20 years to immigrate legally. She didn’t. So this program makes little sense other than to create more hassle for us and the consular officers at the embassy.

Funny enough, all immigrant visa issuance from Mongolia are paused anyways. So even if we wanted to move our family to the US we couldn’t…

Wild Strawberry Jam

Sun, 02 Aug 2026 09:20:22 +0800

One of the very nice things in Mongolian summer is the wild strawberries! Last night we made jam with about 3 liters of strawberries.

Lecture 1: Economics of the AI Supercycle

Wed, 29 Jul 2026 14:25:24 +0800

In this first lecture instructor, held on April 2, 2026, Apoorv Agrawal (currently Altimeter Capital, prev Palantir) goes over the format of the course and the purpose of the course with some primer concepts for the class. Anything that is my personal opinion will be denoted with italics.

Course Page Materials for this week:

Format of the course

The course is primary guest speakers covering different areas of the current AI supercycle. Speakers are in areas such as GPUs, SaaS, infrastructure, etc.

Why take this course?

We are at the start of a massive supercycle. Agrawal makes the contention that this cycle will be larger than cloud, mobile, and the internet.

Where is the $ in AI today?

Agrawal asks some questions from the class then continues the rest of the time with a discussion.

Q: Is this CAPEX generating revenue?

A lot of money is going to CAPEX right now (capital expenditure), the so called 5 layer cake Jensen Huang has referred to (energy, chips, power, interconnect, memory). This CAPEX is amortized (at least for GPUs) over about 5-6 years. Is it worth it?

Google: $100/user/year
Meta: $70
OpenAI: $10

How do you get to >$10 per user per year? Agrawal makes the point that knowledge work isn’t the answer, advertising is.

How this revolution is different

Software gets 80-90% gross margins. AI services at billions in revenue still isn’t profitable. Average margins for AI companies are much lower, around 30%.

Amazon took 8 years to build out AWS, and during that time people were asking if Amazon would go bankrupt. This is probably because Amazon had famously near 0 net profit for many years post IPO, instead focusing on reinvesting to increase revenue.

In cloud most value accrues in apps, but in AI its at the semiconductor layer. Agrawal states that gross margins for semis is 75% (0-30% for AI apps layer). This is the inverse of the cloud period. Also, semis are built for 5-6 year depreciation cycles, but apps generate revenue now. Mobile supercycle had the same inflated CAPEX early in the cycle.

Don’t underestimate Google

A student made a point that Google is one of the only complete AI stacks today, they have TPUs, apps, existing customer base, etc.

Agrawal makes the point to pay attention to earnings calls. CAPEX has an equilibrium (limited to free cashflow and cash reserves and also the expected revenue gains from additional CAPEX), so earnings calls will signal a change in this equilibrium.

This lecture was in April 2026 and this is somewhat prescient but perhaps underestimated the frothiness of the AI CAPEX splurge. Google just had its very first period of negative free cashflow since 2004.

Agrawal says:

Google won the internet supercycle
Apple won mobile
Meta won social
Oligopoly in cloud

Because of Google’s power in chips, apps, and existing customer base it might be likely that Google wins here.

Training and inference are unpredictable

Inference is dependent on when humans are awake, so this causes the usage to be unpredictable. This was how AWS got started (by using excess capacity since Amazon didn’t have traffic at night), so I feel like this isn’t as big of a problem as he states it is.

Where are all the ASICs?

ASICs are application specific integrated circuits. Groq (acquired by Nvidia in Dec 2025) and Cerebras (Jan 2026 deal with OpenAI) are two that come to mind.

There is 300 billion in revenue to go around. There wasn’t much discussion here around this topic, perhaps more in a later lecture.

Where will value accrue in AI?

2024: 90B vs 2026: 435B in revenue. But the shape of the revenue hasn’t changed much.

Yet profits remain in the semis later.

AI Apps - Weekly active users

ChatGPT has most users, then Gemini. Consumer apps are dwarfing ChatGPT, even with all of the hype.

ChatGPT is closer to the niche app category (like Spotify and X) than it is to Netflix or YouTube.

Is knowledge work where most people are?

In ChatGPT you have to go and ask a question. This is very active and most people don’t prefer to engage in apps like this. To increase revenue Agrawal makes the point that these companies should use ads rather than subscriptions, and he thinks this is where it is heading.

Intent is much cleared in AI chat apps, so sure revenue per ad will be higher, but the inventory will be much lower as well.

Course notes

That’s it for this week. I’ll keep posting my course notes and comments on this page.

Timeback cache shenanigans

Wed, 29 Jul 2026 12:22:47 +0800

My son is doing GT Anywhere, which is an offshoot of Alpha School. The Timeback platform kept crashing on him for two of the lessons. I suspected a browser cache issue because of how it loads (via an embedded Electron browser in the app).

Deleted the cache in the container and it worked. Had some help from Codex to tell me where to look.

I truly think Timeback, Alpha, and GT School are the way forward in education (it’s working great for my son). I expected there would be some rough edges with something so new.

Even with all of these rough edges my son loves it. After class his current favorite thing to do is play table tennis.

Broken machine day

Mon, 27 Jul 2026 13:36:13 +0800

Last Friday was broken machine day. Two of our six-sided hole machines, used for inserting holes and slots into furniture pieces, broke down.

For the first machine one of the spindles wouldn’t raise or lower. These are controlled electronically from the computer via an pneumatic system. We suspected either a computer issue, electrical, or a solenoid valve which controls the pneumatic air flow.

The offending spindle.

After a lot of troubleshooting I was convinced it was the solenoid valve. Our factory manager climbed up and checked on it.

Solenoid valves have these little buttons that allow you to activate them manually. This way you can see if the valve is working without an electrical trigger.

They have these little red LEDs telling you when it is being triggered. This is a “two headed” (no idea if that is what they are actually called) which allows airflow two ways. Often these are just open closed situations but this one allows airflow to move the spindle up, and another to move it down.

The electrical part was working, but the solenoid valve itself was borked. We heard it clicking but no movement.

We bought a new one at a local supply store but it was the wrong electrical type! These come in AC or DC varieties. The one we bought was for AC 220V but we needed it to work with DC 20V. Well it turns out we can just take the electrical ends from the broken valve and put it on the new valve. I’m going to research later to see why this is.

Anyways after a few hours we got that fixed. On to the next machine!

The next machine gave an error code, which is usually very helpful. The sensor on the lower spindle wasn’t responding, which usually mean the sensor is broken. These are little sensor that determine if a pneumatic cylinder is in the up or down position (or extended or collapsed depending on the orientation).

We replaced the sensor with a spare we have, but it still didn’t work! So time to trace wires.

We ended up finding out where one sensor wire was split into two, and one of the connections came off. This was nicely spliced too, but I think wear over repeated use made it come loose. We spliced it again and taped it up. I need to remember to bring my wire shrink insulation to the factory for the future.

Oh well it’s all good now! We’ve gotten quite good at troubleshooting these machines over the past year or so. We keep backup parts because a new part that isn’t available locally (like sensors) would take about 10 days to get here from China.

I’m quite surprised at how the old troubleshooting skills I learned from building PCs back in the day, or by working at an IT outsourcing company nearly 20 years ago helps with this stuff. It’s pretty much all the same.

AI requires a builder mindset to get value

Mon, 27 Jul 2026 08:41:38 +0800

Only 3% of American households are paying for an AI subscription. If LLMs make it so much faster and easier to get work done, why isn’t everyone using them?

This X post from Sam Altman sums it up nicely:

chatgpt work is remarkable, and “work” undersells it.

from my phone i sent:

“use all my chat history to figure out ideas for a long weekend trip with 8 friends, plan the best three options, make a full-stack site where the 9 of us can coordinate on what we would want to do in each place and decide where to go, and then after we get to group agreement make reservations. draft an email in my gmail i can send out to my friends when the site is ready.”

it…just worked.

I’m sure it did work. I’ve made so many personal utilities that are either single-use (i.e., throwaway) or that I use every week. I’m struck by the prompt.

Sam had to think through his plans, the range of options he wanted, the output he wanted, whether he needed coordination, and how he wanted to get the results. In other words, he thought like a builder. This is a mindset that, in my experience, is pretty uncommon.

When judging whether a product or service is techie, builder-oriented, or non-mainstream, I usually check with my wife. She is super smart, high-agency, and can generally look at something and immediately tell whether it will help her in life or work. I’ve been trying to teach her how to use ChatGPT Work to get things done for the past week or so. She has used ChatGPT for quick answers, brainstorming, and similar queries for some time now. But she hasn’t really used it to produce work yet.

I convinced her to try making a series of presentations for a course she is teaching. The process has been relatively bumpy, I must say. First, her prompt was too vague, and the LLM dutifully made everything in one go, with predictably middling results. Then I recommended that she start with one slide deck and, once she was happy with the result, ask ChatGPT to use it as a guide for the others. That worked better, but it took 30 minutes to make a single slide deck because of all the checks. It worked, but she was pretty underwhelmed. In her mind, she could have done a better job herself in maybe 45 minutes, so why wait 30? After making it herself, she would also understand the thought process behind the slides and be better able to present them. Fair point.

I think a better, chunkier, and more specific prompt at the beginning would have gotten her 90% of the way there, with perhaps 10 minutes spent writing out what she specifically wanted. But then I realized that she didn’t exactly know what she wanted until she started doing it. Well, I should say that she knew what she wanted, but she didn’t know how to articulate it as a series of instructions an LLM would understand.

Without a lot of coaching, I just can’t see more than a small percentage of people thinking this way. It’s weird. When they want to pay their bills, most people don’t think about the mechanics of finding the bills, paying them through a payment system, and recording that each bill was paid. Builders think about those things.

Until LLMs can do that builder-style thinking for ordinary users, I don’t see my wife, or people like her, choosing to turn to one most of the time.

LLMs are bad at blog posts

Thu, 23 Jul 2026 10:12:37 +0800

Excellent post today from wakamoleguy about how LLms are bad at writing blog posts. Quick and simple argument about information theory that I believe is correct, and more importantly, this great snippet:

So let’s go back to the beginning: you have an idea, but writing is hard! Write anyways; write poorly. Use an LLM for research and feedback. Do not simply hand it your idea and expect it to generate something enjoyable or effective to read. And while you’re at it, lean into the imperfections, because even if they aren’t information-theoretical depictions of your ideas, they help make your writing interesting, unexpected, and yes, surprising. And that is what makes it fun to read.

I’m working on a longer form data driven blog post over on my nascent Substack publication, and I’m finding it hard to write like I used to. I’m writing then asking the LLM to give feedback. It often wants to make my data points overly specific, which I find annoying. Nevertheless it finds typos and areas I didn’t make all that much sense, so it’s helpful! This way I don’t have to bug a colleague or my wife to read it first!

Get to writing!

America's ice cream consumption

Fri, 17 Jul 2026 18:35:39 +0800

Neil Howe and Christian Ford write about ice cream consumption:

Over the last 50 years, US ice cream consumption has steadily declined. In 1975, the average American consumed 18.2 pounds of ice cream. In 2025, that figure fell to only 12.0 pounds. That’s a -34.1% decline.

This is quite sad to me. One of my fondest memories with my grandparents is eating dinner and having a scoop of vanilla ice cream afterwards (usually with chocolate syrup on top). My grandfather would have a scoop as well. Some doctors even believe it helps prevent Type II diabetes.

Wed, 15 Jul 2026 20:43:33 +0800

It’s my father in laws birthday. He enjoys a card game called 13 (which he is very good at).

You add up the number of cards you have left at the end of the round. Once you reach 25 you are out. I held my own and was second to leave the game out of four. It’s not really my game but it’s good fun.

Long running task in Codex

Wed, 15 Jul 2026 12:49:46 +0800

20 hour in and I’m getting apprehensive as to the utility of this run. Lots of code changes and testing. The process is a well defined one, at least conceptually. It is still on step 1 of 5. The first step being scraping government websites in a country (not the US) that ostensibly has a lot of open data, but practically tries to restrict scraping by any means necessary.

Codex is dutifully writing, running, and updating numerous scrapers to deal with the random 403s that shouldn’t 403, data inconsistencies on government websites, and connecting records via trial and error.

But I heard a long time ago never to attribute to malice what is more likely simply incompetence.

Tue, 14 Jul 2026 19:56:05 +0800

Fascinating point from antirez:

I believe many programmers at this point have less impact they could have because they look at the code

I couldn’t agree more. The thing should do what it is supposed to do. The person directing the LLM agent is responsible for checking that manually or autonomously. But looking at the code is a suboptimal use of time.

Tue, 14 Jul 2026 18:57:21 +0800

This is our fourth time coming to this part of Bulgan Aimag in Mongolia. This little Soum center called Khutag-Undur has a nice hill next to it. Mongolian summer is a brief respite from a harsh environment.

Tue, 14 Jul 2026 18:04:55 +0800

From Numeric Citizen Blog:

I don’t know if I should trust the news, but it seems that opposition to data centers and artificial intelligence is growing.

I get the same feeling. The sense I feel is that people genuinely think AI companies are trying to take peoples jobs, and that data centers are causing increases in electricity prices as a result.

Mon, 13 Jul 2026 19:20:55 +0800

Not a bad way to end the day.

Sat, 11 Jul 2026 11:35:14 +0800

Traveling in Mongolia for Naadam and testing out GPT5.6 from the room.

Fixing stuff day

Mon, 06 Jul 2026 16:09:54 +0800

Today is fixing stuff day.

For one of the hats I wear I’m a university administrator. Over the past few years our admissions system has become something of a hydra. We have four different deployed applications for our admissions system and one for our student information system. I’m going to put them all together into a big mono repo today so it’s easier to manage and maintain them.

For my other hat I’m ostensibly the CEO of a furniture company, but because no one else on the staff knows how to terminate ethernet, that’s my job today. Needed to move a security camera a few meters one way in the factory. I made a standard straight through cable and it didn’t work! Powered up but no data to the NVR. So I looked at the source cable and I see it’s split! The installer decided to use 1 ethernet cable for two cameras (IP cameras use only pins 1, 2, 3, and 6 typically) so he wouldn’t need to run another cable (yay for laziness). Rather than re-terminate both ends of the extension I just cut one end and only put in the wires for 1, 2, 3, and 6 pins. Then it worked!

It doesn’t matter your title, if you are good at technical stuff it will always end up being your job.

Fable and GPT 5.5

Sat, 04 Jul 2026 10:08:20 +0800

Fable seems to be a great model in my testing. Currently I’m building a Mac desktop app, and I’m working hard to be faithful to the look that Apple has for their apps. Fable seems to be pretty good at it. But so does GPT 5.5 as long as I’m clear. Maybe my problems aren’t hard enough.

Fable has a tendency to use jargon a lot more in a dense, PhD student sort of way. It’s able to elegantly explain things though if asked. It shares the Opus tendency to jump in and get started even when something isn’t clear. I suppose that is good if you want to move fast but not great if you are a deliberate person.

Do I feel like Fable is a step change? Perhaps yes. But it still isn’t good enough for me to let it run on its own. Is the extra cost worth the extra intelligence, given that I still have to baby sit it for tasks? No, probably not.

GT Anywhere

Thu, 02 Jul 2026 11:46:54 +0800

My son started GT Anywhere yesterday, and offshoot of Alpha. After learning that by meeting his daily XP goals he gets points in their online store for students, he is extremely focused on completing lessons on the Timeback platform today.

He already has a savings account, a debit card, and cash in his Steam account. But this opportunity to get more points he could use to buy things (and compete on the leaderboard) is addictive for him. I’m hoping his motivation stays strong.

He also thinks the lessons are more fun than his old international school based on the IB system (he was in the PYP program).

Keep in mind it’s the summer and he would be happy spending his time flying his FPV drone, building custom game configs, or on Minecraft mods.