More compute required

I’m currently building a few projects, and I’ve underestimated the amount of local compute I need for them. This isn’t for LLM inference. Quite the contrary, I’m running code simply requires a good amount of CPU cycles and RAM.

One is for a chess project where I’m using the wonderful Maia-3 model (aka Chessformer) to compute positions and scores. The idea is to develop a test that can accurately assess a players rating quickly. It’s experimental in that I don’t know if it will work with real players, but I’m pretty sure it will. This project alone has been utilizing 12 of the 16 cores on my closet machine for a good portion of the past month.

Another is for image processing for my wifes jewelry company. Our supplier provides images and we want to make those available in a system our customers can see. Unfortunately they use white backgrounds, and we use black backgrounds for all of our product photos. So now the closet machine is running a PyTorch model to swap out backgrounds and will take a few days using a single thread for over 14,000 images. Perfect for Modal, but I’m in no rush and I’m running it on a machine that is already there.

Now I want to run another project to take the last year of orders from my wife’s company and vectorize them so I can identify pieces from our supplier we are most likely to order. This could save their team tens of hours every few months. This requires yet another compute run to vectorize the images, then a relatively quick run to compute the similarity.

Part of me wants to build a compute cluster now…

Robert Ritz @robertritz