Information retrieval / vector search

Find pictures that look alike

Click a picture and a Sparsr device compares it against 4,096 others - one instruction per comparison, whole picture at a time.

Sparsr VM starting a software model of the processor, running in this tab

This is a software model. The same kernels run unchanged through the Sparsr SDK on your own machine.

Pick any picture in the grid. A Sparsr processor running in your browser compares it against all 4,096 in the library and shows you the twelve that look most like it.

The comparison, and what it costs

Each picture is reduced to a short string of bits. Comparing two pictures means counting how many of those bits differ - and on Sparsr that is one instruction, because a register is 4,096 bits wide and a whole picture fits inside one.

A machine with 32-bit registers needs 128 operations to compare the same pair.

So a search over the library is about two instructions per picture: one to load the stored code, one to compare it against the query. That is 8,192 instructions for 4,096 pictures.

This is deliberately not the trick the molecule demo uses. There, the library is stored sideways and the whole thing is screened in about twenty instructions. Here the codes are stored one per register, the ordinary way, so the win is the register width and nothing more. Both are honest, and they are different sizes of win - the sideways layout is worth roughly two hundred times more, which is the most useful thing these two pages say when read together.

The padding is real, and the page says so

A code is 756 bits and a register is 4,096, so the other 3,340 bits are zero.

The machine compares those zero bits anyway, and the instruction count above does not get cheaper because most of the register is empty. So the demo reports bits-per-instruction beside the picture rather than only the instruction count.

What "looks alike" means here

The code is a difference hash: one bit per neighbouring pair of pixels, saying which of the two is brighter. It is computed straight from the pixels, so it never learns anything and never identifies an object.

So it finds pictures that look alike, and it will cheerfully put a coat next to a pullover - because they do look alike. That is the measure behaving correctly, not a bug.

The picture library

4,096 items from Fashion-MNIST, published by Zalando Research under the MIT licence. They are 28×28 and greyscale, which is why they look the way they do: the entire library has to be small enough to send to a browser.

Why there is no timing figure

Nothing about speed. The processor here is a software model running in your browser, so how long a search takes is a property of your laptop rather than of Sparsr. The instruction counts are exact. Timing is left out on purpose.