For a while now I have wanted a “visual search engine”. A search engine that will answer when I ask ‘What was that site? The one with the gross burgundy bar down the right-hand side?’. And yesterday I came upon a workable answer to this problem!
It’s a bit of a whirlwind, so hold on:
It starts with an attention recorder, a piece of software that keeps track of the web sites I view. Normally, the recorder would store the site addresses in an attention vault of some sort. But what if I stored images from snap.com along with the addresses? You know, site preview images, those little thought bubbles that pop up when you hold the mouse over a link (this weblog uses them).
That small site preview image from Snap.com is just large enough to recognize large blocks of colour and contrast. We can use simple techniques from desktop optical character recognition software to look for patterns in the Snap images. The user would draw a simple picture using a trivial drawing program (simpler than Microsoft Paint); a coloured box, maybe a line. The drawing is broken into patterns, and the search software looks for those patterns in the attention vault images. The user gets back a list of sites they have visited that look similar to the picture they drew.
Pretty cool. This technique lets the user perform vague searches like “the page was green down left”, or “it had a circle in the middle”, or “there was a distinct blue and orange theme”. Mix in some of the artistic and design skill necessary for good data representation, and you have a real application.
So there you go. Snap.com, my attention vault, and OCR technology shine a new light on my past distractions.