Know who the person is who wrote what might be the largest index on the web?

Her name is Anna Patterson, and she indexed 30 billion web pages for the Internet Archive. It’s not the only index she’s put up on the web.

She recently wrote an article for ACM Queue on search called Why Writing Your Own Search Engine is Hard. Interestingly, she starts off by saying that such a project might only be effectively possible by a small group of people - one to four of them, optimally working in an environment like a garage or basement.

And their key attributes will be, possibly more than anything, time and patience.

If you enjoyed that one, make sure that you read Building Nutch: Open Source Search. It’s about two guys, with lots of time and patience, who set out to build a search engine…