Wednesday, September 7, 2011

The State of Desktop Search

Everyone might be talking about the cloud buzz and all that but there are still a significant lot carrying around laptops and desktops with monstrous hard disks. The Register paper has an interesting article on local search  in the light of Google's dropping support and EOL'ing of Google Desktop software.  The author bemoans the lack of good alternatives or rather the options one has,(if one was not happy with Google Desktop) that at present don't seem to be good enough for the task at hand.

It's not pretty.

We get 2TB HDDs for about $100 and given the hoarder mentality that makes people fill it faster and faster compared to the earlier HDDs, we have a big problem.  An insane amount of file clutter and no good methods to find the document with a specific text, photo or information.

On the XP machine that I use, the local search interface is....well...sad and could do with a dedicated tool to help users search.  A bit strange that Microsoft does not seem to have anything to address this. hmmm...though I vaguely remember seeing some papers on semantic filesystems and other stuff by Microsoft back in the days when NTFS was being hyped about.

The unix find, locate, slocate, rlocate  custom built scripts seem to be what passes for cutting edge for rooting through your HDDs; makes you look victorious in the smugly, sad, nerdy way. Which, truth be told, is what I end up doing when I'm not working with Microsoft files.

I looked up a few pieces of software mentioned in the forums but it all seemed a bit much while a few posters in that forum carped, "organise the stuff beforehand".

Very helpful, that.

What do you use to root through your hard disk?

Seagate FreeAgent GoFlex Desk 2 TB USB 2.0 External Hard Drive STAC2000100 (Black)Western Digital My Passport Essential SE 1 TB USB 3.0/2.0 Ultra Portable External Hard Drive (Black)


camilohe said...

You could try everything or locate32 both are free and reasonably fast I think.

elarson said...

As I use a mac, there is spotlight, but I rarely use it. I used to try to use Beagle in linux, but really, the best solution is to keep as organized as possible. I also found services that work on specific areas of your data (email) can be helpful. For example, is a pretty nice service that has saved me a little time. The em email index tool is also blazing fast and really powerful.

GCU Prosthetic Conscience said...

On GNU systems, the current desktop search de-facto standard is Tracker, perhaps combined with Zeitgeist. As far as I know, Tracker works quite well, but for me, locate is normally good enough.

Scott Henson said...

I think the better question is, is there any information of value in that 2TB drive? I think the simple answer is, no. Most people end up with vast amounts of data and no way of getting any useful information out of it. Also, a lot of that data is going to be media that is consumed from other sources. The media that is produced by individuals is almost always uploaded to some service on the web that stores and keeps it.

So the reality of the situation is that it doesn't matter. The average consumer doesn't need to store all that much data and doesn't need help finding what they want.

Anonymous said...

QuickSilver, oh how I miss you.

Anything.el + tracker, I guess.

tycho said...

I think there's a couple of tools: Recoll and Pinot that are promising. I'm a big fan of the Xapian search.

Second, if you can't organize it well going into things, begin organizing your things well.

Third, find some sort of recursive file-renaming tool and do what you can to reduce hierarchy, which might help you migrate old system to new system.

I think it's true that: there doesn't tend to be a lot of value in these huge file repositories, bigger drives more often represent collections of bigger files rather than bigger collections of more files.

Unknown said...

On my XP box, i've been using 'everything' search app and i think it is awesome. visit for more information. (In brief, it supports command line tool named 'es.exe' and gui front end too. you can search file path with regexp pattern)

BTW, in emacs, rubikitch wrote a his anything mode plugin in his 'anything-config.el' file.

Greg said...

Great post. Just a few comments.

1) The cloud is good if it has stuff you want (as opposed to stuff you stored there) but many of us use lots of documents that are behind paid firewalls that we only occassionally get behind. My collection of files is hard earned and only available my hard drive or wherever I put it. (This is not true of everything. My images, used for teaching and so on, were formerly a colletion I had put together in meat space over the years but which has been utterly obviated by Google Image Search).

2) The volume of material one needs to store for many of us is far beyond what we can afford to store in any cloud or other weather phenomenon! The hard drive is the only option.

3) The best way to find stuff (and I say this as a rather disorganized person) is to put it where you can find it. STuff on hard drives is organized into these things called "directories" which can be stored into each other as well in a hierarchical organized fashion. I know this seems obvious now but when everyone is using only iPads which appear to have tossed the file system we all grew up with, any hope of organizing our stuff will be doomed. Doomed I say!