What is Omni-Explorer?
Some folks are starting to notice a new web crawler sniffing around their web sites. Their web logs point people to a site for Omni-Explorer, "a venture-backed startup based in Silicon Valley.'' Do we have another search company in our midst? (via Scoble)
UPDATE: Dan Primack, in a comment below, points to Naval Ravikant as one of the execs. Ravikant, among other things, founded Epinions, which became Shopping.com, which was just bought by eBay. (There's a great story by Po Bronson from 1999 about the hyper-fast launch of Epinions here.) Also, it appears that Omni-Explorer Technologies has been around as a company for some time, with a variety of web sites.
I probably should just wait and publish this on my own site, but since you asked...
Omni Explorer Technologies Inc. is indeed an Internet search startup, based in Palo Alto. It's raised around $5m ($4.8m in equity) from firms like Clearstone Venture Partners and Leapfrog Ventures.
Execs include Naval Ravikant (genoa corp., epinions, entrepreneur-in-residence w/ august capital); Borislav Agapiev and Kevin Laws (currently w/ Pacrim Ventures).
Comment link
With all that venture capital you would think they could afford a web site designer. If you were trying to look like a bogus company you couldn't do any better than this: http://www.omni-explorer.com
Comment link
Verdamt thing downloaded a gig of data off my server yesterday.
Lingster on November 23, 2005 9:54 AMComment link
may be its a search engines which wud hit on web very soon,crawling seems to be very aggresive.thanks
vitamin on December 8, 2005 10:56 PMComment link
Yes, I have noticed this Omni-Explorer Bot visiting my site and gateway sites about 10 times in the past week.
Chris
Chris Keenan on December 20, 2005 2:19 PMComment link
I does appear to be an aggressive little crawler. :)
Chris
www.bizmancan.com
Comment link
Would be useful if they updated that page, or actually showed how the data is to be used. Its been hitting my site for many many months, and I still don't know much about it...
Comment link
Aggressive doesn't even begin to address the problem. It crashed my site three times in as many weeks before we were able to identify and block it. Why would it need to archive my entire site, multiple times per visit, and repeat these visits over a period of only a few days?
That's not aggressive; it's shoddy programming. The other bots barely make a tick in our traffic.
Jood on March 21, 2006 8:48 AMComment link
another search bot thats crawling slowly but very deep , when does it comes on web anybody there??
krish on May 8, 2006 12:53 AMComment link
not very speed as the google bot but surely aggresive than other regular bots say like genko and others dontwant to specify..thanks
james on May 8, 2006 7:09 PMComment link
today again the bot has come indeed very deep and in new way as world indexer? what does it mean? any guess
mathews on May 11, 2006 9:06 PMComment link
This bot comes every hour and every time takes at least a hundred pages, same page over and over again, I am afraid it can eat up all of my bandwidth
al buk on June 2, 2006 12:44 AMComment link
This crawler ignores robots.txt files. It consumes massive amounts of bandwidth.
More info is available at:
http://www.sitepoint.com/forums/showthread.php?p=2919998#post2919998
http://www.webmasterworld.com/forum11/2800.htm
Alan Cates on July 18, 2006 10:56 AMComment link
