Wikia Acquires Web crawler

Wikia's project to develop an open source search engine got another boost with its acquisition of the Grub distributed Web crawler, the company announced Friday. Wikia acquired Grub from LookSmart and released it under an open source license, adding a significant component to Search Wikia, scheduled to debut in this year's fourth quarter. The Search […]

Wikia's project to develop an open source search engine got another boost with its acquisition of the Grub distributed Web crawler, the company announced Friday.

Wikia acquired Grub from LookSmart and released it under an open source license, adding a significant component to Search Wikia, scheduled to debut in this year's fourth quarter.

The Search Wikia project seeks to create a search engine based on open source search protocols and human collaboration, drawing from the concept of the Wikipedia online encyclopedia, which is written and edited by a community of volunteer collaborators.

As such, it will provide a better search experience than the ones offered by commercial, proprietary search engines like those from Google, Yahoo, and Microsoft, said Jimmy Wales, Wikia's co-founder and chairman, and Wikipedia founder.

However, people shouldn't expect Search Wikia's first version to deliver on this lofty goal. "When I opened Wikipedia with only three articles in it, it was a pretty bad encyclopedia," Wales said. "That's where we're going to be in December with the search engine: We'll tell people upfront: 'It's not very good yet.' But we'll open it up to get feedback and community involvement to help us make it better."

Wikia, which has about 35 full-time employees, is developing its own relevancy technology to rank search results, he said. At this point, it's not clear whether at launch Search Wikia will offer only general Web searching or if it will have specialty engines for things such as images and news.

Visually, Search Wikia will have the standard search-engine interface. "The differences will be that at various points all along that search process, there'll be opportunities for people to engage with the community and join and participate in the construction of the search results," Wales said.

More info: Yahoo News - InfoWorld

Wikia, Grub, Grub Web Crawler, Web Crawler, bot, Search Engine