Information Discovery

Information discovery goes beyond search, answering questions you couldn't otherwise ask.  Discovery can lead to more effective search by exploring and then defining new relationships with your particular target data.


What web pages link to my company?Of sites that link to my company, what other (kinds of) pages do they link to?
Where is my product mentioned?Of places where my product is mentioned, what are the most frequent other products mentioned?
What companies have the same primary NAICS code as mine?What demographic slices have companies with the same NAICS code as mine? What kinds don't?
How many bookstores are there within 100 miles?What types of businesses tend to be near bookstores, and what localities are exceptions?


The computational requirements of information discovery tends to rise exponentially with the number of possible relationships, and this is where general search engines are at a disadvantage in their attempt to do all things for all people.  But for focused research and targeted information, such questions are quite tractible with the right tools and methods.

Prospecting for Web Development Clients

A web development company sought additional clients. Their immediate difficulty was not too few prospects, but too many -- it seems everybody has a web site these days.  But how best to select prospective clients in the best position to benefit from their services?

With Aware Research they defined criteria including front page length, quantity of links, key phrases, possible spelling errors, readability indices, presence of flash or video content, JavaScript, meta-tags, and whether the site passes the W3C tests.

A visual screenshot of the full-length front page of each prospect's web site was included in the sortable rankings report, adding quick convenience and providing a reference for later comparison.

Now that they could effectively review their prospect base, they realized that their list of local web site URLs (based mainly on business survey data) was incomplete.  With Aware Research they created a project for automated discovery of web pages of locally owned or managed businesses.  We won't say here how we did it, but we discovered about twice as many as were available from the highly respected business directory.

The client saved hundreds of hours and thousands of dollars focusing first on their most highly ranked prospects, of which they now have three times as many.

Playing to Stay Ahead

A small business found a lucrative niche making custom carrying cases for the latest toy fad sweeping the world. The toy, with various accessories, was being sold at more and more novelty and game stores, and it was impossible to keep up a list, let alone assess which outlets were growing versus declining in sales and popularity.

With Aware Research, they specified a weekly analysis of mentions of the toy in the context of outlets selling similar items.  Additional outlets were easily highlighted for follow-up. Interested also in tracking mentions and popular sentiment of their product, they subscribed to game-player's discussion forums from which relevant text was extracted and summarized.

Working smarter, they've managed to stay ahead in a fast-paced market

Market Research and Exploration

A Marketing Manager is asked by her company president to prepare and present a report next Monday (!) exploring the competitive landscape for a potential new product line. 

A quick Google search shows hundreds of related pages.  What companies, individuals, products and technologies are mentioned in those pages?  How are they linked, and which predominate? 

She calls Aware Research and has the analysis, including company profiles, in her inbox when she gets to work the next morning

Event Promotion on the Web

A group organizing a new conference on emerging technologies needed to boost awareness and excitement.

With Aware Research, they identified web-sites, blogs and discussion groups relevant to their theme. Analysis of link clusters between sites identified key influencers, as well as additional relevant links. Focusing promotional activities on these key influencers worked well, and daily monitoring showed the message spreading. 

The conference was a big success, presenting a new challenge of finding an even bigger venue for the next one.

Mining the Deep Web

"Searching on the Internet today can be compared to dragging a net across the surface of the ocean", said Michael Bergman in a study back in 2000.

Nearly a decade later, some estimates show that up to 500 times more online information remains hidden below the shallow nets cast by the general search engines. There's a wealth of information that is deep and therefore missed, unless you know how to go after it.

What are these information resources?

Searching Smarter

There's a lot of chatter these days about making online search easier, friendlier, serving more of the public in more ways.

For example:

  • natural language search, where we'll just type (or speak) in plain language
  • using prior search, adjusting future results to more closely reflect our past selections
  • social search, where the preferences of our network of friends will influence our results
  • emphasis on optimizing the very first search result since that's the one we'll want to click
  • and so on...

That's all fine, but do we see an underlying assumption emerging? 

Perhaps a problem (and an opportunity) in disguise?

Syndicate content