|
Comprehensive site for mining industry covering exploration, projects,
education, technology, joint ventures, countries, research, news and
environment, world wide |
|
 |
|
|
|
|
|
|
| |
| |
|
MINING THE WEB
N.Subramanian |
Everyone knows that the Web is a balloon of information. Everybody
also knows that this balloon is big, very, very big. No matter what you
need to know, chances are that it’s already on the Web waiting for you
to tap it. Logically therefore, whenever you need any information on say
“the mining industry”, all you need to do is pick it from the Web. But
it isn’t that easy always, is it? While the Web is accessible, getting
it to supply the information you require is a tough ask if you don’t
know how. This article looks at various tools available today that help
query the Web effectively and zero down on the information we want. Of
course, the emphasis is on mining industry-related information.
Start the search:
Information on the Web is accessed primarily through four avenues:
Search engines
Meta search engines
Directories
Invisible Web
Discussion groups
As with everything in life, understanding the specifications of your
requirement is most important for deciding the right search tool. Lets
look at each of the above in some detail highlighting when each is most
useful.
SEARCH ENGINES
The most used search tool on the Web, the typical search engine is a
three-part tool consisting of a spider, an index and the search engine
software. The spider (or crawler) searches the web and builds a net of
web pages. These are then filed into a database containing copies of web
pages gathered by the spider called the index. Finally the search engine
software enables users to query the index and returns the results in the
order it considers most relevant.
So why is all this important? The simple reason is if you understand
what portion of the Web is crawled by a spider and how the search engine
software interprets your queries, you can very well estimate your
chances of obtaining the right answers to your query.
In terms of coverage, Google is the clear leader with a crawled base of
about 1,060,000,000 web pages. It is also unique in that it computes
page-ranks for each page, which is calculated depending on the number of
links from authoritative pages and documents with higher page ranks
corresponding to your queries are returned as primary results. Google is
a prime candidate to begin your search with.
Search engines are generally either keyword based in which case they try
to find if your queried words appear in the documents in its indexed
database or concept based (meaning they try to infer what you are
actually looking for from the combination and context of your keywords).
You may also try using Alta vista, Excite or Ask Jeeves as these not
only search for the queried documents in their indexed database but also
words with similar meaning.
Specialized mining search engines are probably the best bet though.
These are specific to the mining industry and have sufficiently large
databases to answer most of your queries. Two of the most popular in use
today are minelinks and geoindex. Searchenginewatch.com provides a very
handy chart to help you decide which search engine to use.
META SEARCH ENGINES
Meta search engines are the grand daddies of search engines, being
entities that search the search engines. Some of them simply return the
results from the individual search engines, leaving you shouting for
mercy by the sheer volume of documents returned. Others however adopt a
method of clustering where they remove redundant documents leaving you
with a more relevant, smaller set of documents.
WebCrawler, Metaspider and Dogpile are all examples of meta search
engines. For focused search on mining topics they may turn out to be too
cumbersome for use.
DIRECTORIES
Directories are links categorized by topic. Experts in a particular
subject with a basic knowledge of taxonomy are preferred for compilation
of directories. Since experts usually compile directories, the Web pages
available are of higher relevance than in an indexed search engine
database. The coverage however is usually only a fraction of the search
engine crawled area. One of the best examples of a directory is your own
About. COM! Categorized by people who have worked in the industry for
quite a while, you are very unlikely to leave an About site without
finding what you came for. You may also want to try the Dmoz and
minelinks directories for mining-related information.
This list of major Web directories will come in handy.
INVISIBLE WEB
The World Wide Web (WWW) as we know it today forms only a fraction of
the Internet. There is a lot of information ‘out there” which is beyond
most of the traditional search engine spiders’ reach. This portion of
the Internet, which is invisible to the spiders, has come to be known as
the Invisible Web. Getting to the Invisible Web requires us to dig a
little more into the Internet. This is a little difficult to do on your
own, but tools like and engines like Northernlight come to our rescue by
making available significant parts of the Invisible Web for search.
For a detailed explanation on the nuances of the Invisible Web, please
refer to About. COM.
DISCUSSION GROUPS
Discussion groups are just that – groups on the net which hold
discussions on a particular topic. The most popular group on the Web
today is Google Groups (formerly Deja news). The groups do not place any
restriction on entry or exit and allow anyone to post a question/
observation as well as refer previous postings. A look at the geology
and mining groups should provide a good idea as to the utility of this
resource. The entry of the mining browser, a desktop software
application, has brought a world of mining company information to your
fingertips. Searching for global company-specific material has never
been so easy, the only jarring note being that this tool comes with a
monthly subscription of about US$ 20 a month. But then, into each life,
some rain must fall!
Thus, there are many tools to help us get to the information we require
on the Web. Each tool has its own purpose and speaks its own language.
For making best use of the Web, it is necessary that we spend some time
honing our searching skills. After all even looking for that needle in
the proverbial haystack is not all that difficult if we have the right
metal detector to guide us!
|
| |
|
