Search Submissions (direct links to site submission
pages):
Search engines that really do crawl and index entire websites:
Google
Google requires your URL and some descriptive comments, and makes no claim
regarding how soon you will be listed. The Google crawler is called "Googlebot".
Unlike many of the search engines listed here, Googlebot has no trouble
indexing sites using frames (even if your "domain/index.html" page is a
framese page, you can get listed). Google results are displayed on Yahoo
search, and Netscape search, dramatically increasing the value of Google
listings to your site. For browsing, they use the DMOZ Open Directory,
which is a separate free submission.
Googlebot FAQs
FAST (All the Web)
The FAST (alltheweb.com) search engine needs your URL and email, and makes
no claim regarding how soon you will be listed. They crawled this site
within 2 weeks of submission, and listed it in less than a month. They
have also recrawled quickfound.net (over 500 pages with frequent changes)
monthly. On 7/23/2001, FAST announced that they would begin recrawling
their entire index (about 625 million pages) every 9 to 12 days, to
provide fresher listings. Unlike many of the search engines listed here,
FAST has no trouble indexing sites using frames. Their crawler is called
simply "FAST-WebCrawler". FAST results are also displayed on Lycos and
numerous other search engines worldwide, increasing the value of your FAST
listings.
FAST Crawler FAQs
WiseNut
The Korean company Samsung owns a relatively new search engine called
WiseNut. They may index your website with their crawler
"Zyborg", which seems to have no trouble with frames. They are now
accepting submissions. They ask only for your URL, and provide a box for
your comments.
Search engines that do NOT crawl, but will list your pages if you
submit each separately:
AltaVista
AltaVista requires that you read and type in a randomly generated list of
characters to prove that you are not an autosubmission "bot". Then they
require your URL only, but they will not crawl your website starting from
the URL you submit. Instead, you must submit EACH URL indiviually, and
every 5 URLs you have to type in the randomly generated characters again.
-- In late June, 2001 Alta Vista began taking paid submissions called "Express
Inclusion". If you pay $39.00 for the first URL you submit (and
somewhat less for additional URLs, up to 500), they will expedite your
listing, and reindex your submitted pages weekly. You have to pay again
every 6 months.
-- Before instituting their paid submission system, AltaVista had crawled
pages submitted by quickfound.net as quickly as 4 days after submission,
and listed them as quickly as 7 days (though they made no claim).
-- Introduction of the "express" service has resulted in a slowdown in the
listing of standard submissions. AltaVista now states that free
submissions will take 4-6 weeks to be listed (and they do NOT guarantee
listing). Actual experience: page submissions from Quickfound.net that
previously took as little as a week to be listed now require over a month
(but they ARE listed). Although AltaVista has clearly degraded their free
submission service, it must be pointed out that this leaves AltaVista
trailing only Google and FAST among major search engines in "submission
friendliness".
-- The AltaVista crawler, called "Scooter", does NOT do frames (that is,
if the page you submit is a frameset page, they will not automatically
index the pages IN the frameset).
-- AltaVista uses the LookSmart directory for browsing, which is a
separate submission and costs $199.
Lycos
Lycos displays results from FAST, but they also have their own crawler and
submission page. Currently it is difficult to detect any advantage in
submitting to Lycos if you are already listed by FAST, but their crawler
still indexes individually submitted pages, and it can't hurt to try.
Lycos requires your URL and email, and says you will be listed in 2 to 3
weeks. For browsing, they use the DMOZ Open Directory, which is a separate
free submission. The Lycos crawler is called "T-Rex", and it does NOT do
frames. Lycos appears to index only those pages that have been separately
submitted, but it DOES appear to list those.
Search engines that do NOT crawl, but will list your pages if you PAY
to submit them:
Inktomi Search/Submit
Inktomi Seach/Submit is a subscription service that will ensure your
customer's Web pages (URLs) are included in and stay in the Inktomi search
index. This index powers the premier search sites on the Web including
HotBot, MSN, AOL, iWon and Looksmart, and many more. Once a customer
subscribes to the service, their Web pages can be found on all of these
search engines within approximately 48 hours. Inktomi Search/Submit
automatically re-indexes the Web pages multiple times per week. The price
is $30.00 per year for the first URL, and $15.00/year each for additional
URLs.
You can submit to Inktomi free of charge via the HotBot or MSN links
below, but there is no guarantee you will be listed, or how long it will
take, or how often the index will be updated. QuickFound.net has
repeatedly submitted dozens of pages to the Inktomi index the free way,
but listings only appear after a delay of several months (if then), and
the site is never fully "crawled" by Slurp, the Inktomi spider.
Slurp -- Inktomi's Web Robot
HotBot
HotBot uses the Inktomi index for their search, which you can submit to
here. They require your URL and email, and say they will list you in about
60 days. The Inktomi crawler is called "Slurp", and like AltaVista, it
does NOT do frames (that is, if your "domain/index.html" page is a
frameset page, you will NOT get listed). For browsing, they use the DMOZ
Open Directory, which is a separate free submission.
HotBot Webmaster FAQs
MSN
Microsoft also uses the Inktomi index for their search, which you can
submit to here. They require only your URL and email address. The Inktomi
crawler is called "Slurp", and it does NOT do frames. For browsing, they
use the LookSmart directory, which is a paid separate submission.
Search engines that do NOT currently appear to list new submissions:
Northern Light
Northern Light needs your URL, name, and email, and says listings take 6
weeks on average. The Northern Light crawler is called Gulliver, and it
does NOT do frames. Gulliver may also crawl your site under the alias of
Marvin, Scooby, Taz, Sherman, Natasha, or Dudley. A list of the IP numbers
for Gulliver's aliases can be found on the submission page. Northern Light
does not APPEAR to be listing recent submissions, but it may be that they
are VERY slow.
Northern Light Webmaster FAQ's
WebTop
WebTop has yet to crawl the quickfound.net pages, although submitted
months ago. They claim to index entire sites with only one page submitted.
Direct Hit
Direct hit requires your URL, email, and a few keywords. They say it takes
several weeks or longer to get listed. The Direct Hit crawler is called
"Grabber", and, although we have submitted several times, we have never
seen it in our website's stats. They claim it does do frames.
Direct Hit Spider Information
whatUseek
whatUseek requires your URL and email, and makes no claim regarding how
soon you will be listed. Their crawler is called "Winona", and we have
never seen it in our stats, despite several submissions. They do send me
their advertising-packed newsletter frequently, however.
Search engines that are not currently accepting submissions:
Euroseek is not currently accepting site submissions. And like many other
search engines recently, Go.com (InfoSeek), Excite and Canada.com have all
abandoned their own indexes, and substituted paid search (provided by
Overture). Thus, you can no longer submit your site to Go.com, Excite, or
Canada.com.
Other Crawlers:
Singingfish.com Multimedia Search
Singingfish would like you to submit your video, MP3, or other media
files. They ask for your homepage, media directory URL, your content
category, how often you update, your email address, and where you heard of
them.
-- You may find your website being crawled by "ia-archiver", which appears
to handle frames correctly. This is Alexa, a toolbar site-description
browser plug-in. They do not accept submissions.
UK Paid Directory Submission:
looksmart.co.uk
With a LookSmart UK directory submission, you will get your site listed on
the MSN, BTopenworld, Tiscali, NTL and AltaVista directories, and reach
over 60% of UK Internet users.
Directory Submissions:
DMOZ Open Directory
The DMOZ Open Directory is used for browsing by Lycos, HotBot, Google, and
numerous other sites. It is extremely important to get listed on, and can
also be rather difficult. You need to select a category first, and then
click on 'Add URL' on THAT page. But some categories have NO "Add URL"
link!
When you do find a category you wish to submit to, they require your URL,
site title, a 25 to 30 word site description, and your email. It takes
from 2 weeks to several months to get listed, if you ever will. Don't
resubmit for at least 3 weeks.
Yahoo Directory
Like DMOZ, with Yahoo's well-known directory you need to select a category
first, and then click on "Suggest a Site" on THAT page. There is no
guarantee you will be listed.
Galaxy
This page fully explains submission to Galaxy, which also requires that
you select a category first, and then click on the "Submit Your Site to
this Directory" link at the bottom of the page. In addition to their
directory, Galaxy now has categorical "vertical" search engines, which
require separate submissions. Galaxy has 3 submission options, free (no
guarantees), $20 for review within 20 business days (does not guarantee
inclusion), or $100 for review within 2 business days (still does not
guarantee inclusion). Galaxy also allows you to pay extra to be a
"Featured Listing", at the top of the list. They do NOT accept "adult"
sites in their directory.
WebZone Directory
The WebZone directory, formerly InfoHiway, requires your URL, name, email,
title, site description (254 characters), and keywords (80 characters).
They say they list sites monthly, near the end of the month.
Jayde Directory
The Jayde directory requires your URL, name, email, site name, site
category, and site description (175 characters). They say they list sites
in 3 to 4 weeks.
PowerSearch.com
PowerSearch.com bills itself as a search engine, but functions like a
automated directory. PowerSearch.com requires your URL and email (and
instantly obtains more data from your website). They make no claim
regarding how soon you will be listed.
Paid Search Engines ( Pay Per Click, PPC or CPC ):
Overture.com Paid Search
Overture, the search engine until recently known as GoTo.com, is the
leading paid search engine. They recently concluded an agreement with
Yahoo!, which will place their top 3 listings for all keywords at the top
of Yahoo!'s results. Their listings also appear on America Online, Lycos,
AltaVista, Hotbot, Netscape, CNET and many others, reaching 85% of all
Internet users. Of course, you only pay for the traffic you actually
receive.
More Search Engine Information:
The Web Robots Pages
Contains information about crawlers in general and lists most known web
robots.
Search Engine World
Information and tools to boost search engine rankings.
Search Engine Watch
Information to boost search engine rankings.
PC Magazine reviews of top search engines
