Tools FAQ

- Spider Glossary

Depending on your settings, density is either

Occurences of the word / Number of Unique words
or
Occurence of the word / Total number of words

Density is the ratio of one word or phrase to the total number of words in a text.

The density of one word in a ten word sentence having unique words only is 10%.
The density of one word, occuring twice in a ten word sentence having 9 unique words, is 20%

Top

Depending on your settings, prominence is either based on the unique words in a file, or on all the words.
Prominence is the ratio of the position of one word or phrase to the positons of the other words in a text.

The formula we use is
prominence = ($totalwords - (($positionsum - 1) / $positionsnum)) * (100 / $totalwords)

Where
$totalwords = total number of words in string

$positionsum = the sum of each position of the word we are analyzing
(example: if a word occurs on position 2 and 5 $positionsum is 7)

$positionsnum = The number of positions

The prominence of one word in the first position, in a ten word sentence having unique words only is
(10 - ((1 - 1) / 1)) * (100 / 10)) = 100%.

If that same word would be the last word in the sentence, it's prominence would be
(10 - ((10 - 1) / 1)) * (100 / 10)) = 10%.

If that same word would occur twice, on position 1 and 10, it's prominence would be
(10 - ((11 - 1) / 2)) * (100 / 10)) = 50%

Top

That's totally up to you and depends on your goals. For example, if you are targeting a search engine that does not index title attributes you should not incorporate these in calculations of course. The minimum and maximum density settings only serve as a visual aid, and a tool to see if pages are within density policies. These settings are also useful when comparing two pages, as you could compare one pages keywords with a density between x and y to another pages keywords with density between x and y.

Top

This may be due to html which does not validate, your settings, or a bug. Please note you will not see any words of less then 3 characters in the default setting, nor words that occur only once on the page. If you think it's a bug and it is not mentioned in the next question, please let me know and i'll look into it. Don't forget to mention how I can recreate the problem.

Top

Yes.
Pages using multiple title, description or keyword tags are not detected as such.
HTML which does not validate may produce reports that are not accurate about page elements.
Unquoted attributes are not picked up. (as in alt=something or a href=something)

Top

In most cases this will have no influence on the final density and prominence calculations, as this uses all text on the page as a basis. (Depending on your settings stopwords may be stripped). Even pages that use very bad html are usually stripped properly.
Page elements are harder to extract from bad html. If someone mixes < b > bold or < i > italic tags with < a href > link tags the script will still be able to extract these. I'm not sure what the script will do in other bad html situations. The most likely effect of bad html in other cases is some page elements missing or not being complete on the report page.

Top

Some search engines may have one field in their database for the keywords of each url. If this is the case, the maximum number of characters for the collective keywords can not be more then the maximum number of characters in this field. A default maximum for one field in a database is often 255 characters.
This function probably does not serve any purpose for search engines that index all text on a page.

Top

As we do not know how most search engines index and rank pages, we should stay open to all options. It is conceivable that a search engine decides that only the top x number of keywords from a page are to be indexed or rewarded with a higher rank for that term. There could be a fixed maximum to the number of keywords / search terms one url can achieve good rankings on.

Top

I think the last word in a sentence should have a different prominence depending on the number of words in a sentence. My theory is that it will make a big difference to a search engine wheter your keyword is the last in a five word title or in a ten word title. In my calculation the prominence of a last word in a ten word sentence is 10%, in a five word sentence 20%

Top

Not sure at all. In any case it will be different for each search engine. Our default lists are just working examples at this point, if you think you have a good list I recommend using that list. (Send me a copy please ?)

Top

- A warning system to point out html that does not validate for the most important page elements.
- A data export function in delimited text format
- A warning system for items that possibly trigger spam filters

Top