Glossary

Crawler Error

Crawler error reports are useful for sites to see what needs to be fixed. The report lists site errors separately to URL errors as each type can tell the site owner different things. The site error category lists what the main issues were that prevented the crawler from accessing the entire site whereas the URL error lists all the errors that the crawler encountered while in the process of reviewing desktop and mobile pages.

If a site shows an error rate that is high there is something wrong with it and a 100% error rate means that the site is down or totally malfunctioning. If the error rate comes in at 20% or less it could mean either that the site is in a transient stage or that it is improperly configured or overloaded. There should be no site errors reported. Any site errors that are found will be reported to the site owner or administrator.