.Google.com's Gary Illyes discloses Googlebot picks up links as opposed to "following" them, negating main paperwork.Googlebot gathers hyperlinks instead of observing them in real-time.Gary Illyes discussed this info in Google's 'Explore Off The Document' podcast.Google.com's records contradicts the actual crawler habits.