Whenever search engine bots crawl your website, they will to start with seek out this file. It stands as the gatekeeper, so to talk, so in the event you don’t want your website to generally be indexed or accessed, it recognizes that within the get-go.In case you are observing a doc or outcome for somebody aside from your self or your authorized r