To accomplish its job, these scanners need to know what precisely to find on an online server. Usually, these instruments are scanning Net servers looking for susceptible information or directories that may have sample code or vulnerable documents.) accessibility internet pages of a internet site, they Verify to determine if a robots.txt file exist