Robots.txt tells hackers the places you don't want them to look

When you say 'move along, nothing to see here', the bad guys get interested

Melbourne penetration tester Thiebauld Weksteen is warning system administrators that robots.txt files can give attackers valuable information on potential targets by giving them clues about directories their owners are trying to protect."