Gary Illyes shared a nice little tidbit on LinkedIn about robots.txt files. He said that only a tiny number of robots.txt files are over 500 kilobytes. I mean, most robots.txt files have a few lines ...
Some results have been hidden because they may be inaccessible to you
Show inaccessible results
Feedback