Tag: robots.txt

W3C Link Checker and Robots.txt Exclusion

I was ready. I had just finished updating the last few links (and some content) on my clients website and I was ready to check for broken links. I opened the trusty link checker over at W3C Schools and got

Posted in: Development  |  Tagged with: , ,  |  Leave a comment