# www.robotstxt.org/ # User-agent: Twitterbot Disallow: # Allow crawling of all content User-agent: * Disallow: foto/*