Search & Digital Marketing · via Search Engine Journal

Cloudflare Will Write Your Robots.txt, And It Has A Point

8h ago · curated by Catalyst Web Trendz

Bot Preference Sync sets AI crawler policy per category, not per crawler. Allow GPTBot and block Bytespider, and no setting describes what you do.

This is a curated summary. The full story was reported by Search Engine Journal.

Read the full story at Search Engine Journal

Put this to work

What it means for your brand.

Stories like this are exactly the terrain we work in every day. If this trend touches your growth plans, these are the services to look at:

Related headlines.

Search Engine Journaljust now

Google Discover Test Adds Topic Overviews, Starting With Videos

Google is testing a "Dive deeper" button in Discover, starting with videos.

Read on our site

Search Engine Journal3h ago

Checking A Page Is Part Of A Retrieval Pipeline For AI

No GSC for ChatGPT? Paste a distinctive snippet into a chatbot and ask for exact matches. If your URL comes back, retrieval isn't your problem.

Read on our site

Search Engine Journal5h ago

Google Adds Local Businesses To EEA Search Result Units

Google updated its Search documentation to add local business queries to the two result units rolled out in the EEA for directories and direct providers.

Read on our site

All of today's digest