Keep preview crawlers out of challenges

Link previews fail when crawlers receive a challenge page, a blocked response or a login wall instead of public HTML and images.

Allow verified social crawlers through bot protection. User-agent allowlists alone are easy to spoof and should not be the trust boundary.

Keep the page HTML, Open Graph image and structured-data images public. A crawler must be able to fetch them without cookies, JavaScript or a session.

If the browser sees the page but crawler-like requests fail, check WAF rules, bot scores, geofencing, hotlink protection and image CDN rules.