How to spot OpenAI's crawler bot and stop it slurping sites for training data

Aww, c'mon, let us scrape your pages, we've got billions at stake

OpenAI, the maker of machine learning models trained on public web data, has published the specifications for its web crawler so that publishers and site owners can opt out of having their content scraped....