How Twitter is (probably) crawling the Internet for AI
As a bored sysadmin you might SSH into one of your servers and run htop or tail -f access.log.
Last Sunday was one of those tail -f access.log days.
You take a cursory look at your web server logs slowly scrolling past and see a bunch of requests come in. They look a bit odd.
