Robots, Graphs and Binary Search
Post on Code as Craft on monitoring and detecting robotic traffic through origin IP addresses
Over at the Code as Craft blog by Etsy is a new article on detecting and monitoring robotic traffic and the use of a IP categorization database. Here’s the intro:
We love our human customers. That said we get a lot of traffic from robots too. This is great, especially when they use the Etsy API. However, they sometimes misbehave. And they misbehave frequently in the late hours, not unlike a legendary East Village nightclub. This time Craig Ferguson isn’t at the door to keep an eye on things. Instead we monitor the robots attending our night club with graphs.
With an intro like that, I’ve always been disappointed that the only comments are “why didn’t you use a hashtable” or some other nonsense. I’m still hoping someone writes in saying they went to the Save the Robots as well, or something on using BGP tables.