Anyone want to help me design a _rough_ algorithm for re-crawling the Twtxt network for the Yarns (_Yarn Search_) engine hosted at search.twtxt.net -- This is the _only_ experience I have with building search engines, and it would be good to bounce ideas off you smart folks 🙃 -- How often should it recrawl? How often should it rescrape a feed? etc...