Spinning Plates
Had to ease up the amount of blog crawling. The crawler looks ahead and gathers links, but there’s only so many I can get through in a day. There are many tasks Bluebloomer has to deal with, ie. page finding, crawling and indexing text, crawling images, phrase finding, category creation, caching… All these tasks vary in time needed, it’s a balancing act getting everything to work in harmony.
I’d say its going to take a few weeks measuring and sorting out the best balance. Also as I add new crawlers I’ll need to readjust things to suit. But if the overall ratios are ok, generally things should be fine.
I’m hoping to activate category creation today. This means things will start appearing on the front page. We’ll see how we go. In the meantime gotta keep on crawling.
