Thoughts on Googlebot vs Bingbot

W e use a few scripts that run throughout the day to monitor site activity. One of the things we monitor is various bots visiting the site. In particular, the Googlebot and the Bingbot are closely watched. As more content is added, sitemaps are updated, and sometimes submitted directly to the respective Google and Bing search consoles, it's helpful to see when we get their house calls.

I'll cut to the chase. It's obvious that Googlebot is far more savvy about the current social media and sharing landscape than Bingbot. And Googlebot is much faster to pickup on new links from larger sites that point to us. In fact, as far as this small site is concerned, it's not really clear that Bing is keeping tabs on these things at all.

Here's an example. We published a new article about the history of Adobe Photoshop on July 18th. On the 19th, it was mentioned on Reddit and a few other places, and ended up getting about 1400 page views. Googlebot came by 3 times that day to keep tabs on the page, while Bingbot didn't even visit once. Are we not on good terms, Microsoft?

My favorite example came a few days later on July 22nd. We were fortunate to get some coverage for the Photoshop article on a French magazine site, a Japanese Photoshop site, and others. Traffic was really heavy on the Photoshop page that day -- way more page views than the site normally gets. Googlebot seemed to be completely aware of this and was re-indexing the Photoshop page all day. In fact, it indexed the page 29 times that day! And visited almost all the other pages multiple times.

Unfortunately, Bing was nowhere to be found on that page at all. It lazily indexed about 10 of our existing pages, including the homepage 3 times. I've seen this pattern frequently during the few months our re-launched site has been live.

Is it possible that Version Museum is just too small for Bing to care about right now? Sure. We're just getting started. Is it surprising that Google has better search infrastructure than Microsoft? Not really. But if Microsoft wants to make inroads in the search space, they need to do a better job staying aware of new content. Even Baidu's web crawler is far more aware than Bing, from what I've seen here.