Category Archives: Search Engines

How To Get A New Site Crawled in 2 Hours

How To Get A Brand New Site Crawled in Less Than 2 Hours

Here’s a totally free technique that I’ve been using for what seems like forever for attracting googlebot (the google web crawler or spider) to my sites, but never really officially told anybody about it. The reason I’ve never mentioned it is because I just thought everybody who’s been using wordpress has been using it already – but it seems not – so here it is.

If you use wordpress you probably already know that within the wordpress admin panel you have the option to set a “ping list”, and if you search google for “wordpress ping list” you’ll get thousands of pages all suggesting their list of ping sites to include in this list. In fact I just found one page that was proposing I entered a total of 91 different ping URLs in my blog!

The problem with having such a massive list is that every time you create a new post, wordpress will automatically attempt to contact (or ping) every one of these URLs – and if you have 91 of them in there it’s going to take forever before your post gets saved – or worse yet your host will simply time out. The end result would be an extremely slow performing blog.

But this ping list is where the magic is – if you do it right. I have just 5 URLs (or services) in my ping list, and between them they can attract the much coveted googlebot to your site literally within a couple of hours at most.

Don’t confuse this technique with getting your site indexed (added to the search engine lists), as that all depends on what the spiders find once they get to your site. If they find crap or spammy content, chances are your site wont be indexed at all, but if they find good quality unique content, you’ll get indexed very quickly and will stay there too.

So, here’s my list of ping URLs:

http://rpc.pingomatic.com/

http://api.my.yahoo.com/rss/ping

http://blogsearch.google.com/ping/RPC2

http://ping.feedburner.com

http://rpc.technorati.com/rpc/ping

That’s it. Short, to the point and very effective. No need for massive ping lists that will slow your blog to a crawl. Try it out. Create a brand new blog, add these URLs in your ping list (don’t add any others), then create one new post on your blog with good quality and unique content and watch for the spiders to show up. I use a WP plugin called Crawl Rate Tracker which will show up under the “Dashboard” within the WP admin panel. When you load it, it shows how many times each of the top three spiders (googlebot, yahooslurp and msnbot) have visited your site each day. Give it an hour or so after making your first post and I’ll bet good money that at least googlebot has been over to check you out. Easy.