Run A Crawl Before & After You Launch A New Site

  • Facebook
  • Twitter
  • Reddit

A new site is always a dangerous and exciting time in marketing. All the stake-holders get their input, old becomes new and opportunity is right there for the taking. Unfortunately, this is also a time where disaster can lurk in the shadows waiting for a deadline being pushed up to jump out and say BOO!

SEO: Know Thy Site

The best way to avoid problems is to get to know your old site better. Run a thorough site crawl, and check out the results no matter how ugly they are currently (Here’s a guide on fixing crawl errors). I recently did this for a client and found out that a contest they had run in the past was creating hundreds of pages that were supposedly redirected, but were still out there in the wild!

Create a Redirection Map

Export that crawl of your URLs into a spreadsheet and make sure that your new planned sitemap is matched up correctly. This can be a bit of a pain, but making sure your new site captures all existing traffic is key to not losing out in the process!

Launch, Fetch, Correct!

Once you launch that new website, go to Google Search Console, fetch & render and submit the site and all related URLs to the index. Give it a day and check out the crawl errors in Search Console. Sometimes it’s caused by new code being wrong on the site, a missed redirect or even a link on another site to a truly non-existent page, fix those and “Mark as fixed” as quickly as possible.

That’s it! New site launched! Time for a mojito and some sun!

Add comment


  1. Barbara DiLucchio
    06 . Oct . 2016

    Help Please!

    I ran a crawl of this new site but i don’t see how to export the results into excel. It just shows something like I am trying to select files on my drive. I want it to write the xls files onto the file location. Please tell me what stupid thing I am doing that is making this export not work.

    Thanks very much for your help
    Barbara DiLucchio

    • William Gomes
      10 . Jun . 2016

      You are trying to select a folder to save it onto your drive I believe?

  2. mm shaikh
    11 . Sep . 2020

    thank you for your service make free

  3. nikki
    09 . Dec . 2022

    Is there an seo site crawler for mobile p h ones

    • G
      22 . Dec . 2022

      I don’t think so.


SEO Crawls: The Two Most Actionable Errors To Fix
3 months ago

When you have a site and you want to improve your ability to rank in ...

Why I learned not to trust what anyone says in SEO
4 years ago

I used to read SEO blogs religiously and try and be as white hat as ...

A Guide to Auditing Your Google Webmaster Tools
10 years ago

Let us take you on a fantastic journey through Google’s Webmaster tools that will help ...

Site Audit Checklist #1: Headers
10 years ago

We’re going to start a series of blog posts to build up the perfect checklist ...