Getting your site on Google
How Google finds a website, how to see whether yours is listed, and what usually stops it.
How Google finds a site
Google does three things. It crawls: a program called Googlebot visits pages and follows the links on them. It indexes: it works out what each page is about and stores it. Then it ranks: when someone searches, it picks the stored pages that best answer them.
You do not pay to be listed and you do not have to apply. If other sites link to yours, Google will usually find it. What you can do is make that faster and make sure nothing is blocking it.
Check whether you are listed
Search Google for site: followed by your address, for example site:example.com.au. If pages from your site come back, Google has them. If nothing comes back, either it has not found you yet or something is in the way.
Set up Search Console
Search Console is Google’s free tool for site owners. It shows which of your pages are listed, which are not and why, what people searched for to find you, and any problems Google has hit. It is the most useful thing on this page.
To set it up you prove you own the site, usually by adding a record at your domain registrar or a small file to the site. Then:
- Submit your sitemap, so Google has a list of your pages.
- Use URL Inspection to see one page the way Google sees it, and to ask for it to be crawled again after a change.
- Look at the Pages report now and then for anything marked as not indexed.
The things that keep a site out
A noindex tag. This is an instruction on the page that tells search engines not to list it. Sites are often built with it switched on and launched with it still there. In WordPress it is the “Discourage search engines” box under Settings, Reading.
A robots.txt that blocks everything. This file sits at /robots.txt on your site and tells crawlers where they may go. A line reading Disallow: / under User-agent: * shuts them out of the whole site.
A login or password. Google cannot read what it cannot open.
No links in. A brand new site that nothing links to can take a while to be found. A sitemap in Search Console fixes that.
More than one address. If your site opens at both www and without it, or on both http and https, pick one and redirect the others to it, so Google is not choosing between copies.
A sitemap
A sitemap is a file that lists the pages you want found. Most website platforms make one for you, often at /sitemap.xml. It does not guarantee a page will be listed, but it tells Google the page exists.