Technical SEO can be overwhelming. Coding seems hard to learn and risky to implement. Yet, it doesn’t have to be this way 🤗. Learn how to setup your site for success by following our easy-to-implement Technical SEO Checklist ✔️.
In this blogpost, the following technical fundamentals are covered:
After that, you can navigate through the basics:
- Accessibility
- URL structure
- Breadcrumbs
- Meta data
- Facets & parameters
- Robots.txt
- XML Sitemaps
- Canonicals & duplicate content
- Sitespeed
Finally, use our technical checklist and learn how to perform a technical audit:
What is technical SEO?
Technical SEO is the process of optimizing technical aspects of your website so that Google is able to (1) crawl & discover and (2) index your pages. These two are the foundation of your technical SEO efforts. Once you understand both concepts, it is a breeze to optimize them! Let’s find out what they mean.
1. Crawling
Crawling is the discovery process in which millions of Google-bots (also known as spiders or crawlers) browse the World Wide Web. Simply put, the process in which Google discovers new and updated content on your website. To simplify this concept, think of it as a subway network. Google is the train, each station is a webpage and the rails are the links between those pages.
Now, you want to make this process as efficient and affective as possible. Why? Because the web has become so big nowadays, that Google is not able to crawl everything anymore. This simply takes up too much time and resources.
Instead, Google spends a limited amount of time on your website. This concept is also known as crawl-budget. A crawl-budget can be seen as the amount of time Google spends on your site to discover new and updated content. To simplify this concept, compare this crawl-budget with a the traindriver, working from 9 to 5 each day.
With all this knowledge, you want to make sure that this train driver (Google) spends his time visiting the most important stations (pages) via the fastest route on rails (links). While visiting the stations, its important to provide essential information about the station:
- What is the name of the station? (page title)
- Why should I visit? (meta description)
- What can I do there? (body content + headings)
- What other stations can I visit? (internal links)
As your website grows, you might delete old pages. Suddenly, Google reaches a dead-end because you are still linking to that old page somewhere on your website. Or you redirect a page to a new one. Now Google has to first visit that page to find out you have moved the content to a new destination. What a waste of time!
But if you fix both of these errors, Google is able to visit all of your pages again in the most efficient manner.
You see? Understanding crawling doesn’t have to be hard 😄.
2. Indexation
Indexation is the process in which Google indexes your page into their massive database. When someone performs a search all relevant sites are retrieved from this database, which are ranked from most relevant to least relevant by their algorithm.
To simplify this concept, think of the Google index as a library. Here you have different sections, bookcases, shelves, books, chapters and pages. The books refer to websites. The pages in these books refer to webpages.
Now let’s say you want to find out what things you can do in Bali, Indonesia. You would walk to the travel section, go to the shelve that contains books on travelling through Asia and finally navigate to the books specifically about Bali. After that, you would compare these books and ultimately compare certain pages to get the most relevant and best answer to your initial question.
With this knowledge, you want Google (library) to index all of your webpages and match them to the right topics (bookcases and shelves). That’s it!
However, not all of your pages are relevant. Take WordPress for example, that works tags for your blogposts. Very useful, but it automatically generates a new page for every tag! These pages are usually pretty empty and do not serve any other purpose. Thus, you want to tell Google to ignore these pages from indexation.
You see, understanding indexation isn’t that hard either 😉.
The following technical basics all have to do with either crawling, indexation or both. Re-read these fundamentals a couple of times and you will be able to optimize your technical SEO a lot easier!
VARIANT | EXAMPLE URL |
---|---|
HTTP | http://example.com/page-1 |
HTTP + SLASH | http://example.com/page-1/ |
HTTP + WWW | http://www.example.com/page-1 |
HTTP + WWW + SLASH | http://www.example.com/page-1/ |
HTTPS | https://example.com/page-1 |
HTTPS + SLASH | https://example.com/page-1/ |
HTTPS + WWW | https://www.example.com/page-1 |
HTTPS + WWW + SLASH | https://www.example.com/page-1/ |
As you can see, Google can access your website in eight different ways.
Why is this important? Because of three reasons
- Crawling
- Indexing
- Link equity
If you run an ecommerce store with 1.000 products, this means Google could crawl 8.000 URLs to fully understand your website. In worst case you end up with 8.000 duplicate URLs indexed. But more importantly, if other pages or external websites link to your domain, the link equity gets dilluted over all these variations.
The solution?
Point all possible combinations to one preferred variation 🥰. You will do this using a 301-redirect (permanent) so Google knows this is the only URL variation you want indexed. Let’s look at the possible ways to do this:
- .Htaccess file
- WordPress settings
- Webhosting settings (cPanel)
1. Htaccess file
The .htaccess file is a file which is hosted on your server that Google will read each time it requests a page. Within this document, you can set certain rules. If you run a WordPress website, you should be able to see it with FTP software in your root folder:
Apply these rules to redirect all URL variations to your preferred one:
RewriteEngine On RewriteCond %{HTTPS} off [OR] RewriteCond %{HTTP_HOST} ^www. [NC] RewriteCond %{HTTP_HOST} ^(?:www.)?(.+)$ [NC] RewriteRule ^ https://%1%{REQUEST_URI} [L,NE,R=301]
2. WordPress settings
In your WordPress dashoard, go to Settings > Reading > and make sure it’s set to the same preferred variation as you want it to be:
3. Webhosting settings (optional)
Some webhosting providers allow you to setup certain redirects within their cPanel. Go the the control panel of your webhosting and look for domains or redirects. Here, set the settings to the preffered route or force HTTPS:
It is important to have all settings in sync. If you set your wordpress settings to https://example.com you should also use this in your htaccess redirect rules (HTTPS/NON-WWW). These two should already do the job. Forcing HTTPS in cPanel is not always neccessary.
URL Stucture
A URL structure refers to the what your URLs look like. Google crawls your URLs and tries to understand it by using the words in your URL. By doing this solely from the URL, Google is able to provide the latest information on the world wide web in the fastest way possible.
A proper URL structure should be meeting two criteria:
- As short as possible;
- You should understand what a page is about by only reading its URL.
A few other general rules of thumb are:
- Use hyphens (-) for spaces;
- Don’t reuse the same keywords over and over;
- Avoid stopwords (a, the, for, etc.).
However, things can get complicated for Google. Perhaps you have seen URLs like this before 🤔:
- yourdomain.com/blog?post-id=1234
- yourdomain.com/post-1234.html
- yourdomain.com/site/18jh_z/post-1234
Let’s say you have a page about a tripple layer chocolate cake recipe. Can you tell what the pages are about by only reading the URLs mentioned above? No. Instead, use URLs like this:
- yourdomain.com/cake-recipe (mwah 😕)
- yourdomain.com/chocolate-cake-recipe (ok 😀 …)
- yourdomain.com/tripple-layer-chocolate-cake-recipe (great 😎!)
If you are using WordPress, go to Settings > Permalinks and use the following settings:
or alternatively, use the category tag + post tag:
Breadcrumbs
Breadcrumbs are internal links that allow users to track of their location on a website. Visually presented in the form on a trail. It also serves as navigational aid to quickly go back to a previous section. An example:
Example.com > Shoes > Men’s shoes > Nike > Air Max
However, they also matter for SEO.
Why?
Because it creates relationships between pages. By following the breadcrumbs, Google is able to understand which pages relate to the main topic, which pages are subtopics and so on. The more relevant and complete your pages are about a specific subject, the higher you will rank.
Make sure to enable your breadcrumbs and that they are working properly. Each link should directly point to a final destination and the hierarchy should match your setup. It depends on your WordPress theme if you can and how you can enable breadcrumbs. Alternatively, you can use the Yoast plugin to enable them. Go to Search Appearance > Breadcrumbs 👇:
Meta data
Meta data are pieces of information about your website. It refers to certain elements that have value for SEO because Google uses it to understand what a webpage is about and how to index it. Let ‘s have a look at the most important ones:
- Meta title
- Meta description
- Meta robots
1. Meta title
You can recognize the meta title from the browser tab’s title or in the source code (CTRL+U).
It’s the most important on-page element since it tells Google what your page is about. Use keywords wisely in your title tags to get an extra boost in rankings. Together with the meta description (as described below), it is what is being showed in the search results to convice potential customers to click on your result.
Let’s look at a few bad and good examples:
Mwah 😕 …
- Dog toys | DogToys101.com
- Toys | Shop | Dog Toys 101
- Toys for dogs | DogToys101.com
Great 🧐!
- Top 10 Best Dog Toys 2021 | DogToys101.com
- Buy Indestructible Chew Dog Toys | DogToys101.com
- Cheap Dog Toys: Save 25% in Sale | DogToys101
We have wrote a blog about page title tag tricks that you should checkout to get the most out of your page titles.
2. Meta description
A meta description is a piece of HTML code that describes and summarizes the contents of your page. You can recognize the meta description in the source-code (CTRL+U):
Together with the page title it is being shown in the search results. It is important to spend some time optimizing your meta descriptions to persuade potential customer to click on your result. Let’s look at a few best practices:
- Use keywords: Repeat keywords in your meta description as the will be bolded in your result when they match the user’s query;
- Use Unique Selling Points: Use Unique Selling Points in your meta description to distinguish yourself from your competitors;
- Use special characters: use special characters to stand out in the search results, emoticons such as ✅ or special symbols like ᐅ;
- Use a call-to-action: tell people what to do by ending with a strong call-to-action.
Here is an example of a heavily optimized meta title and description using all kinds of special characters and persuasion techniques:
3. Meta robots
Meta robots is a piece of HTML code that informs Google two things, (1) whether to index the page and (2) whether to follow the page or not. You can recognize this piece of code as follows:
This can be helpful when you have pages with parameters (?=xyz) that you don’t want to index but it provides a gateway to other pages you want Google to discover for new or updated content.
Final note on meta tags, please don’t bother using the meta keywords tag. Google stopped using them since 2009 and you can do perfectly fine without them.
Facets & Parameters
If you run an e-commerce store, play close attention. Facets and parameters can greatly influence your SEO if you optimize them. Remember how we talked about the concept of crawling? Let’s assume you sell 100 shoes in four colours and fize sizes. URLs like these will be created:
- yourdomain.com/shop/nike-air-force-1?color=black&size=40
- yourdomain.com/shop/nike-air-force-1?color=black&size=41
but also
- yourdomain.com/shop/nike-air-force-1?color=white&size=40
- yourdomain.com/shop/nike-air-force-1?color=white&size=40
In other words, it multiplies your URLs with 100 * 4 * 5 = 2.000 URLs.
Why is this important?
Again, because of crawling and indexation. First of all you don’t want to waste your crawl-budget on these URLs. Secondly, you don’t want these URLs to be indexed as they are usually duplicates 👟👟👟. Thirdly, you want to consolidate all link equity to one preferred URL.
Title | URL |
---|---|
Nike Air Force 1 | yourdomain.com/shop/nike-air-force-1?color=black&size=40 |
Nike Air Force 1 | yourdomain.com/shop/nike-air-force-1?color=black&size=41 |
Nike Air Force 1 | yourdomain.com/shop/nike-air-force-1?color=white&size=40 |
Nike Air Force 1 | yourdomain.com/shop/nike-air-force-1?color=white&size=41 |
You can optimize crawling and indexation by using the robots.txt file 📝.
Robots.txt
The robots.txt file is a textfile that contains instructions for Google-bots. It can tell which pages Google should crawl and which it should not. The robots.txt file is always located in the root folder of a website. You can see if you have one by going to yourdomain.com/robots.txt in your browser.
Let’s have a look at the most common setups:
You want to add a few things here:
- A line that prevents Google-bots from crawling search pages. Usually Disallow: /*?s=
- A line that prevents Google-bots from crawling facetted pages. Usually Disallow: /*?color=
- A line that provides Google-bots with a list of all your pages. Ususally Sitemap: https://www.yourdomain.com/sitemap.xml
Here’s a simple and optimized robots.txt file for an ecommerce store:
Canonicals (& Duplicate content)
A canonical tag is a piece of HTML code that tells Google what the main version of a set of similar pages is. It helps to prevent duplicate content and it bundles link equity to one page (this is never confirmed by Google, but its very likely). You can recognize this piece of code by looking in the source code:
So when is this useful? Let’s look at an e-commerce store, that might contain the following pages:
- yourdomain.com/shop/nike-air-force-1?size=40
- yourdomain.com/shop/nike-air-force-1?size=41
- yourdomain.com/shop/nike-air-force-1?size=42
- etc.
Now if you don’t have unique elements in place for every URL, these pages are 99% duplicates of each other, the only difference is the products being shown in their corresponding size. Yet, it is better to index the main page only. Otherwise you have lots of pages that Google might index, but are all the same.
Hence, the canonical tag for these URLs would be:
- yourdomain.com/shop/nike-air-force-1
In a few cases, I have also found that when you use a specific WordPress template it is smart double checking your homepage canonical tag in the source-code. If you make a new page called ‘Home’, you will have two homepages:
- yourdomain.com/
- yourdomain.com/home/
While you are probably redirected from yourdomain.com/home/ to yourdomain.com, the canonical tag for your homepage still is: yourdomain.com/home/
We don’t want this because we have yourdomain.com as our homepage. So we need to adjust this manually. You can do this with the Yoast SEO plugin. Edit your homepage, scroll all the way down to the Yoast plugin. Click on ‘Advanced’ and edit the ‘Canonical URL’ manually:
You can easily create or edit a robots.txt file yourself by using Yoast. Go to SEO > Tools > File editor and edit the robots.txt here. Or if you don’t use Yoast, you can edit the robots.txt file via your FTP software.
XML Sitemap
A sitemap is a file where you provide information about the pages, videos and other files on your website as well as the relationship between them. Basically a map of your site, hence the word sitemap.
There are a few guidelines:
- The list should be up-to-date (automatically)
- The list should contain only working URLs
- The list should not exceed 50.000 URLs or 50mb
There are online tools to create a static XML sitemap. This is not recommended, since you do need to update this regularly, in order to instruct Google with the lastest information. Within WordPress, install the Yoast SEO plugin to create and update your sitemaps automatically. Go to SEO > General > Features and enable the sitemap:
You can visit your sitemap by browsing to: yourdomain.com/sitemap.xml or yourdomain.com/sitemap_index.xml. Click around to see what pages are in your sitemap.
By default, Yoast adds all on your pages to your XML sitemap. Depending on your website, you want to exclude certain pages from being indexed. In order to remove them from the sitemap, go to SEO > Search Appearance > Taxonomies and disable the type of pages you don’t use. For example for tags:
Furthermore, don’t worry too much about canonicals at first. Only when you run a big webshop with lots of filter options, its worth diving into a bit more. If not, make sure your canonical is always self-referencing. Meaning it is the same as the permalink (URL) you see in the browser.
Sitespeed
Sitespeed has become an important ranking factor in 2018 and it will be even more important when the Core Web Vitals will become part of the algorithm in may (2021) this year 🚀.
First, lets check your pagespeed with Pagespeed Insights:
Scroll down to the recommendations and see what you can improve. Google even suggests using certain WordPress plugins.
Secondly, although entire blogposts can be dedicated towards sitespeed, here are a few basic tips to optimize for it:
- Caching: Install a caching plugin to make your pages load faster. WP Rocket is the best but paid, a good free alternative is WP fastest Cache;
- Compressing: Compress your images before you upload them with TinyPNG.com or with the Imagify plugin if you already have uploaded them;
- Resizing: Resize your images properly. For example with Pixlr. Do you need a 400 by 400 pixels image? Upload it accordingly. Do not use a 1.000 by 1.000 pixels image for this;
- Lazy loading: Use lazy loading for your images with a plugin like A3 Lazy load;
- Fonts: Use Web Safe Fonts, which you can check here if you do. Otherwise, Google or the users needs to load these first.
Technical SEO Checklist (Mainly WordPress)
We have created a simple Google Sheets Technical SEO checklist you can use to start optimizing your website for Google. Download it here (File > Create a Copy). You can read more about each topic in depth in this blogpost.
Performing a quick technical SEO audit for your site
How do you quickly audit your website on technical SEO, with all the knowledge you have now? Start with these two free tools:
- Detailed SEO Chrome Extension
- Screaming Frog Spider
1. Detailed SEO Chrome Extension
This extension is an All-in-One extension that quickly provides you with all the information you need. Just install it, go to any webpage and click the icon:
Here you can quickly check the most important elements mentioned in this post:
- Meta data
- Canonical tag
- Robots.txt
- XML Sitemap
Also make sure to check our blogpost about The 25 Best and Free SEO Chrome Extensions.
2. Screaming Frog Web Spider
In order to analyse how Google crawls and indexes your website, install Screaming Frog. The free version allows you to crawl up to 500 URLs. That should be enough to start with. You can find the most important errors already by crawling a sample of your website.
- Put in your domain name at the top;
- Hit ‘Start’ to start crawling your website;
- After its finished crawling , you can analyse your website for errors like statuscodes or missing content elements.
That’s a wrap 🤠! There you have it, a simple yet powerful checklist ✔️ to optimize your website . Please let us know if you have any questions in the comments below.
Leave a Reply
Want to join the discussion?Feel free to contribute!