Search Engine Optimisation (SEO) remains one of the most important online marketing and visibility strategies for businesses. While there’s plenty of SEO you’ll see on the frontend of a website, particularly in the form of optimised content, there’s also a plethora of technical SEO strategies that all work behind the scenes to strengthen a website’s authority signals and drive more qualified traffic in.
What is technical SEO?
Technical SEO is focused on optimising your website’s infrastructure, much of which is not immediately visitor-facing. Unlike content-related SEO, technical is akin to the core foundations of your website.
It includes optimising your site for search engine crawlers and indexing through the integration of XML sitemaps, a Robots.txt file, and canonical tags (I’ll touch on these more later). Along with this, improving key components of the website experience such as load speeds, mobile-friendliness, logical structure, backlinks, and a whole lot more funnel into the realm of technical SEO.
Why technical SEO matters
Strong technical SEO helps search engines crawl, render, understand and index your website effectively, while providing a solid foundation for your content and other SEO efforts. As with all forms of SEO, the technical side is an ongoing process that requires regular refinements and strategies. Search engines like Google roll out a significant number of ranking algorithm changes every year, so your SEO needs to address these changes over time, as well.
As technical SEO is foundational to your website’s online visibility, you cannot afford to ignore its integration into your site as a whole.
How search engines crawl, index, and rank your website
There are three key steps to how search engines such as Google bring your website into their results. These three steps are crawling, indexing, and ranking.
1. Crawl
Google periodically sends out crawlers to websites across the internet to see if they can find anything new. Googlebot discovers URLs through previously known pages, links, sitemaps and other sources. It then crawls eligible URLs to retrieve their content and resources.
2. Index
Once the crawl phase has been sorted, all of the resulting new data (new website pages and websites) is then logically indexed by the search engine. Indexing essentially helps map out the structure and logic of a website so that Google (and other search engines) can provide more relevant results.
3. Rank
The final step in the process is ranking. After a site (or site pages) has been successfully crawled and indexed, search engines analyse a range of ranking signals within those pages to determine where results should sit in the Search Engine Results Page (SERP) in accordance with different search enquiries.
It’s important to note that even if Google crawls and indexes your website, it’s not a guarantee that every page it indexed will be ranked. If your fundamental SEO is lacking and your site isn’t actively optimised, it may not be considered authoritative or relevant enough to surface in search results.
A lot goes into technical SEO – and covering everything in one blog will make for a dense read. So, instead, let’s just take a look at some of the top-level components of technical SEO and the role they play.
SEO-friendly site architecture – Sitemaps, Robots.txt, breadcrumbs & schema markup
Let’s start by touching on sitemaps, robots.txt, breadcrumbs, and schema markups.
Sitemaps
Sitemaps are essentially a machine-readable file (typically in XML format) that lists out all of the important pages and files on your website. It’s akin to the index you’d find in the back of a book, and it tells search engines the exact pages that exist, when they were last updated, and their importance. You can submit a sitemap directly to Google Search Console (GSC), through your website’s CMS, or by including it in a robots.txt file.
While a sitemap does not guarantee that the listed URLs will be crawled or indexed, it does provide guidance to search engines about new or updated pages and improves the likelihood they may be crawled/indexed.
Robots.txt
Robots.txt is a plain text file that should be located in the root directory of your website (e.g. www.example.com/robots.txt). It’s a file where you can provide directives to automated bots and search engine crawlers on which areas they’re allowed to access and index, and which sections they must ignore. This is helpful because it prevents crawling of unimportant or private pages (such as login pages, shopping carts, etc.). A good robots.txt also tends to contain a reference to your sitemap.
Breadcrumbs
While users will rarely look at sitemaps or robots.txt files, breadcrumbs are purpose-designed to help simplify on-site navigation for both users and search engines. Breadcrumbs are essentially navigational paths that are displayed towards the top of webpages. A common example is if you’re browsing an ecommerce site, going from the Home page to a product category, a subcategory, and so on.
You may see a navigational path (breadcrumb) displayed that shows: Home > Men’s > Shoes > Running Shoes. Each of these page paths will be hyperlinked so that users can easily backtrack along the path. This type of navigation also makes it easier for search engine crawlers to make sense of your website structure and how different pages lead to the next.
Structured data (Schema markup)
Structured data is a standardised code (often JSON-LD) that can be added to a website’s HTML and will explicitly tell search engines what the content on individual pages means. By providing a clearer means for search engines to categorise the content on your site more accurately, it leads to richer results. On Google results, these are known as Rich Snippets and can include star ratings, images, pricing, recipe layouts, etc.
When Google displays a rich result, the enhanced search appearance can make a result more prominent and potentially improve click-through performance, especially from qualified leads. It also reduces ambiguity for search engines, resulting in better categorisation and local results.
JavaScript rendering
A significant number of modern websites continue to rely on JavaScript to create interactive elements and dynamically load content as it comes into view. While Google Search can process JavaScript, how it’s implemented on your site still affects how efficiently search engines discover, render, and index your content.
Google Search generally processes JavaScript through three key stages: crawling, rendering, and indexing. First, Googlebot crawls the URL and retrieves its resources. In the second step, the page may be placed in a rendering queue, where Google’s Web Rendering Service executes JavaScript and generates the rendered version of the page. Finally, with the third step, Google can then use the rendered HTML to understand and index the content.
If your website’s important content, links or metadata depend heavily on JavaScript, technical issues can prevent Google from accessing or understanding them correctly. This is why server-side rendering (SSR) or pre-rendering can be beneficial, particularly for important SEO content. Google notes that server-side or pre-rendering can improve performance for users and crawlers and can also help when other crawlers cannot execute JavaScript.
If you use JavaScript to generate important SEO elements, test the rendered page using Google’s URL Inspection Tool or Rich Results Test (via GSC) to confirm that Google can access and understand the final content.
Backlinks
Although backlinks are generally considered off-page SEO rather than technical SEO, they’re worth mentioning because they can contribute to discovery and overall search visibility. A backlink is whenever an external website links back to a page on your website.
The more authority the referring website has, the more trust this will signal to Google and, by extension, boost your rankings. The more external websites link to you, the more opportunities it creates for Google crawlers to discover more of your content and enhance your presence.
Mobile-friendliness
I’ll keep this section brief. In case you didn’t know, when Google crawls any website, they do this on a mobile-first basis. In other words, they crawl and index from a mobile viewport first. Any modern website that doesn’t have a responsive, mobile-friendly design will be impacted in their rankings. The good news is that almost every current CMS or website builder includes mobile-responsive design by default. So, make sure your website renders and adapts to various device types without issue.
Core Web Vitals
Finally, I want to touch on three website performance metrics – which Google calls Core Web Vitals. These three metrics are as follows:
- Largest Contentful Paint (LCP): This measures how long your website/webpages take to load the largest image, text block, and/or video that is visible in the viewport (what’s on screen) when a user first navigates to the page. You should ideally aim for an LCP of 2.5 seconds or less.
- Interaction to Next Paint (INP): This measures the responsiveness of your website to user interaction across devices (e.g. how long it takes between a mouse click on a tab and the loading of the tab or how quickly typed words in a website search bar show up, etc.). The ideal INP is 200 milliseconds (ms).
- Cumulative Layout Shift (CLS): This final Core Web Vitals metric is all about the visual stability of your website. It essentially looks to measure how much things jump around on a page as it loads (text, images, different frames, etc.). The less movement, the better. Google sets a good CLS score as 0.1. This essentially means a movement/shift of no more than 10% within the viewport of how the user’s viewing the site (laptop, desktop, mobile, etc.).
You can assess these three vitals across Google’s PageSpeed Insights and GSC.
As you can see, technical SEO covers a lot of ground, but there’s even more that goes into this behind-the-scenes facet of search engine optimisation. If you think your website needs a technical SEO overhaul, Reform Digital can make it happen. To find out more about how we can help your business, get in touch with us today.
