BrightonSEO
Managing Crawling & Indexing

Managing Crawling & Indexing Training

Want to make spiders your SEO friends? Now you can. In just one day, you’ll learn the strategies, tools and diagnostics that help you spot inefficiencies, prioritise fixes and build a site that’s easier for search engines to understand, crawl and index, plus analyse which of your pages are being indexed. A ticket to the UK course includes one day of training, plus two days entry to all the brightonSEO talks at the conferences, plus extra perks.

Charlie Williams

Charlie Williams

SEO & Content Architecture Consultant at Chopped Digital

Courses: 01

Charlie is a trainer for the team at Screaming Frog, the agency famous for the SEO Spider and Log File Analyser. He’s a regular industry speaker who loves to share ideas on content design & strategy, technical SEO, and creative user experiences. A search veteran with over a dozen years in the industry, Charlie is also an independent consultant. His SEO & content strategy consultancy, Chopped Digital, is dedicated to helping create better sites and smarter SEOs. Fascinated by the challenges of building an online business, you can find Charlie talking content, UX, SEO & food at the bar, or likely cooking something up in the kitchen..

What to expect...

This detailed, fast-paced, but easy-to-follow training course is a deep dive into the fundamentals of technical SEO and reviewing content structure and quality. Shared in plain language that anyone can understand, you’ll come away confident in how to manage and optimise how search engines crawl, render and index your site.

Whether you are mastering the basics, troubleshooting a complex site or scaling SEO processes in your team, this course gives you practical skills you can put to work. Learn to crawl JavaScript-based website frameworks and large sites, interpret log files, diagnose content-quality issues, use canonical tags, and manage sitemaps, robots directives, and pagination. All in a friendly environment where you can ask all the questions you can think of.

By the end of the day, you’ll be able to use crawl data, indexing metrics and log-file insights to shape a smarter SEO roadmap - helping you build a leaner, more efficient and higher quality site. And possibly leave with the urge to immediately jump straight back to your desk and start making changes.

Who should attend this course?

If you’ve ever wondered how to get your pages crawled or diagnose why pages aren’t being indexed, this course is for you

Join this Managing Crawling and Indexing training course if you want to…

  1. Control how search engines interact with your site
  2. Create efficient, easily crawlable sites
  3. Confidently identify, diagnose and fix content quality issues
  4. And get answers to all your crawling and indexing questions!

This course is ideal for SEO teams managing large or complex websites, as well as agency-side junior and intermediate digital marketers looking to deepen their technical SEO skills, and anyone who needs to diagnose content-quality or crawl-performance issues.

Course Content

This jam-packed training course will be split into four key sessions plus breaks for coffee, lunch and post-training drinks.

Here’s a look at what you can expect to learn in this course:

1: Introductions & what crawl budget means

  • An introduction to crawling & indexing
  • How Google crawls, assesses & indexes your pages
  • What happens when indexing goes wrong & how you can take control
  • Crawling vs indexing problems
  • Crawl budget – what is it, do I need to worry about it & how can I improve it?
  • Different crawler types – the advantages of each & how to choose one
  • Your Screaming Frog SEO Spider questions answered

2: Crawl analysis & indexing management

  • What to analyse in your crawl data
  • How to find crawl errors & the different ways of handling them
  • How to audit your site architecture & internal link structure to optimise crawling
  • The basics of log file analysis – what it is and what is suitable for testing
  • Exploring Search Console’s Crawl Stats report
  • Spotting crawl traps
  • Using the robots.txt file to control which URLs can be crawled
  • Using Robots tags to control crawling & indexing
  • Using XML sitemaps to help content discovery & encourage crawling
  • Using nofollow to stop links from being followed
  • Faceted navigation – the risks, best practices & what to do
  • Crawling JavaScript-based websites (and why it’s not as scary as you might think)
  • How to use page rendering to mimic the way Google ‘sees’ & understands your content
  • Mobile-first indexing
  • Canonical tags – how they work & when to use them
  • Pagination controls – to do or not to do? (and how to do it)
  • The different ways you can “ask” Google to index your pages

3: Managing Indexing

  • When & how to deindex, retire, or redirect pages
  • Tools you can use for indexing analysis
  • Monitoring indexing in Google & Bing
  • Using the URL Inspection tool in Search Console
  • How to spot crawling and indexing patterns for your site - and what we can learn about potential content quality issues
  • Search Console’s Page Indexing reports – everything you need to know about Google’s classifications of your pages, and when to take action
  • Why doesn’t Google want to index my page?
  • How to find the poor-quality content plaguing your site

4: Dealing with poor-performing or poor-quality content

  • How to communicate content quality issues and recommendations to stakeholders
  • The power of content inventories
  • Understanding Google's quality signals and how they impact crawling/indexing decisions
  • Strategies for improving thin or low-value content vs. when to remove it entirely
  • Group activities analysing real-world content quality signals together using live site examples or prioritising which non-indexed pages to improve, remove, or consolidate

What's included in your ticket?

Book onto this training course and you will get:

  • A full day's training on your chosen topic
  • Lunch included on your training day
  • Full access to brightonSEO
  • Bag of merch
  • Video bundle to stream all the conference talks post event (UK only)
  • Access to conference networking events and activities


--

We may cancel courses which don't have enough enrolments, and if so we’ll let you know and give you the opportunity to swap to an alternative course or have a refund.

Share this course:

Available Tickets

Brighton April 2026

Wed 29 Apr, 2026 | 10:00 AM - 04:30 PM

£650+vat£1150Prices rise 29th January
Book Now
100% Satisfaction Guaranteed

100% satisfaction guaranteed

We do all we possibly can to make our courses the best you’ve attended but if you ever feel we didn’t live up to your expectations, we’ll give you a total refund, no questions asked.

Don't just take it from us

What others have to say about our Managing Crawling & Indexing workshop

Jon Sharman

Jon Sharman

SEO Specialist, Bio-Rad AbD Serotec Ltd

Essential for all SEOs - There is no point in creating great content if Google can't index it! This course ensures that you get your content 'in the game' and is tailored to suit your needs.