Appearance

XML Sitemap Requirements ​

What is a Sitemap? ​

A sitemap is a file on your website that shows the structure of your website, most importantly, a sitemap shows all existing pages and content, including relationships between pages.

Search engines use sitemaps to crawl websites, mostly because sitemaps reveal all existing pages (URLs) on any given website.

Sitemap visualization

XML Sitemap ​

When you hear the word sitemap, in 99% of the cases people will refer to an XML sitemap used by search engines.

XML sitemap is an extensible markup language file that lists all URLs on the domain, including some relevant information like page relationships, update times, relevance, etc.

This is how the sitemap looks and in most cases, it can be found by adding /sitemap.xml after any domain name.

XML Sitemap Importance for SEO ​

Part of successful SEO steps is making sure that all the most important pages on your website are crawled by search engines.

SEOs and website owners achieve that by submitting a sitemap.xml file to search engines and keeping it up to date as new pages are created.

While you could make the argument that with perfect internal linking, advanced taxonomy, and great website structure, a sitemap.xml file is unnecessary. Because, if all three of these things are done correctly, crawlers will crawl your website fully without it.

In reality, everyone makes mistakes, therefore information within the sitemap.xml file acts as a "source of truth" and a backlog of published pages that search engines can rely on whenever they need it.

Also submitting a sitemap.xml file to webmaster tools allows search engines to "discover your website".

Be sure to read our guide on website crawling if you want to learn more about building crawlable websites in Webflow.

XML Sitemaps in Webflow ​

Sitemaps in Webflow can be generated automatically or updated and published manually.

Automatic Sitemap ​

To generate an automatic sitemap in Webflow:

  1. Visit website settings
  2. Under SEO, find the Sitemap section
  3. Make sure the Automatic Sitemap option is turned on
  4. Click on the Save Changes button
  5. Publish the website if you made any changes (turned on or off)

Your sitemap will now be reachable by visiting this URL:

[your-domain.com]/sitemap.xml

Manual Sitemap ​

There are SEO cases where you need to maintain and update your sitemap manually.

Webflow allows you to turn on or off sitemap and indexing within page settings. This will remove the URL from the sitemap and will add a noindex tag to the head code.

For even more control, Webflow allows you to manually manage your sitemap file in Website SEO settings.

It's tedious and slightly unnecessary work, but because we have little to no control in Webflow regarding which pages will be added or not added to the sitemap…

Sometimes manual sitemap.xml maintenance is mandatory.

In cases like that, you should follow previous instructions, but turn off the auto-generated sitemap option.

If you do that, you will get a text field where you can add your sitemap version.

This means that you can manually add hreflang information, lastmod attributes, or page priority properties to your sitemap.

Of course, you'll also need to update this information when changing anything on your website.

Therefore, the custom sitemap option should not be considered by beginners as sitemap maintenance can get messy as the site becomes bigger and bigger.

Submitting Your Sitemap to Search Engines ​

After you release your website, it is mandatory to submit your sitemap.xml to search engines like Google.

How to Submit Your Add Your Sitemap to Search Console? ​

For Google to find your sitemap, you should create your GSC (Google Search Console) account.

After you approve your domain ownership inside the Search Console, you must submit your sitemap to it.

To submit your sitemap simply:

  1. Go to Search Console and find Sitemaps
  2. Add your sitemap.xml URL under "Add New Sitemap"
  3. Click Submit and wait for submission confirmation

Note! Below the submission field, you will find data about the submitted sitemap status, discovered pages, and other valuable information.

After submitting your sitemap, Google will be aware of your website's pages and periodically check for updates, ensuring their index remains current.

Adding Your Sitemap to Robots.txt ​

Robots.txt file allows crawlers (not only search engine crawlers!) to know which pages on your website should not be crawled.

You should also add your sitemap location to robots.txt as specified by Google's robots.txt guidelines.

To add your sitemap information to robots.txt in Webflow:

  1. Go to website settings and under SEO find the robots.txt section
  2. Add Sitemap: https://your-domain.com/sitemap.xml into the text field

Sitemap Best Practices for SEO ​

Here are a few tips and tricks to keep in mind.

Only Public-facing pages ​

Include only the pages you want indexed in your sitemap.

Exclude URLs that shouldn't be picked up by Google.

While search engines may still index pages not in the sitemap, it signals that you don't consider them important.

Avoid Duplicate Content ​

If you have landing pages or marketing campaign pages with direct or close content duplicates, you should never add them to your sitemap.

That's why most marketing sites having loads of similar campaigns should maintain their manual sitemap to keep it SEO-friendly.

Pro tip: if you have duplicate content like similar landing pages, you shouldn't add them to your sitemap. In addition to that, add a no-index tag and disallow crawling within the robots.txt file (after deindexing the site).

Define Localized Pages ​

If you're using Webflow's Localization, or Linguana, the sitemap will be taken care of automatically. Otherwise, add your hreflang tags manually.

ConsentPro Sitemap Requirements ​

ConsentPro uses your website's XML sitemap to discover and scan all pages for tracking providers, cookies, and compliance requirements. A properly configured sitemap is essential for accurate website scanning and consent management.

How ConsentPro Uses Your Sitemap ​

ConsentPro's scanning engine relies on your sitemap.xml file to:

  1. Discover all website pages - The scanner reads your sitemap to find all URLs that need to be analyzed
  2. Comprehensive tracking detection - Each page listed in your sitemap is scanned for tracking scripts, cookies, and third-party providers
  3. Compliance assessment - Pages are evaluated for GDPR, CCPA, and other privacy regulation requirements
  4. Configuration generation - Based on the scan results, ConsentPro generates appropriate consent banners and privacy configurations

Required Sitemap Format ​

Your sitemap must be accessible at one of these standard locations:

  • https://your-domain.com/sitemap.xml
  • Referenced in your robots.txt file with: Sitemap: https://your-domain.com/sitemap.xml

Common Issues and Solutions ​

Sitemap Not Found (404 Error) ​

Error Message: "Sitemap required - No sitemap.xml found at https://your-domain.com/sitemap.xml"

Causes:

  • Sitemap is not published or enabled in Webflow
  • Website is not published to the custom domain
  • Sitemap URL is incorrect or moved

Solutions:

  1. Enable Automatic Sitemap in Webflow:

    • Go to Website Settings → SEO → Sitemap
    • Turn on "Automatic Sitemap" option
    • Publish your website

  2. Verify Website Publication:

    • Ensure your website is published to your custom domain
    • Check that your domain is properly connected and SSL is active

  3. Check Sitemap Accessibility:

    • Visit https://your-domain.com/sitemap.xml directly in your browser
    • Ensure the sitemap loads without errors

Empty or Invalid Sitemap ​

Error Message: "No URLs found in sitemap - Sitemap exists but contains no valid URLs"

Causes:

  • All pages are set to "noindex" in Webflow
  • Sitemap contains only excluded or draft pages
  • Sitemap XML format is corrupted

Solutions:

  1. Review Page Settings:

    • Check individual page SEO settings in Webflow
    • Ensure important pages are not set to "noindex"
    • Verify pages are published (not drafts)

  2. Validate Sitemap Format:

    • Use online XML sitemap validators
    • Check for proper XML syntax and structure
    • Ensure URLs are properly formatted with full domains

Draft or Password-Protected Pages ​

Warning: "Contains draft pages - Some pages in your sitemap are not publicly accessible"

Impact:

  • Draft pages cannot be scanned for tracking providers
  • Password-protected pages may not be fully analyzed
  • This may result in incomplete consent configuration

Solutions:

  1. Publish All Important Pages:

    • Review your Webflow page statuses
    • Publish any draft pages that should be included in scanning
    • Remove draft pages from sitemap if they're not ready

  2. Handle Password-Protected Content:

    • Consider temporarily removing password protection for scanning
    • Use manual tracker configuration for protected pages
    • Schedule regular rescans after content updates

Optimizing Your Sitemap for ConsentPro ​

Include All Public Pages ​

Ensure your sitemap contains:

  • ✅ All published website pages
  • ✅ Landing pages with tracking scripts
  • ✅ E-commerce product pages
  • ✅ Blog posts and articles
  • ❌ Admin pages or internal tools
  • ❌ Test or development pages

Regular Sitemap Maintenance ​

  1. Update Frequency:

    • Regenerate sitemap when adding new pages
    • Remove outdated or deleted pages
    • Rescan your website monthly or after major updates

  2. Monitor Sitemap Health:

    • Check sitemap accessibility regularly
    • Validate XML format and structure
    • Ensure all URLs return 200 status codes

Troubleshooting Checklist ​

If ConsentPro cannot access your sitemap:

  • [ ] Verify website is published to custom domain
  • [ ] Check sitemap is enabled in Webflow SEO settings
  • [ ] Test sitemap URL directly in browser
  • [ ] Confirm no redirects or authentication blocks sitemap access
  • [ ] Ensure sitemap contains public, published pages only
  • [ ] Validate XML format using sitemap testing tools

Getting Help ​

If you continue experiencing sitemap-related issues:

  1. Check ConsentPro scan logs for detailed error messages
  2. Use browser developer tools to test sitemap accessibility
  3. Contact ConsentPro support with your domain and specific error messages
  4. Review Webflow documentation for sitemap configuration guidance

For additional assistance, reach out to our support team with your sitemap URL and any error messages you're encountering.