As an SEO Company, we often get this question, and the answer can be a bit tricky because duplicate content can exist not only on your website but other websites as well. For instance, if your content matches too closely between pages of your website (most say 90% match), then it can be flagged for duplicate content. Another example would be viral or syndicated blog posts; the original content lives on your website. However, other bloggers and websites may have posted it to their websites as well essentially creating duplicate content. These are just a couple of examples, and surely there are more. The reason you’re probably researching duplicate content is that you received a red flag for duplicate content.
Google Recommendations Regarding Duplicate Content
Google has provided SEOs some steps to address duplicate content issues to ensure visitors see the content you want them to see:
- Use 301 redirects: If you’ve redesigned your website, you can use 301 redirects in your .htaccess file to redirect visitors, Googlebot, and other spiders. This will prevent broken links and help you keep your SEO link Juice.
- Be consistent: Keep internal linking consistent. For example, don’t link to http://www.example.com/page/ and http://www.example.com/page
- Use top-level domains: To help serve the most appropriate version.
- Syndicate carefully: Google will always show the version they think is the most appropriate for users given each search, which may or may not is the version you prefer. You can ask those who use your content to use the noindex meta tag to prevent search engines from indexing their version.
- Use Search Console to index your preferred site: You can tell Google your preferred domain (http://www.example.com or http://example.com).
- Minimize boilerplate repetition: Instead of including lengthy copyright content on the bottom of every page, include a summary and then link to a page with more details.
- Avoid publishing stubs: Users don’t like seeing “empty” pages, so avoid placeholders where possible. For example, don’t publish pages for which you don’t yet have real content. If you create a placeholder pages, use the noindex meta tag to block these pages from being indexed.
- Understand your Content Management System (CMS): Make sure to familiarize yourself with how content is display on your website. Blogs, forums, and related systems often show the same content in multiple formats. For instance, a blog entry may appear on the home page of the blog, in an archive page, and in a page of other entries with the same label.
- Minimize similar content: if you have many pages that are similar, consider expanding each page or consolidating the pages into one.
- Rel=”canonical” link element: Helps identify duplicate content and point search engines to the original piece of content.
Do Not Block Crawler Access
One approach Google does not recommend is blocking crawler access to duplicate content on your website. If search engines can’t crawl pages, they can’t automatically detect that these URLs point to the same content and will, therefore, treat them as separate pages.
Regardless, duplicate content doesn’t usually result in a penalty; unless, it is deemed to be deceptive and manipulates search engine results. If Google’s review indicates your website has engaged in deceptive practices, they will remove your website from the search results; until it no longer violates their guidelines.
If you’re having issues with duplicate content penalties, an SEO Agency like Actuate Media might be able to help you.