Easy Content Crawler

This plugin is free. Basic support and updates forever.

Overview

Crawls specified websites and automatically fetches selected content using XPath selectors for easy content population. Easy Content Crawler is a powerful yet simple tool for content automation. It allows you to define any website as a source and use precise XPath selectors to fetch links to articles, news, products, or any other structured content. Forget about manual copy-pasting. With this plugin, you can populate your site with fresh content automatically and efficiently.

Features

Easy Content Crawler best features:

Unlimited Sources: Monitor as many websites as you need.

Precise Targeting with XPath: Get exactly the data you want.

Automated Crawling (CRON): Set an interval (e.g., every 12 hours), and the plugin will handle everything on its own.

Flexible Post Creation: Create drafts or publish posts, pages, or any custom post type directly.

Automatic Category Assignment: You can set a default category for each post creation target.

Detailed Help Section: The integrated help page guides you in getting the right XPath, even if you’re a beginner.

PREMIUM Modules

Activate your premium modules with a license key to unlock powerful features.

Detail Content Crawler: Don’t just fetch links—get the full content from the target pages! You can choose whether to insert the content with its original HTML formatting or as clean, plain text.

Source Display (Shortcode): Want to elegantly show your readers where you source your content? With the `[easy_content_crawler_sources]` shortcode, you can display a stylish list of your active sources on any page, styled like Google search results.

Screenshots

What does this plugin look like? Check out detailed screenshots directly from the plugin.

Show all screenshots

FAQ - Easy Content Crawler

We answer your most frequently asked questions.

XPath is a query language for selecting nodes from an HTML or XML document. It's a way to tell the plugin, "On this page, I'm interested in all links that are inside an H2 heading." The easiest way to get it is to right-click on the desired element on a webpage, select "Inspect," and find a common attribute in the code (e.g., `class="article-title"`). A detailed guide is available directly in the plugin's 'Help' section, including tips on how to generate XPath using AI.

First, check the crawl status on the 'Sources' page. If the status is 'Error' or 'Empty', try the following: * **Verify the XPath selector:** This is the most likely cause of the issue. Try simplifying it or using a different one. * **Check the URL:** Make sure the source URL is correct and accessible. * **The target website might be blocking crawlers:** Some websites have protections against automated scraping. In such cases, the plugin may not work correctly.

The plugin uses WordPress's internal system called WP-Cron. You can enable it in 'Settings' and choose an interval in hours. WP-Cron is triggered only when someone visits your site. This means that on websites with very low traffic, there might be delays. For 100% reliability, we recommend setting up a server-side CRON job.

After purchasing a premium module from our website, you will receive a license key. Go to **Content Crawler > Licenses** in your WordPress admin area, enter the key in the corresponding field for the module, and click "Activate License".

hanks to XPath technology, the plugin is extremely flexible. You can use it to fetch any kind of structured data from websites. The goal is always to get a list of links that lead to a detail page (an article, product, event, etc.). The premium "Detail Content Crawler" module can then also fetch the full content from these detail pages.

News and Article Aggregator, Job Offer Tracking, Affiliate Marketing and Product Reviews, Real Estate or Car Listings, Recipe Database, Event and Action Overviews, Deal and Coupon Aggregation or Building Business Directories. Basically, if the data on a page is displayed in a repeating list (where each item has the same HTML structure), you can track and import it with this plugin.

Yes, there are situations where the crawler may not be able to fetch content. Most of these are related to how the target website is built and protected. Here are the most common reasons why the crawler might fail: Content Loaded with JavaScript, Anti-Bot Protection (Cloudflare, Firewalls), Login Required, Very Complex or Unstable HTML Structure, IP Address Blocking.

Version:

2.4.0

Last update:

12. Sep 2025

Tested Upto:

WordPress 6.2

Compability:

WordPress 6+

Size:

161 kB