News crawler

[tab name=”Solution”]

News crawler

News crawler is Datacol-based module, which implements automatic news extraction from Dailymail website. Generally, data are saved to xlsx file (open in Excel to explore). You can also adjust Datacol export settings to publish data to website (WordPress, DLE, Joomla), database etc.

News extractor: data extraction results
Click image to enlarge

Main advantages of Datacol-based news crawler are listed below:

[/tab]
[tab name=”Test NOW!”]

Step by Step test of news crawler

To test news extractor:

1. Install Datacol trial version;
2. Choose news-parsers/news-crawler.par in the campaign tree and click Start button to launch news extractor campaign.

News crawler: starting data extraction
Click image to enlarge

Before launching news-parsers/news-crawler.par you can adjust the Input data. Select the campaign in the campaign tree for this purpose. In this way you can setup links to news categories you need to extract data from.

Please contact us if the news extractor will not collect data after you have made changes to the Starting URL list.

News scraper: setting Starting URL list
Click image to enlarge

3. Wait for data extraction results to appear. When you see the first results, you can force running campaign to stop (click Stop button).

News harvester: working process
Click image to enlarge

4. After campaign is finished/stopped you can find news-crawler.xlsx file in Documents folder.

News parser: data extraction results
Click image to enlarge

Datacol Trial VS Activated

Feature Trial License (Full version)
Preset default configuration for data extraction
50+
50+
Maximum data extraction results
Maximum 25
Unlimited
Free software updates
Free email tech support
Paid skype+teamviewer consultations
Paid setup

[spoiler show=”What if the news extractor is blocked (banned) by the source website?” hide=”What if the news extractor is blocked (banned) by the source website?”]
If the source website blocks your IP-address (after blocking you will get no more extraction results), use proxy.
[/spoiler]

[/tab]

[tab name=”Data processing and Export”]
Data processing options for data, harvested by news extractor:

Data export options for data, harvested by news extractor:

  • Basic: CSV/TXT/Database/Excel;
  • Online stores: Magento/PrestaShop/osCommerce/OpenCart/ZENCart/VirtueMart;
  • Content CMS: WordPress/Joomla/DLE;
  • All options.

[/tab]
[tab name=”Ask your question!”]
If you have any questions, related to news extractor, please ask via the contact form.
[/tab]

[end_tabset]

Scroll to Top