Service for tracking changes on a page with a password. Alternatives to replace Page Monitor

Habrahabr, hello!

Have you ever had the need to track the appearance of some information on a website, especially if there is no newsletter or RSS feed!? Did you have to wait for exam results to be published, new vacancies, tenders, and price lists to be updated?

In our company, which spends most of its time in pension and insurance consulting, such a need arises constantly. At the same time, it’s sad to waste time on regularly visiting sites, and it was impossible to find a suitable tool that could be entrusted with this kind of task: either it was necessary to install separate program, or the display of the result was not satisfactory. Therefore, it was decided to create a service that would allow tracking all changes on sites. And not only track and receive notifications about changes, but also see the changes themselves directly on the site.

The result of the development was the service Websvodka.ru, which I’ll talk about briefly here. Perhaps it can be useful to you too.

Main functions Websvodka.ru:
- Track and display changes on web pages.
- Control of adding and deleting website pages (hyperlinks on existing pages are monitored).
- Notification about changes in results for search queries.
- Tracking the size of files posted on the Internet.

1. Changes on the page
After placing a page under control, you can see all the changes happening to it. In the screenshot below, the added phrase is highlighted in green, and the deleted phrase is highlighted in red. Controlled page rbc.ru:

Changes to habrahabr.ru/posts/software/:

In addition, the service can be tasked with tracking instances of appearance or disappearance on the page. individual words and phrases.

2. Site monitoring
This is what the report on adding and deleting pages on the site looks like: ixbt.com:

Pages that interest you can be opened in a new tab by clicking on the eye image, or added to the list of controlled pages by clicking on the checkbox. They will automatically appear in the appropriate section.

3. Search queries
The “Add” button is pressed, the query of interest is entered, and the service will show retired and new search results, as well as changes in their positions in the search results.

4. File sizes
You can add files for monitoring and receive tracking results in the same way as for other types of content. A link to the file is specified, and the service will inform about changes in its size, the dates of changes are recorded.

The four options considered for tracking changes in website content reflect the main existing functionality of the service. Websvodka.ru. To increase reliability and speed of operation, modules for loading content, analyzing pages and storing results operate in parallel and are located on different servers. Work to expand capabilities continues.

We will be glad if the service helps you solve your problems!

P.S. How it works:
After registering on Websvodka.ru and login (to simply get acquainted with the functionality, you don’t have to register, but select the “Demo” mode, but then your actions on the site will not be saved), go to the “Desktop” section, select the desired category, click the “Add” button and enter the address of what you want to monitor. Further Websvodka.ru regularly checks the object of interest and notifies you about its changes (you can set up notifications by e-mail). Using the basic functionality is free. The changes themselves can be seen by entering the desired category of the “Desktop” section.

We often encounter that a client updates a website and does not tell us about it. Of course, he seems to have to inform us, but on the other hand, everything is supposedly already on the website, go and look. And here the question arises: how to instantly learn about changes on the sites of all your projects at once? Luckily, there is a simple solution.

It's worth starting by understanding what exactly we want to track: this could be a price list that is updated online as soon as it is added new product, or it’s just a list of categories to keep abreast of new sections on the client’s website - whatever. For example, let's take any store that is not one of our clients and that has a price list. Let it be a children's goods store. Just as shown in the example below, you can select and set up change tracking for a list of site categories, for products separate section, for blog updates, for comments, and so on.

Setting up email notifications about website updates using Kimono

We follow the points:

1. Install the Chrome app - Kimono.
2. Go to the price list page and click on the Kimono icon on the right top corner browser.
3. Click on any product name from the list. Additionally, click the checkmarks on other products to highlight them all. Since it is the change in the list of goods that is important to us, we may not highlight prices. 4. Scroll down the page and check that all fields are highlighted. If not, highlight it additionally.
   5. Click " done" in the upper right corner.
   6. Enter the API name and follow the link. For example, let's call it “PriceUpdate”.
   
   7. In the Kimono interface, go to the Crawl Setup tab and select the API update frequency. It is enough to check the changes once a day.
   8. Set up email notifications. To do this, go to the Use Data tab and click Email Alerts.
   9. Enter your email. The letter will only come if something has changed in the price list. Please note that you can enter multiple addresses Email. For example, indicate the email addresses of all project participants who should know about changes to the site.
   An email from Kimono will look like this:
   Congratulations! Kimono will now notify you if there are changes to the items being tracked on your site. If we're talking about about a site with five categories and 20 products, one notification may be enough for you. But if we are talking about an online store with a huge amount products and categories, changes here cannot be determined “by eye”, and a comparison of data is needed.

How to determine what exactly has changed on the site: comparing data in Google SpreadSheets

The first thing you need to do is install the Kimono add-on for Google Spreadsheets. To do this, go to Add-ons → Install add-ons in the table and enter “Kimono” in the search field.
Kimono installed. What's next? Follow: Add-ons → Kimono → Connect to Kimono and select API.
2. Data from the API will be downloaded to separate sheet. We create a copy of this sheet and rename it, for example, to “0”. This is our starting point - it is this data that we will compare with updated data on the site.
3. On the first sheet we place a formula for comparing data from sheet “0” and sheet “Kimono_PriceUpdate_collection1”. There can be three situations here:

• The URL has changed, but the name remains the same - for example, if a problem has arisen with the site’s CMS and unnecessary GET parameters have been added to the URL;
• The name has changed, but not the URL - the category or product has been renamed;
• both have changed, that is, it is a new product or a new category.

We are interested in all three situations, so we will take them all into account in the product comparison formula.
The formula itself and how everything looks can be viewed at the link. To have editing rights to the document, select File → Make a Copy. You can get acquainted with the formulas for Google SpreadSheets.

I received a notification by email. What's next?

If you receive a notification from Kimono that a change has been detected on your site, you need to compare the new API with the old data. To do this, you go into the table to the data sheet from Kimono (in our case, to the “Kimono_PriceUpdate_collection1” sheet, not to “0”!) and update the data - like this:

Having detailed data on all products, you can take action necessary actions— provide technical specifications for correcting the URL if errors appear in the CMS, or expand semantic core for new products or categories that have appeared on the site, or add new products to the current one advertising campaign. The main thing is, do not forget to copy the changed data to sheet “0”, since now they become the starting point.

Distill is a web monitoring tool. It can track RSS feeds, web page or part of a web page. Alerts may be received in the form of pop-ups, audio or email.

Free (with restrictions) Mac Windows Linux Android Chrome Opera Firefox

What's on this list?

The list contains programs that can be used to replace Page Monitor on the Internet, Chrome, and Chrome OS platforms. This list contains 12 apps similar to Page Monitor.

If we consider the reasons why sites suddenly begin to lose traffic, they can be divided into 3 main groups: algorithmic, internal and external. The first includes changing algorithms search engines, which we cannot influence. We influence all other reasons, consciously or unconsciously. And if with conscious influence everything is more or less clear, it all depends on the degree of awareness and sanity of the optimizer, then a serious problem is not knowing what is happening with the site.

One example: my abandoned the site was hacked and posted malicious code, later confirmed the rights to the domain in panels for webmasters and put a bunch of spam pages into the index. I only found out about this from the panel notification when Google sent the message “you have granted access to the site...”. It is clear that traffic from search to the site disappeared due to a decrease in trust. If I could quickly track changes in the page code, I would react more quickly. In my case, the site was abandoned, but many live sites experience similar hacking situations and the business can suffer a lot of damage.

There are many other examples when a team is working on the site and one person on the team does not always know what the other is doing. And it’s even more interesting when all changes in the code are overwritten after CMS updates . This happened to my friends who couldn’t release for more than a year new version site, since closed tasks were re-opened after regular updates and edits (they did one thing, another broke and vice versa). And it’s good here when you can find out all this in test mode, but what to do when you have already rolled out the project to the public and a week later you automatically updated the CMS or some plugin for security reasons?

The answer is obvious - it is necessary to monitor changes on the site. This is especially true for big projects, and especially where several people work on projects. Although, I recommend everyone to do this. At least occasionally, to insure yourself against various accidents. With the help of monitoring, you will be able to respond in a timely manner to unnecessary (or failed) changes on the site, plus you will be able to better understand the reasons for sudden changes in the positions of the site or its individual pages.

What exactly to monitor?

Ideally, you need to monitor everything :) but there are many nuances for sites with dynamic content, so it’s better to concentrate on important areas:

• TITLE and H1 tag
• Meta description tag
• Dedicated content areas
• HTTP status code
• Robots.txt file

It will also be useful to monitor the meta robots tag and canonical links (rel canonical).

Can this be automated and how?

Constantly tracking changes manually is a thankless task. Monitoring needs to be automated, and you need to look at reports only when one of the monitored parameters changes.

There are several tools available to automate this process. One of them is fresh, from the Mail.ru company - SeoSan. The advantages are that the service is free, and in addition to meta tags, it tracks changes to the robots.txt file. The downsides are that there are restrictions on the number of pages (the maximum you can add is 50 pages), and you can’t monitor sites that you can’t verify ownership of. I would like to track changes and competitors as well.

Do you want to know when prices will drop on shopping sites? When someone sent a new message to the forum? When your favorite update site doesn't offer a channel? All these use cases and more are a cry for programs that can control website changes.

A program that controls site updates? She obviously has to check the site at certain time intervals and compare active state With previous state Online. Site updates must be sent to the user's computer. It's nice to have capabilities for site monitoring, highlighting changes to websites, the ability to ignore minor site changes (such as date and time that change every minute), or logging functionality.
Below is a list of five tools for monitoring websites. Three of these Firefox add-on tools, one online service latest software for operating system Windows.

Check4Change Firefox add-on changing site behavior. These are probably the most basic website monitoring tools covered in this article. The user can select text on websites and open context menu Check 4 Change on click right click mice. It is there that you can determine the interval for monitoring website changes. Website updates will be displayed in the tab of the site that is under control. This is one of the biggest weaknesses of this expansion. The site must be open to be controlled by the Firefox add-on. Several add-on options are provided. You can change the intervals and configure several options that include notification sounds, pop-ups, or bring window to foreground. The supplement is best used for website monitoring that require authentication. For example, the add-on may be interesting for monitoring web email accounts.
Update Scanner

Update Scanner takes a different approach than Check 4 Change. The program automatically monitors the entire website for changes with the ability to ignore numbers and minor changes. Another benefit of using this site monitoring is that sites that are monitored do not have to be opened in order to perform the update check.
Check interval for site changes has a value of once per day by default. This value can be changed to a minimum of every 5 minutes. The add-on has an icon in the Firefox status bar. Left click on this icon to open side panel, which displays all the sites that are controlled by the add-on. The sidebar contains buttons to perform manual check, open one of website monitoring and access to add-on preferences, which allow the user to customize notifications.

Available sounds and pop-up notifications.

SiteDelta the third (and final) Firefox add-on for website monitoring. The add-on will seem a little confusing at first, since there is no obvious way to add sites that need to be verified. The add-on begins monitoring website changes if the user clicks an icon in the Firefox status bar. The active URLs will then be added to the list of pages that are controlled by the add-on. The icon will glow green, if the active site is controlled by it. Right-clicking opens a context menu that can be used to configure monitoring in detail.
Users can customize scanning parameters in two tabs. The add-on will by default check for text and graphic changes. Alternatively, you can ignore case or numbers and create backup copy pages when changes are acknowledged.

When used for scanning tab for setting scanning intervals. You can set a specific scanning frequency or let the add-on select the frequency itself. The add-on uses an algorithm that checks the controlled site for update frequency.

Change Detection

ChangeDetection it's free online service who can control website changes. The website URL and email address are provided as the first step in this process. The service will try scan website. If the scan is successful, step 2 is displayed.

At the second stage, we configure notifications and type of changes. Notifications will be provided by sending a letter to an electronic account or by providing a channel that is updated with changes to the site. Service online monitoring,can be configured to only send notifications if significant changes ,have occurred while monitoring websites. You can also define text that should trigger notifications (or if they are added or removed from the page that may be of interest for link building purposes).
WebMon

WebMonis the only software V this review, which can be used for website monitoring. Software website monitoring has several advantages over Firefox add-ons. The program can run in background, even if all web browser windows are closed. Firefox add-ons can check for site updates if the web browser is running.
The program comes with a simple in use of the interface. She starts work with adding URL to the program. Any URL that is currently buffered Windows Exchange, will be automatically inserted into the form right now. This makes it possible to configure usage check intervals of minutes, hours and days in the interface. Process full check just on the page that needs to be checked. Users who want to define specific text or elements on a website can do so by clicking on the “define content to review” button in the configuration window.

This basically allows the user indicate the range of content on the site that should be included in the program. Software website monitoring notifies the user with a pop-up alert or sounds. The program may start if the update command is not found.
Software security website monitoring has a lot additional options provided that proxy server support, registration or import and export of the list of monitored sites is enabled.
Verdict
Software For website monitoring, add-ons or services depends greatly on the use case. Users who work in Firefox web browser all the time may prefer additions for monitoring website changes. The online service, on the other hand, will control sites remotely so that computer system The user does not need to be turned on always to monitor changes (you can even send notifications to a smartphone, for example).
Software website monitoring, on the other hand, is the most flexible tool that provides the ability to execute commands if changes are detected.
And what do you prefer?