Popular search engines on the Internet. Internet search engines: an overview of existing solutions
We present to your attention a new course from the team The Codeby- "Web Application Penetration Testing from Scratch". General Theory, Working Environment Preparation, Passive Fuzzing and Fingerprinting, Active Fuzzing, Vulnerabilities, Post-Exploitation, Tools, Social Engineering and more.
For me, the most surprising thing about the Yandex search engine is that a huge number of people still use it. My acquaintance with the Internet, like most at that time, began through the Yandex search engine. There were many popular search engines, including Russian ones, at that time. I searched for what I needed using different search engines, and for quite a long time my empirical experience confirmed that if Yandex did not find something, then others certainly would not find anything useful for this query. I also liked the slogan of that Yandex that “Yandex is not the censor of the Internet, but its mirror.” It meant that Yandex does not filter the issue, even if some of its results are not liked by someone.
At that time, it was common to go up to 7 and up to 11 pages of issue, etc. either the Runet was still small, or the search engines worked so-so. It is now the vast majority rarely for any query that enters at least the second page of search results.
Some time later, I noticed that Google was catching up with Yandex in terms of the quality of search results. But I continued to use Yandex even when I realized that Google had become better. It's all about the conservatism inherent in most people: I had mail on Yandex, and in general, people have some fears about changes. In general, I decided and completely switched to Google search, and after a couple of weeks my fears about the transition began to seem naive and ridiculous.
Sometimes I come back, “test” the Yandex search results and only once again make sure how right I was by stopping using it. Runet, through the eyes of Yandex, is somehow shabby, incomplete, with some kind of skewed priorities.
Some time ago, it's hard to say exactly when, but Google has ceased to cause a feeling of complete satisfaction. Those. In the vast majority, Google is perfect and gives exactly what you need in the very first positions. But when searching, for example, for movies or something else counterfeit, Google censors the results. Google directly reports this right on the issuance page. This, as well as the constant "Google is watching users" has always pushed for experiments with other search engines.
I began my note by saying that most of us are very conservative and very reluctant to change our habits. To date, judging by the number of clicks to my sites, there are very few users of duckduckgo.com. Someone simply does not know about its existence, someone tried it before and did not like it. It's time to try something new. Everyone who wants to look at the Internet (Runet) with a clean, uncluttered censorship or delusional “ranking” look, I invite you to try the search engine
The best search engines of the Internet. Internet search engine – these are special search programs installed on a whole complex of specialized machines. And in a simple way - this is the same site with a set of programs, only on a special search engine (server). It is with the help of search engines that you find all the information you need. There are a lot of search engines.
1. What is an Internet search engine
2. Popular search engines of our country
3. Popular search engines abroad
4. Unusual search engines
5. How to search for information on the Internet
Most best nsearch systems in our country:
http://www.yandex.ru
http://www.google.com
http://www.aport.ru
http://www.rambler.ru/
http://go.mail.ru
http://www.webalta.ru/
The most unloved and intrusive search engine.
Popular foreign search engines
http://www.altavista.com
http://www.alltheweb.com
http://www. bing.com
http://www.google.com
http://www.excite.com
http://www.lycos.com
http://www.mamma.com
http://www.yahoo.com
http://www.dmoz.com
http://www.hotbot.com
http://www.dogpile.com
http://www.netscape.com
http://www.msn.com
http://www.webcrawler.com
http://www.jayde.com
http://www.aol.com
http://www.euroseek.com
http://www.teoma.com
http://www.about.com
http://www.ixquick.com
http://www.lookle.com
http://www.metaeureka.com
http://www.searchspot.com
http://www.slider.com
http://www.allthesites.com
http://www.clickey.com
http://www.galaxy.com
http://brainysearch.com
http://www.orura.com
Each country has its own popular search engines.
Fancy Search Engines
- DuckDuckGo (https://duckduckgo.com/) - a hybrid search engine with a privacy policy for the user and his search queries.
- TinEye (http://tineye.com/) - a search engine that specializes in finding images on the Internet. Recently, it has lost its relevance, after Google introduced the same feature in its image search.
- Guénon (http://www.genon.ru/) is a search engine that collects and creates content on its website.
In almost every search engine, in addition to the search box, there are links on the most popular news sites, and sites of a certain subject.
How to search for information on the Internet
Each search engine has its own algorithms (rules) for information retrieval.
In order to find some information on the Internet through a search engine, you need to enter in the search field request. If you enter any one word, then you will be given thousands of links to sites in which this word is mentioned on this request.
Therefore, it is necessary to enter as specific a query as possible, consisting of two, three or more phrases.
Let's look at an example query in a search engine Yandex.
Let's say you want to find information about buying a computer. If you write in the search box one word " A computer”, then you will be given 133 million answers
You need to ask a more specific request. It is better to indicate which computer you want to buy and where (in which city).
Then the search engine will give you much fewer answers to your query.
The search engine does not care whether you enter a query in large or small letters.
Yandex distinguishes between nouns and adjectives, but completely ignores endings.
Also, he is completely indifferent to cases, plurals and the like.
In order for the search to be more accurate, you must enclose the query in quotation marks or put an exclamation point before the word.
Now look at the same query, but without the exclamation points.
See the difference? With exclamation points, the number of answers is not 2 million, but 186 thousand.
If you put an exclamation point before a word with a capital letter, then you will be given answers in which exactly this word with a capital letter occurs.
If the word is in the nominative case, and you need information on exactly that word, and exactly the way you wrote it, then put two exclamation marks before this word. For example: !!Ball .
The search will give you answers for this word " Ball' the way you wrote it. Not " ball", not " balls", and with a capital letter.
If you write a phrase with the word " on the", then Yandex will ignore " on the". For example: " on the shelf". The search will be performed only on the word " a shelf ».
In order for him to take it into account and not ignore it, it is necessary before the word " on the" put a plus sign - " + on ».
Each search engine has its own search algorithm, so if you use a particular search engine and want to learn how to correctly compose queries, then you just need to type in the search bar " search rules inGoogle " or " search rules in Yandex ”, follow the link of the answer to your request and read the necessary information.
A search engine is a database of specific information on the Internet. Many users believe that as soon as they enter a query into a search engine, the entire Internet is immediately crawled, but this is not at all the case. Internet scanning occurs constantly, many programs, data about sites are entered into a database, where, according to certain criteria, all sites and all their pages are distributed into various lists and databases. That is, it is a kind of data file, and the search takes place not on the Internet, but on this file.
Popular search engines
Yandex is the largest search engine in Runet.
In addition to the search engine, Yandex offers 77 additional services, the most popular of which are the Yandex mail service, Yandex browser, Yandex disk, traffic and weather information, Yandex money, and much more. The search engine considers your location when providing search results. Also, the search program is constantly being upgraded to provide more correct results, designed for the greatest information content for the user.
Google is the most popular search engine in the world.
In addition to the search engine, Google offers many additional services, software and hardware, including the mail service, the Google Chrome browser, the largest youtube video library and many other projects. Google is confidently buying up many projects that bring large profits. Most of the services are not aimed at a direct user, but at making money on the Internet and are integrated with a focus on the interests of European and American users.
Mail is a search engine popular mainly because of the mail service.
There are many additional services, the key of which is mail Mail, at the moment Mail owns the Odnoklassniki social network, its own My World network, the Money-mail service, many online games, three almost identical browsers with different names. All applications and services have a lot of advertising content. The social network "VKonatkte" blocks direct transitions to Mail services, arguing with a large number of viruses.
Wikipedia.
Wikipedia is a searchable reference system.
A non-profit search engine that exists on private donations, therefore it does not fill the pages with advertising. A multilingual project whose goal is to create a complete reference encyclopedia in all languages of the world. It has no specific authors, is filled in and managed by volunteers from all over the world. Each user can both write and edit an article.
The official page is www.wikipedia.org.
Youtube is the largest video library.
Video hosting with elements of a social network, where each user can add a video. From the moment they were acquired by Google Ink, a separate registration for YouTube is not required, it is enough to register in the Google mail service.
The official page is youtube.com.
Yahoo! is the second most important search engine in the world.
There are additional services, the most famous of which is Yahoo mail. As part of improving the quality of the search engine, Yahoo transmits data about users and their queries to Microsoft. From these data, an idea of the interests of users is formed, as well as a market for advertising content. The Yahoo search engine, as well as, is engaged in the absorption of other companies, for example, Yahoo owns the Altavista search service and the Alibaba e-commerce site.
The official page is www.yahoo.com.
WDL is a digital library.
The library collects books of cultural value in digital form. The main goal is to increase the level of cultural content of the Internet. Access to the library is free.
The official page is www.wdl.org/ru/.
Bing is a search engine from Microsoft.
The official website is www.baidu.com.
Search engines in Russia
Rambler is a "pro-American" search engine.
It was originally created as a media Internet portal. Like many other search engines, it has image search services, video files, maps, weather forecast, news section and much more. Publishers also offer a free browser Rambler-Nichrome.
The official page is www.rambler.ru.
Nigma is an intelligent search engine.
A more convenient search engine due to the presence of many filters and settings. The interface allows you to include or exclude suggested similar values in the search to get better results. Also, when receiving a search result, it allows you to use information from other major search engines.
The official page is www.nigma.ru.
Aport - online catalog of goods.
In the past, the search engine, but after the fact that developments and innovations were discontinued, quickly lost ground and . At the moment, Aport is a trading platform, where goods from more than 1500 companies are presented.
The official page is www.aport.ru.
Sputnik is a national search engine and Internet portal.
Created by Rostelecom. It is currently in the testing phase.
The official website is www.sputnik.ru.
Metabot is a developing search engine.
The tasks of Metabot is to create a search engine for all other search engines, creating positions for issuing results, taking into account the data of the entire list of search engines. That is, it is a search engine for search engines.
The official page is www.metabot.ru.
The search engine has been suspended.
The official page is www.turtle.ru.
KM - multiportal.
Initially, the site was a multi-portal with the subsequent introduction of a search engine. The search can be carried out both within the site and on all tracked Runet sites.
The official page is www.km.ru.
Gogo - does not work, redirects to a search engine.
The official page is www.gogo.ru.
The Russian multiportal, which is not very popular, needs to be improved. The search engine includes news, TV, games, map.
The official page is www.zoneru.org.
The search engine does not work, the developers suggest using the search engine.
The official page is www.au.ru.
Let's continue the master class on anonymous search engines that do not harass the user.
everything is clear on Google, on Russian bottling products too, so we are discussing really serious resources
great article on fresh data, just copying.
DuckDuckGo
DuckDuckGo was once billed as an "anti-spyware" alternative to Google, but even a quick analysis with utilities like lightbeam (an extension for Mozilla Firefox) shows that it doesn't respect your privacy all that much. The reason for this is his business model, which relies heavily on commissions from partners. That is, it is not a fact that he manages to earn money without spying on users.
In addition, DuckDuckGo uses Amazon cloud services as its infrastructure. Although this decision is technically justified, it still raises questions given that Amazon is the main provider of cloud services for the CIA and has already obeyed the order of the US government (in violation of the first amendment) to remove Wikileaks from the network ... In short, with DuckDuckGo everything is far from being as clean as they want to show you.
Ixquick is an alternative that keeps your privacy out of the way and offers a DuckDuckGo-like interface. It was inspired by Google, but it is not a search engine in the literal sense of the word, but an aggregator of results from other search engines. This approach, of course, is interesting, but it can hardly be called a real alternative. It relies on other search engines and does nothing to create a Google-free world.
In addition, Qwant, a “made in France” search engine, is among the systems that do not seek to stick their nose into your personal life. Unlike Ixquick, it is a real search engine and uses its own search technology. However, even six months ago, he valued privacy hardly more than DuckDuckGo and even used Google Analytics to measure traffic. And this, you see, is a very controversial move if you want to protect the privacy of users.
But after people drew their attention to this issue (your obedient servant, as well as many hackers, had a hand in this, by the way), they nevertheless corrected themselves and installed an alternative to Google Analytics with anonymous proxy servers (they become intermediaries in user interaction with the Internet), which allowed everyone to search the Internet and social networks without transferring any data to Facebook and Twitter.
Like DuckDuckgo, Qwant seeks to establish itself in the growing niche of privacy-friendly resources and takes the technical side of the issue seriously. To do this, they set a proxy (as a result, users can make all requests completely anonymously), introduce an autonomous and independent of Google traffic evaluation system and use logs (the information that remains on the server after visiting) that do not associate your IP with search queries in any way. . It turns out that even if the authorities confiscate the equipment, there is no way they will be able to restore your search history by IP address. Some hackers have criticized Qwant for not being too precise in protecting users' personal data, representatives of the search engine have established ties with them and are actively developing cooperation to improve their existing technologies. In short, the efforts they have made show a serious attitude to the privacy of users. And this is the right approach, because in this area there is no way Google can outshine Qwant, and DuckDuckGo did not take things seriously and in the end will pay dearly for it.
Finally, among the alternatives to Google, it is worth noting the technology developed by YaCy, a decentralized search engine. It is not for everyone, but in terms of infrastructure, we see a really innovative approach. The downside of a decentralized system is that its effectiveness depends on the number of users (such as the speed of downloading files via P2P protocol), in contrast to the centralized systems mentioned above. So far, this technology, of course, is interesting, but remains the lot of "nerds". Be that as it may, it is quite possible to imagine a re-decentralization of the Internet, in which such a technological approach would take on a completely different meaning. In general, this is the only way out of the society of total control that is being formed before our eyes, in connection with which YaCy deserves our support in any case.
Google is a hard drug?
However, it will be very difficult for a mere mortal to make such a transition, because everyone has long and irrevocably been accustomed to Google. Personally, I try to use another search engine more often in parallel with Google, just like I puff on an electronic cigarette in order to smoke less real ones. When I need to find some "sensitive" information that is somehow related to my journalistic investigations, I usually use only Qwant and Ixquick, or I use a whole technological arsenal to achieve complete anonymity on the network. But, as you understand, it is not available to the average user.
Fabrice Epelboin is an entrepreneur and lecturer at the Institute for Political Studies.
Original post: Les alternatives pour ceux qui veulent se passer de Google
Hello, dear readers of the blog site. , then its few users had enough of their own bookmarks. However, as you remember, it happened exponentially, and very soon it became more difficult to navigate in all its diversity.
Then directories appeared (Yahu, Dmoz and others), in which their authors added and sorted into categories various sites. This immediately made life easier for the then, not yet very numerous users of the global network. Many of these directories are still alive today.
But after a while, the size of their databases became so large that the developers first thought about creating a search inside them, and then about creating an automated indexing system for all Internet content to make it available to everyone.
The main search engines of the Russian-speaking segment of the Internet
As you can imagine, this idea was realized with resounding success, but, however, everything turned out well only for a handful of selected companies that managed to not disappear on the Internet. Almost all the search engines that appeared on the first wave have now either disappeared, or vegetate, or were bought by more successful competitors.
The search system is a very complex and, importantly, a very resource-intensive mechanism (meaning not only material resources, but also human ones). Behind the outwardly simple, or its ascetic counterpart from Google, there are thousands of employees, hundreds of thousands of servers, and many billions of investments that are necessary for this colossus to continue to work and remain competitive.
Entering this market now and starting everything from scratch is more of a utopia than a real business project. For example, one of the richest corporations in the world, Microsoft, has been trying to gain a foothold in the search market for decades, and only now their Bing search engine is slowly starting to justify their expectations. And before that there was a whole series of failures and failures.
What can we say about entering this market without any special financial influences. For example, our domestic search engine Nigma has a lot of useful and innovative things in its arsenal, but their attendance is thousands of times inferior to the leaders of the Russian market. For example, take a look at the Yandex daily audience:
In this regard, we can assume that the list of the main (best and most successful) search engines of the Runet and the entire Internet has already been formed, and the whole intrigue lies only in who will eventually devour whom, well, or how their percentage will be distributed if they all survive and stay afloat.
Russian search engine market is very well visible, and here, probably, two or three main players and a couple of secondary ones can be distinguished. In general, a rather unique situation has developed in Runet, which, as I understand it, has been repeated only in two more countries in the world.
I'm talking about the fact that the Google search engine, having come to Russia in 2004, has not yet been able to take the lead. In fact, they tried to buy Yandex around this period, but something didn’t work out there, and now “our Russia”, together with the Czech Republic and China, are the places where the almighty Google, if not defeated, then, in any case, met with serious resistance.
In fact, see the current state of affairs among best search engines in runet anyone can. It will be enough to paste this URL into the address bar of your browser:
http://www.liveinternet.ru/stat/ru/searches.html?period=month;total=yes
The fact is that most of them use .
After entering the given Url, you will see a picture that is not very attractive and presentable, but it reflects the essence of the matter well. Pay attention to the top five search engines from which sites in Russian receive traffic:
Yes, of course, not all resources with Russian-language content are hosted in this zone. There are also SU, and the Russian Federation, and common areas such as COM or NET are full of Internet projects focused on Runet, but still, the sample is quite representative.
This dependency can be drawn in a more colorful way, as, for example, someone online did for his presentation:
It doesn't change the essence. There are a couple of leaders and a few very, very far behind search engines. By the way, I have already written about many of them. Sometimes it is quite entertaining to plunge into the history of success or, conversely, delve into the reasons for the failures of once promising search engines.
So, in order of importance for Russia and the Runet as a whole, I will list them and give them brief characteristics:
Google search has already become a household name for many inhabitants of the planet - you can read about that at the link. I liked the “results translation” option in this search engine, when you received answers from all over the world, but in your own language, but now, unfortunately, it is not available (at least on google.ru).
Also, recently I have been puzzled by the quality of their issuance (Search Engine Result Page). Personally, I always first use the search engine of the Runet mirror (there is, well, I got used to it) and only if I don’t find an intelligible answer there, I turn to Google.
Usually their issuance made me happy, but lately it has only puzzled me - sometimes such nonsense comes out. It is possible that their struggle to increase their PPC revenue and their constant shuffling of SERPs to discredit Seo promotion could backfire. In any case, this search engine has a competitor in RuNet, and even what one.
I think that hardly anyone will specifically go to Go.mail.ru to search in Runet. Therefore, traffic on entertainment projects from this search engine can be significantly more than ten percent. Owners of such projects should pay attention to this system.
However, apart from the prominent leaders in the market of search engines in the Russian-speaking segment of the Internet, there are several other players whose share is quite low, but nevertheless the very fact of their existence makes me say a few words about them.
Runet search engines from the second echelon
Internet-wide search engines
By and large, on the scale of the entire Internet, there is only one serious player - Google. This is the undisputed leader, but he still has some competition.
First, it's still the same bing, which, for example, has very good positions in the US market, especially considering that its engine is also used on all Yahoo services (almost a third of the entire US search market).
Well, and secondly, due to the huge share that users from China make up in the total number of Internet users, their main search engine called Baidu wedged into the distribution of places on the world Olympus. He was born in 2000 and now his share is about 80% of the entire national audience in China.
It is difficult to say something more intelligible about Baidu, but on the Internet there are judgments that the places in his Top are occupied not only by the most relevant sites, but also by those who paid for it (directly to the search engine, and not to the Seo office). Of course, this applies primarily to commercial issuance.
In general, looking at the statistics, it becomes clear why Google easily goes there to worsen its issuance in exchange for increasing profits from contextual advertising. In fact, they are not afraid of the outflow of users, because in most cases they have nowhere to go and nowhere to go. This situation is somewhat sad, but let's see what happens next.
By the way, in order to further complicate the life of optimizers, and perhaps to maintain the peace of mind of users of this search engine, Google has recently been using encryption when transmitting requests from users' browsers to the search string. Soon it will no longer be possible to see in the statistics of visitor counters what queries users from Google came to you for.
Of course, in addition to the search engines voiced in this publication, there are more than one thousand others - regional, specialized, exotic, etc. It will not be possible to try to list and describe them all within the framework of one article, and, probably, it is not necessary. Let's say a few words about how easy it is to create a search engine and how not easy and not cheap to keep it up to date.
The vast majority of systems work on similar principles (read about that and about) and pursue the same goal - to give users an answer to their question. Moreover, this answer should be relevant (corresponding to the question), exhaustive and, which is not unimportant, relevant (of the first freshness).
Solving this problem is not so easy, especially considering that the search engine will need to analyze the content of billions of Internet pages on the fly, weed out unnecessary ones, and form a list (output) from the rest, where the most appropriate answers to the user's question will first go.
This extremely complex task is solved by preliminary collection of information from these pages using various indexing robots. They collect links from already visited pages and load information from them into the search engine database. There are text-indexing bots (normal and fast bots that live on news and frequently updated resources so that the most recent data is always presented in the results).
In addition, there are robots indexing images (for their subsequent output to), favicons, site mirrors (for their subsequent comparison and possible gluing), bots that check the performance of Internet pages that users or through tools for webmasters (here you can read about, and) .
The indexing process itself and the subsequent process of updating index databases are quite time-consuming. Although Google does this much faster than its competitors, at least Yandex, which takes a week or two to do this (read about).
Usually, the search engine breaks the text content of a web page into separate words, which leads to the basic foundations, so that later you can give the correct answers to questions asked in different morphological forms. All unnecessary body kit in the form of Html tags, spaces, etc. things are removed, and the remaining words are sorted alphabetically and next to them their position in this document is indicated.
This shnyaga is called a reverse index and allows you to search no longer on web pages, but on structured data located on the servers of the search engine.
The number of such servers at Yandex (which searches mainly only on Russian-language sites and a bit on Ukrainian and Turkish) is in the tens or even hundreds of thousands, and Google (which searches in hundreds of languages) is in the millions.
Many servers have copies that serve both to increase the safety of documents and help to increase the speed of processing a request (due to load distribution). Estimate the cost of maintaining this entire economy.
The user's request will be directed by the load balancer to the server segment that is currently least loaded. Then the analysis of the region is carried out, from where the user of the search engine sent his request, and its morphological analysis is done. If a similar request was recently entered in the search line, then the data from the cache is slipped to the user so as not to load the server once again.
If the request has not yet been cached, then it is passed to the area where the index base of the search engine is located. The response will be a list of all web pages that have at least some relation to the request. Not only direct occurrences are taken into account, but also other morphological forms, as well as, etc. things.
Them needs to be sorted and at this stage, the algorithm (artificial intelligence) comes into play. In fact, the user's request is multiplied due to all possible options for its interpretation and answers to many requests are searched simultaneously (due to the use of query language operators, some of which are available to ordinary users).
As a rule, in the issue there is one page from each site (sometimes more). are now very complex and take into account many factors. In addition, for their correction, and are used, which manually evaluate the reference sites, which allows you to adjust the operation of the algorithm as a whole.
In general, it is clear that the matter is dark. You can talk about this for a long time, but even so it is understood that user satisfaction with a search engine is achieved, oh, how not easy. And there will always be those who do not like something, like, for example, you and me, dear readers.
Good luck to you! See you soon on the blog pages site
You can watch more videos by going to");">
You may be interested
Yandex People - how to search for people on social networks Apometr is a free service for tracking changes in the issuance and updates of search engines DuckDuckGo is a search engine that doesn't follow you. 
How to check Internet speed - online connection test on a computer and phone, SpeedTest, Yandex and other meters 
Yandex and Google pictures, as well as search by image file in Tineye and Google