Search engines on the World Wide Web. How does a search engine work? History of the development of search engines

Just recently, erudition was at a high price. But today, knowledge of all the world's capitals and the smallest historical details is absolutely useless. Instead of feeding your brain to no one the necessary facts, you just need to know what search engines there are on the Internet. Everything else will be decided by the smartest software algorithm.

Brief description of search engines

A search engine is software system, which is designed to search for information on the World Wide Web. Query results are usually presented in the form of a search results page consisting of a dozen short text blocks (snippets) combined with multimedia content.

Today there are about a dozen international data search tools. They all operate on the same principles:

1. Responsible for collecting information crawler- robot (simpler - special program), which “crawls” from site to site and enters into the index Various types data (page content, program scripts, meta tags, etc.);
2. Once added to the index, the information becomes available to the average user through the search bar interface;
3. The results on the results page are ranked in a special way. Key indicator is relevance- compliance with the initial request.

How to use the search engine?

The developers of these systems did everything possible to ensure that everyone, even children, could work with them. And, I must say, they succeeded:

1. In order to appeal to the mind of a search engine, just go to its official website. Design start pages may differ, but the line for entering text will definitely be in the most visible place;
2. An even simpler way is to specify a request via address bar browser. Most modern browsers support this feature;
3. If typing text in a particular situation is difficult or impossible, you can use the function voice input(launched by clicking the microphone icon);
4. A split second after the call, the system will display the results;
5. To speed up the search you can use logical operators. Thus, adding a minus sign in front of a word will exclude it from the search results, while adding a plus sign will indicate search engine, What given word must be in the searched text, and the phrase in quotation marks launches a search for the exact phrase;
6. If found necessary information If it didn’t work, it doesn’t matter. No one is stopping you from using another service. Here is an overview of the main players in this market.

Overview of key services

At the turn of the century, almost every state had its own national search engine. However, after several years, competition has left only a few global players:

• Google- the undisputed market leader. It accounts for the lion's share of the world's search traffic. In developed countries, its share ranges from 80 to 90%. In Russia, the car’s success is not so impressive: only 40% share;
• Yandex- the main competitor for Google in Russia, on this moment occupies a leading position, due to the change search algorithm in 2017, it gradually began to lose its position;
• ‒ ranks third in RuNet (about 5%), search from a well-known Russian company, is part of the Mail.Ru Group, which owns the social networks “Vkontakte”, “Odnoklassniki”, the search engine Nigma (we will describe it below), payment system Qiwi and much more.
• Bing- a service from Microsoft. top scores issues primarily in English. The Russian-speaking segment is practically undeveloped. This, however, does not prevent Bing from being the No. 2 search engine in the world (7%);
• Next, with a small gap, follows Chinese company "Baidu", which primarily serves the East Asian markets (China and Japan). It is little known in the West and in Russia;
• Yahoo!- stood at the origins of the modern sphere high technology, but lost in the competition. Today he uses a solution from Microsoft;
• Nigma- a once popular search engine based Russian programmers in 2005, now almost losing traffic share;
• Duckduckgogo- positions itself as a service that takes care of users’ personal data (there is no cache collection or personalization).

Yandex: “our everything” in RuNet

A conversation about searching for information on the Internet would be incomplete without mentioning a key player in Russian market- Yandex. Its founders, Arkady Volozh and Ilya Segalovich, launched a joint venture long before the Google brand was born. Over many decades, the search engine has become one of the largest search engines in the world (No. 4 since 2014).

The main reason for this phenomenon is highest quality service:

• Yandex was the first to launch a search taking into account Russian morphology. This innovation allowed him to overtake the then IT industry giant Rambler in 2001;
• Since 2009, the service began to take into account the user’s region of residence. Thanks to this, regional sites received an impetus for development;
• The same year the algorithm was launched machine learning Matrixnet, which significantly increased the speed and quality of the service;
• At the same time, an English-language website was developing, for which Yandex rented an office in California;
• In 2012, query results began to take into account the personal preferences of a particular user.

The popularity of the system is constantly growing: from 2001 to 2017, the volume of traffic increased 280 times. But, as we wrote earlier, after a radical change in the algorithm in 2017, Yandex began to lose the share of its visitors to Google, see the full statistics of RuNet traffic on this moment you can follow the link.

How to remove information about yourself from a search engine?

The Internet is not only a benefit for all humanity, but also a potential source of threats. Some of them pose a significant threat to the personal data of ordinary users. Once online, even the most offensive information cannot be deleted.

Yes, according to at least, it was until recently. Since January 2016, the law “On the right to be forgotten” has been in force in Russia. Now every person who is not satisfied with his reputation in virtual world, can clean it completely in a legal way. To do this you need:

1. Submit an appropriate application to the organization providing search services;
2. Attach all possible evidence of the falsity and irrelevance of the data that is proposed to be removed from the search results;
3. Most search engines make it easier for users. You can send a corresponding request through a special online form (for Yandex);
4. Within ten days, the company will check the application for legal grounds;
5. If the outcome is positive, the information is subject to removal from the index;
6. If no grounds for removal are established, the organization notifies the applicant.

The permanent host of the program “Clever Men and Clever People,” Yuri Vyazemsky, mentioned in one of his interviews how he discovered an unusual word from Bulgakov and tried to find its meaning. Working in the library took whole year(!). Today it is impossible to imagine such a situation: there are many search engines on the Internet. One click - and everyone can connect to the world's mind.

Video review of anonymous search engines

In this video, Anton Morozov will tell you what search engines exist that maintain complete user confidentiality:

At first glance it may seem that there is better than Google maybe only Yandex, and even that is not a fact. These companies invest huge amounts of money in innovation and development. Does anyone really have a chance not only to compete with the leaders, but also to win? Lifehacker's answer: “Yes!” There are several search engines that have succeeded. Let's look at our heroes.

What is this

This is a fairly well-known search engine with open source code. Servers are located in the USA. In addition to its own robot, the search engine uses results from other sources: Yahoo! Search BOSS, Wikipedia, Wolfram|Alpha.

The better

DuckDuckGo positions itself as a search engine that provides maximum privacy and confidentiality. The system does not collect any data about the user, does not store logs (no search history), use cookies as limited as possible.

DuckDuckGo doesn't collect personal information users and does not share it. This is our privacy policy.
Gabriel Weinberg, founder of DuckDuckGo

Why do you need this

All major search engines are trying to personalize search results based on data about the person in front of the monitor. This phenomenon is called the “filter bubble”: the user sees only those results that are consistent with his preferences or that the system deems as such.

DuckDuckGo creates an objective picture that does not depend on your past behavior on the Internet, and eliminates thematic Google advertising and Yandex, based on your requests. With DuckDuckGo, it’s easy to search for information in foreign languages: Google and Yandex by default give preference to Russian-language sites, even if the query is entered in another language.

What is this

"" is a Russian metasearch system developed by Moscow State University graduates Viktor Lavrenko and Vladimir Chernyshov. It searches through the indexes of Google, Bing, Yandex and others, and also has its own search algorithm.

The better

Searching through the indexes of all major search engines allows you to generate relevant results. In addition, Nigma breaks the results into several thematic groups(clusters) and prompts the user to narrow the search field by discarding unnecessary ones or highlighting priority ones. Thanks to the Mathematics and Chemistry modules, you can solve mathematical problems and request the results of chemical reactions directly in the search bar.

Why do you need this

Eliminates the need to search for the same query in different search engines. The cluster system makes it easy to manipulate search results. For example, Nigma collects results from online stores into a separate cluster. If you do not intend to buy anything, then simply exclude this group. By selecting the “English-language sites” cluster, you will receive results only in English. The Mathematics and Chemistry modules will help schoolchildren.

Unfortunately, the project is not currently being developed, as the developers have transferred their activity to the Vietnamese market. Nevertheless, “Nigma” is not only not outdated yet, but in some things it still gives Google a head start. Let's hope development resumes.

What is this

not Evil - a system that searches by anonymous Tor networks. To use, you need to log into this network, for example by running specialized browser with the same name. not Evil is not the only search engine of its kind. There is LOOK (default search in the Tor browser, accessible from regular internet) or TORCH (one of the oldest search engines on the Tor network) and others. We settled on not Evil because of the clear allusion to Google itself (just look at the start page).

The better

It searches where Google, Yandex and other search engines are generally closed.

Why do you need this

The Tor network contains many resources that cannot be found on the law-abiding Internet. And as government control over the content of the Internet tightens, their number will grow. Tor is a kind of Network within the Network: with its social networks, torrent trackers, media, trading platforms, blogs, libraries and so on.

YaCy

What is this

YaCy is a decentralized search engine that works on the principle of P2P networks. Each computer on which the main one is installed software module, scans the Internet independently, that is, it is an analogue search robot. The results obtained are collected in common base, which is used by all YaCy members.

The better

It’s difficult to say whether this is better or worse, since YaCy is a completely different approach to organizing search. The absence of a single server and owner company makes the results completely independent of anyone's preferences. The autonomy of each node eliminates censorship. YaCy is capable of searching the deep web and non-indexed public networks.

Why do you need this

If you are a supporter of open source software and a free internet, not subject to influence government agencies and large corporations, then YaCy is your choice. It can also be used to organize a search within a corporate or other autonomous network. And even though YaCy is not very useful in everyday life, it is worthy alternative to Google in terms of the search process.

Pipl

What is this

Pipl is a system designed to search for information about a specific person.

The better

The authors of Pipl claim that their specialized algorithms search more efficiently than “regular” search engines. In particular, the priority sources of information are profiles social networks, comments, lists of participants and various databases where information about people is published, such as databases of court decisions. Pipl's leadership in this area is confirmed by assessments from Lifehacker.com, TechCrunch and other publications.

Why do you need this

If you need to find information about a person living in the US, then Pipl will be much more effective than Google. The databases of Russian courts are apparently inaccessible to the search engine. Therefore, he does not cope so well with Russian citizens.

What is this

Another specialized search engine. Looking for various sounds(house, nature, cars, people, etc.) in open sources. The service does not support queries in Russian, but there is an impressive list of Russian-language tags that you can search for.

The better

The output contains only sounds and nothing extra. In the search settings you can set the desired format and sound quality. All sounds found are available for download. There is a search for sounds by pattern.

Why do you need this

If you need to quickly find the sound of a musket shot, the blows of a suckling woodpecker, or the cry of Homer Simpson, then this service is for you. And I chose this only from the available Russian-language queries. In English the spectrum is even wider. But seriously, a specialized service requires a specialized audience. But what if it comes in handy for you too?

The life of alternative search engines is often fleeting. Lifehacker asked the former general director of the Ukrainian branch of Yandex, Sergei Petrenko, about the long-term prospects of such projects.

As for the fate of alternative search engines, it is simple: to be very niche projects with a small audience, therefore without clear commercial prospects or, conversely, with complete clarity of their absence.

If you look at the examples in the article, you can see that such search engines either specialize in a narrow but popular niche, which, perhaps, has not yet grown enough to be noticeable on the radars of Google or Yandex, or they are testing an original hypothesis in ranking, which is not yet applicable in regular search.

For example, if a search on Tor suddenly turns out to be in demand, that is, results from there are needed by at least a percentage of Google’s audience, then, of course, ordinary search engines will begin to solve the problem of how to find them and show them to the user. If the behavior of the audience shows that for a significant proportion of users in a significant number of queries, results given without taking into account factors depending on the user seem more relevant, then Yandex or Google will begin to produce such results.

“Be better” in the context of this article does not mean “be better at everything.” Yes, in many aspects our heroes are far from Google and Yandex (even far from Bing). But each of these services gives the user something that the search industry giants cannot offer.

Until recently found necessary information in World Wide Web it wasn't that easy. But thanks rapid development and the emergence of new search engines, this process has become much more convenient and faster. In our today's review, the 10 most best services to search for various types of data that can find everything you need in the depths of the Internet.

1. Internet search engine - Bing

Bing is a famous Internet search engine that was created in June 2009. This service available in 40 languages. It has approximately 350 million monthly unique visitors.

2. Internet metasearch engine - AOL Search.com

AOL Search.com is an Internet metasearch engine that was founded in the United States of America in 1985. The monthly number of visitors to this service is approximately 75 million people. The position of CEO is occupied by Tim Armstrong.

3. Internet search engine - DuckDuckGo

DuckDuckGo- This internet search engine open source, which was launched in 2008 by Gabriel Weinberg. The company's headquarters are located in Valley Forge, Pennsylvania. The number of unique visitors using DuckDuckGo is about 13 million people per month.

4. Internet search engine - Ask.com

Ask.com is an internet search engine main feature which is the search for answers to questions. The service was founded in June 1996 in Berkeley, California by Garrett Gruner and David Wharton. The monthly number of unique visitors to this service is approximately 145 million people.

5. Internet search engine - Google

Google is the most famous Internet search engine, headquartered in Mountain View, California, United States of America. The company was founded on September 4, 1998 by Larry Page and Sergei Mikhailovich Brin. Its monthly number of unique visitors is about 1,100,000,000 people.

6. Internet metasearch system - MyWebSearch.com

MyWebSearch.com is an Internet metasearch engine that ranks 73rd on the list of most popular websites. It has approximately 60 million monthly unique visitors.

7. Internet metasearch engine - Infospace.com

The presented search engine was founded in 1996. Its headquarters are located in Bellevue, Washington, United States. Monthly number of unique visitors Infospace.com is approximately 24 million people.

8. Internet metasearch engine - WebCrawler.com

WebCrawler.com is an internet metasearch engine powered by Yahoo and Google. The company was founded in April 1994. It has approximately 65 million monthly unique visitors.

9. Internet metasearch engine - Info.com

Info.com is a metasearch engine headquartered in the UK capital London. Info.com has approximately 13 million monthly unique visitors.

10. Internet search engine – Yahoo

Internet search engine system called - Yahoo, which was founded in January 1994 by Jerry Yang and David Feal. General Director company is Marissa Mayer. Its monthly number of unique visitors is approximately 300 million people. The service's headquarters is located in Santa Clara, California, United States of America.

And for the lovers mobile devices And computer technology, it will certainly be interesting to look at

The undisputed leader in terms of queries in the world is the search engine. Google system. The search engine processes more than a billion user requests every day. The company has the largest share (about 62%) of the entire search engine market and offers users a variety of online services and tools that allow them to provide the most relevant results. Googlebot crawls about 25 billion web pages per month, which is also the largest figure for web search. According to some data, the search engine is able to work with information posted on the Internet in 195 languages ​​and search for it equally effectively.

"Yandex"

Yandex ranks 4th in the world in terms of the number of requests processed per day.

The first most popular search engine in Russia. Originally built on Google engine, today Yandex offers its own search algorithm aimed at Russian-speaking users in Russia and the CIS countries. The search engine successfully copes with its task and offers both visitors and webmasters a variety of services that can not only improve the quality of the results provided, but also make surfing the Internet more convenient.

Other search engines

There is a great variety popular search engines: Yahoo, AOL, Ask, Mail.ru, Rambler. Some search engines use mechanisms borrowed from other systems (for example, QIP.ru uses the Yandex engine).

Among other search engines, we can note the no less popular Baidu, whose main audience is located in China. The search engine ranks 3rd in the world in terms of the number of queries processed. The site has its own services, for example, an encyclopedia, antivirus program, translator, etc. The Bing project from Microsoft is also gaining increasing popularity, which also has its share in the market and ranks 2nd in the world after Google in terms of traffic. The search engine has not yet been officially launched in Russia, but is capable of processing Russian-language results. Bing search is the default in Internet Explorer and on phones and tablets running on Windows platform Phone and Windows 8.

There are also highly specialized search engines. For example, we can highlight image search engines (for example, TinEye), grabbers (for example, Guenon, which shows the contents of other sites on its pages). There are also search resources with a registration system (DuckDuckGo).

Let's look at the most popular search engines in Russia and the World.

The most popular search engines in Russia

Yandex

Yandex is the most popular system searching for information in RuNet. It ranks first in terms of safety! 27,000,000 people were chosen.

"Yandex" Russian search service appeared in 1997. The undoubted leader of the Russian-speaking part of the Internet. It has one of the best regional search systems in the world. Eat the ability to find people on social networks. This system features a huge number of useful additions:

• Yandex.Disk
• Cards
• Flight, train and bus schedules
• Weather and much more.

It is in good demand among neighboring countries, since the service management takes into account the requirements of users from the CIS countries.

Mail

Mail.ru is promising Post service, which appears a large number of fans.

Not long ago it began to position itself as a search engine. It can be set by default when downloading various applications.

Aport

A fairly common search engine in the Russian-speaking part of the Internet. In terms of popularity, this service ranks second in Russia.

Advantages of the Aport search engine:

1. Russian-language morphology is taken into account.
2. You can highlight the context of displayed words.
3. Search by category.
4. There is a function for refining search results in accordance with servers and available catalog sections.
5. Any of the requests can be translated into English language and back.

Rambler

Represents the first information retrieval system, which became available in Russia. Previously, it was the main competitor of Yandex as it is used search technology Yandex. The morphology of the Russian language is taken into account. It took an honorable third place in our ranking.

The most popular search engines in the world

Google

Google is a global giant. Therefore, it is not surprising that this search engine accounts for the bulk of the world's traffic.

Google search engine

Very simple and high-quality live issuance and great amount on useful tools:

• Documentation;
• Blogs;
• YouTube;
• Social media;
• Google + and a lot of interesting things.

Baidu

Baidu is a Chinese search engine. China is the most populated place on Earth. It should also be noted in full closed access to the World Wide Web.

This “search engine” was preferred by millions of people. We will assume that they simply have no choice, but, nevertheless, Judging by the reviews, this system finds a lot of interesting and useful information.

Bing

Bing is a search engine from Microsoft. In demand in Europe and the USA.

Compared to Google, has low traffic. Installed as standard search V .

Yahoo!

Today it is a catalog of resources and a “search engine” rolled into one; the service operates as an independent project. In terms of popularity, it took fourth position in our ranking.

Advantages of Yahoo search engine:

1. Can search for various types of information in Russian.
2. Directory used useful links to network resources in various countries of the world.

AltaVista

Another search development that searches for the necessary information on the Internet. The level of popularity allowed it to rank fifth.

Advantages of the AltaVista search engine:

1. There is support for the Russian-speaking space.

Lycos

Common Directory network resources. The site is based on a search engine along with a rubricator for web portals. Provides links to various localized editions of the catalog. English language used. In terms of popularity it is in sixth place in our ranking.

Ukrainian search engine Meta

Unified search system for Ukrainian origin. The level of fame allowed it to take seventh place.

Advantages of the Meta search engine:

1. The service takes into account the morphological features of both Russian and Ukrainian languages.
2. If necessary, you can highlight various found words in context.
3. It will also be possible to translate search results and search queries into Russian, Ukrainian and English.

FTP Search

"File Search System". It is one of the hundred largest FTP servers in the Russian Federation. The level of popularity corresponds to eighth place.