Sas rotation speed. SATA drives: household and industrial sectors

This article will talk about what allows you to connect HDD to the computer, namely, about the hard drive interface. More precisely, about hard drive interfaces, because a great many technologies have been invented for connecting these devices throughout their existence, and the abundance of standards in this area can confuse an inexperienced user. However, first things first.

Hard drive interfaces (or strictly speaking, interfaces external drives, since they can be used not only by , but also by other types of drives, for example, drives for optical disks) are designed to exchange information between these external memory devices and the motherboard. Hard drive interfaces, no less than the physical parameters of the drives, affect many of the operating characteristics of the drives and their performance. In particular, drive interfaces determine such parameters as the speed of data exchange between the hard drive and the motherboard, the number of devices that can be connected to the computer, the ability to create disk arrays, the possibility of hot plugging, support for NCQ and AHCI technologies, etc. . It also depends on the hard drive interface which cable, cord or adapter you will need to connect it to the motherboard.

SCSI - Small Computer System Interface

The SCSI interface is one of the oldest interfaces designed for connecting storage devices in personal computers. This standard appeared in the early 1980s. One of its developers was Alan Shugart, also known as the inventor of the floppy disk drive.

Appearance of the SCSI interface on the board and the cable connecting to it

SCSI standard (traditionally this abbreviation read in Russian transcription as “skazi”) was originally intended for use in personal computers, as evidenced by even the name of the format - Small Computer System Interface, or system interface for small computers. However, it so happened that the drives of this type were used mainly in top-class personal computers, and subsequently in servers. This was due to the fact that, despite the successful architecture and wide set of commands, technical implementation The interface was quite complex and not suitable for the cost of mass PCs.

However, this standard had a number of features that were not available for other types of interfaces. For example, the cord for connecting Small Computer System Interface devices can have a maximum length of 12 m, and the data transfer speed can be 640 MB/s.

Like the IDE interface that appeared a little later, the SCSI interface is parallel. This means that the interface uses buses that transmit information over several conductors. This feature was one of the limiting factors for the development of the standard, and therefore a more advanced, consistent SAS standard (from Serial Attached SCSI) was developed as its replacement.

SAS - Serial Attached SCSI

This is what the SAS server disk interface looks like

Serial Attached SCSI was developed as an improvement to a fairly old interface connecting hard Small Computers System Interface drives. Despite the fact that Serial Attached SCSI uses the main advantages of its predecessor, it nevertheless has many advantages. Among them it is worth noting the following:

• Use of a common bus by all devices.
• The serial communication protocol used by SAS allows for fewer signal lines to be used.
• There is no need for bus termination.
• Virtually unlimited number of connected devices.
• Higher throughput (up to 12 Gbit/s). Future implementations of the SAS protocol are expected to support data transfer rates of up to 24 Gbit/s.
• Possibility of connecting drives with Serial ATA interface to the SAS controller.

As a rule, Serial Attached SCSI systems are built on the basis of several components. The main components include:

• Target devices. This category includes the actual drives or disk arrays.
• Initiators are chips designed to generate requests to target devices.
• Data delivery system - cables connecting target devices and initiators

Serial Attached SCSI connectors come in different shapes and sizes, depending on the type (external or internal) and SAS versions. Below are the SFF-8482 internal connector and the SFF-8644 external connector designed for SAS-3:

On the left is an internal SAS connector SFF-8482; On the right is an external SAS SFF-8644 connector with a cable.

A few examples of the appearance of SAS cords and adapters: HD-Mini SAS cord and SAS-Serial ATA adapter cord.

On the left is the HD Mini SAS cable; On the right is an adapter cable from SAS to Serial ATA.

Firewire - IEEE 1394

Today you can often find hard disks with Firewire interface. Although the Firewire interface can connect any type of peripheral devices to a computer, and it is not a specialized interface designed exclusively for connecting hard drives, Firewire nevertheless has a number of features that make it extremely convenient for this purpose.

FireWire - IEEE 1394 - view on a laptop

The Firewire interface was developed in the mid-1990s. The development began with the well-known company Apple, which needed its own bus, different from USB, for connecting peripheral equipment, primarily multimedia. The specification describing the operation of the Firewire bus is called IEEE 1394.

Firewire is one of the most commonly used high-speed serial external bus formats today. The main features of the standard include:

• Possibility of hot connection of devices.
• Open bus architecture.
• Flexible topology for connecting devices.
• Data transfer speeds vary widely – from 100 to 3200 Mbit/s.
• The ability to transfer data between devices without a computer.
• Possibility of organizing local networks using a bus.
• Power transmission via bus.
• A large number of connected devices (up to 63).

To connect hard drives (usually via external hard drive enclosures) via the Firewire bus, as a rule, a special SBP-2 standard is used, which uses the Small Computers System Interface protocol command set. It is possible to connect Firewire devices to a regular USB connector, but this requires a special adapter.

IDE - Integrated Drive Electronics

The abbreviation IDE is undoubtedly known to most personal computer users. The interface standard for connecting IDE hard drives was developed by a well-known hard drive manufacturer - Western Digital. The advantage of the IDE over other interfaces that existed at the time, in particular the Small Computers System Interface, as well as the ST-506 standard, was that there was no need to install hard controller drive to the motherboard. The IDE standard implied installing a drive controller on the drive itself, and only a host interface adapter for connecting IDE drives remained on the motherboard.

IDE interface on motherboard

This innovation has improved the operating parameters of the IDE drive due to the fact that the distance between the controller and the drive itself has been reduced. In addition, installing an IDE controller inside the hard drive case made it possible to somewhat simplify both motherboards and the production of hard drives themselves, since the technology gave freedom to manufacturers in terms of optimal organization of the logic of the drive.

The new technology was initially called Integrated Drive Electronics. Subsequently, a standard was developed to describe it, called ATA. This name is derived from the last part of the name of the PC/AT family of computers by adding the word Attachment.

For connecting hard drive or other device, such as an optical drive that supports Integrated Drive Electronics technology, to the motherboard is used special cable IDE. Since ATA refers to parallel interfaces (therefore it is also called Parallel ATA or PATA), that is, interfaces that provide for simultaneous data transmission over several lines, its data cable has a large number of conductors (usually 40, and in recent versions of the protocol it was possible to use 80-core cable). Regular data cable for this standard has a flat and wide appearance, but there are also round cables. The power cable for Parallel ATA drives has a 4-pin connector and is connected to the computer's power supply.

Below are examples of IDE cable and round PATA data cable:

Appearance of the interface cable: on the left - flat, on the right in a round braid - PATA or IDE.

Due to the comparative low cost of Parallel ATA drives, the simplicity of implementing the interface on the motherboard, as well as the ease of installation and configuration of PATA devices for the user, Integrated Drive Electronics drives long time devices of other types of interface were forced out of the market of hard drives for budget-level personal computers.

However, the PATA standard also has a number of disadvantages. First of all, this is a limitation on the length that a Parallel ATA data cable can have - no more than 0.5 m. In addition, the parallel organization of the interface imposes a number of restrictions on the maximum data transfer speed. It does not support the PATA standard and many of the advanced features that other types of interfaces have, such as hot plugging of devices.

SATA - Serial ATA

View of the SATA interface on the motherboard

The SATA (Serial ATA) interface, as the name suggests, is an improvement over ATA. This improvement consists, first of all, in converting the traditional parallel ATA (Parallel ATA) into a serial interface. However, the differences between the Serial ATA standard and the traditional one are not limited to this. In addition to changing the data transmission type from parallel to serial, the data and power connectors also changed.

Below is the SATA data cable:

Data cable for SATA interface

This made it possible to use a much longer cord and increase the data transfer speed. However, the downside was the fact that PATA devices, which were present on the market in huge quantities before the advent of SATA, became impossible to connect directly to the new connectors. True, most new motherboards still have old connectors and support connecting older devices. However, the reverse operation - connecting a new type of drive to an old motherboard usually causes more problems. For this operation, the user usually requires a Serial ATA to PATA adapter. The power cable adapter usually has a relatively simple design.

Serial ATA to PATA power adapter:

On the left is a general view of the cable; Enlarged on the right appearance PATA and Serial ATA connectors

However, the situation is more complicated with such a device as an adapter for connecting a device serial interface into the parallel interface connector. Typically, an adapter of this type is made in the form of a small microcircuit.

Appearance of a universal bidirectional adapter between SATA - IDE interfaces

Currently, the Serial ATA interface has practically replaced Parallel ATA, and PATA drives can now be found mainly only in fairly old computers. Another feature of the new standard that ensured its wide popularity was support.

Type of adapter from IDE to SATA

You can tell us a little more about NCQ technology. The main advantage of NCQ is that it allows you to use ideas that have long been implemented in the SCSI protocol. In particular, NCQ supports a system for sequencing read/write operations across multiple drives installed in a system. Thus, NCQ can significantly improve the performance of drives, especially arrays of hard disks.

Type of adapter from SATA to IDE

To use NCQ, technology support is required on the hard drive side, as well as on the motherboard host adapter. Almost all adapters that support AHCI also support NCQ. In addition, some older proprietary adapters also support NCQ. Also, for NCQ to work, it requires support from the operating system.

eSATA - External SATA

It is worth mentioning separately the eSATA (External SATA) format, which seemed promising at the time, but never became widespread. As you can guess from the name, eSATA is a type of Serial ATA designed for connecting exclusively external drives. The eSATA standard offers most of the capabilities of the standard for external devices, i.e. internal Serial ATA, in particular, the same system of signals and commands and the same high speed.

eSATA connector on a laptop

However, eSATA also has some differences from the internal bus standard that gave birth to it. In particular, eSATA supports a longer data cable (up to 2 m) and also has higher power requirements for drives. In addition, eSATA connectors are slightly different from standard connectors Serial ATA.

Compared to other external buses such as USB and Firewire, eSATA, however, has one significant drawback. While these buses allow the device to be powered via the bus cable itself, the eSATA drive requires special connectors for power. Therefore, despite the relatively high data transfer speed, eSATA is currently not very popular as an interface for connecting external drives.

Conclusion

Information stored on the hard drive cannot become useful to the user and accessible to application programs until it gets access CPU computer. Hard drive interfaces provide a means of communication between these drives and the motherboard. Today there are many various types hard interfaces disks, each of which has its own advantages, disadvantages and characteristic features. We hope that the information provided in this article will be largely useful to the reader, because the choice of a modern hard drive is largely determined not only by its internal characteristics, such as capacity, cache memory, access and rotation speed, but also by the interface for which it was developed.

Everyone knows the performance parameters of disk subsystems in theory. But what in practice? Many people ask this question, some build their own hypotheses. I decided to conduct a series of tests and determine “Who is who”. I started testing with all the well-known utilities dd, hdparm, then moved on to fio, sysbench. A number of tests were also carried out using UnixBench and several other analogues. A number of graphs were generated, but with further testing it was discovered that most of the software was not suitable for adequately comparing different drives.
Using fio, it was possible to create a comparative table or graph for SAS, SATA, but when testing SSDs it turned out that the results obtained were completely unusable. Of course, I respect the developers of all this software, but at that moment it was decided to create a series of not synthetic tests, but ones closer to the real situation.

I’ll say right away that the test parameters and the machines themselves were selected in such a way that the test results were not distorted by the type of processor, its frequency or other parameters.

Test 1

Creating files

Over the course of eight cycles, the creation was generated small files with chaotic content and a gradual increase in the number of files per cycle. The execution time was measured for each cycle.

The graph shows that SSD KINGSTON SV300S3 has a higher speed of file creation and is almost independent of their number. It is also worth noting that these particular discs have a more linear scale
Looking at SAS disks in Hardware RAID, it can be seen that the speed depends on the type of raid, but does not depend at all on the number of disks.
But more time is spent not on creating files, as it turned out, but on rewriting them. With this in mind, let's move on to the second test.

Test 2

Overwriting files

The same operations as in the first test were repeated, but new files were not created each time, but the same file was used, into which new information was written each time.


The terrible picture of the SATA 7,200 rpm MB2000GCVBR drives immediately catches your eye. Slow recording and 2x 300GB SAS SEAGATE. Therefore, I decided to remove them from the schedule for clarity on the rest.


The fastest subsystem turned out to be a single SSD KINGSTON. Second and third places went to 8x SEAGATE ST3300657SS and 4x SEAGATE ST3300657SS. We also see that as the number of SSDs in the array increases, the speed drops slightly.

Test 3

MySQL. Combining sql queries INSERT, SELECT, UPDATE, DELETE

An InnoDB table was created with the following structure:
CREATE TABLE `table` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`time` int(11) NOT NULL,
`uid` int(11) NOT NULL,
`status` varchar(32) NOT NULL,
PRIMARY KEY (`id`),
FULLTEXT KEY `status` (`status`)
) ENGINE=InnoDB DEFAULT CHARSET=cp1251;

Several requests were generated simultaneously:
- INSERT;
- UPDATE with sampling by PRIMARY KEY;
- UPDATE with selection by FULLTEXT (search by 4 characters out of 24): WHERE `status` LIKE "%(string)%";
- DELETE FROM with selection by PRIMARY KEY;
- DELETE FROM with selection without using a key: WHERE `time`>(int);
- SELECT with selection without using a key: WHERE `time`>(int);
- SELECT with selection by PRIMARY KEY;
- SELECT with selection by FULLTEXT (search by 4 characters out of 24): WHERE `status` LIKE "%(string)%";
- SELECT with selection without using a key: WHERE `uid`>(int).

And again we see the same picture as in the second test.

In the following tests I use the sysbench utility, which generates large files:
128 files, total sizes 10 GB, 30 GB and 50 GB.
Block size 4 KB.
I would like to immediately draw your attention to the fact that on some graphs, for some servers there is no data for 10 GB. This is due to the fact that these machines have random access memory more than 10 GB and data caching is performed. The lack of some results for 50 GB is due to lack of disk space, in the case of the KINGSTON SV300S3 SSD.

Test 4

Linear recording (file creation)

It's clear that best performance available for all variations with SSD KINGSTON SV300S3, as well as for 8x SEAGATE ST3300657SS in RAID10. The increase in speed with an increase in the number of SAS disks is very clearly visible.
Here is the very moment where you can clearly see that SSDs are completely different. The difference is 4 times!

Test 5

Linear recording (file overwriting)

The leaders are still the same. If we compare 2x SSD from INTEL and 2x SAS there is practically no difference.

Test 6

Linear reading

Here we see a slightly different picture. The leaders are 4x SSD KINGSTON RAID10, with minimal changes in results as the file size increases, and 8x SEAGATE in RAID10, with a gradual decrease in speed, at speeds of 700 Mbit/s and 600 Mbit/s.
The lines for 1x SSD KINGSTON and 2x SSD KINGSTON RAID1 matched. Simply put, for linear reading it is better to take either RAID10 or a single disk. The use of RAID1 is not justified.
It is clearly seen that 2x SAS RAID1 and 4x SAS RAID10 showed very similarities. But when the number of disks is doubled, a huge increase in speed is visible.
2x Intel SSD RAID1 has a significant drop in speed in the range of 10 GB - 30 GB, and then they go at the same speed as SATA RAID1.

Test 7

Random reading

All SSDs are in the lead:
- 4x KINGSTON RAID10;
- 2x KINGSTON RAID1, 2x INTEL RAID1;
- 1 KINGSTON.

I copied everyone else onto the following graph for clarity.


Naturally, 8x SAS RAID10 has the highest speed among these, but the speed drops sharply. But based on the data for 2x SAS and 4x SAS, I will assume that with further growth in volume the speed will stabilize.

Test 8

Random entry

2x 120GB SSD INTEL SSDSC2CT12 Hardware RAID1 SAS1068E with a stable speed of 30 Mbit/s has excellent performance. According to KINGSTON, as the number of disks increases, the speed, oddly enough, decreases. In fourth place is 8x SAS SEAGATE.

Test 9

Combined random read and write operations

We all know that no server is read-only or write-only. Both operations are always performed. And in most cases these are just random operations, not linear ones. So, let's see what we got.


Due to its excellent write speed, 2x SSD INTEL comes in far ahead, followed by SSD KINGSTON. Third place was shared by 2x SSD KINGSTON and 8x SAS SEAGATE.

Test 10

After carrying out all these tests, I decided that it would be convenient to derive the dependence of speed on the ratio of random read and random write operations.


Some have an increase in speed, some have a decrease, and 8x SAS RAID10 has a straight line.

Test 11

I also made a comparison of large arrays of SAS disks, which shows that it depends more on the speed of the disk than on their number.

It's time to take stock.
There were a lot of cars, but not enough. Unfortunately, I was not able to determine whether the indicators for the INTEL SSDSC2CT12 SSD are their feature or a feature of the raid controller. But I believe that it is the controller.

1. As the number of SAS disks in the array increases, all indicators only improve.
2. For MySQL, the slow subsystems are SATA RAID1 and SAS RAID1. For the rest there are differences, but they are not so significant.
3. For linear recording, both large arrays of SAS disks in RAID10 and SSDs are good. There is no point in using SSD arrays. The cost is rising, but the performance is stagnant.
4. Any large arrays are good for linear reading. But in practice lin. Reading without writing is almost unheard of here.
5. Random reading from single SSDs or Software RAID.
6. For random recording, it is better to use Hardware RAID from an SSD, although single SSDs do not sacrifice much.
7. Random read/write, that is, one of the most important indicators, has top scores to Hardware RAID from SSD.
8. Summarizing all of the above, for most tasks it is better to use large arrays (>=8) of SAS or Hardware RAID of SSD. But for some tasks it would be more correct to use single SSDs.
9. Based SSD volumes, which are mainly offered on our market, should be used for VDS nodes maximum performance processors paired with large SAS arrays or mediocre processors and single SSDs. I think that using hw raid for two SSDs will be a bit expensive.
10. If you need fast system and there is no need for large disk space 2x SSD in Hardware RAID will be best choice. If you want to save a little at the expense of performance, then you can take a single SSD or two SSDs in a software raid.

Questions that remain unanswered:

1. What happens when you increase the number of SSDs in a Hardware RAID?
2. Which is cheaper under virtual servers: expensive machines and one large SAS array or several mediocre servers with single SSDs? In this matter, you should also take into account the reliability/durability of SAS and SSD, since there are various rumors about the latter.

In addition to the listed tests and servers, there were many more, but they were not included in the results, since the tests were “calibrated” on them and many of them were found to be incorrect.
RAMDisk testing was also carried out. The results were pretty good, but not the best. Probably due to the fact that it was a virtual machine.

All tests, except the last one, were performed only on dedicated servers.

Why SAS?

The Serial Attached SCSI interface is not just a serial implementation of the SCSI protocol. It does much more than simply port SCSI features such as TCQ (Tagged Command Queuing) through new connector. If we wanted the greatest simplicity, then we would use the Serial ATA (SATA) interface, which is a simple point-to-point connection between the host and an end device such as a hard drive.

But SAS is based on object model, which defines a "SAS domain" - a data delivery system that can include optional expanders and SAS end devices, such as hard drives and host bus adapters (HBA). Unlike SATA, SAS devices can have multiple ports, each of which can use multiple physical connections to provide faster (wider) SAS connections.Moreover, any given target can be accessed by multiple initiators, and the cable length can be up to eight meters (for the first generation of SAS) vs. one meter for SATA. Clearly, this provides many opportunities for creating high-performance or redundant storage solutions. In addition, SAS supports the SATA Tunneling Protocol (STP), allowing you to connect SATA devices to the SAS controller.

The second generation SAS standard increases connection speeds from 3 to 6 Gbps. This speed increase is very important for complex environments where high performance is required due to high-speed storage. The new version of SAS also aims to reduce cabling complexity as well as the number of connections per Gbps of bandwidth by increasing possible cable lengths and improving expander performance (zoning and auto-discovery). We'll talk about these changes in detail below.

Increase SAS speed up to 6 Gbps

To bring the benefits of SAS to a wider audience, the SCSI Trade Association (SCSI TA) presented a primer on SAS technology at the Storage Networking World Conference earlier this year in Orlando, Florida, USA. The so-called SAS Plugfest, where 6 Gbps SAS operation, compatibility and functions were demonstrated, took place even earlier in November 2008. LSI and Seagate were the first on the market to introduce hardware compatible with 6 Gbps SAS, but other manufacturers should catch up soon. In our article we will look at the current state of SAS technologies and some new devices.

SAS Features and Basics

SAS Fundamentals

Unlike SATA, the SAS interface operates on a full duplex basis, providing full bandwidth in both directions. As mentioned earlier, SAS connections are always established via physical connections using unique device addresses. In contrast, SATA can only address port numbers.

Each SAS address can contain multiple physical layer (PHY) interfaces, allowing for broader connections via InfiniBand (SFF-8470) or mini-SAS cables (SFF-8087 and -8088). Typically, four SAS interfaces with one PHY each are combined into one wide SAS interface, which is already connected to the SAS device. Communication can also be done through expanders, which act more like switches than SAS devices.

Features such as zoning now allow administrators to associate specific SAS devices with initiators. This is where the increased throughput of 6Gbps SAS will come in handy, as a quad-link connection will now have twice the speed. Finally, SAS devices can even have multiple SAS addresses. Since SAS drives can use two ports, with one PHY on each, the drive can have two SAS addresses.

Connections and Interfaces

Click on the picture to enlarge.

Addressing of SAS connections occurs through SAS ports using SSP (Serial SCSI Protocol), but communication at the lower level from PHY to PHY is done using one or more physical connections for reasons of increased bandwidth. SAS uses 8/10-bit encoding to convert 8 bits of data into 10-character transmissions for the purposes of timing recovery, DC balance, and error detection. As a result, we get an effective throughput of 300 MB/s for 3 Gb/s transfer mode and 600 MB/s for 6 Gb/s connections. Technologies Fiber Channel, Gigabit Ethernet, FireWire and others operate using a similar encoding scheme.

The power and data interfaces of SAS and SATA are very similar to each other. But if SAS has data and power interfaces combined into one physical interface (SFF-8482 on the device side), then SATA requires two separate cables. The gap between the power and data pins (see illustration above) in the case of SAS is closed, which does not allow connecting a SAS device to a SATA controller.

On the other hand, SATA devices can work fine on a SAS infrastructure thanks to STP or in native mode if expanders are not used. STP adds additional latency to the expanders because they need to establish a connection, which is slower than a direct SATA connection. However, the delays are still very small.

Domains, expanders

SAS domains can be thought of as tree structures, much like complex Ethernet networks. SAS expanders can handle a large number of SAS devices, but they use circuit switching rather than the more common packet switching. Some expanders contain SAS devices, others do not.

SAS 1.1 recognizes edge expanders, which allow a SAS initiator to bind to up to 128 additional SAS addresses. In a SAS 1.1 domain, you can only use two edge expanders. However, one fanout expander can connect up to 128 edge expanders, which significantly increases the infrastructure capabilities of your SAS solution.

Click on the picture to enlarge.

Compared to SATA, the SAS interface may seem complex: different initiators access target devices through expanders, which implies laying out appropriate routes. SAS 2.0 simplifies and improves routing.

Remember that SAS does not allow loops or multiple paths. All connections must be point-to-point and exclusive, but the connection architecture itself is highly scalable.

New SAS 2.0 Features: Expanders, Performance

	SAS 1.0/1.1
Function	Retains legacy SCSI support SATA compatible	Compatible with 3 Gbps Improved speed and signal transmission Zone management Improved scalability
Storage functions	RAID 6 Small form factor HPC High Capacity SAS Drives Ultra320 SCSI replacement Choice: SATA or SAS Blade servers	RAS (data security) Safety (FDE) Cluster support Support for larger topologies SSD Virtualization External storage 4K sector size
Data transfer speed and cable bandwidth	4 x 3 Gbit/s (1.2 GB/s)	4 x 6 Gbit/s (2.4 GB/s)
Cable type	Copper	Copper
Length of cable	8 m	10 m

Expander zones and automatic configuration

Edge and fanout expanders are almost a thing of history. This is often attributed to updates in SAS 2.0, but the reason actually lies in SAS zones introduced in 2.0, which remove the separation between edge and extension expanders. Of course, zones are usually implemented specifically for each manufacturer, and not as a single industry standard.

In fact, now several zones can be located on one information delivery infrastructure. This means that storage targets (drives) can be accessed by different initiators through the same SAS expander. Domain segmentation is done through zones and access is exclusive.

Hard drive for a server, features of choice

The hard drive is the most valuable component in any computer. After all, it stores information that the computer and the user work with, if we are talking about personal computer. Every time a person sits down at a computer, he expects that the operating system loading screen will now run through, and he will begin working with his data, which the hard drive will produce “on the mountain” from its depths. If we are talking about a hard drive, or even an array of them as part of a server, then there are tens, hundreds and thousands of users who expect to gain access to personal or work data. And all their quiet work or rest and entertainment depends on these devices, which constantly store data. Already from this comparison it is clear that the demands placed on home and industrial-class hard drives are unequal - in the first case, one user works with it, in the second - thousands. It turns out that the second hard drive should be many times more reliable, faster, and more stable than the first, because many users work with it and rely on it. This article will look at the types of hard drives used in the corporate sector and the features of their design that allow them to achieve the highest reliability and performance.

SAS and SATA drives - so similar and so different

Until recently, the standards of industrial-class and household hard drives differed significantly and were incompatible - SCSI and IDE, but now the situation has changed - the overwhelming majority of hard drives on the market are SATA and SAS (Serial Attached SCSI). The SAS connector is universal in form factor and is compatible with SATA. This allows you to directly connect to the SAS system both high-speed, but small-capacity (at the time of writing - up to 300 GB) SAS drives, and lower-speed, but many times more capacious, SATA drives(at the time of writing up to 2 TB). Thus, in one disk subsystem, you can combine mission-critical applications that require high performance and rapid access to data, and more economical applications with a lower cost per gigabyte.

Such design compatibility benefits both manufacturers rear panels, and to end users, because this reduces equipment and design costs.

That is, you can connect both SAS and SATA devices to the SAS connectors, and to SATA connectors Only SATA devices are connected.

SAS and SATA - high speed and large capacity. What to choose?

SAS drives, which replaced SCSI drives, completely inherited their main properties that characterize a hard drive: spindle speed (15,000 rpm) and volume standards (36,74,147 and 300 GB). However, SAS technology itself is significantly different from SCSI. Let's briefly look at the main differences and features: The SAS interface uses a point-to-point connection - each device is connected to the controller by a dedicated channel, in contrast, SCSI operates over a common bus.

SAS supports a large number of devices (>16384), while SCSI supports 8, 16, or 32 devices per bus.

The SAS interface supports data transfer rates between devices at speeds of 1.5; 3; 6 Gb/s, while the interface SCSI speed The bus is not allocated to each device, but is divided between them.

SAS supports connecting slower SATA devices.

SAS configurations are much easier to install and install. Such a system is easier to scale. In addition, SAS hard drives inherited the reliability of SCSI hard drives.

When choosing a disk subsystem - SAS or SATA, you need to be guided by what functions will be performed by the server or workstation. To do this, you need to decide on the following questions:

1. How many simultaneous diverse requests will the disk process? If it's large, your clear choice is SAS disks. Also, if your system will serve a large number of users, choose SAS.

2. How much information will be stored on the disk subsystem of your server or workstation? If it is more than 1-1.5 TB, you should pay attention to a system based on SATA hard drives.

3. What is the budget allocated for the purchase of a server or workstation? It should be remembered that in addition to SAS disks, you will need a SAS controller, which also needs to be taken into account.

4. Do you plan to subsequently increase the volume of data, increase productivity, or increase system fault tolerance? If yes, then you will need a SAS-based disk subsystem; it is easier to scale and more reliable.

5. Your server will work with critical data and applications - Your choice - SAS drives, designed for harsh operating conditions.

A reliable disk subsystem includes not only high-quality hard drives from a renowned manufacturer, but also an external disk controller. They will be discussed in one of the following articles. Let's look at SATA drives, what types of these drives there are and which ones should be used when building server systems.

SATA drives: household and industrial sectors

SATA drives, used everywhere, from consumer electronics and home computers to high-performance workstations and servers, are divided into subtypes, there are drives for use in household appliances, with low heat generation, power consumption, and, as a result, reduced performance, there are middle-class disks for home computers, and there are disks for high-performance systems. In this article we will look at the class of hard drives for high-performance systems and servers.

Performance characteristics

	Server class HDD	HDD desktop class
Rotational speed	7,200 rpm (nominal)	7,200 rpm (nominal)
Cache size
Average delay time	4.20 ms (nominal)	6.35 ms (nominal)
Data transfer rate
Reading from drive cache (Serial ATA)	maximum 3 Gb/s	maximum 3 Gb/s
physical characteristics
Capacity after formatting	1,000,204 MB	1,000,204 MB
Capacity
Interface	SATA 3 Gb/s	SATA 3 Gb/s
Number of sectors available to the user	1 953 525 168	1 953 525 168
Dimensions
Height	25.4 mm	25.4 mm
Length	147 mm	147 mm
Width	101.6 mm	101.6 mm
	0.69 kg	0.69 kg
Impact resistance
Impact resistance in working condition	65G, 2ms	30G; 2 ms
Impact resistance when not in use	250G, 2ms	250G, 2ms
Temperature
In working order	-0°C to 60°C	-0°C to 50°C
Inoperative	-40°C to 70°C	-40°C to 70°C
Humidity
In working order		relative humidity 5-95%
Inoperative	relative humidity 5-95%	relative humidity 5-95%
Vibration
In working order
Linear	20-300 Hz, 0.75 g (0 to peak)	22-330 Hz, 0.75 g (0 to peak)
free	0.004 g/Hz (10 - 300 Hz)	0.005 g/Hz (10 - 300 Hz)
Inoperative
Low frequency	0.05 g/Hz (10 - 300 Hz)	0.05 g/Hz (10 - 300 Hz)
High frequency		20-500 Hz, 4.0G (0 to peak)

The table shows the characteristics of hard drives from one of the leading manufacturers; one column shows SATA data server class hard drive, another regular SATA hard drive.

From the table we see that disks differ not only in performance characteristics, but also in operational characteristics, which directly affect the life expectancy and successful operation of the hard drive. Please note that these hard drives differ only slightly in appearance. Let's look at what technologies and features allow us to do this:

The reinforced shaft (spindle) of the hard drive is fixed at both ends by some manufacturers, which reduces the influence of external vibration and facilitates precise positioning of the head unit during read and write operations.

Application of special intelligent technologies allowing to take into account both linear and angular vibration, which reduces head positioning time and increases disk performance by up to 60%

The function of eliminating errors during operation in RAID arrays prevents hard drives from falling out of RAID, which is a characteristic feature of conventional hard drives.

Adjustment of the flight height of the heads in combination with technology to prevent contact with the surface of the platters, which leads to a significant increase in the life of the disk.

A wide range of self-diagnosis functions that allow you to predict in advance the moment when the hard drive fails and warn the user about it, which allows you to have time to save information to a backup drive.

Features that reduce the rate of unrecoverable read errors, which increases the reliability of the server hard drive compared to conventional hard drives.

Speaking about the practical side of the issue, we can confidently say that specialized hard drives in servers “behave” much better. There are significantly fewer calls to the technical service due to instability of work RAID arrays and hard drive failures. Manufacturer support for the server segment of hard drives occurs much more quickly than conventional hard drives, due to the fact that the priority area of ​​work for any manufacturer of data storage systems is the industrial sector. After all, it is here that the most advanced technologies are used to protect your information.

Analogue of SAS disks:

Hard drives from Western Digital VelociRaptor. These drives have a disk rotation speed of 10 thousand rpm, equipped with a SATA 6 Gb/s interface and 64 MB of cache memory. The time between failures of these drives is 1.4 million hours.
More details on the manufacturer's website www.wd.com

You can order the assembly of a server based on SAS or an analogue of SAS hard drives from our company "Status" in St. Petersburg; also, you can buy or order SAS hard drives in St. Petersburg:

• call +7-812-385-55-66 in St. Petersburg
• write to the address
• leave an application on our website on the "Online application" page

Last time we looked at everything related to SCSI technology in a historical context: who invented it, how it developed, what varieties it has, and so on. We ended with the fact that the most modern and relevant standard is Serial Attached SCSI; it appeared relatively recently, but has undergone rapid development. The first implementation “in silicon” was shown by LSI in January 2004, and in November of the same year, SAS entered the top popular queries site storagesearch.com.

Let's start with the basics. How do devices using SCSI technology work? The SCSI standard is all about the client/server concept.

The client, called the initiator, sends various commands and waits for their results. Most often, of course, the SAS controller acts as the client. Today, SAS controllers are HBA and RAID controllers, as well as storage controllers located inside external storage systems.

The server is called a target device, its task is to accept the initiator’s request, process it and return data or confirmation of the command back. The target device can be either a separate disk or an entire disk array. In this case, the SAS HBA inside the disk array (the so-called external storage system), designed to connect servers to it, operates in Target mode. Each target device is assigned a separate SCSI Target ID.

To connect clients with the server, a data delivery subsystem is used (English Service Delivery Subsystem), in most cases, this tricky name hides just cables. Cables are available for external connections, and for connections within servers. Cables change from generation to generation of SAS. Today there are three generations of SAS:

SAS-1 or 3Gbit SAS
- SAS-2 or 6Gbit SAS
- SAS-3 or 12 Gbit SAS – is being prepared for release in mid-2013

Internal and external SAS cables

Sometimes this subsystem may include SAS extenders or expanders. Expanders (English Expanders, expanders, but the word “expander” has taken root in Russian) are understood as devices that help deliver information from initiators to targets and back, but are transparent to target devices. One of the most typical examples is an expander, which allows you to connect several target devices to one initiator port, for example, an expander chip in a disk shelf or in the backplane of a server. Thanks to this organization, servers can have more than 8 disks (controllers that are used today by leading server manufacturers are usually 8-port), and disk shelves can have any required number.

The initiator connected to the target device by the data delivery system is called a domain. Any SCSI device contains at least one port, which can be an initiator port, a target port, or a combination of both. Ports can be assigned identifiers (PIDs).

Target devices consist of at least one Logical Unit Number or LUN. It is the LUN that identifies which of the disks or partitions of this target device the initiator will work with. The target is sometimes said to provide the initiator with a LUN. Thus, for complete addressing to the required storage SCSI Target ID + LUN pair is used.

As in the well-known joke (“I don’t lend money, and First National Bank doesn’t sell seeds”), the target device usually does not act as the “sender of commands,” and the initiator does not provide a LUN. Although it is worth noting that the standard allows for the fact that one device can be both an initiator and a target, this is rarely used in practice.

For the “communication” of devices in SAS, there is a protocol, according to the “good tradition” and according to the OSI recommendation, divided into several layers (from top to bottom): Application, Transport, Link, PHY, Architecture and Physical.

SAS includes three transport protocols. Serial SCSI Protocol (SSP) - used to work with SCSI devices. Serial ATA Tunneling Protocol (STP) - for interaction with SATA drives. Serial Management Protocol (SMP) - for managing the SAS fabric. Thanks to STP, we can connect SATA drives to SAS controllers. Thanks to SMP, we can build large (up to 1000 disk/SSD devices in one domain) systems, and also use SAS zoning (more about this in the article about the SAS switch).

The link layer is used to manage connections and transfer frames. PHY layer - used for things like setting connection speed and encoding. At the architectural level there are issues of expanders and topology. The physical layer defines voltage, connection waveforms, etc.

All communication in SCSI is based on commands that the initiator sends to the target device and waits for their results. These commands are sent in the form of command description blocks (Command Description Block or CDB). A block consists of one byte of command code and its parameters. The first parameter is almost always LUN. A CDB can be anywhere from 6 to 32 bytes in length, although recent versions of SCSI allow variable-length CDBs.

After receiving the command, the target device returns a confirmation code. 00h means that the command was received successfully, 02h means an error, 08h - busy device.

Teams are divided into 4 large categories. N, from English “non-data”, are intended for operations not directly related to data exchange. W, from “write” - recording the data received by the target device from the initiator. R, as you might guess from the word “read”, is used for reading. Finally B - for two-way data exchange.

There are quite a lot of SCSI commands, so we will list only the most frequently used ones.

Test unit ready (00h) - check whether the device is ready, whether there is a disk in it (if it is a tape drive), whether the disk has spun up, and so on. It is worth noting that in this case the device does not perform full self-diagnosis; there are other commands for this.
Inquiry (12h) - get the main characteristics of the device and its parameters
Send diagnostic (1Dh) - perform self-diagnosis of the device - the results of this command are returned after diagnostics with the Receive Diagnostic Results (1Ch) command
Request sense (03h) - the command allows you to get the execution status of the previous command - the result of this command can be either a message like “no error” or various failures, from the absence of a disk in the drive to serious problems.
Read capacity (25h) - allows you to find out the capacity of the target device
Format Unit (04h) - serves to destructively format the target device and prepare it for data storage.
Read (4 options) - reading data; exists in the form of 4 different commands, differing in CDB length
Write (4 options) - record. Same as for reading in 4 versions
Write and verify (3 options) - data recording and verification
Mode select (2 options) - installation various parameters devices
Mode sense (2 options) - returns current device parameters

Now let's look at a few typical examples of organizing data storage on SAS.

Example one, data storage server.

What is it and what is it eaten with? Large companies such as Amazon, Youtube, Facebook, Mail.ru and Yandex use servers of this type to store content. Content means video, audio information, pictures, results of indexing and information processing (for example, Hadoop, so popular recently in the USA), mail, etc. To understand the task and correctly select equipment for it, you need to additionally know a few introductory information, without which it is absolutely impossible. First and most importantly, the more disks, the better.

Data center of one of the Russian Web 2.0 companies

Processors and memory in such servers are not used much. Secondly, in the world of Web 2.0, information is stored geographically distributed, with several copies on different servers. 2-3 copies of information are stored. Sometimes, if it is requested frequently, more copies are stored to balance the load. Well, thirdly, based on the first and second, the cheaper the better. In most cases, all of the above leads to the use of high-capacity Nearline SAS or SATA drives. As a rule, Enterprise-level. This means that such drives are designed to operate 24x7 and are significantly more expensive than their counterparts used in desktop PCs. The case is usually chosen one where you can insert more disks. If it is 3.5’’, then 12 disks in 2U.

Typical 2U storage server

Or 24 x 2.5’’ in 2U. Or other options in 3U, 4U, etc. Now, having the case, the number of disks and their type, we must select the connection type. Actually, the choice is not very large. And it comes down to using an expander or non-expander backplane. If we use an expander backplane, then the SAS controller can be 8-port. If expander-free, then the number of SAS controller ports must be equal to or exceed the number of disks. And finally, the choice of controller. We know the number of ports, 8, 16, 24, for example, and select a controller based on these conditions. There are 2 types of controllers, RAID and HBA. They differ in that RAID controllers support RAID levels 5,6,50,60 and have a fairly large amount of memory (512MB-2GB today) for caching. The HBA either has no memory at all or very little of it. In addition, HBAs either do not know how to do RAID at all, or they can only do simple levels that do not require a large amount of calculations. RAID 0/1/1E/10 is a typical set for HBA. Here we need an HBA, they are much cheaper, so we don’t need data protection at all and we strive to minimize the cost of the server.

16-port SAS HBA

Example two, mail server Exchange. As well as MDaemon, Notes and other similar servers.

Here everything is not as obvious as in the first example. Depending on how many users the server needs to serve, the recommendations will vary. In any case, we know that the Exchange database (the so-called Jet database) is best stored on RAID 5/6 and is cached well using an SSD. Depending on the number of users, we determine the required storage volumes “today” and “for growth.” We remember that the server lives for 3-5 years. Therefore, “for growth” can be limited to a 5-year perspective. Then it will be cheaper to completely change the server. Depending on the volume of the disks, we will choose the case. It’s easier with the backplane; it is recommended to use expanders, since the price requirements are not as strict as in the previous case, and in general, we can easily tolerate an increase in the cost of a server by $50-$100, and sometimes more, for the sake of reliability and functionality. We will choose SAS or NL-SAS/Enterprise SATA disks depending on the volume. Next, data protection and caching. Let's choose a modern 4/8-port controller that supports RAID 5/6/50/60 and SSD caching. For LSI, this is any MegaRAID except 9240 with the CacheCade 2.0 caching function, or Nytro MegaRAID with an on-board SSD. For Adaptec, these are controllers that support MAX IQ. For caching in both cases (except for Nytro MegaRAID), you will need to take a pair of SSDs based on Enterprise-class e-MLC technology. Intel, Seagate, Toshiba, etc. have these. Prices and companies are up to you to choose from. If you don’t mind paying extra for the brand, then in the server lines of IBM, Dell, HP, find similar products and go ahead!

Nytro MegaRAID SSD caching RAID controller

Example three, do-it-yourself external data storage system.

So, the most serious knowledge of SAS, of course, is required for those who produce data storage systems or want to make them themselves. We will focus on a fairly simple storage system, software for which it is produced by Open-E. Of course, you can make storage systems on Windows Storage Server, and on Nexenta, and on AVRORAID, and on Open NAS, and on any other software suitable for these purposes. I just outlined the main directions, and then the manufacturers’ websites will help you. So, if it's an external system, we almost never know how many disks the end user will need. We must be flexible. For this there are so-called JBODs - external disk shelves. They include one or two expanders, each of which has an input (4-port SAS connector), an output to the next expander, the remaining ports are routed to connectors intended for connecting disks. Moreover, in two-expander systems, the first port of the disk is routed to the first expander, the second port is routed to the second expander. This allows you to build fault-tolerant chains of JBODs. The head server may have internal disks or not have them at all. In this case, “external” ones are used SAS controllers. That is, controllers with ports “outside”. The choice between a SAS RAID controller or a SAS HBA depends on the management software you choose. In the case of Open-E, this is a RAID controller. You can also take care of the caching option on SSD. If your storage system will have a lot of disks, then the Daisy Chain solution (when each subsequent JBOD connects to the previous one, or to the head server) is not suitable for many reasons. In this case, the head server is either equipped with several controllers, or a device called a SAS switch is used. It allows you to connect one or more servers to one or more JBODs. We will look at SAS switches in more detail in the following articles. For external data storage systems, it is strongly recommended to use only SAS disks (including NearLine) due to increased requirements for fault tolerance. The fact is that the SAS protocol includes much more features than SATA. For example, control of written-read data along the entire path using checksums (T.10 End-to-End protection). And the path, as we already know, can be very long.

Multi-disk JBOD

This concludes our excursion into the world of history and theory of SCSI in general and SAS in particular, and next time I will tell you in more detail about the use of SAS in real life.