How to create an alternative ntfs file stream. Four Kinds of NTFS Metadata

Useful tips

In this topic, I will look at four types of metadata that can be attached to a file or directory using the file system NTFS. I will describe for what purposes this or that type of metadata can be used, and I will give an example of its use in some Microsoft technology or third-party software.

We will talk about reparse points, object ids and other types of data that a file may contain in addition to its main content.

Object ID this is 64 bytes that can be attached to a file or directory. Of these, the first 16 bytes allow you to uniquely identify a file within the volume and access it not by name, but by identifier. The remaining 48 bytes can contain arbitrary data.

Object IDs have existed in NTFS since Windows 2000. In the system itself, they are used to track the location of the file that a shortcut (.lnk) refers to. Let's say the file referenced by the shortcut has been moved within the volume. When you launch the shortcut, it will still open. Special Windows service if the file is not found, it will attempt to open the file not by its name, but by a previously created and saved identifier. If the file was not deleted and did not leave the volume, it will open, and the shortcut will again point to the file.

Object identifiers were used in the iSwift technology of Kaspersky Anti-Virus version 7. This is how this technology is described: The technology was developed for the NTFS file system. In this system, each object is assigned an NTFS identifier. This identifier is compared with values in a special iSwift database. If the database values with the NTFS identifier do not match, then the object is checked or rechecked if it has been modified.

However, an overabundance of created identifiers caused problems with scanning the disk with a standard utility chkdsk checks, it's been going on for too long. In subsequent versions of Kaspersky Anti-Virus, the use of NTFS Object Id was abandoned.

Reparse Point

In the NTFS file system, a file or directory may contain a reparse point, which is translated into Russian as "reprocess point". Special data is added to a file or directory, the file ceases to be regular file and only a special file system filter driver can process it.

Windows contains reparse point types that can be processed by the system itself. For example, reparse points in Windows implement symbolic links (symlinks) and junction points (junction points), as well as mount points for volumes in a directory.
A reparse buffer appended to a file is a buffer that has maximum size 16 kilobytes. It is characterized by the presence of a tag that tells the system what type the reparse point belongs to. When using a reparse buffer own type You also need to set the GUID in a special field, and it may not be available in Microsoft reparse buffers.

What types of reprocessing points are there? I will list the technologies that use reparse points. These are Single Instance Storage (SIS) and Cluster Shared Volumes in Windows Storage Server 2008 R2, Hierarchical Storage Management, Distributed File System (DFS), Windows Home Server Drive Extender. These are Microsoft technologies technologies not mentioned here third party companies, using reprocessing points, although there are also such.

Extended Attributes

Extended file attributes. It was about them. It is only worth mentioning here that this technology is practically not used under Windows. From what I know software only Cygwin uses extended attributes to store POSIX permissions. A single file on NTFS can have either extended attributes or a reparse point buffer. Simultaneous installation of both is impossible. The maximum size of all extended attributes in one file is 64 KB.

Alternate Data Streams

Additional file streams. Probably everyone already knows about them. I will list the main features of this type of metadata: naming (that is, a file can have several streams, and each has its own name), direct access from the file system (they can be opened using the format “file name, colon, stream name”), unlimited size , the ability to run a process directly from a thread (and the ability to implement it through it).

Used in iStream technology of Kaspersky Anti-Virus. They are used in Windows itself, for example, when downloading a file from the Internet, a Zone.Identifier stream is attached to it, containing information about the location from which it was received this file. After running the executable file, the user may see a message “Unable to verify publisher. Do you really want to run this program?.

This gives the user additional protection against the thoughtless launch of programs obtained from the Internet. This is just one use of streams, and they can store a wide variety of data. The mentioned Kaspersky Anti-Virus was stored there checksums each file, but later this technology was also abandoned for some reason.

Anything else?

Is there some more security id, plus standard file attributes that are not directly accessible, even though they are also implemented as file streams. And they, and extended attributes, and reparse and object id - all these are file streams from the point of view of the system. There is no point in directly changing the security identifier, shown in the following picture as::$SECURITY_DESCRIPTOR; let the system deal with changing it. The system itself does not provide direct access to other types of streams. So that's it.

Viewing the contents of object id, reparse points, as well as working with extended attributes and alternative file streams is possible using the program

Visibly-invisibly

Blog reader Victor was unable to run the PowerShell script he downloaded from the Internet. Carefully reading my instructions avoided the problem, but the root of it was not PowerShell's strict security policies.

Victor downloaded an archive with the PSWindowsUpdate.zip script from the TechNet gallery for Windows management Update that I talked about. However, the unpacked script refused to work. When I suggested to the reader that the first point of my instructions talked about the need to unlock the archive, everything went like clockwork.

Victor asked to explain why the system blocked the script, and how it knew that the archive was downloaded from another computer.

To be honest, today's topic is not new, but I decided to cover it on my blog for several reasons:

Many articles were written back in Windows times XP or Windows 7 and do not take into account the built-in capabilities of newer Microsoft operating systems.
One of the articles planned for the near future touches on this topic, and it is easier for me to refer to material for the relevance and correctness of which I myself am responsible.
The blog has a large audience, and for many readers this topic will still be new :)

Today on the program

NTFS data streams

Windows gets information about the file's source from the alternative data stream (ADS) of the NTFS file system. In the file properties, she modestly writes that it is from another computer, but in reality she knows a little more, as you will see later.

From an NTFS point of view, a file is a collection of . The contents of the file is a data attribute named $DATA. For example, a text file with the line “Hello, World!” has the data attribute “Hello, World!”

IN NTFS attribute$DATA is a data stream and is called main or unnamed because... it has no name. Formally, it looks like this:

$DATA:""

$DATA- Name attribute
: - delimiter
"" - Name flow(in this case there is no name - there is nothing between the quotes)

Interesting Features of Alternative Data Streams

In the context of the examples above, I want to make a few interesting points.

Invisible changes

Having created a text file with the first command, you can open it in text editor and make sure that all further manipulations do not affect the contents of the file in any way.

It gets interesting when the file is opened, say, in Notepad++. This editor can warn you about file changes. And it will do this when you write an alternative stream to the file, but the content will remain the same!

Record and view ADS from CMD

ADS can be created and displayed from command line. The following commands write hidden text to a second ADS named MyStream2 and then display it.

Echo Hidden Text > C:\temp\test.txt:MyStream2 more< C:\temp\test.txt:MyStream2

Viewing ADS in text editors

The same Notepad++ will show you the contents of ADS if you specify the name of the stream on the command line

"C:\Program Files (x86)\Notepad++\notepad++.exe" C:\temp\test.txt:MyStream1

Result:

With notepad, this trick will work only if there is a .txt. The commands below add a third ADS and open it in Notepad.

Echo Hidden Text > C:\temp\test.txt:MyStream3.txt notepad C:\temp\test.txt:MyStream3.txt

Result:

Blocking downloaded files

Let's get back to the question a reader asked me. Whether a file will be blocked depends primarily on the program in which it was downloaded, and secondly on the OS settings. So, all modern browsers support blocking, and it is included in Windows.

Remember that when an archive is locked, all unpacked files will be locked “inheritedly”. Also remember that ADS is a feature of NTFS, i.e. When saving or unpacking an archive on FAT32, no blocking occurs.

View information about the source of a blocked file

In PowerShell, go to the folder with the downloaded file and see information about all the threads.

Get-Item .\PSWindowsUpdate.zip -Stream * FileName: C:\Users\Vadim\Downloads\PSWindowsUpdate.zip Stream Length ------ ------ :$DATA 45730 Zone.Identifier 26

As you already know, $Data is the contents of the file, but ADS also appears in the list Zone.Identifier. This is a clear hint that the file was received from some zone. Do you know where this picture is from?

To find out the zone, you need to read the contents of the ADS.

Get-Content .\PSWindowsUpdate.zip -Stream Zone.Identifier ZoneId=3

Obviously, it is aimed at batch unlocking (for example, when the archive is already unpacked). The command below will unlock in the Downloads folder all files containing in the name PS:

Dir C:\Downloads\*PS* | Unblock-File

Of course, there are all sorts of utilities with a graphical interface, even those that can be integrated into context menu. But, in my opinion, PowerShell or, at worst, streams is quite enough.

How to prevent files from being blocked

The blocking is controlled by the group policy Do not store information about the zone of origin of attached files. As the name suggests, blocking is standard Windows behavior, and the policy allows you to change it.

However, it is not obvious from the name that the policy applies not only to mail attachments, but also files downloaded from the Internet. Read more about the attachment manager in KB883260.

In the editor's home editions group policies no, but no one canceled the registry: SaveZoneInformation.zip.

Other examples of practical application of ADS

The scope of ADS is not limited to adding a downloaded file zone, nor is it necessary to store only text in ADS. Any program can use this NTFS feature to store any kind of data, so I'll just give a couple of examples from different areas.

File Classification Infrastructure

about the author

Interesting material, thank you. I learned something new about PowerShell, which is still a little familiar to me :)

I often use WhatsApp to communicate with my family - so far there have been the fewest problems with this service, even my parents have gotten used to it. Kontaktik is also mainly for family, although the exchange of messages there is mainly around published albums with photos and videos. Some relatives remain faithful to Viber - it didn’t work out for me, I just keep it for them, without giving up trying to drag them to WhatsApp.

For work, mainly Slack, when something is urgent - WhatsApp, very urgent - SMS. VKontakte for communication about working with the outside world.

I use Skype only for video calls, mainly with my family. I would gladly replace it with WhatsApp if there were video calls.

urix

Viber now has video calls, and even video calls for the desktop version. So maybe Viber will be the next Skype... in in a good way

Andrey Kuznetsov

Interesting material, thank you. I knew about the existence of threads, but I didn’t know that it was so easy to work with them through PowerShell.
As for IM: I only have complaints about Skype regarding the launch time Windows Phone. There is no such problem on iPad and Windows. I use it for voice communication when for some reason it is inconvenient to use GSM.
And correspondence via Whatsapp. Having it only on the phone is rather a plus, from a privacy point of view.

Andrey Kuznetsov: And correspondence via Whatsapp. Having it only on the phone is rather a plus, from a privacy point of view.

Andrey, explain what is the plus here?

Pavlovsky Roman

1. I use most often: Skype and Hangouts - for work on a PC, for other correspondence on VKontakte from any device, since work clients usually use Skype, and friends and acquaintances on Social Networks.

2. I would ideally like to use: Jabber - for correspondence and calls from any device. As for me, the client can be installed on any device and correspond with each other wherever the user is, even on a weak Internet connection + in addition, you can deploy your own jabber server and store all correspondence on the server, so that later you can quickly find the necessary correspondence, if the client does not know how to store history, and plugins for calls via Jabber can be found (for example, through the same SIP Asterisk 1.8+)

Andrey Bayatakov

Most often I use WhatsApp (mainly for work), for calls (audio/video/international calls) Skype. Although desktop Skype is terribly infuriating (I have a transformer and at home I use it mainly as a tablet)… Viber has not caught on. To make calls via WhatsApp you just need to have nerves of steel. You say something to your interlocutor and wait a minute or two for him to hear you (50Mbit connection)…
If there was an opportunity, I would completely switch to Skype. On Windows 10 Mobile, after a recent update, messages from Skype come directly to the built-in Messages application (like SMS), which is very convenient.

Maxim

1. Reluctantly, I use ICQ (for retrograde customers) and Slack (for more modern ones).
2. I would like to use Jabber - for the same reasons as Roman Pavlovsky above.

Vladimir Kiryushin

Hello Vadim!
Before this article, I read your article about how to read a check report for everything system disk with the chkdsk command. Great article! Thanks to her, today after checking the system disk with the chkdsk command, I received a text file of the report. And this article also clarifies a lot of things in PowerShell program. Some things are incomprehensible to me as a pensioner, but I try not to panic and read diligently to the end. Thank you for the study you are doing with us! All the best to you!

Lecron

What browsers and downloaders create this stream?

What other options are there for the user to use threads? And in particular, a script writer user? Because, although I knew about them for a long time, I never used them. At real work with a computer you simply don’t remember about them, and because of this, you may use crutches instead convenient tool, and without this work, from memory, it’s impossible to come up with anything.
I only thought of one option. A comment to the file, if there is no possibility or desire to write long text in the file name. But this requires support from the file manager, who previously, and even now, writes them to descript.ion or files.bbs.

Speed Guru

Another garbage technology like USN magazine. How much use will you get from ZoneIdentifier or from a virus attached to a file or folder? Of course not. Moreover, this is cluttering the system with unnecessary “sub-files” that are in no way needed by a normal user. Each extra read in the MFT directory and other operations associated with the maintenance and maintenance of alternative threads are extra spent processor cycles, random access memory, and most importantly, an extra load on the hard drive.
You can tell me that this technology is very necessary for the system. But this is nonsense - the system would work perfectly without threads. But no one asks the user - they sold it (like a USN magazine) and did not give the opportunity to completely disable the maintenance of these flows. But as users, I don’t need them at all, I think like you…
All we can do is “streams -s -d %systemdrive%”. But this also does not make it possible to delete threads on the system partition.

Alexiz Kadev

Named streams are a great thing, and they existed, as far as I remember, from the first release of NTFS. It’s quite convenient to store, for example, document versions in named streams, which, if I’m not mistaken, a number of applications have done. But there remains an ambush with copying to another file system - named streams are simply cut off.

It’s a pity that it was impossible to select several messengers in the poll: I use several, since some of my contacts prefer certain ones. So, I use WhatsUp, ICQ (though, of course, not a native client), Skype, SkypeforBusiness (quiet horror, not a client, however, when it was called Lync it was even worse) and Viber (here there is more spam than in others at least once at 5).
And ideally, use just one, like Miranda with plugins, since finding, if necessary, who said/wrote something when in this whole bunch is simply unrealistic. But alas, a number of manufacturers close their protocols and protect them like Kashchei protects his needle.

Vladimir Kokarev

VSh

Vadim Sterkin: Roman, I did not include Jabber in the survey. I decided that few people use it and there are no prospects.

In vain
For example, I use OpenFire (freeware xmpp) as an office communicator on several domains.

Therefore, my main one is XMPP (Pidgin.exe, Spark.exe), but 99.8% of these messages are intradomain.
Skype - for external IM
WhatsApp and Viber are for “random connections”, the last n months have been nothing but SPAM, I’m thinking – should I delete it?

Artem

For some reason everything is on Viber. And the quality of communication is quite satisfactory. Otherwise there would be a telegram. It's empty there.

hazet

1. Skype (on PC) and Viber (on Mobile). The reasons are basically the same as for most - the number of available contacts and, naturally, the reluctance of these same contacts to switch to another messenger.
2.uTox. Miniature, nothing superfluous, client for Win, Linux, Mac and Android. Positioned as protected.
P.S. I'll start dragging my contacts onto it more closely :-)

Evgeniy Karelov

Thank you for your work!

Regarding the survey, on a PC for correspondence I use QIP 2012, to which I am connected ICQ contacts, VKontakte and others. Personally, it’s convenient for me to use one program to communicate over several protocols. And the ability to view social media feeds from one place is very pleasing. Ideally, the only thing missing is support for Skype, which I use for voice communication, but it obviously won't appear.
Although this program looks “abandoned”, because there have been no updates for a long time, it performs its assigned functions perfectly.

strafer

An interesting mixture of the topic of the post about data flows and the IM survey.

According to the survey: Jabber/Jabber, which you shouldn’t have included in the list, although there is WhatsApp based on XMPP, and even Asechka, which is heading towards success.

Jabber, in general, solves all these problems due to the openness of the protocol, the availability of clients for many platforms, and the availability of servers that can be set up independently. But chewing cacti is more traditional, yes.

The list includes clients, not protocols.
ICQ... well, I didn’t put emoticons there, because it should be clear.
Jabber definitely doesn't solve one problem - no one is there.
- strafer
  
  Vadim Sterkin: Clients are listed, not protocols.
  
  Due to the fact that the protocol and source codes of the official client are closed, a natural identity is established between the only client and the protocol.
  
  Vadim Sterkin: ICQ... well, I didn’t put emoticons there, because it should be clear.
  
  It is not enough for the rotten mail girl that the asechka dies a natural death - they also make additional efforts to make it die faster.
  
  Vadim Sterkin: Jabber definitely doesn't solve one problem - there's no one there.
  
  Nevertheless, you yourself wrote for Telegram
  
  looks great, but it's empty (which can be fixed)
  
  Jabber had every chance of becoming what the e-mail ecosystem is today (complete openness of the protocol, the ability to set up your servers for anyone and ensure interaction between servers, etc.), but corporations do not need this, which is clearly seen in the example of the departure from him Google or proprietary WhatsApp.
  - For Telegram - fixable, for Jabber - very unlikely. Therefore, the first one is on the list, but the second one is not.
    - strafer
      
      Of course, Telegram is stylish, fashionable, youthful, but Jabber is not used by anyone cool like Pasha Durov. What are the prospects here?
      
      Hm... come out of your tank of “the whole world is against free software” conspiracy theories. All much easier
      
      If it’s not clear, this is what a person’s first experience of interacting with the officially recommended Jabber client on the most common mobile platform looks like.
      
      strafer
    - I didn’t understand a little where in my comment about the conspiracy.
      
      Yes, everywhere :) You are trying to attribute the failures of jabber to unfashionability and lack of youth, while its clients from the first screen are not adapted to modern reality.
      
      What should I see in the screenshot?
      
      Prompt to enter a phone number ~~~O~

Recently, due to the reduction in the cost of hardware (in dollar equivalent), an increasing number of computer users have at their disposal resources quite sufficient for work operating system Microsoft Windows NT (i200MMX + 32-64 Mb). The unreliability and unpredictability of Windows 95/98, as well as its inability to manage resources at the proper level modern computers leads many users to think about switching to NT.

At the same time, many inexperienced users do not find anything radically new for themselves. And indeed, having installed Internet Explorer 4 and without taking advantage of NT's many security policy enforcement capabilities, the biggest difference from Windows 98 may seem to be the presence of two Startup folders in the start menu ( current user and common for all users) and the absence of the Add/Remove Hardware applet in the Control Panel. And if you don’t format the disk with the NTFS file system, then you may not find much of a difference.

But this article just describes some of the differences between NTFS and FAT, VFAT, FAT16 and FAT32. Well-known differences: self-healing ability, lazy writing, maximum volume and file size up to 16 Exabytes ( 1 Exabyte = 1,000,000 GB), the ability to compress individual files and folders, set permissions and audit are quite widely covered in the literature and documentation for Windows NT. But there are still little-known and little-used features of NTFS: hard links (hardlinks) and multiple data flows (multiply data flows or forks). Next we will talk about them.

Multiple Data Streams. This term is familiar to Macintosh users. In this system, a file can have two forks: a data stream and a resource stream. The data stream stores the file data - this stream is copied as the only one when transferring a file from a Macintosh to a PC. The second stream of the file is a resource stream containing operating system data - menus, icons, fonts, in general, everything that is commonly called resources. When Windows NT Server serves Macintosh clients and provides them with disk space to store files, the server's file system must support the client's file format. This is one of the reasons for the appearance of multiple data streams in NTFS.

How is this implemented? Any information about a file, starting with its name, permissions and ending with the actual data stored in the file, from the point of view of NTFS is an attribute stored in its own stream. The developers of NTFS considered that it is possible not to be limited to one stream for data - an unnamed one, and added the ability to create several, in addition to the main, named streams. To create multiple threads, you can use the Win32 API function, but there is an easier way.

Since the days of Kernighan and Ritchie, the developers of the C language and the operating system UNIX systems Many operating systems have the ability to generalize input-output operations. From this point of view, any I/O operation can be considered as an operation of input from a stream or output to a stream, regardless of what the data source is (console, i.e. keyboard, file or port) and receiver (again the console, in this case). case already a monitor screen, printer or file). It is also possible to redirect input - program output from the screen to the printer and enter commands not from the keyboard, but from a file. In our time of widespread use of graphic user interface These possibilities are used very rarely, so let’s illustrate this with an example.

The Microsoft operating systems echo command is used to display information on the screen in text mode:

C:>echo Hello, World!

The echo command uses the monitor screen as an output device. The output of this command can be redirected from the console to a file (use the “>” symbol for this):

C:>echo Hello, World! > file

As you can see, the echo command in this case did not display anything on the screen. But in the file file you can find the line “Hello, World!”. Similarly, the output of the echo command can be redirected to a printer:

C:>echo Hello, World! >lpt1

There is nothing on the screen again, but on a sheet of paper in the printer you can find the same line “Hello, world!”, if of course the printer is connected to the lpt1 port. Thus, the output of any text mode program can be redirected to any device that supports streaming information input or to a file, with the exception of those programs that in text mode use direct modification of video memory and other non-standard, from the point of view of classical C, capabilities to output information.

Similarly, you can redirect program input. The more command of Microsoft operating systems is used to buffer the output of commands that display more information than fits on the screen. But this command can also be used to illustrate input redirection:

C:>more Hello, World!

The file file contained the string “Hello, World!”, which was directed to the screen.

Similarly, using I/O redirection, you can create and read multiple data streams:

C:>echo string1 > file:fork1

The entry file:fork1 defines a stream named fork1 in the file file (since it does not exist yet, a new one is created with this name) and redirects the output of the echo command to it. In this case, the file size does not change when viewing its properties, and its existence cannot be determined using standard Windows NT tools without knowing the name of the thread. But, knowing its name, you can use the more command to determine its contents:

In this way, you can create and read the contents of file data streams. The number of threads created in one file is limited only by the availability free space on disk. Similarly, you can create data streams in directories, but to view the contents of the stream you will have to use another means of displaying the stream on the screen, since the more command produces the following error:

If you couldn’t find anything suitable, you can write the following program in any C++ compiler:

while (cin.get(ch)) cout.put(ch);

It is better to build this program as a Win32 console application, and use it as a tool for studying directory streams.

Windows NT does not provide standard means to obtain information about multiple data streams. But what if you still need such information? In this case, you can use the streams program by Mark Russinovich, which, together with source code can be obtained from www.sysinternals.com. This program uses undocumented Windows NT features to obtain information about multiple data streams. Here is the information obtained using the streams program about the file file:

NTFS Streams Enumerator v1.0

Systems Internals - http://www.sysinternals.com

Here you can see both the name of the data stream and its size in bytes (the additional 3 characters are the space after the “>” character, the carriage return and the line feed added by the echo command). Unfortunately, streams does not allow you to define multiple data streams in directories.

What can multiple data streams be used for? In addition to the use Apple has found for them, we can talk about the simplest means of hiding information, for example, to remember the installation date shareware programs. At dawn OLE technologies Microsoft intended to use data streams to store information about embedded objects, but apparently providing data streams on FAT turned out to be more difficult than creating long file names and the idea had to be abandoned. Creating a “resource file” for a script that stores all the output in different languages can also be an interesting possibility to use streams. In addition to the above, there are many more interesting applications for multiple data streams, so do not ignore them.

Hard links. Users of various UNIX clones are familiar with this concept. Unlike the FAT file system, which allows each file to have only one name, UNIX has no such restriction - each file can have multiple names and its data cannot be deleted until the file name count is 0. In UNIX also has symbolic links - an analogue of shortcuts in Windows, but they monitor the movement of the object to which they refer.

Windows NT has limited compliance POSIX standard(Portable Operating System Interface for Computing Environments). One example of limitation is support for hard links and lack of support for symbolic ones. Apparently, it was decided that shortcuts are a worthy analogue of symbolic links.

In NTFS, hard links are organized in a similar way to multiple data streams: if a file has multiple data streams, why can't there be multiple named streams? Multiple file names can be in different directories, but only within the same partition.

To create a hard link, you need a Windows NT POSIX subsystem program. Such a program, along with the source texts, is located on the CD “ Windows Resources NT". By analogy with UNIX, this program is called ln. The syntax for this command is:

C:>Ln file hardlink1

Using this command, we create a second name or hard link hardlink1 for the file file, and by changing the contents of the file file, you can change the contents of hardlink1, more precisely, it is the same file, but with two names. You can similarly change other file attributes. The number of names a file has is not limited, but when you copy a file name, the link is broken and another file is created. It is possible to create a link in another directory:

C:>Ln file ../temp/hardlink2

In this case, you must specify a relative directory name rather than an absolute one.

There are just as many uses for hard links as there are for multiple data streams. For example, create hard links for dll libraries to protect your program from accidentally deleting a necessary file. Other possible applications The best place to look for hard links is in UNIX literature. And, of course, the use of hard links can be combined with the multiple data streams described above.

Alternate Data Stream (AltDS) support was added to NTFS for compatibility with the Macintosh file system HFS, which used a resource stream to store icons and other file information. The use of AltDS is hidden from the user and is not accessible through normal means. Explorer and other applications work with the standard stream and cannot read data from the alternative stream. With AltDS you can easily hide data that cannot be detected standard checks systems. This article will give basic information about the working and definition of AltDS.

Creation of AltDS

Creating an AltDS is very easy. To do this, we will use the command line. First, let's create a base file to which we will attach our streams.

C:\>echo Just a plan text file>sample.txt
C:\>type sample.txt
Just a plan text file

Next, we'll use a colon as an operator to indicate that we'll be using AltDS:

C:\\>echo You can"t see me>sample.txt:secret.txt

You can use the following commands to view the content:

C:\more< sample.txt:secret.txt

C:\notepad sample.txt:secret.txt

If everything works well, you will see the text: You can't see me, but when opened from Explorer, this text will not be visible. You can also attach AltDS not only to a file, but also to a folder. To do this, create a folder and attach some kind of some text:

C:\>md stuff
C:\>cd stuff
C:\stuff>echo Hide stuff in stuff>:hide.txt
C:\stuff>dir
Volume in drive C has no label.
Volume Serial Number is 40CC-B506Directory of C:\stuff
09/28/2004 10:19 AM .
09/28/2004 10:19 AM…
0 File(s) 0 bytes2 Dir(s) 12,253,208,576 bytes free
C:\stuff>notepad:hide.txt

Now you know how to view and edit an attached AltDS using Notepad, as well as how to attach it to files and folders.

Hiding and launching applications

Hiding applications using AltDS is as easy as hiding test files. First, let's create the base file again:

Next, let's place our application in a stream, for example I used notepad.exe:

C:\WINDOWS>type notepad.exe>test.txt:note.exe

Now let’s make sure that our file contains the same text:

C:\WINDOWS>type test.txt
Test

And now the fun part, let’s launch our hidden application:

C:\WINDOWS>start .\test.txt:note.exe
C:\WINDOWS>

Since this article is not a complete translation of the article taken, it is formatted as a simple topic. Additional techniques can be found at the link provided.

UPD:

Utilities for working with AltDS (list taken from the article linked above):

LADS - List Alternate Data Streams by Frank Heyne
www.heysoft.de/Frames/f_sw_la_en.htm

Streams.exe from SysInternals.

Windows operating systems come with two little-known data hiding features: NTFS data streams (also known as alternate data streams) and Access-based Enumeration (ABE) resource enumeration. Alternative streams data allow you to add hidden information to a file, such as file information. You probably won't need to use hidden data streams, but attackers could use this technology against you, so you should be aware of it and how it might work.

As for the ABE method, it can add to your arsenal. This method allows you to do invisible folders and shared resource files for those users who do not have permission to access them.

Here's what you need to know about these funds.

Rivers feeding the sea of data

Alternate data streams are a feature of the NTFS file system. They were once included in Windows NT 3.1 to allow NT and Macintosh users to share files.

An NTFS file consists of data streams. This is the standard $DATA data stream, and possibly one or more alternative data streams. Any user with the necessary permissions to the file can see the existing $DATA data stream, can open it, and read and write data to the stream.

An alternate data stream is additional information or files that a user or application can attach to an NTFS file. Only the user who created it knows about the existence of an alternative data stream. Users typically do not know whether an alternate data stream is attached to a file; The point is that neither the content of this stream nor its name are visible. In addition, there is no way to see the change in file size.

There are many ways to use alternative data streams. On Windows, these streams are used to store summary data for documents created by applications that are not included with Microsoft Office, such as simple text files(.txt). You can enter summary information, such as title, subject, and author information, on the Summary tab of the file's Properties dialog box. This summary data is stored in an alternate data stream, SummaryInformation.

Windows applications such as Encrypting File System (EFS) and Windows Explorer, use alternative data streams to attach data relating to certain files to files that are stored on drives formatted under the NTFS system. EFS appends encoding and decoding information to encrypted files using alternate data streams, allowing for decentralized encryption and decryption by EFS.

Implemented in Windows XP Service Pack 2 (SP2) Microsoft application Internet Explorer (IE) uses an alternative Security.Zone data stream to provide security zone classification of files written to an NTFS volume. As a result, IE has the ability to block user-escalation attacks that can occur in situations where a user downloads malicious code from a non-secure area of the Internet and stores the code on the local hard drive. IE classifies locally stored content into the Local Machine security zone, which provides more rights than the Internet security zone. XP SP2 always checks the Security.Zone data flow before allowing loaded code to take any action on the local system.

Channel for introducing malicious code

What makes alternative data streams noteworthy and dangerous is that their names and contents are not displayed in Windows Explorer. Therefore, the organizers of various types of attacks consider such flows convenient means hiding data or malicious code that has entered the system. An example of the use of these threads is the VBS.Potok@mm worm. Hackers used an alternate data stream to attach several language scripts to an existing ODBC .ini file. Visual Basic(VB).

When activated, the worm creates account with administrative powers and sends himself to addresses that he himself discovers in address book Microsoft Outlook.

Another danger is that the disk space allocated for alternative data streams is not reflected in Windows Explorer's (file) size and unallocated disk space data. A hacker can use alternative data streams to fill disk space file server, and the administrator will only have to rack his brains, trying to get to the bottom of the problem. Additionally, it should be noted that the Dir command line utility does not take alternate data streams into account when calculating size data (files and folders). Currently, there is only one Microsoft tool that can take into account alternative data streams when calculating sizes: the Chkdsk utility.

Adding a new thread

Anyone with write access to an NTFS file can use normal operating system commands to append an alternate data stream to the file. Eg, next command creates an alternate data stream mystream, appends mystream to a file called file.txt, and stores the phrase "top secret" in mystream.

echo top secret > file.txt: mystream

You can view the contents of the mystream using the command

As noted above, you can add to alternative data streams executable files. Thus, it is possible to add a hidden copy of the Windows calculator (calc.exe) to the file file.txt. To do this you just need to enter the command

type calc.exe > file.txt: calc.exe

To launch a hidden calculator, enter the command

start .file.txt: calc.exe

You can verify for yourself that alternative data streams and their contents are not displayed in tools ah Microsoft. Open Windows Explorer and view the properties of file.txt. The actual file size is 112 KB (that's how much space the embedded calc.exe file takes up) - but the program will show the file size as 0 KB: there is no information about the embedded file in the $DATA data stream, and Windows application Explorer does not have the ability to read information from an alternate data stream.

It is clear that there are many threats associated with alternative data streams, especially in networks where the work of issuing permissions to access NTFS resources is not given due attention and strict access control to Windows servers is not established. There is a simple security mechanism that can prevent hackers from trying to take advantage of alternative data streams - the NTFS access control system. If attackers do not have permission to write data to a file, they will not be able to create alternate data streams and append them to the file.

Detecting Changes

If you feel that hackers have bypassed your permissions, use one of the alternative content discovery tools that have been developed to date. System integrity checkers, such as Tripwire Enterprise and Tripwire for Servers, can detect all changes to the NTFS file system that have occurred on a Windows system, including additions or changes to the contents of the data stream.

Sysinternal's Streams program is a free command-line utility that determines the names of alternate data streams attached to files. Figure 1 shows how to use the Streams utility to view the name of the calc.exe data stream that we previously added to the file.txt file. This utility can be downloaded from http://www.sysinternals.com/utilities/streams.html.

Another easy way to detect an alternate data stream is to use Windows Explorer to copy suspicious file to a drive with a file system other than NTFS (say, a FAT drive). Other file systems are not equipped to handle alternative data streams. So if you try to copy an NTFS file with alternate data streams attached to place it on a different file system, NTFS will issue a warning similar to the one shown in Figure 2. But be aware that if you copy this file in the Command Prompt window using the Copy command , Windows will copy it to a non-NTFS file system and delete the data stream without warning.

Hiding shared resources using ABE

ABE is an optional file sharing layer feature that Microsoft first implemented in Windows Server 2003 SP1. It can be used in any Windows shared directory, regardless of the file system on which the shared data is stored. ABE allows administrators to hide folders and files stored on shared resources from users who do not have appropriate permissions to access them at the NTFS level. In other words, we are talking about providing security at the folder level.

In cases where ABE is not used, users connecting to a shared directory will see all files and folders located on the share, including those that they do not have read permissions to read and those that they are blocked from accessing. When a user tries to open a file or folder that they are not allowed to access, the system displays an error message explaining that access is denied. These error messages can be confusing for users, so enabling ABE can help reduce support workload.

However, using ABE also has its disadvantages. Before returning a list of objects in a folder to a client connected to a share, the server must check all ACLs on those objects so that it can determine what data to return. As a result, you may experience a significant decrease in system performance, especially when accessing shared resources that contain many objects.

It is advisable to use ABE tools, for example, to configure public resources in user home directories. Instead of creating a hidden share for each user's home directory, you can create one share that contains the home directories of all users under the root home directory folder. Users will connect to this root directory, and you can use ABE as well as NTFS permissions to control the visibility of all users' home directories.

Activating the ABE function

This function uses the new public resource level flag SHI1005_FLAGS_ENFORCE_NAMESPACE_ ACCESS; at the time these lines are being written, it is implemented only in Windows 2003 SP1 and Release 2 (R2). This flag means that you are applying the ABE function to one of the folders.

You can use property extensions to set the flag Windows folders Explorer or the abecmd.exe command line tool. Microsoft distributes the ABE Explorer extension and abecmd.exe in the ABE installation package, which is an additional module for Windows platforms Server 2003 SP1. Installation package can be downloaded from Microsoft at http://www.microsoft.com/downloads/details.aspx?FamilyId=04A563D 9-78D9-4342-A485-B030AC442084. Since ABE is a server extension, it can be used no matter what Windows version installed on the client.

After installing ABE tools on the server, you can set this flag for a particular folder. Right-click the folder, select Properties, go to the Access-based Enumeration tab, and set the Enable access-based enumeration on this shared folder flag, as Figure 3 shows. To apply the ABE function to all shared resources on the system, set the Apply flag this folder"s setting to all existing shared folders on this computer.

The second method is to use the abecmd.exe command line tool. To apply the ABE function to the shareddocs public resource, enter the following command:

abecmd /enable shareddocs

You can use the /all option to enable ABE on all available resources, or use the /disable option to disable ABE.

Access Control

ABE is a simple tool that allows you to limit user permissions to only those files that they need to do their work. Users can easily find necessary files, because they don't have to wade through unrelated folders, and they don't bother the support team with questions about why files they don't have permission to open won't open.

To protect against alternate data stream hackers, administrators should monitor their public access control settings and use one of the utilities I described to identify hidden alternate data streams as well as changes to the NTFS system.

Jean De Clercq(declercq @hp.com) - employee of the Security Office of Hewlett-Packard. Responsible for identity and security management of Microsoft products. Author of Windows Server 2003 Security Infrastructures (Digital Press).