Pipe (Unix)

In the context of Unix and Unix-like operating systems, as well as MS-DOS, a pipe signifies that the output of one program ("stdout") feeds directly as input ("stdin") to another program. Any error messages from the first program ("stderr") are not passed on through the pipe. The Unix shell uses the pipe character ( | ) to join programs together. A sequence of commands joined together by pipes is known as a pipeline. Often filter programs form the constituent programs in a pipeline -- see pipes and filters. An example of a pipeline, which should print the numbers from 1 to 13: while : ; do echo ; done | head -n 13 | nl -ba A "true" pipe is implemented by a shell on a multi-tasking operating system by launching both the source and the destination process at the same time, and servicing the data read requests by the destination process (on stdin) with the data that the source process writes to stdout. Usually, read and write requests are blocking operations, which means that the execution of the source process, upon writing, is suspended until all data could be written to the destination process, and, likewise, the execution of the destination process, upon reading, is suspended until at least some of the requested data could be obtained from the source process. Obviously, this cannot lead to a deadlock, where both processes would wait indefinitely for each other to respond, since at least one of the two processes will soon thereafter have its request serviced by the operating system, and continue to run. For performance, most operating systems implementing pipes use pipe buffers, which allow the source process to provide more data than the destination process is currently able or willing to receive. Under most Unices and Unix-like operating systems, a special command is also available which implements a pipe buffer of potentially much larger and configurable size, typically called "buffer". This command can be useful if the destination process is significantly slower than the source process, but it is anyway desired that the source process can complete its task as soon as possible. E.g., if the source process consists of a command which reads an audio track from a CD and the destination process consists of a command which compresses the waveform audio data to a format like OGG Vorbis. In this case, buffering the entire track in a pipe buffer would allow the CD drive to spin down more quickly, and enable the user to remove the CD from the drive before the encoding process has finished. It should be noted that such a buffer command can be implemented using nothing but the already available operating system primitives for reading and writing data, however, to avoid wasteful active waiting, additional multithreading capabilities are desirable. On single-tasking operating systems, such as MS-DOS, true pipes could not be implemented, since only one process could execute at a time. As a substitute, MS-DOS provided "pseudo-pipes" with the same syntax, but modified semantics. Upon entering a pipeline into the command line interpreter, COMMAND.COM, DOS would execute the commands one by one in sequential order, and, at each step, redirect their output and/or input to use temporary files (with names implicitly assigned by MS-DOS). For instance, the command "dir | sort | more" would have been executed like this (albeit with more complicated temporary file names):

1. Create temporary file 1.tmp\n# Run command "dir", redirecting its output to 1.tmp\n# Create temporary file 2.tmp\n# Run command "sort", redirecting its input to 1.tmp and its output to 2.tmp\n# Run command "more", redirecting its input to 2.tmp, and presenting its output to the user\n# Delete 1.tmp and 2.tmp, which are no longer needed\n# Return to the command prompt\n \nThus, pseudo-pipes acted like true pipes with a pipe buffer of unlimited size (not withstanding disk space limitations), with the significant restriction that a receiving process could not read any data from the pipe buffer until the sending process finished completely. Besides causing disk traffic that would have been unnecessary under multi-tasking operating systems, this implementation also made pipes unsuitable for applications requiring real-time responsitivity, like, for example, interactive purposes (where the user enters commands that the first process in the pipeline receives via stdin, and the last process in the pipeline presents its output to the user via stdout).