I/O Redirection and Pipes

UNIX Internals





Three Standard File Descriptors (for a process)

0: stdin, standard input - the stream of data to process
1: stdout, standard output - the stream of result data of a process
2: stderr - a stream of error messages

All Unix tools use the file descriptor 0,1,2. All 3 FDs are expected already open for reading, writing and writing, respectively.

What is a file descriptor?
Each process has a collection of files it has open. These open files are stored in an array. A file descriptor is simply an index of an item in that array.

Unix uses "the lowest-available-file-descriptor" principle - when opening a file, you always get the lowest available spot in that array.

3 ways to attach stdin to a file:
  1. close-then-open; e.g. close (0); fd=open("input_file",O_RDONLY);
  2. open-close-dup-close; e.g. open(file), close(0), dup(fd), close(fd);
  3. open-dup2-close

Summary of Redirection to Files:

  • Standard input, output, and error are file descriptors 0, 1 and 2.
  • The kernel always uses the lowest numbered unused file descriptor.
  • The set of file descriptors is passed unchanged across exec calls.
Technical Details of Pipes:
  • Pipes are NOT files.
  • A pipe is a data queue in the kernel with each end attached to a file descriptor. It is created by pipe system call.
  • Both ends of pipe are copied to a child process when the parent calls fork.
  • Pipes can only connect processes that share a common parent.
  • System calls that create file descriptors always use the lowest-numbered free file descriptor
Reading from Pipes
  • When a process tries to read from a pipe, the call blocks until some bytes are written into the pipe. What prevents a process from reading forever?
  • Reading EOF on pipe
  • Multiple readers can cause problem. A pipe is a queue. When a process reads bytes from a pipe, those bytes are no longer in the pipe. In two processes try to read from the same pipe, each process can only get partial of the data.
Writing to Pipes
  • write to a pipe blocks until there is space
  • write fails if no readers



Related System Calls:

int dup(int oldfd);
int dup2(int oldfd, int newfd);

DESCRIPTION
dup and dup2 create a copy of the file descriptor oldfd.

After successful return of dup or dup2, the old and new descriptors
may be used interchangeably. They share locks, file position pointers
and flags; for example, if the file position is modified by using
lseek on one of the descriptors, the position is also changed for the
other.

The two descriptors do not share the close-on-exec flag, however.

dup uses the lowest-numbered unused descriptor for the new descriptor.


dup2 makes newfd be the copy of oldfd, closing newfd first if neces-
sary.

RETURN VALUE
dup and dup2 return the new descriptor, or -1 if an error occurred (in
which case, errno is set appropriately).



int pipe(int filedes[2]);

DESCRIPTION
pipe creates a pair of file descriptors, pointing to a pipe inode, and
places them in the array pointed to by filedes. filedes[0] is for
reading, filedes[1] is for writing.

RETURN VALUE
On success, zero is returned. On error, -1 is returned, and errno is
set appropriately.