This file documents the GNU C library. This is Edition 0.09 DRAFT, last updated 28 Aug 1999, of `The GNU C Library Reference Manual', for Version 2.2 Beta. Copyright (C) 1993, '94, '95, '96, '97, '98, '99 Free Software Foundation, Inc. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the section entitled "GNU Library General Public License" is included exactly as in the original, and provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that the text of the translation of the section entitled "GNU Library General Public License" must be approved for accuracy by the Foundation. Main Menu ********* This is Edition 0.09 DRAFT, last updated 28 Aug 1999, of `The GNU C Library Reference Manual', for Version 2.2 Beta of the GNU C Library. Introduction ************ The C language provides no built-in facilities for performing such common operations as input/output, memory management, string manipulation, and the like. Instead, these facilities are defined in a standard "library", which you compile and link with your programs. The GNU C library, described in this document, defines all of the library functions that are specified by the ISO C standard, as well as additional features specific to POSIX and other derivatives of the Unix operating system, and extensions specific to the GNU system. The purpose of this manual is to tell you how to use the facilities of the GNU library. We have mentioned which features belong to which standards to help you identify things that are potentially non-portable to other systems. But the emphasis in this manual is not on strict portability. Getting Started =============== This manual is written with the assumption that you are at least somewhat familiar with the C programming language and basic programming concepts. Specifically, familiarity with ISO standard C (*note ISO C::), rather than "traditional" pre-ISO C dialects, is assumed. The GNU C library includes several "header files", each of which provides definitions and declarations for a group of related facilities; this information is used by the C compiler when processing your program. For example, the header file `stdio.h' declares facilities for performing input and output, and the header file `string.h' declares string processing utilities. The organization of this manual generally follows the same division as the header files. If you are reading this manual for the first time, you should read all of the introductory material and skim the remaining chapters. There are a _lot_ of functions in the GNU C library and it's not realistic to expect that you will be able to remember exactly _how_ to use each and every one of them. It's more important to become generally familiar with the kinds of facilities that the library provides, so that when you are writing your programs you can recognize _when_ to make use of library functions, and _where_ in this manual you can find more specific information about them. Standards and Portability ========================= This section discusses the various standards and other sources that the GNU C library is based upon. These sources include the ISO C and POSIX standards, and the System V and Berkeley Unix implementations. The primary focus of this manual is to tell you how to make effective use of the GNU library facilities. But if you are concerned about making your programs compatible with these standards, or portable to operating systems other than GNU, this can affect how you use the library. This section gives you an overview of these standards, so that you will know what they are when they are mentioned in other parts of the manual. *Note Library Summary::, for an alphabetical list of the functions and other symbols provided by the library. This list also states which standards each function or symbol comes from. ISO C ----- The GNU C library is compatible with the C standard adopted by the American National Standards Institute (ANSI): `American National Standard X3.159-1989--"ANSI C"' and later by the International Standardization Organization (ISO): `ISO/IEC 9899:1990, "Programming languages--C"'. We here refer to the standard as ISO C since this is the more general standard in respect of ratification. The header files and library facilities that make up the GNU library are a superset of those specified by the ISO C standard. If you are concerned about strict adherence to the ISO C standard, you should use the `-ansi' option when you compile your programs with the GNU C compiler. This tells the compiler to define _only_ ISO standard features from the library header files, unless you explicitly ask for additional features. *Note Feature Test Macros::, for information on how to do this. Being able to restrict the library to include only ISO C features is important because ISO C puts limitations on what names can be defined by the library implementation, and the GNU extensions don't fit these limitations. *Note Reserved Names::, for more information about these restrictions. This manual does not attempt to give you complete details on the differences between ISO C and older dialects. It gives advice on how to write programs to work portably under multiple C dialects, but does not aim for completeness. POSIX (The Portable Operating System Interface) ----------------------------------------------- The GNU library is also compatible with the ISO "POSIX" family of standards, known more formally as the "Portable Operating System Interface for Computer Environments" (ISO/IEC 9945). They were also published as ANSI/IEEE Std 1003. POSIX is derived mostly from various versions of the Unix operating system. The library facilities specified by the POSIX standards are a superset of those required by ISO C; POSIX specifies additional features for ISO C functions, as well as specifying new additional functions. In general, the additional requirements and functionality defined by the POSIX standards are aimed at providing lower-level support for a particular kind of operating system environment, rather than general programming language support which can run in many diverse operating system environments. The GNU C library implements all of the functions specified in `ISO/IEC 9945-1:1996, the POSIX System Application Program Interface', commonly referred to as POSIX.1. The primary extensions to the ISO C facilities specified by this standard include file system interface primitives (*note File System Interface::), device-specific terminal control functions (*note Low-Level Terminal Interface::), and process control functions (*note Processes::). Some facilities from `ISO/IEC 9945-2:1993, the POSIX Shell and Utilities standard' (POSIX.2) are also implemented in the GNU library. These include utilities for dealing with regular expressions and other pattern matching facilities (*note Pattern Matching::). Berkeley Unix ------------- The GNU C library defines facilities from some versions of Unix which are not formally standardized, specifically from the 4.2 BSD, 4.3 BSD, and 4.4 BSD Unix systems (also known as "Berkeley Unix") and from "SunOS" (a popular 4.2 BSD derivative that includes some Unix System V functionality). These systems support most of the ISO C and POSIX facilities, and 4.4 BSD and newer releases of SunOS in fact support them all. The BSD facilities include symbolic links (*note Symbolic Links::), the `select' function (*note Waiting for I/O::), the BSD signal functions (*note BSD Signal Handling::), and sockets (*note Sockets::). SVID (The System V Interface Description) ----------------------------------------- The "System V Interface Description" (SVID) is a document describing the AT&T Unix System V operating system. It is to some extent a superset of the POSIX standard (*note POSIX::). The GNU C library defines most of the facilities required by the SVID that are not also required by the ISO C or POSIX standards, for compatibility with System V Unix and other Unix systems (such as SunOS) which include these facilities. However, many of the more obscure and less generally useful facilities required by the SVID are not included. (In fact, Unix System V itself does not provide them all.) The supported facilities from System V include the methods for inter-process communication and shared memory, the `hsearch' and `drand48' families of functions, `fmtmsg' and several of the mathematical functions. XPG (The X/Open Portability Guide) ---------------------------------- The X/Open Portability Guide, published by the X/Open Company, Ltd., is a more general standard than POSIX. X/Open owns the Unix copyright and the XPG specifies the requirements for systems which are intended to be a Unix system. The GNU C library complies to the X/Open Portability Guide, Issue 4.2, with all extensions common to XSI (X/Open System Interface) compliant systems and also all X/Open UNIX extensions. The additions on top of POSIX are mainly derived from functionality available in System V and BSD systems. Some of the really bad mistakes in System V systems were corrected, though. Since fulfilling the XPG standard with the Unix extensions is a precondition for getting the Unix brand chances are good that the functionality is available on commercial systems. Using the Library ================= This section describes some of the practical issues involved in using the GNU C library. Header Files ------------ Libraries for use by C programs really consist of two parts: "header files" that define types and macros and declare variables and functions; and the actual library or "archive" that contains the definitions of the variables and functions. (Recall that in C, a "declaration" merely provides information that a function or variable exists and gives its type. For a function declaration, information about the types of its arguments might be provided as well. The purpose of declarations is to allow the compiler to correctly process references to the declared variables and functions. A "definition", on the other hand, actually allocates storage for a variable or says what a function does.) In order to use the facilities in the GNU C library, you should be sure that your program source files include the appropriate header files. This is so that the compiler has declarations of these facilities available and can correctly process references to them. Once your program has been compiled, the linker resolves these references to the actual definitions provided in the archive file. Header files are included into a program source file by the `#include' preprocessor directive. The C language supports two forms of this directive; the first, #include "HEADER" is typically used to include a header file HEADER that you write yourself; this would contain definitions and declarations describing the interfaces between the different parts of your particular application. By contrast, #include is typically used to include a header file `file.h' that contains definitions and declarations for a standard library. This file would normally be installed in a standard place by your system administrator. You should use this second form for the C library header files. Typically, `#include' directives are placed at the top of the C source file, before any other code. If you begin your source files with some comments explaining what the code in the file does (a good idea), put the `#include' directives immediately afterwards, following the feature test macro definition (*note Feature Test Macros::). For more information about the use of header files and `#include' directives, *note Header Files: (cpp.info)Header Files.. The GNU C library provides several header files, each of which contains the type and macro definitions and variable and function declarations for a group of related facilities. This means that your programs may need to include several header files, depending on exactly which facilities you are using. Some library header files include other library header files automatically. However, as a matter of programming style, you should not rely on this; it is better to explicitly include all the header files required for the library facilities you are using. The GNU C library header files have been written in such a way that it doesn't matter if a header file is accidentally included more than once; including a header file a second time has no effect. Likewise, if your program needs to include multiple header files, the order in which they are included doesn't matter. *Compatibility Note:* Inclusion of standard header files in any order and any number of times works in any ISO C implementation. However, this has traditionally not been the case in many older C implementations. Strictly speaking, you don't _have to_ include a header file to use a function it declares; you could declare the function explicitly yourself, according to the specifications in this manual. But it is usually better to include the header file because it may define types and macros that are not otherwise available and because it may define more efficient macro replacements for some functions. It is also a sure way to have the correct declaration. Macro Definitions of Functions ------------------------------ If we describe something as a function in this manual, it may have a macro definition as well. This normally has no effect on how your program runs--the macro definition does the same thing as the function would. In particular, macro equivalents for library functions evaluate arguments exactly once, in the same way that a function call would. The main reason for these macro definitions is that sometimes they can produce an inline expansion that is considerably faster than an actual function call. Taking the address of a library function works even if it is also defined as a macro. This is because, in this context, the name of the function isn't followed by the left parenthesis that is syntactically necessary to recognize a macro call. You might occasionally want to avoid using the macro definition of a function--perhaps to make your program easier to debug. There are two ways you can do this: * You can avoid a macro definition in a specific use by enclosing the name of the function in parentheses. This works because the name of the function doesn't appear in a syntactic context where it is recognizable as a macro call. * You can suppress any macro definition for a whole source file by using the `#undef' preprocessor directive, unless otherwise stated explicitly in the description of that facility. For example, suppose the header file `stdlib.h' declares a function named `abs' with extern int abs (int); and also provides a macro definition for `abs'. Then, in: #include int f (int *i) { return abs (++*i); } the reference to `abs' might refer to either a macro or a function. On the other hand, in each of the following examples the reference is to a function and not a macro. #include int g (int *i) { return (abs) (++*i); } #undef abs int h (int *i) { return abs (++*i); } Since macro definitions that double for a function behave in exactly the same way as the actual function version, there is usually no need for any of these methods. In fact, removing macro definitions usually just makes your program slower. Reserved Names -------------- The names of all library types, macros, variables and functions that come from the ISO C standard are reserved unconditionally; your program *may not* redefine these names. All other library names are reserved if your program explicitly includes the header file that defines or declares them. There are several reasons for these restrictions: * Other people reading your code could get very confused if you were using a function named `exit' to do something completely different from what the standard `exit' function does, for example. Preventing this situation helps to make your programs easier to understand and contributes to modularity and maintainability. * It avoids the possibility of a user accidentally redefining a library function that is called by other library functions. If redefinition were allowed, those other functions would not work properly. * It allows the compiler to do whatever special optimizations it pleases on calls to these functions, without the possibility that they may have been redefined by the user. Some library facilities, such as those for dealing with variadic arguments (*note Variadic Functions::) and non-local exits (*note Non-Local Exits::), actually require a considerable amount of cooperation on the part of the C compiler, and implementationally it might be easier for the compiler to treat these as built-in parts of the language. In addition to the names documented in this manual, reserved names include all external identifiers (global functions and variables) that begin with an underscore (`_') and all identifiers regardless of use that begin with either two underscores or an underscore followed by a capital letter are reserved names. This is so that the library and header files can define functions, variables, and macros for internal purposes without risk of conflict with names in user programs. Some additional classes of identifier names are reserved for future extensions to the C language or the POSIX.1 environment. While using these names for your own purposes right now might not cause a problem, they do raise the possibility of conflict with future versions of the C or POSIX standards, so you should avoid these names. * Names beginning with a capital `E' followed a digit or uppercase letter may be used for additional error code names. *Note Error Reporting::. * Names that begin with either `is' or `to' followed by a lowercase letter may be used for additional character testing and conversion functions. *Note Character Handling::. * Names that begin with `LC_' followed by an uppercase letter may be used for additional macros specifying locale attributes. *Note Locales::. * Names of all existing mathematics functions (*note Mathematics::) suffixed with `f' or `l' are reserved for corresponding functions that operate on `float' and `long double' arguments, respectively. * Names that begin with `SIG' followed by an uppercase letter are reserved for additional signal names. *Note Standard Signals::. * Names that begin with `SIG_' followed by an uppercase letter are reserved for additional signal actions. *Note Basic Signal Handling::. * Names beginning with `str', `mem', or `wcs' followed by a lowercase letter are reserved for additional string and array functions. *Note String and Array Utilities::. * Names that end with `_t' are reserved for additional type names. In addition, some individual header files reserve names beyond those that they actually define. You only need to worry about these restrictions if your program includes that particular header file. * The header file `dirent.h' reserves names prefixed with `d_'. * The header file `fcntl.h' reserves names prefixed with `l_', `F_', `O_', and `S_'. * The header file `grp.h' reserves names prefixed with `gr_'. * The header file `limits.h' reserves names suffixed with `_MAX'. * The header file `pwd.h' reserves names prefixed with `pw_'. * The header file `signal.h' reserves names prefixed with `sa_' and `SA_'. * The header file `sys/stat.h' reserves names prefixed with `st_' and `S_'. * The header file `sys/times.h' reserves names prefixed with `tms_'. * The header file `termios.h' reserves names prefixed with `c_', `V', `I', `O', and `TC'; and names prefixed with `B' followed by a digit. Feature Test Macros ------------------- The exact set of features available when you compile a source file is controlled by which "feature test macros" you define. If you compile your programs using `gcc -ansi', you get only the ISO C library features, unless you explicitly request additional features by defining one or more of the feature macros. *Note GNU CC Command Options: (gcc.info)Invoking GCC, for more information about GCC options. You should define these macros by using `#define' preprocessor directives at the top of your source code files. These directives _must_ come before any `#include' of a system header file. It is best to make them the very first thing in the file, preceded only by comments. You could also use the `-D' option to GCC, but it's better if you make the source files indicate their own meaning in a self-contained way. This system exists to allow the library to conform to multiple standards. Although the different standards are often described as supersets of each other, they are usually incompatible because larger standards require functions with names that smaller ones reserve to the user program. This is not mere pedantry -- it has been a problem in practice. For instance, some non-GNU programs define functions named `getline' that have nothing to do with this library's `getline'. They would not be compilable if all features were enabled indiscriminately. This should not be used to verify that a program conforms to a limited standard. It is insufficient for this purpose, as it will not protect you from including header files outside the standard, or relying on semantics undefined within the standard. - Macro: _POSIX_SOURCE If you define this macro, then the functionality from the POSIX.1 standard (IEEE Standard 1003.1) is available, as well as all of the ISO C facilities. The state of `_POSIX_SOURCE' is irrelevant if you define the macro `_POSIX_C_SOURCE' to a positive integer. - Macro: _POSIX_C_SOURCE Define this macro to a positive integer to control which POSIX functionality is made available. The greater the value of this macro, the more functionality is made available. If you define this macro to a value greater than or equal to `1', then the functionality from the 1990 edition of the POSIX.1 standard (IEEE Standard 1003.1-1990) is made available. If you define this macro to a value greater than or equal to `2', then the functionality from the 1992 edition of the POSIX.2 standard (IEEE Standard 1003.2-1992) is made available. If you define this macro to a value greater than or equal to `199309L', then the functionality from the 1993 edition of the POSIX.1b standard (IEEE Standard 1003.1b-1993) is made available. Greater values for `_POSIX_C_SOURCE' will enable future extensions. The POSIX standards process will define these values as necessary, and the GNU C Library should support them some time after they become standardized. The 1996 edition of POSIX.1 (ISO/IEC 9945-1: 1996) states that if you define `_POSIX_C_SOURCE' to a value greater than or equal to `199506L', then the functionality from the 1996 edition is made available. - Macro: _BSD_SOURCE If you define this macro, functionality derived from 4.3 BSD Unix is included as well as the ISO C, POSIX.1, and POSIX.2 material. Some of the features derived from 4.3 BSD Unix conflict with the corresponding features specified by the POSIX.1 standard. If this macro is defined, the 4.3 BSD definitions take precedence over the POSIX definitions. Due to the nature of some of the conflicts between 4.3 BSD and POSIX.1, you need to use a special "BSD compatibility library" when linking programs compiled for BSD compatibility. This is because some functions must be defined in two different ways, one of them in the normal C library, and one of them in the compatibility library. If your program defines `_BSD_SOURCE', you must give the option `-lbsd-compat' to the compiler or linker when linking the program, to tell it to find functions in this special compatibility library before looking for them in the normal C library. - Macro: _SVID_SOURCE If you define this macro, functionality derived from SVID is included as well as the ISO C, POSIX.1, POSIX.2, and X/Open material. - Macro: _XOPEN_SOURCE - Macro: _XOPEN_SOURCE_EXTENDED If you define this macro, functionality described in the X/Open Portability Guide is included. This is a superset of the POSIX.1 and POSIX.2 functionality and in fact `_POSIX_SOURCE' and `_POSIX_C_SOURCE' are automatically defined. As the unification of all Unices, functionality only available in BSD and SVID is also included. If the macro `_XOPEN_SOURCE_EXTENDED' is also defined, even more functionality is available. The extra functions will make all functions available which are necessary for the X/Open Unix brand. If the macro `_XOPEN_SOURCE' has the value 500 this includes all functionality described so far plus some new definitions from the Single Unix Specification, version 2. - Macro: _LARGEFILE_SOURCE If this macro is defined some extra functions are available which rectify a few shortcomings in all previous standards. Specifically, the functions `fseeko' and `ftello' are available. Without these functions the difference between the ISO C interface (`fseek', `ftell') and the low-level POSIX interface (`lseek') would lead to problems. This macro was introduced as part of the Large File Support extension (LFS). - Macro: _LARGEFILE64_SOURCE If you define this macro an additional set of functions is made available which enables 32 bit systems to use files of sizes beyond the usual limit of 2GB. This interface is not available if the system does not support files that large. On systems where the natural file size limit is greater than 2GB (i.e., on 64 bit systems) the new functions are identical to the replaced functions. The new functionality is made available by a new set of types and functions which replace the existing ones. The names of these new objects contain `64' to indicate the intention, e.g., `off_t' vs. `off64_t' and `fseeko' vs. `fseeko64'. This macro was introduced as part of the Large File Support extension (LFS). It is a transition interface for the period when 64 bit offsets are not generally used (see `_FILE_OFFSET_BITS'). - Macro: _FILE_OFFSET_BITS This macro determines which file system interface shall be used, one replacing the other. Whereas `_LARGEFILE64_SOURCE' makes the 64 bit interface available as an additional interface, `_FILE_OFFSET_BITS' allows the 64 bit interface to replace the old interface. If `_FILE_OFFSET_BITS' is undefined, or if it is defined to the value `32', nothing changes. The 32 bit interface is used and types like `off_t' have a size of 32 bits on 32 bit systems. If the macro is defined to the value `64', the large file interface replaces the old interface. I.e., the functions are not made available under different names (as they are with `_LARGEFILE64_SOURCE'). Instead the old function names now reference the new functions, e.g., a call to `fseeko' now indeed calls `fseeko64'. This macro should only be selected if the system provides mechanisms for handling large files. On 64 bit systems this macro has no effect since the `*64' functions are identical to the normal functions. This macro was introduced as part of the Large File Support extension (LFS). - Macro: _ISOC99_SOURCE Until the revised ISO C standard is widely adopted the new features are not automatically enabled. The GNU libc nevertheless has a complete implementation of the new standard and to enable the new features the macro `_ISOC99_SOURCE' should be defined. - Macro: _GNU_SOURCE If you define this macro, everything is included: ISO C89, ISO C99, POSIX.1, POSIX.2, BSD, SVID, X/Open, LFS, and GNU extensions. In the cases where POSIX.1 conflicts with BSD, the POSIX definitions take precedence. If you want to get the full effect of `_GNU_SOURCE' but make the BSD definitions take precedence over the POSIX definitions, use this sequence of definitions: #define _GNU_SOURCE #define _BSD_SOURCE #define _SVID_SOURCE Note that if you do this, you must link your program with the BSD compatibility library by passing the `-lbsd-compat' option to the compiler or linker. *Note:* If you forget to do this, you may get very strange errors at run time. - Macro: _REENTRANT - Macro: _THREAD_SAFE If you define one of these macros, reentrant versions of several functions get declared. Some of the functions are specified in POSIX.1c but many others are only available on a few other systems or are unique to GNU libc. The problem is the delay in the standardization of the thread safe C library interface. Unlike on some other systems, no special version of the C library must be used for linking. There is only one version but while compiling this it must have been specified to compile as thread safe. We recommend you use `_GNU_SOURCE' in new programs. If you don't specify the `-ansi' option to GCC and don't define any of these macros explicitly, the effect is the same as defining `_POSIX_C_SOURCE' to 2 and `_POSIX_SOURCE', `_SVID_SOURCE', and `_BSD_SOURCE' to 1. When you define a feature test macro to request a larger class of features, it is harmless to define in addition a feature test macro for a subset of those features. For example, if you define `_POSIX_C_SOURCE', then defining `_POSIX_SOURCE' as well has no effect. Likewise, if you define `_GNU_SOURCE', then defining either `_POSIX_SOURCE' or `_POSIX_C_SOURCE' or `_SVID_SOURCE' as well has no effect. Note, however, that the features of `_BSD_SOURCE' are not a subset of any of the other feature test macros supported. This is because it defines BSD features that take precedence over the POSIX features that are requested by the other macros. For this reason, defining `_BSD_SOURCE' in addition to the other feature test macros does have an effect: it causes the BSD features to take priority over the conflicting POSIX features. Roadmap to the Manual ===================== Here is an overview of the contents of the remaining chapters of this manual. * *Note Error Reporting::, describes how errors detected by the library are reported. * *Note Language Features::, contains information about library support for standard parts of the C language, including things like the `sizeof' operator and the symbolic constant `NULL', how to write functions accepting variable numbers of arguments, and constants describing the ranges and other properties of the numerical types. There is also a simple debugging mechanism which allows you to put assertions in your code, and have diagnostic messages printed if the tests fail. * *Note Memory::, describes the GNU library's facilities for managing and using virtual and real memory, including dynamic allocation of virtual memory. If you do not know in advance how much memory your program needs, you can allocate it dynamically instead, and manipulate it via pointers. * *Note Character Handling::, contains information about character classification functions (such as `isspace') and functions for performing case conversion. * *Note String and Array Utilities::, has descriptions of functions for manipulating strings (null-terminated character arrays) and general byte arrays, including operations such as copying and comparison. * *Note I/O Overview::, gives an overall look at the input and output facilities in the library, and contains information about basic concepts such as file names. * *Note I/O on Streams::, describes I/O operations involving streams (or `FILE *' objects). These are the normal C library functions from `stdio.h'. * *Note Low-Level I/O::, contains information about I/O operations on file descriptors. File descriptors are a lower-level mechanism specific to the Unix family of operating systems. * *Note File System Interface::, has descriptions of operations on entire files, such as functions for deleting and renaming them and for creating new directories. This chapter also contains information about how you can access the attributes of a file, such as its owner and file protection modes. * *Note Pipes and FIFOs::, contains information about simple interprocess communication mechanisms. Pipes allow communication between two related processes (such as between a parent and child), while FIFOs allow communication between processes sharing a common file system on the same machine. * *Note Sockets::, describes a more complicated interprocess communication mechanism that allows processes running on different machines to communicate over a network. This chapter also contains information about Internet host addressing and how to use the system network databases. * *Note Low-Level Terminal Interface::, describes how you can change the attributes of a terminal device. If you want to disable echo of characters typed by the user, for example, read this chapter. * *Note Mathematics::, contains information about the math library functions. These include things like random-number generators and remainder functions on integers as well as the usual trigonometric and exponential functions on floating-point numbers. * *Note Low-Level Arithmetic Functions: Arithmetic, describes functions for simple arithmetic, analysis of floating-point values, and reading numbers from strings. * *Note Searching and Sorting::, contains information about functions for searching and sorting arrays. You can use these functions on any kind of array by providing an appropriate comparison function. * *Note Pattern Matching::, presents functions for matching regular expressions and shell file name patterns, and for expanding words as the shell does. * *Note Date and Time::, describes functions for measuring both calendar time and CPU time, as well as functions for setting alarms and timers. * *Note Character Set Handling::, contains information about manipulating characters and strings using character sets larger than will fit in the usual `char' data type. * *Note Locales::, describes how selecting a particular country or language affects the behavior of the library. For example, the locale affects collation sequences for strings and how monetary values are formatted. * *Note Non-Local Exits::, contains descriptions of the `setjmp' and `longjmp' functions. These functions provide a facility for `goto'-like jumps which can jump from one function to another. * *Note Signal Handling::, tells you all about signals--what they are, how to establish a handler that is called when a particular kind of signal is delivered, and how to prevent signals from arriving during critical sections of your program. * *Note Program Basics::, tells how your programs can access their command-line arguments and environment variables. * *Note Processes::, contains information about how to start new processes and run programs. * *Note Job Control::, describes functions for manipulating process groups and the controlling terminal. This material is probably only of interest if you are writing a shell or other program which handles job control specially. * *Note Name Service Switch::, describes the services which are available for looking up names in the system databases, how to determine which service is used for which database, and how these services are implemented so that contributors can design their own services. * *Note User Database::, and *Note Group Database::, tell you how to access the system user and group databases. * *Note System Management::, describes functions for controlling and getting information about the hardware and software configuration your program is executing under. * *Note System Configuration::, tells you how you can get information about various operating system limits. Most of these parameters are provided for compatibility with POSIX. * *Note Library Summary::, gives a summary of all the functions, variables, and macros in the library, with complete data types and function prototypes, and says what standard or system each is derived from. * *Note Maintenance::, explains how to build and install the GNU C library on your system, how to report any bugs you might find, and how to add new functions or port the library to a new system. If you already know the name of the facility you are interested in, you can look it up in *Note Library Summary::. This gives you a summary of its syntax and a pointer to where you can find a more detailed description. This appendix is particularly useful if you just want to verify the order and type of arguments to a function, for example. It also tells you what standard or system each function, variable, or macro is derived from. Error Reporting *************** Many functions in the GNU C library detect and report error conditions, and sometimes your programs need to check for these error conditions. For example, when you open an input file, you should verify that the file was actually opened correctly, and print an error message or take other appropriate action if the call to the library function failed. This chapter describes how the error reporting facility works. Your program should include the header file `errno.h' to use this facility. Checking for Errors =================== Most library functions return a special value to indicate that they have failed. The special value is typically `-1', a null pointer, or a constant such as `EOF' that is defined for that purpose. But this return value tells you only that an error has occurred. To find out what kind of error it was, you need to look at the error code stored in the variable `errno'. This variable is declared in the header file `errno.h'. - Variable: volatile int errno The variable `errno' contains the system error number. You can change the value of `errno'. Since `errno' is declared `volatile', it might be changed asynchronously by a signal handler; see *Note Defining Handlers::. However, a properly written signal handler saves and restores the value of `errno', so you generally do not need to worry about this possibility except when writing signal handlers. The initial value of `errno' at program startup is zero. Many library functions are guaranteed to set it to certain nonzero values when they encounter certain kinds of errors. These error conditions are listed for each function. These functions do not change `errno' when they succeed; thus, the value of `errno' after a successful call is not necessarily zero, and you should not use `errno' to determine _whether_ a call failed. The proper way to do that is documented for each function. _If_ the call failed, you can examine `errno'. Many library functions can set `errno' to a nonzero value as a result of calling other library functions which might fail. You should assume that any library function might alter `errno' when the function returns an error. *Portability Note:* ISO C specifies `errno' as a "modifiable lvalue" rather than as a variable, permitting it to be implemented as a macro. For example, its expansion might involve a function call, like `*_errno ()'. In fact, that is what it is on the GNU system itself. The GNU library, on non-GNU systems, does whatever is right for the particular system. There are a few library functions, like `sqrt' and `atan', that return a perfectly legitimate value in case of an error, but also set `errno'. For these functions, if you want to check to see whether an error occurred, the recommended method is to set `errno' to zero before calling the function, and then check its value afterward. All the error codes have symbolic names; they are macros defined in `errno.h'. The names start with `E' and an upper-case letter or digit; you should consider names of this form to be reserved names. *Note Reserved Names::. The error code values are all positive integers and are all distinct, with one exception: `EWOULDBLOCK' and `EAGAIN' are the same. Since the values are distinct, you can use them as labels in a `switch' statement; just don't use both `EWOULDBLOCK' and `EAGAIN'. Your program should not make any other assumptions about the specific values of these symbolic constants. The value of `errno' doesn't necessarily have to correspond to any of these macros, since some library functions might return other error codes of their own for other situations. The only values that are guaranteed to be meaningful for a particular library function are the ones that this manual lists for that function. On non-GNU systems, almost any system call can return `EFAULT' if it is given an invalid pointer as an argument. Since this could only happen as a result of a bug in your program, and since it will not happen on the GNU system, we have saved space by not mentioning `EFAULT' in the descriptions of individual functions. In some Unix systems, many system calls can also return `EFAULT' if given as an argument a pointer into the stack, and the kernel for some obscure reason fails in its attempt to extend the stack. If this ever happens, you should probably try using statically or dynamically allocated memory instead of stack memory on that system. Error Codes =========== The error code macros are defined in the header file `errno.h'. All of them expand into integer constant values. Some of these error codes can't occur on the GNU system, but they can occur using the GNU library on other systems. - Macro: int EPERM Operation not permitted; only the owner of the file (or other resource) or processes with special privileges can perform the operation. - Macro: int ENOENT No such file or directory. This is a "file doesn't exist" error for ordinary files that are referenced in contexts where they are expected to already exist. - Macro: int ESRCH No process matches the specified process ID. - Macro: int EINTR Interrupted function call; an asynchronous signal occurred and prevented completion of the call. When this happens, you should try the call again. You can choose to have functions resume after a signal that is handled, rather than failing with `EINTR'; see *Note Interrupted Primitives::. - Macro: int EIO Input/output error; usually used for physical read or write errors. - Macro: int ENXIO No such device or address. The system tried to use the device represented by a file you specified, and it couldn't find the device. This can mean that the device file was installed incorrectly, or that the physical device is missing or not correctly attached to the computer. - Macro: int E2BIG Argument list too long; used when the arguments passed to a new program being executed with one of the `exec' functions (*note Executing a File::) occupy too much memory space. This condition never arises in the GNU system. - Macro: int ENOEXEC Invalid executable file format. This condition is detected by the `exec' functions; see *Note Executing a File::. - Macro: int EBADF Bad file descriptor; for example, I/O on a descriptor that has been closed or reading from a descriptor open only for writing (or vice versa). - Macro: int ECHILD There are no child processes. This error happens on operations that are supposed to manipulate child processes, when there aren't any processes to manipulate. - Macro: int EDEADLK Deadlock avoided; allocating a system resource would have resulted in a deadlock situation. The system does not guarantee that it will notice all such situations. This error means you got lucky and the system noticed; it might just hang. *Note File Locks::, for an example. - Macro: int ENOMEM No memory available. The system cannot allocate more virtual memory because its capacity is full. - Macro: int EACCES Permission denied; the file permissions do not allow the attempted operation. - Macro: int EFAULT Bad address; an invalid pointer was detected. In the GNU system, this error never happens; you get a signal instead. - Macro: int ENOTBLK A file that isn't a block special file was given in a situation that requires one. For example, trying to mount an ordinary file as a file system in Unix gives this error. - Macro: int EBUSY Resource busy; a system resource that can't be shared is already in use. For example, if you try to delete a file that is the root of a currently mounted filesystem, you get this error. - Macro: int EEXIST File exists; an existing file was specified in a context where it only makes sense to specify a new file. - Macro: int EXDEV An attempt to make an improper link across file systems was detected. This happens not only when you use `link' (*note Hard Links::) but also when you rename a file with `rename' (*note Renaming Files::). - Macro: int ENODEV The wrong type of device was given to a function that expects a particular sort of device. - Macro: int ENOTDIR A file that isn't a directory was specified when a directory is required. - Macro: int EISDIR File is a directory; you cannot open a directory for writing, or create or remove hard links to it. - Macro: int EINVAL Invalid argument. This is used to indicate various kinds of problems with passing the wrong argument to a library function. - Macro: int EMFILE The current process has too many files open and can't open any more. Duplicate descriptors do count toward this limit. In BSD and GNU, the number of open files is controlled by a resource limit that can usually be increased. If you get this error, you might want to increase the `RLIMIT_NOFILE' limit or make it unlimited; *note Limits on Resources::. - Macro: int ENFILE There are too many distinct file openings in the entire system. Note that any number of linked channels count as just one file opening; see *Note Linked Channels::. This error never occurs in the GNU system. - Macro: int ENOTTY Inappropriate I/O control operation, such as trying to set terminal modes on an ordinary file. - Macro: int ETXTBSY An attempt to execute a file that is currently open for writing, or write to a file that is currently being executed. Often using a debugger to run a program is considered having it open for writing and will cause this error. (The name stands for "text file busy".) This is not an error in the GNU system; the text is copied as necessary. - Macro: int EFBIG File too big; the size of a file would be larger than allowed by the system. - Macro: int ENOSPC No space left on device; write operation on a file failed because the disk is full. - Macro: int ESPIPE Invalid seek operation (such as on a pipe). - Macro: int EROFS An attempt was made to modify something on a read-only file system. - Macro: int EMLINK Too many links; the link count of a single file would become too large. `rename' can cause this error if the file being renamed already has as many links as it can take (*note Renaming Files::). - Macro: int EPIPE Broken pipe; there is no process reading from the other end of a pipe. Every library function that returns this error code also generates a `SIGPIPE' signal; this signal terminates the program if not handled or blocked. Thus, your program will never actually see `EPIPE' unless it has handled or blocked `SIGPIPE'. - Macro: int EDOM Domain error; used by mathematical functions when an argument value does not fall into the domain over which the function is defined. - Macro: int ERANGE Range error; used by mathematical functions when the result value is not representable because of overflow or underflow. - Macro: int EAGAIN Resource temporarily unavailable; the call might work if you try again later. The macro `EWOULDBLOCK' is another name for `EAGAIN'; they are always the same in the GNU C library. This error can happen in a few different situations: * An operation that would block was attempted on an object that has non-blocking mode selected. Trying the same operation again will block until some external condition makes it possible to read, write, or connect (whatever the operation). You can use `select' to find out when the operation will be possible; *note Waiting for I/O::. *Portability Note:* In many older Unix systems, this condition was indicated by `EWOULDBLOCK', which was a distinct error code different from `EAGAIN'. To make your program portable, you should check for both codes and treat them the same. * A temporary resource shortage made an operation impossible. `fork' can return this error. It indicates that the shortage is expected to pass, so your program can try the call again later and it may succeed. It is probably a good idea to delay for a few seconds before trying it again, to allow time for other processes to release scarce resources. Such shortages are usually fairly serious and affect the whole system, so usually an interactive program should report the error to the user and return to its command loop. - Macro: int EWOULDBLOCK In the GNU C library, this is another name for `EAGAIN' (above). The values are always the same, on every operating system. C libraries in many older Unix systems have `EWOULDBLOCK' as a separate error code. - Macro: int EINPROGRESS An operation that cannot complete immediately was initiated on an object that has non-blocking mode selected. Some functions that must always block (such as `connect'; *note Connecting::) never return `EAGAIN'. Instead, they return `EINPROGRESS' to indicate that the operation has begun and will take some time. Attempts to manipulate the object before the call completes return `EALREADY'. You can use the `select' function to find out when the pending operation has completed; *note Waiting for I/O::. - Macro: int EALREADY An operation is already in progress on an object that has non-blocking mode selected. - Macro: int ENOTSOCK A file that isn't a socket was specified when a socket is required. - Macro: int EMSGSIZE The size of a message sent on a socket was larger than the supported maximum size. - Macro: int EPROTOTYPE The socket type does not support the requested communications protocol. - Macro: int ENOPROTOOPT You specified a socket option that doesn't make sense for the particular protocol being used by the socket. *Note Socket Options::. - Macro: int EPROTONOSUPPORT The socket domain does not support the requested communications protocol (perhaps because the requested protocol is completely invalid). *Note Creating a Socket::. - Macro: int ESOCKTNOSUPPORT The socket type is not supported. - Macro: int EOPNOTSUPP The operation you requested is not supported. Some socket functions don't make sense for all types of sockets, and others may not be implemented for all communications protocols. In the GNU system, this error can happen for many calls when the object does not support the particular operation; it is a generic indication that the server knows nothing to do for that call. - Macro: int EPFNOSUPPORT The socket communications protocol family you requested is not supported. - Macro: int EAFNOSUPPORT The address family specified for a socket is not supported; it is inconsistent with the protocol being used on the socket. *Note Sockets::. - Macro: int EADDRINUSE The requested socket address is already in use. *Note Socket Addresses::. - Macro: int EADDRNOTAVAIL The requested socket address is not available; for example, you tried to give a socket a name that doesn't match the local host name. *Note Socket Addresses::. - Macro: int ENETDOWN A socket operation failed because the network was down. - Macro: int ENETUNREACH A socket operation failed because the subnet containing the remote host was unreachable. - Macro: int ENETRESET A network connection was reset because the remote host crashed. - Macro: int ECONNABORTED A network connection was aborted locally. - Macro: int ECONNRESET A network connection was closed for reasons outside the control of the local host, such as by the remote machine rebooting or an unrecoverable protocol violation. - Macro: int ENOBUFS The kernel's buffers for I/O operations are all in use. In GNU, this error is always synonymous with `ENOMEM'; you may get one or the other from network operations. - Macro: int EISCONN You tried to connect a socket that is already connected. *Note Connecting::. - Macro: int ENOTCONN The socket is not connected to anything. You get this error when you try to transmit data over a socket, without first specifying a destination for the data. For a connectionless socket (for datagram protocols, such as UDP), you get `EDESTADDRREQ' instead. - Macro: int EDESTADDRREQ No default destination address was set for the socket. You get this error when you try to transmit data over a connectionless socket, without first specifying a destination for the data with `connect'. - Macro: int ESHUTDOWN The socket has already been shut down. - Macro: int ETOOMANYREFS ??? - Macro: int ETIMEDOUT A socket operation with a specified timeout received no response during the timeout period. - Macro: int ECONNREFUSED A remote host refused to allow the network connection (typically because it is not running the requested service). - Macro: int ELOOP Too many levels of symbolic links were encountered in looking up a file name. This often indicates a cycle of symbolic links. - Macro: int ENAMETOOLONG Filename too long (longer than `PATH_MAX'; *note Limits for Files::) or host name too long (in `gethostname' or `sethostname'; *note Host Identification::). - Macro: int EHOSTDOWN The remote host for a requested network connection is down. - Macro: int EHOSTUNREACH The remote host for a requested network connection is not reachable. - Macro: int ENOTEMPTY Directory not empty, where an empty directory was expected. Typically, this error occurs when you are trying to delete a directory. - Macro: int EPROCLIM This means that the per-user limit on new process would be exceeded by an attempted `fork'. *Note Limits on Resources::, for details on the `RLIMIT_NPROC' limit. - Macro: int EUSERS The file quota system is confused because there are too many users. - Macro: int EDQUOT The user's disk quota was exceeded. - Macro: int ESTALE Stale NFS file handle. This indicates an internal confusion in the NFS system which is due to file system rearrangements on the server host. Repairing this condition usually requires unmounting and remounting the NFS file system on the local host. - Macro: int EREMOTE An attempt was made to NFS-mount a remote file system with a file name that already specifies an NFS-mounted file. (This is an error on some operating systems, but we expect it to work properly on the GNU system, making this error code impossible.) - Macro: int EBADRPC ??? - Macro: int ERPCMISMATCH ??? - Macro: int EPROGUNAVAIL ??? - Macro: int EPROGMISMATCH ??? - Macro: int EPROCUNAVAIL ??? - Macro: int ENOLCK No locks available. This is used by the file locking facilities; see *Note File Locks::. This error is never generated by the GNU system, but it can result from an operation to an NFS server running another operating system. - Macro: int EFTYPE Inappropriate file type or format. The file was the wrong type for the operation, or a data file had the wrong format. On some systems `chmod' returns this error if you try to set the sticky bit on a non-directory file; *note Setting Permissions::. - Macro: int EAUTH ??? - Macro: int ENEEDAUTH ??? - Macro: int ENOSYS Function not implemented. This indicates that the function called is not implemented at all, either in the C library itself or in the operating system. When you get this error, you can be sure that this particular function will always fail with `ENOSYS' unless you install a new version of the C library or the operating system. - Macro: int ENOTSUP Not supported. A function returns this error when certain parameter values are valid, but the functionality they request is not available. This can mean that the function does not implement a particular command or option value or flag bit at all. For functions that operate on some object given in a parameter, such as a file descriptor or a port, it might instead mean that only _that specific object_ (file descriptor, port, etc.) is unable to support the other parameters given; different file descriptors might support different ranges of parameter values. If the entire function is not available at all in the implementation, it returns `ENOSYS' instead. - Macro: int EILSEQ While decoding a multibyte character the function came along an invalid or an incomplete sequence of bytes or the given wide character is invalid. - Macro: int EBACKGROUND In the GNU system, servers supporting the `term' protocol return this error for certain operations when the caller is not in the foreground process group of the terminal. Users do not usually see this error because functions such as `read' and `write' translate it into a `SIGTTIN' or `SIGTTOU' signal. *Note Job Control::, for information on process groups and these signals. - Macro: int EDIED In the GNU system, opening a file returns this error when the file is translated by a program and the translator program dies while starting up, before it has connected to the file. - Macro: int ED The experienced user will know what is wrong. - Macro: int EGREGIOUS You did *what*? - Macro: int EIEIO Go home and have a glass of warm, dairy-fresh milk. - Macro: int EGRATUITOUS This error code has no purpose. - Macro: int EBADMSG - Macro: int EIDRM - Macro: int EMULTIHOP - Macro: int ENODATA - Macro: int ENOLINK - Macro: int ENOMSG - Macro: int ENOSR - Macro: int ENOSTR - Macro: int EOVERFLOW - Macro: int EPROTO - Macro: int ETIME _The following error codes are defined by the Linux/i386 kernel. They are not yet documented._ - Macro: int ERESTART - Macro: int ECHRNG - Macro: int EL2NSYNC - Macro: int EL3HLT - Macro: int EL3RST - Macro: int ELNRNG - Macro: int EUNATCH - Macro: int ENOCSI - Macro: int EL2HLT - Macro: int EBADE - Macro: int EBADR - Macro: int EXFULL - Macro: int ENOANO - Macro: int EBADRQC - Macro: int EBADSLT - Macro: int EDEADLOCK - Macro: int EBFONT - Macro: int ENONET - Macro: int ENOPKG - Macro: int EADV - Macro: int ESRMNT - Macro: int ECOMM - Macro: int EDOTDOT - Macro: int ENOTUNIQ - Macro: int EBADFD - Macro: int EREMCHG - Macro: int ELIBACC - Macro: int ELIBBAD - Macro: int ELIBSCN - Macro: int ELIBMAX - Macro: int ELIBEXEC - Macro: int ESTRPIPE - Macro: int EUCLEAN - Macro: int ENOTNAM - Macro: int ENAVAIL - Macro: int EISNAM - Macro: int EREMOTEIO - Macro: int ENOMEDIUM - Macro: int EMEDIUMTYPE Error Messages ============== The library has functions and variables designed to make it easy for your program to report informative error messages in the customary format about the failure of a library call. The functions `strerror' and `perror' give you the standard error message for a given error code; the variable `program_invocation_short_name' gives you convenient access to the name of the program that encountered the error. - Function: char * strerror (int ERRNUM) The `strerror' function maps the error code (*note Checking for Errors::) specified by the ERRNUM argument to a descriptive error message string. The return value is a pointer to this string. The value ERRNUM normally comes from the variable `errno'. You should not modify the string returned by `strerror'. Also, if you make subsequent calls to `strerror', the string might be overwritten. (But it's guaranteed that no library function ever calls `strerror' behind your back.) The function `strerror' is declared in `string.h'. - Function: char * strerror_r (int ERRNUM, char *BUF, size_t N) The `strerror_r' function works like `strerror' but instead of returning the error message in a statically allocated buffer shared by all threads in the process, it returns a private copy for the thread. This might be either some permanent global data or a message string in the user supplied buffer starting at BUF with the length of N bytes. At most N characters are written (including the NUL byte) so it is up to the user to select the buffer large enough. This function should always be used in multi-threaded programs since there is no way to guarantee the string returned by `strerror' really belongs to the last call of the current thread. This function `strerror_r' is a GNU extension and it is declared in `string.h'. - Function: void perror (const char *MESSAGE) This function prints an error message to the stream `stderr'; see *Note Standard Streams::. If you call `perror' with a MESSAGE that is either a null pointer or an empty string, `perror' just prints the error message corresponding to `errno', adding a trailing newline. If you supply a non-null MESSAGE argument, then `perror' prefixes its output with this string. It adds a colon and a space character to separate the MESSAGE from the error string corresponding to `errno'. The function `perror' is declared in `stdio.h'. `strerror' and `perror' produce the exact same message for any given error code; the precise text varies from system to system. On the GNU system, the messages are fairly short; there are no multi-line messages or embedded newlines. Each error message begins with a capital letter and does not include any terminating punctuation. *Compatibility Note:* The `strerror' function is a new feature of ISO C. Many older C systems do not support this function yet. Many programs that don't read input from the terminal are designed to exit if any system call fails. By convention, the error message from such a program should start with the program's name, sans directories. You can find that name in the variable `program_invocation_short_name'; the full file name is stored the variable `program_invocation_name'. - Variable: char * program_invocation_name This variable's value is the name that was used to invoke the program running in the current process. It is the same as `argv[0]'. Note that this is not necessarily a useful file name; often it contains no directory names. *Note Program Arguments::. - Variable: char * program_invocation_short_name This variable's value is the name that was used to invoke the program running in the current process, with directory names removed. (That is to say, it is the same as `program_invocation_name' minus everything up to the last slash, if any.) The library initialization code sets up both of these variables before calling `main'. *Portability Note:* These two variables are GNU extensions. If you want your program to work with non-GNU libraries, you must save the value of `argv[0]' in `main', and then strip off the directory names yourself. We added these extensions to make it possible to write self-contained error-reporting subroutines that require no explicit cooperation from `main'. Here is an example showing how to handle failure to open a file correctly. The function `open_sesame' tries to open the named file for reading and returns a stream if successful. The `fopen' library function returns a null pointer if it couldn't open the file for some reason. In that situation, `open_sesame' constructs an appropriate error message using the `strerror' function, and terminates the program. If we were going to make some other library calls before passing the error code to `strerror', we'd have to save it in a local variable instead, because those other library functions might overwrite `errno' in the meantime. #include #include #include #include FILE * open_sesame (char *name) { FILE *stream; errno = 0; stream = fopen (name, "r"); if (stream == NULL) { fprintf (stderr, "%s: Couldn't open file %s; %s\n", program_invocation_short_name, name, strerror (errno)); exit (EXIT_FAILURE); } else return stream; } Virtual Memory Allocation And Paging ************************************ This chapter describes how processes manage and use memory in a system that uses the GNU C library. The GNU C Library has several functions for dynamically allocating virtual memory in various ways. They vary in generality and in efficiency. The library also provides functions for controlling paging and allocation of real memory. Memory mapped I/O is not discussed in this chapter. *Note Memory-mapped I/O::. Process Memory Concepts ======================= One of the most basic resources a process has available to it is memory. There are a lot of different ways systems organize memory, but in a typical one, each process has one linear virtual address space, with addresses running from zero to some huge maximum. It need not be contiguous; i.e. not all of these addresses actually can be used to store data. The virtual memory is divided into pages (4 kilobytes is typical). Backing each page of virtual memory is a page of real memory (called a "frame") or some secondary storage, usually disk space. The disk space might be swap space or just some ordinary disk file. Actually, a page of all zeroes sometimes has nothing at all backing it - there's just a flag saying it is all zeroes. The same frame of real memory or backing store can back multiple virtual pages belonging to multiple processes. This is normally the case, for example, with virtual memory occupied by GNU C library code. The same real memory frame containing the `printf' function backs a virtual memory page in each of the existing processes that has a `printf' call in its program. In order for a program to access any part of a virtual page, the page must at that moment be backed by ("connected to") a real frame. But because there is usually a lot more virtual memory than real memory, the pages must move back and forth between real memory and backing store regularly, coming into real memory when a process needs to access them and then retreating to backing store when not needed anymore. This movement is called "paging". When a program attempts to access a page which is not at that moment backed by real memory, this is known as a "page fault". When a page fault occurs, the kernel suspends the process, places the page into a real page frame (this is called "paging in" or "faulting in"), then resumes the process so that from the process' point of view, the page was in real memory all along. In fact, to the process, all pages always seem to be in real memory. Except for one thing: the elapsed execution time of an instruction that would normally be a few nanoseconds is suddenly much, much, longer (because the kernel normally has to do I/O to complete the page-in). For programs sensitive to that, the functions described in *Note Locking Pages:: can control it. Within each virtual address space, a process has to keep track of what is at which addresses, and that process is called memory allocation. Allocation usually brings to mind meting out scarce resources, but in the case of virtual memory, that's not a major goal, because there is generally much more of it than anyone needs. Memory allocation within a process is mainly just a matter of making sure that the same byte of memory isn't used to store two different things. Processes allocate memory in two major ways: by exec and programmatically. Actually, forking is a third way, but it's not very interesting. *Note Creating a Process::. Exec is the operation of creating a virtual address space for a process, loading its basic program into it, and executing the program. It is done by the "exec" family of functions (e.g. `execl'). The operation takes a program file (an executable), it allocates space to load all the data in the executable, loads it, and transfers control to it. That data is most notably the instructions of the program (the "text"), but also literals and constants in the program and even some variables: C variables with the static storage class (*note Memory Allocation and C::). Once that program begins to execute, it uses programmatic allocation to gain additional memory. In a C program with the GNU C library, there are two kinds of programmatic allocation: automatic and dynamic. *Note Memory Allocation and C::. Memory-mapped I/O is another form of dynamic virtual memory allocation. Mapping memory to a file means declaring that the contents of certain range of a process' addresses shall be identical to the contents of a specified regular file. The system makes the virtual memory initially contain the contents of the file, and if you modify the memory, the system writes the same modification to the file. Note that due to the magic of virtual memory and page faults, there is no reason for the system to do I/O to read the file, or allocate real memory for its contents, until the program accesses the virtual memory. *Note Memory-mapped I/O::. Just as it programmatically allocates memory, the program can programmatically deallocate ("free") it. You can't free the memory that was allocated by exec. When the program exits or execs, you might say that all its memory gets freed, but since in both cases the address space ceases to exist, the point is really moot. *Note Program Termination::. A process' virtual address space is divided into segments. A segment is a contiguous range of virtual addresses. Three important segments are: * The "text segment" contains a program's instructions and literals and static constants. It is allocated by exec and stays the same size for the life of the virtual address space. * The "data segment" is working storage for the program. It can be preallocated and preloaded by exec and the process can extend or shrink it by calling functions as described in *Note Resizing the Data Segment::. Its lower end is fixed. * The "stack segment" contains a program stack. It grows as the stack grows, but doesn't shrink when the stack shrinks. Allocating Storage For Program Data =================================== This section covers how ordinary programs manage storage for their data, including the famous `malloc' function and some fancier facilities special the GNU C library and GNU Compiler. Memory Allocation in C Programs ------------------------------- The C language supports two kinds of memory allocation through the variables in C programs: * "Static allocation" is what happens when you declare a static or global variable. Each static or global variable defines one block of space, of a fixed size. The space is allocated once, when your program is started (part of the exec operation), and is never freed. * "Automatic allocation" happens when you declare an automatic variable, such as a function argument or a local variable. The space for an automatic variable is allocated when the compound statement containing the declaration is entered, and is freed when that compound statement is exited. In GNU C, the size of the automatic storage can be an expression that varies. In other C implementations, it must be a constant. A third important kind of memory allocation, "dynamic allocation", is not supported by C variables but is available via GNU C library functions. Dynamic Memory Allocation ......................... "Dynamic memory allocation" is a technique in which programs determine as they are running where to store some information. You need dynamic allocation when the amount of memory you need, or how long you continue to need it, depends on factors that are not known before the program runs. For example, you may need a block to store a line read from an input file; since there is no limit to how long a line can be, you must allocate the memory dynamically and make it dynamically larger as you read more of the line. Or, you may need a block for each record or each definition in the input data; since you can't know in advance how many there will be, you must allocate a new block for each record or definition as you read it. When you use dynamic allocation, the allocation of a block of memory is an action that the program requests explicitly. You call a function or macro when you want to allocate space, and specify the size with an argument. If you want to free the space, you do so by calling another function or macro. You can do these things whenever you want, as often as you want. Dynamic allocation is not supported by C variables; there is no storage class "dynamic", and there can never be a C variable whose value is stored in dynamically allocated space. The only way to get dynamically allocated memory is via a system call (which is generally via a GNU C library function call), and the only way to refer to dynamically allocated space is through a pointer. Because it is less convenient, and because the actual process of dynamic allocation requires more computation time, programmers generally use dynamic allocation only when neither static nor automatic allocation will serve. For example, if you want to allocate dynamically some space to hold a `struct foobar', you cannot declare a variable of type `struct foobar' whose contents are the dynamically allocated space. But you can declare a variable of pointer type `struct foobar *' and assign it the address of the space. Then you can use the operators `*' and `->' on this pointer variable to refer to the contents of the space: { struct foobar *ptr = (struct foobar *) malloc (sizeof (struct foobar)); ptr->name = x; ptr->next = current_foobar; current_foobar = ptr; } Unconstrained Allocation ------------------------ The most general dynamic allocation facility is `malloc'. It allows you to allocate blocks of memory of any size at any time, make them bigger or smaller at any time, and free the blocks individually at any time (or never). Basic Memory Allocation ....................... To allocate a block of memory, call `malloc'. The prototype for this function is in `stdlib.h'. - Function: void * malloc (size_t SIZE) This function returns a pointer to a newly allocated block SIZE bytes long, or a null pointer if the block could not be allocated. The contents of the block are undefined; you must initialize it yourself (or use `calloc' instead; *note Allocating Cleared Space::). Normally you would cast the value as a pointer to the kind of object that you want to store in the block. Here we show an example of doing so, and of initializing the space with zeros using the library function `memset' (*note Copying and Concatenation::): struct foo *ptr; ... ptr = (struct foo *) malloc (sizeof (struct foo)); if (ptr == 0) abort (); memset (ptr, 0, sizeof (struct foo)); You can store the result of `malloc' into any pointer variable without a cast, because ISO C automatically converts the type `void *' to another type of pointer when necessary. But the cast is necessary in contexts other than assignment operators or if you might want your code to run in traditional C. Remember that when allocating space for a string, the argument to `malloc' must be one plus the length of the string. This is because a string is terminated with a null character that doesn't count in the "length" of the string but does need space. For example: char *ptr; ... ptr = (char *) malloc (length + 1); *Note Representation of Strings::, for more information about this. Examples of `malloc' .................... If no more space is available, `malloc' returns a null pointer. You should check the value of _every_ call to `malloc'. It is useful to write a subroutine that calls `malloc' and reports an error if the value is a null pointer, returning only if the value is nonzero. This function is conventionally called `xmalloc'. Here it is: void * xmalloc (size_t size) { register void *value = malloc (size); if (value == 0) fatal ("virtual memory exhausted"); return value; } Here is a real example of using `malloc' (by way of `xmalloc'). The function `savestring' will copy a sequence of characters into a newly allocated null-terminated string: char * savestring (const char *ptr, size_t len) { register char *value = (char *) xmalloc (len + 1); value[len] = '\0'; return (char *) memcpy (value, ptr, len); } The block that `malloc' gives you is guaranteed to be aligned so that it can hold any type of data. In the GNU system, the address is always a multiple of eight on most systems, and a multiple of 16 on 64-bit systems. Only rarely is any higher boundary (such as a page boundary) necessary; for those cases, use `memalign', `posix_memalign' or `valloc' (*note Aligned Memory Blocks::). Note that the memory located after the end of the block is likely to be in use for something else; perhaps a block already allocated by another call to `malloc'. If you attempt to treat the block as longer than you asked for it to be, you are liable to destroy the data that `malloc' uses to keep track of its blocks, or you may destroy the contents of another block. If you have already allocated a block and discover you want it to be bigger, use `realloc' (*note Changing Block Size::). Freeing Memory Allocated with `malloc' ...................................... When you no longer need a block that you got with `malloc', use the function `free' to make the block available to be allocated again. The prototype for this function is in `stdlib.h'. - Function: void free (void *PTR) The `free' function deallocates the block of memory pointed at by PTR. - Function: void cfree (void *PTR) This function does the same thing as `free'. It's provided for backward compatibility with SunOS; you should use `free' instead. Freeing a block alters the contents of the block. *Do not expect to find any data (such as a pointer to the next block in a chain of blocks) in the block after freeing it.* Copy whatever you need out of the block before freeing it! Here is an example of the proper way to free all the blocks in a chain, and the strings that they point to: struct chain { struct chain *next; char *name; } void free_chain (struct chain *chain) { while (chain != 0) { struct chain *next = chain->next; free (chain->name); free (chain); chain = next; } } Occasionally, `free' can actually return memory to the operating system and make the process smaller. Usually, all it can do is allow a later call to `malloc' to reuse the space. In the meantime, the space remains in your program as part of a free-list used internally by `malloc'. There is no point in freeing blocks at the end of a program, because all of the program's space is given back to the system when the process terminates. Changing the Size of a Block ............................ Often you do not know for certain how big a block you will ultimately need at the time you must begin to use the block. For example, the block might be a buffer that you use to hold a line being read from a file; no matter how long you make the buffer initially, you may encounter a line that is longer. You can make the block longer by calling `realloc'. This function is declared in `stdlib.h'. - Function: void * realloc (void *PTR, size_t NEWSIZE) The `realloc' function changes the size of the block whose address is PTR to be NEWSIZE. Since the space after the end of the block may be in use, `realloc' may find it necessary to copy the block to a new address where more free space is available. The value of `realloc' is the new address of the block. If the block needs to be moved, `realloc' copies the old contents. If you pass a null pointer for PTR, `realloc' behaves just like `malloc (NEWSIZE)'. This can be convenient, but beware that older implementations (before ISO C) may not support this behavior, and will probably crash when `realloc' is passed a null pointer. Like `malloc', `realloc' may return a null pointer if no memory space is available to make the block bigger. When this happens, the original block is untouched; it has not been modified or relocated. In most cases it makes no difference what happens to the original block when `realloc' fails, because the application program cannot continue when it is out of memory, and the only thing to do is to give a fatal error message. Often it is convenient to write and use a subroutine, conventionally called `xrealloc', that takes care of the error message as `xmalloc' does for `malloc': void * xrealloc (void *ptr, size_t size) { register void *value = realloc (ptr, size); if (value == 0) fatal ("Virtual memory exhausted"); return value; } You can also use `realloc' to make a block smaller. The reason you would do this is to avoid tying up a lot of memory space when only a little is needed. In several allocation implementations, making a block smaller sometimes necessitates copying it, so it can fail if no other space is available. If the new size you specify is the same as the old size, `realloc' is guaranteed to change nothing and return the same address that you gave. Allocating Cleared Space ........................ The function `calloc' allocates memory and clears it to zero. It is declared in `stdlib.h'. - Function: void * calloc (size_t COUNT, size_t ELTSIZE) This function allocates a block long enough to contain a vector of COUNT elements, each of size ELTSIZE. Its contents are cleared to zero before `calloc' returns. You could define `calloc' as follows: void * calloc (size_t count, size_t eltsize) { size_t size = count * eltsize; void *value = malloc (size); if (value != 0) memset (value, 0, size); return value; } But in general, it is not guaranteed that `calloc' calls `malloc' internally. Therefore, if an application provides its own `malloc'/`realloc'/`free' outside the C library, it should always define `calloc', too. Efficiency Considerations for `malloc' ...................................... As opposed to other versions, the `malloc' in the GNU C Library does not round up block sizes to powers of two, neither for large nor for small sizes. Neighboring chunks can be coalesced on a `free' no matter what their size is. This makes the implementation suitable for all kinds of allocation patterns without generally incurring high memory waste through fragmentation. Very large blocks (much larger than a page) are allocated with `mmap' (anonymous or via `/dev/zero') by this implementation. This has the great advantage that these chunks are returned to the system immediately when they are freed. Therefore, it cannot happen that a large chunk becomes "locked" in between smaller ones and even after calling `free' wastes memory. The size threshold for `mmap' to be used can be adjusted with `mallopt'. The use of `mmap' can also be disabled completely. Allocating Aligned Memory Blocks ................................ The address of a block returned by `malloc' or `realloc' in the GNU system is always a multiple of eight (or sixteen on 64-bit systems). If you need a block whose address is a multiple of a higher power of two than that, use `memalign', `posix_memalign', or `valloc'. These functions are declared in `stdlib.h'. With the GNU library, you can use `free' to free the blocks that `memalign', `posix_memalign', and `valloc' return. That does not work in BSD, however--BSD does not provide any way to free such blocks. - Function: void * memalign (size_t BOUNDARY, size_t SIZE) The `memalign' function allocates a block of SIZE bytes whose address is a multiple of BOUNDARY. The BOUNDARY must be a power of two! The function `memalign' works by allocating a somewhat larger block, and then returning an address within the block that is on the specified boundary. - Function: int posix_memalign (void **MEMPTR, size_t ALIGNMENT, size_t SIZE) The `posix_memalign' function is similar to the `memalign' function in that it returns a buffer of SIZE bytes aligned to a multiple of ALIGNMENT. But it adds one requirement to the parameter ALIGNMENT: the value must be a power of two multiple of `sizeof (void *)'. If the function succeeds in allocation memory a pointer to the allocated memory is returned in `*MEMPTR' and the return value is zero. Otherwise the function returns an error value indicating the problem. This function was introduced in POSIX 1003.1d. - Function: void * valloc (size_t SIZE) Using `valloc' is like using `memalign' and passing the page size as the value of the second argument. It is implemented like this: void * valloc (size_t size) { return memalign (getpagesize (), size); } *Note Query Memory Parameters:: for more information about the memory subsystem. Malloc Tunable Parameters ......................... You can adjust some parameters for dynamic memory allocation with the `mallopt' function. This function is the general SVID/XPG interface, defined in `malloc.h'. - Function: int mallopt (int PARAM, int VALUE) When calling `mallopt', the PARAM argument specifies the parameter to be set, and VALUE the new value to be set. Possible choices for PARAM, as defined in `malloc.h', are: `M_TRIM_THRESHOLD' This is the minimum size (in bytes) of the top-most, releasable chunk that will cause `sbrk' to be called with a negative argument in order to return memory to the system. `M_TOP_PAD' This parameter determines the amount of extra memory to obtain from the system when a call to `sbrk' is required. It also specifies the number of bytes to retain when shrinking the heap by calling `sbrk' with a negative argument. This provides the necessary hysteresis in heap size such that excessive amounts of system calls can be avoided. `M_MMAP_THRESHOLD' All chunks larger than this value are allocated outside the normal heap, using the `mmap' system call. This way it is guaranteed that the memory for these chunks can be returned to the system on `free'. `M_MMAP_MAX' The maximum number of chunks to allocate with `mmap'. Setting this to zero disables all use of `mmap'. Heap Consistency Checking ......................... You can ask `malloc' to check the consistency of dynamic memory by using the `mcheck' function. This function is a GNU extension, declared in `mcheck.h'. - Function: int mcheck (void (*ABORTFN) (enum mcheck_status STATUS)) Calling `mcheck' tells `malloc' to perform occasional consistency checks. These will catch things such as writing past the end of a block that was allocated with `malloc'. The ABORTFN argument is the function to call when an inconsistency is found. If you supply a null pointer, then `mcheck' uses a default function which prints a message and calls `abort' (*note Aborting a Program::). The function you supply is called with one argument, which says what sort of inconsistency was detected; its type is described below. It is too late to begin allocation checking once you have allocated anything with `malloc'. So `mcheck' does nothing in that case. The function returns `-1' if you call it too late, and `0' otherwise (when it is successful). The easiest way to arrange to call `mcheck' early enough is to use the option `-lmcheck' when you link your program; then you don't need to modify your program source at all. Alternatively you might use a debugger to insert a call to `mcheck' whenever the program is started, for example these gdb commands will automatically call `mcheck' whenever the program starts: (gdb) break main Breakpoint 1, main (argc=2, argv=0xbffff964) at whatever.c:10 (gdb) command 1 Type commands for when breakpoint 1 is hit, one per line. End with a line saying just "end". >call mcheck(0) >continue >end (gdb) ... This will however only work if no initialization function of any object involved calls any of the `malloc' functions since `mcheck' must be called before the first such function. - Function: enum mcheck_status mprobe (void *POINTER) The `mprobe' function lets you explicitly check for inconsistencies in a particular allocated block. You must have already called `mcheck' at the beginning of the program, to do its occasional checks; calling `mprobe' requests an additional consistency check to be done at the time of the call. The argument POINTER must be a pointer returned by `malloc' or `realloc'. `mprobe' returns a value that says what inconsistency, if any, was found. The values are described below. - Data Type: enum mcheck_status This enumerated type describes what kind of inconsistency was detected in an allocated block, if any. Here are the possible values: `MCHECK_DISABLED' `mcheck' was not called before the first allocation. No consistency checking can be done. `MCHECK_OK' No inconsistency detected. `MCHECK_HEAD' The data immediately before the block was modified. This commonly happens when an array index or pointer is decremented too far. `MCHECK_TAIL' The data immediately after the block was modified. This commonly happens when an array index or pointer is incremented too far. `MCHECK_FREE' The block was already freed. Another possibility to check for and guard against bugs in the use of `malloc', `realloc' and `free' is to set the environment variable `MALLOC_CHECK_'. When `MALLOC_CHECK_' is set, a special (less efficient) implementation is used which is designed to be tolerant against simple errors, such as double calls of `free' with the same argument, or overruns of a single byte (off-by-one bugs). Not all such errors can be protected against, however, and memory leaks can result. If `MALLOC_CHECK_' is set to `0', any detected heap corruption is silently ignored; if set to `1', a diagnostic is printed on `stderr'; if set to `2', `abort' is called immediately. This can be useful because otherwise a crash may happen much later, and the true cause for the problem is then very hard to track down. There is one problem with `MALLOC_CHECK_': in SUID or SGID binaries it could possibly be exploited since diverging from the normal programs behaviour it now writes something to the standard error desriptor. Therefore the use of `MALLOC_CHECK_' is disabled by default for SUID and SGID binaries. It can be enabled again by the system administrator by adding a file `/etc/suid-debug' (the content is not important it could be empty). So, what's the difference between using `MALLOC_CHECK_' and linking with `-lmcheck'? `MALLOC_CHECK_' is orthogonal with respect to `-lmcheck'. `-lmcheck' has been added for backward compatibility. Both `MALLOC_CHECK_' and `-lmcheck' should uncover the same bugs - but using `MALLOC_CHECK_' you don't need to recompile your application. Memory Allocation Hooks ....................... The GNU C library lets you modify the behavior of `malloc', `realloc', and `free' by specifying appropriate hook functions. You can use these hooks to help you debug programs that use dynamic memory allocation, for example. The hook variables are declared in `malloc.h'. - Variable: __malloc_hook The value of this variable is a pointer to the function that `malloc' uses whenever it is called. You should define this function to look like `malloc'; that is, like: void *FUNCTION (size_t SIZE, const void *CALLER) The value of CALLER is the return address found on the stack when the `malloc' function was called. This value allows you to trace the memory consumption of the program. - Variable: __realloc_hook The value of this variable is a pointer to function that `realloc' uses whenever it is called. You should define this function to look like `realloc'; that is, like: void *FUNCTION (void *PTR, size_t SIZE, const void *CALLER) The value of CALLER is the return address found on the stack when the `realloc' function was called. This value allows you to trace the memory consumption of the program. - Variable: __free_hook The value of this variable is a pointer to function that `free' uses whenever it is called. You should define this function to look like `free'; that is, like: void FUNCTION (void *PTR, const void *CALLER) The value of CALLER is the return address found on the stack when the `free' function was called. This value allows you to trace the memory consumption of the program. - Variable: __memalign_hook The value of this variable is a pointer to function that `memalign' uses whenever it is called. You should define this function to look like `memalign'; that is, like: void *FUNCTION (size_t SIZE, size_t ALIGNMENT, const void *CALLER) The value of CALLER is the return address found on the stack when the `memalign' function was called. This value allows you to trace the memory consumption of the program. You must make sure that the function you install as a hook for one of these functions does not call that function recursively without restoring the old value of the hook first! Otherwise, your program will get stuck in an infinite recursion. Before calling the function recursively, one should make sure to restore all the hooks to their previous value. When coming back from the recursive call, all the hooks should be resaved since a hook might modify itself. - Variable: __malloc_initialize_hook The value of this variable is a pointer to a function that is called once when the malloc implementation is initialized. This is a weak variable, so it can be overridden in the application with a definition like the following: void (*__MALLOC_INITIALIZE_HOOK) (void) = my_init_hook; An issue to look out for is the time at which the malloc hook functions can be safely installed. If the hook functions call the malloc-related functions recursively, it is necessary that malloc has already properly initialized itself at the time when `__malloc_hook' etc. is assigned to. On the other hand, if the hook functions provide a complete malloc implementation of their own, it is vital that the hooks are assigned to _before_ the very first `malloc' call has completed, because otherwise a chunk obtained from the ordinary, un-hooked malloc may later be handed to `__free_hook', for example. In both cases, the problem can be solved by setting up the hooks from within a user-defined function pointed to by `__malloc_initialize_hook'--then the hooks will be set up safely at the right time. Here is an example showing how to use `__malloc_hook' and `__free_hook' properly. It installs a function that prints out information every time `malloc' or `free' is called. We just assume here that `realloc' and `memalign' are not used in our program. /* Prototypes for __malloc_hook, __free_hook */ #include /* Prototypes for our hooks. */ static void *my_init_hook (void); static void *my_malloc_hook (size_t, const void *); static void my_free_hook (void*, const void *); /* Override initializing hook from the C library. */ void (*__malloc_initialize_hook) (void) = my_init_hook; static void my_init_hook (void) { old_malloc_hook = __malloc_hook; old_free_hook = __free_hook; __malloc_hook = my_malloc_hook; __free_hook = my_free_hook; } static void * my_malloc_hook (size_t size, const void *caller) { void *result; /* Restore all old hooks */ __malloc_hook = old_malloc_hook; __free_hook = old_free_hook; /* Call recursively */ result = malloc (size); /* Save underlaying hooks */ old_malloc_hook = __malloc_hook; old_free_hook = __free_hook; /* `printf' might call `malloc', so protect it too. */ printf ("malloc (%u) returns %p\n", (unsigned int) size, result); /* Restore our own hooks */ __malloc_hook = my_malloc_hook; __free_hook = my_free_hook; return result; } static void * my_free_hook (void *ptr, const void *caller) { /* Restore all old hooks */ __malloc_hook = old_malloc_hook; __free_hook = old_free_hook; /* Call recursively */ free (ptr); /* Save underlaying hooks */ old_malloc_hook = __malloc_hook; old_free_hook = __free_hook; /* `printf' might call `free', so protect it too. */ printf ("freed pointer %p\n", ptr); /* Restore our own hooks */ __malloc_hook = my_malloc_hook; __free_hook = my_free_hook; } main () { ... } The `mcheck' function (*note Heap Consistency Checking::) works by installing such hooks. Statistics for Memory Allocation with `malloc' .............................................. You can get information about dynamic memory allocation by calling the `mallinfo' function. This function and its associated data type are declared in `malloc.h'; they are an extension of the standard SVID/XPG version. - Data Type: struct mallinfo This structure type is used to return information about the dynamic memory allocator. It contains the following members: `int arena' This is the total size of memory allocated with `sbrk' by `malloc', in bytes. `int ordblks' This is the number of chunks not in use. (The memory allocator internally gets chunks of memory from the operating system, and then carves them up to satisfy individual `malloc' requests; see *Note Efficiency and Malloc::.) `int smblks' This field is unused. `int hblks' This is the total number of chunks allocated with `mmap'. `int hblkhd' This is the total size of memory allocated with `mmap', in bytes. `int usmblks' This field is unused. `int fsmblks' This field is unused. `int uordblks' This is the total size of memory occupied by chunks handed out by `malloc'. `int fordblks' This is the total size of memory occupied by free (not in use) chunks. `int keepcost' This is the size of the top-most releasable chunk that normally borders the end of the heap (i.e. the high end of the virtual address space's data segment). - Function: struct mallinfo mallinfo (void) This function returns information about the current dynamic memory usage in a structure of type `struct mallinfo'. Summary of `malloc'-Related Functions ..................................... Here is a summary of the functions that work with `malloc': `void *malloc (size_t SIZE)' Allocate a block of SIZE bytes. *Note Basic Allocation::. `void free (void *ADDR)' Free a block previously allocated by `malloc'. *Note Freeing after Malloc::. `void *realloc (void *ADDR, size_t SIZE)' Make a block previously allocated by `malloc' larger or smaller, possibly by copying it to a new location. *Note Changing Block Size::. `void *calloc (size_t COUNT, size_t ELTSIZE)' Allocate a block of COUNT * ELTSIZE bytes using `malloc', and set its contents to zero. *Note Allocating Cleared Space::. `void *valloc (size_t SIZE)' Allocate a block of SIZE bytes, starting on a page boundary. *Note Aligned Memory Blocks::. `void *memalign (size_t SIZE, size_t BOUNDARY)' Allocate a block of SIZE bytes, starting on an address that is a multiple of BOUNDARY. *Note Aligned Memory Blocks::. `int mallopt (int PARAM, int VALUE)' Adjust a tunable parameter. *Note Malloc Tunable Parameters::. `int mcheck (void (*ABORTFN) (void))' Tell `malloc' to perform occasional consistency checks on dynamically allocated memory, and to call ABORTFN when an inconsistency is found. *Note Heap Consistency Checking::. `void *(*__malloc_hook) (size_t SIZE, const void *CALLER)' A pointer to a function that `malloc' uses whenever it is called. `void *(*__realloc_hook) (void *PTR, size_t SIZE, const void *CALLER)' A pointer to a function that `realloc' uses whenever it is called. `void (*__free_hook) (void *PTR, const void *CALLER)' A pointer to a function that `free' uses whenever it is called. `void (*__memalign_hook) (size_t SIZE, size_t ALIGNMENT, const void *CALLER)' A pointer to a function that `memalign' uses whenever it is called. `struct mallinfo mallinfo (void)' Return information about the current dynamic memory usage. *Note Statistics of Malloc::. Allocation Debugging -------------------- A complicated task when programming with languages which do not use garbage collected dynamic memory allocation is to find memory leaks. Long running programs must assure that dynamically allocated objects are freed at the end of their lifetime. If this does not happen the system runs out of memory, sooner or later. The `malloc' implementation in the GNU C library provides some simple means to detect such leaks and obtain some information to find the location. To do this the application must be started in a special mode which is enabled by an environment variable. There are no speed penalties for the program if the debugging mode is not enabled. How to install the tracing functionality ........................................ - Function: void mtrace (void) When the `mtrace' function is called it looks for an environment variable named `MALLOC_TRACE'. This variable is supposed to contain a valid file name. The user must have write access. If the file already exists it is truncated. If the environment variable is not set or it does not name a valid file which can be opened for writing nothing is done. The behaviour of `malloc' etc. is not changed. For obvious reasons this also happens if the application is installed with the SUID or SGID bit set. If the named file is successfully opened, `mtrace' installs special handlers for the functions `malloc', `realloc', and `free' (*note Hooks for Malloc::). From then on, all uses of these functions are traced and protocolled into the file. There is now of course a speed penalty for all calls to the traced functions so tracing should not be enabled during normal use. This function is a GNU extension and generally not available on other systems. The prototype can be found in `mcheck.h'. - Function: void muntrace (void) The `muntrace' function can be called after `mtrace' was used to enable tracing the `malloc' calls. If no (succesful) call of `mtrace' was made `muntrace' does nothing. Otherwise it deinstalls the handlers for `malloc', `realloc', and `free' and then closes the protocol file. No calls are protocolled anymore and the program runs again at full speed. This function is a GNU extension and generally not available on other systems. The prototype can be found in `mcheck.h'. Example program excerpts ........................ Even though the tracing functionality does not influence the runtime behaviour of the program it is not a good idea to call `mtrace' in all programs. Just imagine that you debug a program using `mtrace' and all other programs used in the debugging session also trace their `malloc' calls. The output file would be the same for all programs and thus is unusable. Therefore one should call `mtrace' only if compiled for debugging. A program could therefore start like this: #include int main (int argc, char *argv[]) { #ifdef DEBUGGING mtrace (); #endif ... } This is all what is needed if you want to trace the calls during the whole runtime of the program. Alternatively you can stop the tracing at any time with a call to `muntrace'. It is even possible to restart the tracing again with a new call to `mtrace'. But this can cause unreliable results since there may be calls of the functions which are not called. Please note that not only the application uses the traced functions, also libraries (including the C library itself) use these functions. This last point is also why it is no good idea to call `muntrace' before the program terminated. The libraries are informed about the termination of the program only after the program returns from `main' or calls `exit' and so cannot free the memory they use before this time. So the best thing one can do is to call `mtrace' as the very first function in the program and never call `muntrace'. So the program traces almost all uses of the `malloc' functions (except those calls which are executed by constructors of the program or used libraries). Some more or less clever ideas .............................. You know the situation. The program is prepared for debugging and in all debugging sessions it runs well. But once it is started without debugging the error shows up. A typical example is a memory leak that becomes visible only when we turn off the debugging. If you foresee such situations you can still win. Simply use something equivalent to the following little program: #include #include static void enable (int sig) { mtrace (); signal (SIGUSR1, enable); } static void disable (int sig) { muntrace (); signal (SIGUSR2, disable); } int main (int argc, char *argv[]) { ... signal (SIGUSR1, enable); signal (SIGUSR2, disable); ... } I.e., the user can start the memory debugger any time s/he wants if the program was started with `MALLOC_TRACE' set in the environment. The output will of course not show the allocations which happened before the first signal but if there is a memory leak this will show up nevertheless. Interpreting the traces ....................... If you take a look at the output it will look similar to this: = Start [0x8048209] - 0x8064cc8 [0x8048209] - 0x8064ce0 [0x8048209] - 0x8064cf8 [0x80481eb] + 0x8064c48 0x14 [0x80481eb] + 0x8064c60 0x14 [0x80481eb] + 0x8064c78 0x14 [0x80481eb] + 0x8064c90 0x14 = End What this all means is not really important since the trace file is not meant to be read by a human. Therefore no attention is given to readability. Instead there is a program which comes with the GNU C library which interprets the traces and outputs a summary in an user-friendly way. The program is called `mtrace' (it is in fact a Perl script) and it takes one or two arguments. In any case the name of the file with the trace output must be specified. If an optional argument precedes the name of the trace file this must be the name of the program which generated the trace. drepper$ mtrace tst-mtrace log No memory leaks. In this case the program `tst-mtrace' was run and it produced a trace file `log'. The message printed by `mtrace' shows there are no problems with the code, all allocated memory was freed afterwards. If we call `mtrace' on the example trace given above we would get a different outout: drepper$ mtrace errlog - 0x08064cc8 Free 2 was never alloc'd 0x8048209 - 0x08064ce0 Free 3 was never alloc'd 0x8048209 - 0x08064cf8 Free 4 was never alloc'd 0x8048209 Memory not freed: ----------------- Address Size Caller 0x08064c48 0x14 at 0x80481eb 0x08064c60 0x14 at 0x80481eb 0x08064c78 0x14 at 0x80481eb 0x08064c90 0x14 at 0x80481eb We have called `mtrace' with only one argument and so the script has no chance to find out what is meant with the addresses given in the trace. We can do better: drepper$ mtrace tst errlog - 0x08064cc8 Free 2 was never alloc'd /home/drepper/tst.c:39 - 0x08064ce0 Free 3 was never alloc'd /home/drepper/tst.c:39 - 0x08064cf8 Free 4 was never alloc'd /home/drepper/tst.c:39 Memory not freed: ----------------- Address Size Caller 0x08064c48 0x14 at /home/drepper/tst.c:33 0x08064c60 0x14 at /home/drepper/tst.c:33 0x08064c78 0x14 at /home/drepper/tst.c:33 0x08064c90 0x14 at /home/drepper/tst.c:33 Suddenly the output makes much more sense and the user can see immediately where the function calls causing the trouble can be found. Interpreting this output is not complicated. There are at most two different situations being detected. First, `free' was called for pointers which were never returned by one of the allocation functions. This is usually a very bad problem and what this looks like is shown in the first three lines of the output. Situations like this are quite rare and if they appear they show up very drastically: the program normally crashes. The other situation which is much harder to detect are memory leaks. As you can see in the output the `mtrace' function collects all this information and so can say that the program calls an allocation function from line 33 in the source file `/home/drepper/tst-mtrace.c' four times without freeing this memory before the program terminates. Whether this is a real problem remains to be investigated. Obstacks -------- An "obstack" is a pool of memory containing a stack of objects. You can create any number of separate obstacks, and then allocate objects in specified obstacks. Within each obstack, the last object allocated must always be the first one freed, but distinct obstacks are independent of each other. Aside from this one constraint of order of freeing, obstacks are totally general: an obstack can contain any number of objects of any size. They are implemented with macros, so allocation is usually very fast as long as the objects are usually small. And the only space overhead per object is the padding needed to start each object on a suitable boundary. Creating Obstacks ................. The utilities for manipulating obstacks are declared in the header file `obstack.h'. - Data Type: struct obstack An obstack is represented by a data structure of type `struct obstack'. This structure has a small fixed size; it records the status of the obstack and how to find the space in which objects are allocated. It does not contain any of the objects themselves. You should not try to access the contents of the structure directly; use only the functions described in this chapter. You can declare variables of type `struct obstack' and use them as obstacks, or you can allocate obstacks dynamically like any other kind of object. Dynamic allocation of obstacks allows your program to have a variable number of different stacks. (You can even allocate an obstack structure in another obstack, but this is rarely useful.) All the functions that work with obstacks require you to specify which obstack to use. You do this with a pointer of type `struct obstack *'. In the following, we often say "an obstack" when strictly speaking the object at hand is such a pointer. The objects in the obstack are packed into large blocks called "chunks". The `struct obstack' structure points to a chain of the chunks currently in use. The obstack library obtains a new chunk whenever you allocate an object that won't fit in the previous chunk. Since the obstack library manages chunks automatically, you don't need to pay much attention to them, but you do need to supply a function which the obstack library should use to get a chunk. Usually you supply a function which uses `malloc' directly or indirectly. You must also supply a function to free a chunk. These matters are described in the following section. Preparing for Using Obstacks ............................ Each source file in which you plan to use the obstack functions must include the header file `obstack.h', like this: #include Also, if the source file uses the macro `obstack_init', it must declare or define two functions or macros that will be called by the obstack library. One, `obstack_chunk_alloc', is used to allocate the chunks of memory into which objects are packed. The other, `obstack_chunk_free', is used to return chunks when the objects in them are freed. These macros should appear before any use of obstacks in the source file. Usually these are defined to use `malloc' via the intermediary `xmalloc' (*note Unconstrained Allocation::). This is done with the following pair of macro definitions: #define obstack_chunk_alloc xmalloc #define obstack_chunk_free free Though the memory you get using obstacks really comes from `malloc', using obstacks is faster because `malloc' is called less often, for larger blocks of memory. *Note Obstack Chunks::, for full details. At run time, before the program can use a `struct obstack' object as an obstack, it must initialize the obstack by calling `obstack_init'. - Function: int obstack_init (struct obstack *OBSTACK-PTR) Initialize obstack OBSTACK-PTR for allocation of objects. This function calls the obstack's `obstack_chunk_alloc' function. If allocation of memory fails, the function pointed to by `obstack_alloc_failed_handler' is called. The `obstack_init' function always returns 1 (Compatibility notice: Former versions of obstack returned 0 if allocation failed). Here are two examples of how to allocate the space for an obstack and initialize it. First, an obstack that is a static variable: static struct obstack myobstack; ... obstack_init (&myobstack); Second, an obstack that is itself dynamically allocated: struct obstack *myobstack_ptr = (struct obstack *) xmalloc (sizeof (struct obstack)); obstack_init (myobstack_ptr); - Variable: obstack_alloc_failed_handler The value of this variable is a pointer to a function that `obstack' uses when `obstack_chunk_alloc' fails to allocate memory. The default action is to print a message and abort. You should supply a function that either calls `exit' (*note Program Termination::) or `longjmp' (*note Non-Local Exits::) and doesn't return. void my_obstack_alloc_failed (void) ... obstack_alloc_failed_handler = &my_obstack_alloc_failed; Allocation in an Obstack ........................ The most direct way to allocate an object in an obstack is with `obstack_alloc', which is invoked almost like `malloc'. - Function: void * obstack_alloc (struct obstack *OBSTACK-PTR, int SIZE) This allocates an uninitialized block of SIZE bytes in an obstack and returns its address. Here OBSTACK-PTR specifies which obstack to allocate the block in; it is the address of the `struct obstack' object which represents the obstack. Each obstack function or macro requires you to specify an OBSTACK-PTR as the first argument. This function calls the obstack's `obstack_chunk_alloc' function if it needs to allocate a new chunk of memory; it calls `obstack_alloc_failed_handler' if allocation of memory by `obstack_chunk_alloc' failed. For example, here is a function that allocates a copy of a string STR in a specific obstack, which is in the variable `string_obstack': struct obstack string_obstack; char * copystring (char *string) { size_t len = strlen (string) + 1; char *s = (char *) obstack_alloc (&string_obstack, len); memcpy (s, string, len); return s; } To allocate a block with specified contents, use the function `obstack_copy', declared like this: - Function: void * obstack_copy (struct obstack *OBSTACK-PTR, void *ADDRESS, int SIZE) This allocates a block and initializes it by copying SIZE bytes of data starting at ADDRESS. It calls `obstack_alloc_failed_handler' if allocation of memory by `obstack_chunk_alloc' failed. - Function: void * obstack_copy0 (struct obstack *OBSTACK-PTR, void *ADDRESS, int SIZE) Like `obstack_copy', but appends an extra byte containing a null character. This extra byte is not counted in the argument SIZE. The `obstack_copy0' function is convenient for copying a sequence of characters into an obstack as a null-terminated string. Here is an example of its use: char * obstack_savestring (char *addr, int size) { return obstack_copy0 (&myobstack, addr, size); } Contrast this with the previous example of `savestring' using `malloc' (*note Basic Allocation::). Freeing Objects in an Obstack ............................. To free an object allocated in an obstack, use the function `obstack_free'. Since the obstack is a stack of objects, freeing one object automatically frees all other objects allocated more recently in the same obstack. - Function: void obstack_free (struct obstack *OBSTACK-PTR, void *OBJECT) If OBJECT is a null pointer, everything allocated in the obstack is freed. Otherwise, OBJECT must be the address of an object allocated in the obstack. Then OBJECT is freed, along with everything allocated in OBSTACK since OBJECT. Note that if OBJECT is a null pointer, the result is an uninitialized obstack. To free all memory in an obstack but leave it valid for further allocation, call `obstack_free' with the address of the first object allocated on the obstack: obstack_free (obstack_ptr, first_object_allocated_ptr); Recall that the objects in an obstack are grouped into chunks. When all the objects in a chunk become free, the obstack library automatically frees the chunk (*note Preparing for Obstacks::). Then other obstacks, or non-obstack allocation, can reuse the space of the chunk. Obstack Functions and Macros ............................ The interfaces for using obstacks may be defined either as functions or as macros, depending on the compiler. The obstack facility works with all C compilers, including both ISO C and traditional C, but there are precautions you must take if you plan to use compilers other than GNU C. If you are using an old-fashioned non-ISO C compiler, all the obstack "functions" are actually defined only as macros. You can call these macros like functions, but you cannot use them in any other way (for example, you cannot take their address). Calling the macros requires a special precaution: namely, the first operand (the obstack pointer) may not contain any side effects, because it may be computed more than once. For example, if you write this: obstack_alloc (get_obstack (), 4); you will find that `get_obstack' may be called several times. If you use `*obstack_list_ptr++' as the obstack pointer argument, you will get very strange results since the incrementation may occur several times. In ISO C, each function has both a macro definition and a function definition. The function definition is used if you take the address of the function without calling it. An ordinary call uses the macro definition by default, but you can request the function definition instead by writing the function name in parentheses, as shown here: char *x; void *(*funcp) (); /* Use the macro. */ x = (char *) obstack_alloc (obptr, size); /* Call the function. */ x = (char *) (obstack_alloc) (obptr, size); /* Take the address of the function. */ funcp = obstack_alloc; This is the same situation that exists in ISO C for the standard library functions. *Note Macro Definitions::. *Warning:* When you do use the macros, you must observe the precaution of avoiding side effects in the first operand, even in ISO C. If you use the GNU C compiler, this precaution is not necessary, because various language extensions in GNU C permit defining the macros so as to compute each argument only once. Growing Objects ............... Because memory in obstack chunks is used sequentially, it is possible to build up an object step by step, adding one or more bytes at a time to the end of the object. With this technique, you do not need to know how much data you will put in the object until you come to the end of it. We call this the technique of "growing objects". The special functions for adding data to the growing object are described in this section. You don't need to do anything special when you start to grow an object. Using one of the functions to add data to the object automatically starts it. However, it is necessary to say explicitly when the object is finished. This is done with the function `obstack_finish'. The actual address of the object thus built up is not known until the object is finished. Until then, it always remains possible that you will add so much data that the object must be copied into a new chunk. While the obstack is in use for a growing object, you cannot use it for ordinary allocation of another object. If you try to do so, the space already added to the growing object will become part of the other object. - Function: void obstack_blank (struct obstack *OBSTACK-PTR, int SIZE) The most basic function for adding to a growing object is `obstack_blank', which adds space without initializing it. - Function: void obstack_grow (struct obstack *OBSTACK-PTR, void *DATA, int SIZE) To add a block of initialized space, use `obstack_grow', which is the growing-object analogue of `obstack_copy'. It adds SIZE bytes of data to the growing object, copying the contents from DATA. - Function: void obstack_grow0 (struct obstack *OBSTACK-PTR, void *DATA, int SIZE) This is the growing-object analogue of `obstack_copy0'. It adds SIZE bytes copied from DATA, followed by an additional null character. - Function: void obstack_1grow (struct obstack *OBSTACK-PTR, char C) To add one character at a time, use the function `obstack_1grow'. It adds a single byte containing C to the growing object. - Function: void obstack_ptr_grow (struct obstack *OBSTACK-PTR, void *DATA) Adding the value of a pointer one can use the function `obstack_ptr_grow'. It adds `sizeof (void *)' bytes containing the value of DATA. - Function: void obstack_int_grow (struct obstack *OBSTACK-PTR, int DATA) A single value of type `int' can be added by using the `obstack_int_grow' function. It adds `sizeof (int)' bytes to the growing object and initializes them with the value of DATA. - Function: void * obstack_finish (struct obstack *OBSTACK-PTR) When you are finished growing the object, use the function `obstack_finish' to close it off and return its final address. Once you have finished the object, the obstack is available for ordinary allocation or for growing another object. This function can return a null pointer under the same conditions as `obstack_alloc' (*note Allocation in an Obstack::). When you build an object by growing it, you will probably need to know afterward how long it became. You need not keep track of this as you grow the object, because you can find out the length from the obstack just before finishing the object with the function `obstack_object_size', declared as follows: - Function: int obstack_object_size (struct obstack *OBSTACK-PTR) This function returns the current size of the growing object, in bytes. Remember to call this function _before_ finishing the object. After it is finished, `obstack_object_size' will return zero. If you have started growing an object and wish to cancel it, you should finish it and then free it, like this: obstack_free (obstack_ptr, obstack_finish (obstack_ptr)); This has no effect if no object was growing. You can use `obstack_blank' with a negative size argument to make the current object smaller. Just don't try to shrink it beyond zero length--there's no telling what will happen if you do that. Extra Fast Growing Objects .......................... The usual functions for growing objects incur overhead for checking whether there is room for the new growth in the current chunk. If you are frequently constructing objects in small steps of growth, this overhead can be significant. You can reduce the overhead by using special "fast growth" functions that grow the object without checking. In order to have a robust program, you must do the checking yourself. If you do this checking in the simplest way each time you are about to add data to the object, you have not saved anything, because that is what the ordinary growth functions do. But if you can arrange to check less often, or check more efficiently, then you make the program faster. The function `obstack_room' returns the amount of room available in the current chunk. It is declared as follows: - Function: int obstack_room (struct obstack *OBSTACK-PTR) This returns the number of bytes that can be added safely to the current growing object (or to an object about to be started) in obstack OBSTACK using the fast growth functions. While you know there is room, you can use these fast growth functions for adding data to a growing object: - Function: void obstack_1grow_fast (struct obstack *OBSTACK-PTR, char C) The function `obstack_1grow_fast' adds one byte containing the character C to the growing object in obstack OBSTACK-PTR. - Function: void obstack_ptr_grow_fast (struct obstack *OBSTACK-PTR, void *DATA) The function `ob