CPIO(5) File Formats Manual CPIO(5)
NAME
cpio — format of cpio archive files
DESCRIPTION
The cpio archive format collects any number of files, directories, and other file system objects (symbolic links, device nodes, etc.) into a single stream of bytes.
General Format
Each file system object in a cpio archive comprises a header record with basic numeric metadata followed by the full pathname of the entry and the file data. The header record stores a series of integer values that generally follow the fields in struct stat. (See stat(2) for details.) The variants differ primarily in how they store those integers (binary, octal, or hexadecimal). The header is followed by the pathname of the entry (the length of the pathname is stored in the header) and any file data. The end of the archive is indicated by a special record with the pathname “TRAILER!!!”.
PWB format
The PWB binary cpio format is the original format, when cpio was introduced as part of the Programmer’s Work Bench system, a variant of 6th Edition UNIX. It stores numbers as 2-byte and 4-byte binary values. Each entry begins with a header in the following format:
struct
header_pwb_cpio {
short h_magic;
short h_dev;
short h_ino;
short h_mode;
short h_uid;
short h_gid;
short h_nlink;
short h_majmin;
long h_mtime;
short h_namesize;
long h_filesize;
};
The short fields here are 16-bit integer values, while the long fields are 32 bit integers. Since PWB UNIX, like the 6th Edition UNIX it was based on, only ran on PDP-11 computers, they are in PDP-endian format, which has little-endian shorts, and big-endian longs. That is, the long integer whose hexadecimal representation is 0x12345678 would be stored in four successive bytes as 0x34, 0x12, 0x78, 0x56. The fields are as follows:
h_magic
The integer value octal 070707.
h_dev, h_ino
The device and inode numbers from the disk. These are used by programs that read cpio archives to determine when two entries refer to the same file. Programs that synthesize cpio archives should be careful to set these to distinct values for each entry.
h_mode
The mode specifies both the regular permissions and the file type, and it also holds a couple of bits that are irrelevant to the cpio format, because the field is actually a raw copy of the mode field in the inode representing the file. These are the IALLOC flag, which shows that the inode entry is in use, and the ILARG flag, which shows that the file it represents is large enough to have indirect blocks pointers in the inode. The mode is decoded as follows:
0100000
IALLOC flag - irrelevant to cpio.
0060000
This masks the file type bits.
0040000
File type value for directories.
0020000
File type value for character special devices.
0060000
File type value for block special devices.
0010000
ILARG flag - irrelevant to cpio.
0004000
SUID bit.
0002000
SGID bit.
0001000
Sticky bit.
0000777
The lower 9 bits specify read/write/execute permissions for world, group, and user following standard POSIX conventions.
h_uid, h_gid
The numeric user id and group id of the owner.
h_nlink
The number of links to this file. Directories always have a value of at least two here. Note that hardlinked files include file data with every copy in the archive.
h_majmin
For block special and character special entries, this field contains the associated device number, with the major number in the high byte, and the minor number in the low byte. For all other entry types, it should be set to zero by writers and ignored by readers.
h_mtime
Modification time of the file, indicated as the number of seconds since the start of the epoch, 00:00:00 UTC January 1, 1970.
h_namesize
The number of bytes in the pathname that follows the header. This count includes the trailing NUL byte.
h_filesize
The size of the file. Note that this archive format is limited to 16 megabyte file sizes, because PWB UNIX, like 6th Edition, only used an unsigned 24 bit integer for the file size internally.
The pathname immediately follows the fixed header. If h_namesize is odd, an additional NUL byte is added after the pathname. The file data is then appended, again with an additional NUL appended if needed to get the next header at an even offset.
Hardlinked files are not given special treatment; the full file contents are included with each copy of the file.
New Binary Format
The new binary cpio format showed up when cpio was adopted into late 7th Edition UNIX. It is exactly like the PWB binary format, described above, except for three changes:
First, UNIX now ran on more than one hardware type, so the endianness of 16 bit integers must be determined by observing the magic number at the start of the header. The 32 bit integers are still always stored with the most significant word first, though, so each of those two, in the struct shown above, was stored as an array of two 16 bit integers, in the traditional order. Those 16 bit integers, like all the others in the struct, were accessed using a macro that byte swapped them if necessary.
Next, 7th Edition had more file types to store, and the IALLOC and ILARG flag bits were re-purposed to accommodate these. The revised use of the various bits is as follows:
0170000
This masks the file type bits.
0140000
File type value for sockets.
0120000
File type value for symbolic links. For symbolic links, the link body is stored as file data.
0100000
File type value for regular files.
0060000
File type value for block special devices.
0040000
File type value for directories.
0020000
File type value for character special devices.
0010000
File type value for named pipes or FIFOs.
0004000
SUID bit.
0002000
SGID bit.
0001000
Sticky bit.
0000777
The lower 9 bits specify read/write/execute permissions for world, group, and user following standard POSIX conventions.
Finally, the file size field now represents a signed 32 bit integer in the underlying file system, so the maximum file size has increased to 2 gigabytes.
Note that there is no obvious way to tell which of the two binary formats an archive uses, other than to see which one makes more sense. The typical error scenario is that a PWB format archive unpacked as if it were in the new format will create named sockets instead of directories, and then fail to unpack files that should go in those directories. Running bsdcpio -itv on an unknown archive will make it obvious which it is: if it’s PWB format, directories will be listed with an ’s’ instead of a ’d’ as the first character of the mode string, and the larger files will have a ’?’ in that position.
Portable ASCII Format
Version 2 of the Single UNIX Specification (“SUSv2”) standardized an ASCII variant that is portable across all platforms. It is commonly known as the “old character” format or as the “odc” format. It stores the same numeric fields as the old binary format, but represents them as 6-character or 11-character octal values.
struct
cpio_odc_header {
char c_magic[6];
char c_dev[6];
char c_ino[6];
char c_mode[6];
char c_uid[6];
char c_gid[6];
char c_nlink[6];
char c_rdev[6];
char c_mtime[11];
char c_namesize[6];
char c_filesize[11];
};
The fields are identical to those in the new binary format. The name and file body follow the fixed header. Unlike the binary formats, there is no additional padding after the pathname or file contents. If the files being archived are themselves entirely ASCII, then the resulting archive will be entirely ASCII, except for the NUL byte that terminates the name field.
New ASCII Format
The "new" ASCII format uses 8-byte hexadecimal fields for all numbers and separates device numbers into separate fields for major and minor numbers.
struct
cpio_newc_header {
char c_magic[6];
char c_ino[8];
char c_mode[8];
char c_uid[8];
char c_gid[8];
char c_nlink[8];
char c_mtime[8];
char c_filesize[8];
char c_devmajor[8];
char c_devminor[8];
char c_rdevmajor[8];
char c_rdevminor[8];
char c_namesize[8];
char c_check[8];
};
Except as specified below, the fields here match those specified for the new binary format above.
magic
The string “070701”.
check
This field is always set to zero by writers and ignored by readers. See the next section for more details.
The pathname is followed by NUL bytes so that the total size of the fixed header plus pathname is a multiple of four. Likewise, the file data is padded to a multiple of four bytes. Note that this format supports only 4 gigabyte files (unlike the older ASCII format, which supports 8 gigabyte files).
In this format, hardlinked files are handled by setting the filesize to zero for each entry except the first one that appears in the archive.
New CRC Format
The CRC format is identical to the new ASCII format described in the previous section except that the magic field is set to “070702” and the check field is set to the sum of all bytes in the file data. This sum is computed treating all bytes as unsigned values and using unsigned arithmetic. Only the least-significant 32 bits of the sum are stored.
HP variants
The cpio implementation distributed with HPUX used XXXX but stored device numbers differently XXX.
Other Extensions and Variants
Sun Solaris uses additional file types to store extended file data, including ACLs and extended attributes, as special entries in cpio archives.
XXX Others? XXX
SEE ALSO
cpio(1), tar(5)
STANDARDS
The cpio utility is no longer a part of POSIX or the Single Unix Standard. It last appeared in Version 2 of the Single UNIX Specification (“SUSv2”). It has been supplanted in subsequent standards by pax(1). The portable ASCII format is currently part of the specification for the pax(1) utility.
HISTORY
The original cpio utility was written by Dick Haight while working in AT&T’s Unix Support Group. It appeared in 1977 as part of PWB/UNIX 1.0, the “Programmer’s Work Bench” derived from Version 6 AT&T UNIX that was used internally at AT&T. Both the new binary and old character formats were in use by 1980, according to the System III source released by SCO under their “Ancient Unix” license. The character format was adopted as part of IEEE Std 1003.1-1988 (“POSIX.1”). XXX when did "newc" appear? Who invented it? When did HP come out with their variant? When did Sun introduce ACLs and extended attributes? XXX
BUGS
The “CRC” format is mis-named, as it uses a simple checksum and not a cyclic redundancy check.
The binary formats are limited to 16 bits for user id, group id, device, and inode numbers. They are limited to 16 megabyte and 2 gigabyte file sizes for the older and newer variants, respectively.
The old ASCII format is limited to 18 bits for the user id, group id, device, and inode numbers. It is limited to 8 gigabyte file sizes.
The new ASCII format is limited to 4 gigabyte file sizes.
None of the cpio formats store user or group names, which are essential when moving files between systems with dissimilar user or group numbering.
Especially when writing older cpio variants, it may be necessary to map actual device/inode values to synthesized values that fit the available fields. With very large filesystems, this may be necessary even for the newer formats. Debian December 23, 2011 CPIO(5)
CPIO(5) BSD File Formats Manual CPIO(5)
d27 2 a28 2cpio — format of cpio archive files
d32 1 a32 1The cpio archive format d37 2 a38 21
General
Format
Each file system object in a cpio archive comprises a
header record with basic numeric metadata followed by the
full pathname of the entry and the file data. The header
record stores a series of integer values that generally
follow the fields in struct stat. (See stat(2) for
details.) The variants differ primarily in how they store
those integers (binary, octal, or hexadecimal). The header
is followed by the pathname of the entry (the length of the
pathname is stored in the header) and any file data. The end
of the archive is indicated by a special record with the
pathname “TRAILER!!!”.
PWB
format
The PWB binary cpio format is the original format,
when cpio was introduced as part of the Programmer’s
Work Bench system, a variant of 6th Edition UNIX. It stores
numbers as 2-byte and 4-byte binary values. Each entry
begins with a header in the following format:
struct d78 1 a78 1
The short d90 1 a90 1
The integer value octal d95 1 a95 1
The device and inode numbers d104 1 a104 1
The mode d116 1 a116 1
IALLOC flag - d121 1 a121 1
This masks the d126 1 a126 1
File type value d131 1 a131 1
File type value d136 1 a136 1
File type value d141 1 a141 1
ILARG flag - d146 1 a146 1
SUID bit.
d150 1 a150 1SGID bit.
d154 1 a154 1Sticky bit.
d158 1 a158 1The lower 9 d164 1 a164 1
The numeric user id and group d169 1 a169 1
The number of links to this d176 1 a176 1
For block special and character d184 1 a184 1
Modification time of the file, d190 1 a190 1
The number of bytes in the d196 1 a196 1
The size of the file. Note that d201 1 a201 1
The pathname d208 1 a208 1
Hardlinked files d212 7 a218 6
New Binary
Format
The new binary cpio format showed up when cpio was
adopted into late 7th Edition UNIX. It is exactly like the
PWB binary format, described above, except for three
changes:
First, UNIX now d231 1 a231 1
Next, 7th d238 1 a238 1
This masks the d243 1 a243 1
File type value d248 1 a248 1
File type value d254 1 a254 1
File type value d259 1 a259 1
File type value d264 1 a264 1
File type value d269 1 a269 1
File type value d274 1 a274 1
File type value d279 1 a279 1
SUID bit.
d283 1 a283 1SGID bit.
d287 1 a287 1Sticky bit.
d291 1 a291 1The lower 9 d295 1 a295 1
Finally, the d300 1 a300 1
Note that there d314 10 a323 9
Portable
ASCII Format
Version 2 of the Single UNIX Specification
(“SUSv2”) standardized an ASCII variant that is
portable across all platforms. It is commonly known as the
“old character” format or as the
“odc” format. It stores the same numeric fields
as the old binary format, but represents them as 6-character
or 11-character octal values.
struct d340 1 a340 1
The fields are d349 2 a350 5
New ASCII
Format
The "new" ASCII format uses 8-byte hexadecimal
fields for all numbers and separates device numbers into
separate fields for major and minor numbers.
struct d375 1 a375 1
Except as d381 1 a381 1
The string d386 1 a386 1
This field is d390 1 a390 1
The pathname is d397 1 a397 1
In this format, d402 25 a426 20
New CRC
Format
The CRC format is identical to the new ASCII format
described in the previous section except that the magic
field is set to “070702” and the check
field is set to the sum of all bytes in the file data. This
sum is computed treating all bytes as unsigned values and
using unsigned arithmetic. Only the least-significant 32
bits of the sum are stored.
HP
variants
The cpio implementation distributed with HPUX used
XXXX but stored device numbers differently XXX.
Other
Extensions and Variants
Sun Solaris uses additional file types to store extended
file data, including ACLs and extended attributes, as
special entries in cpio archives.
XXX Others? d433 2 a434 1
cpio(1), tar(5)
d438 1 a438 1The cpio utility is no d442 3 a444 2 standards by pax(1). The portable ASCII format is currently part of the specification for the pax(1) utility.
d448 1 a448 1The original cpio utility was d464 1 a464 1
The “CRC” format is d468 1 a468 1
The binary d474 1 a474 1
The old ASCII d479 1 a479 1
The new ASCII d482 1 a482 1
None of the cpio d487 1 a487 1
Especially when d491 2 a492 4 be necessary even for the newer formats.
BSD December 23, 2011 BSD
@ 1.11 log @libarchive: updated to 3.7.4 Libarchive 3.7.4 is a bugfix and security release Security fixes: rar: Fix OOB in rar e8 filter (CVE-2024-26256) zip: Fix out of boundary access Important bugfixes: 7zip: Limit amount of properties bsdtar: Fix error handling around strtol() usages passphrase: Improve newline handling on Windows passphrase: Never allow empty passwords rar: Fix "File CRC Error" when extracting specific rar4 archives xar: Avoid infinite link loop zip: Update AppleDouble support for directories zstd: Implement core detection @ text @d2 1 a2 1 @ 1.10 log @libarchive: updated to 3.7.3 Libarchive 3.7.3 is a feature, security and bugfix release. New features: PCRE2 support add trailing letter b to bsdtar(1) substitute pattern add support for long options "--group" and "--owner" to tar(1) Security fixes: Fix possible vulnerability in tar error reporting introduced in f27c173 Important bugfixes: ISO9660: preserve the natural order of links rar5: fix decoding unicode filenames on Windows rar5: fix infinite loop if during rar5 decompression the last block produced no data xz filter: fix incorrect eof at the end of an lzip member zip: fix end-of-data marker processing when decompressing zip archives multiple bsdunzip(1) fixes filetime truncation fix on Windows @ text @d2 1 a2 1 @ 1.9 log @libarchive: updated to 3.7.2 Libarchive 3.7.2 is a security, bugfix and feature release. Security fixes: Multiple vulnerabilities have been fixed in the PAX writer (1b4e0d0) Important bugfixes: bsdunzip(1) now correctly handles arguments following an -x after the zipfile New features: bsdunzip(1) now supports the "--version" flag 7-zip reader now translates Windows permissions into UNIX permissions uudecode filter in raw mode now supports file name and file mode zstd filter now supports the "long" write option Libarchive 3.7.1 is a security, feature and bugfix release. Security fixes: SEGV and stack buffer overflow in verbose mode of cpio Feature updates: bsdunzip updated to match latest upstream code Important bugfixes: miscellaneous functional bugfixes build fixes on multiple platforms Libarchive 3.7.0 is a feature and bugfix release. New features: bsdunzip: new tool ported from FreeBSD drop-in replacement for Info-ZIP unzip, not yet ported for Windows 7zip reader: support for Zstandard compression 7zip reader: support for ARM64 filter zstd filter: support for multi-frame zstd archives Other notable bugfixes and improvements: pax: fix year 2038 problem on platforms with 64-bit time_t Windows: Universal Windows Platform (UWP) fixes and improvements Windows: bcrypt usage fixes and improvements Windows: time function usage fixes and improvements @ text @d2 1 a2 1 @ 1.8 log @libarchive: Update to 3.4.3 Libarchive 3.4.3 is a feature and bugfix release. New features: support for pzstd compressed files (#1357) support for RHT.security.selinux tar extended attribute (#1348) Important bugfixes: various zstd fixes and improvements (#1342 #1352 #1359) child process handling fixes (#1372) Libarchive 3.4.2 is a feature and security release. New features: support for atomic file extraction (bsdtar -x --safe-writes) (#1289) support for mbed TLS (PolarSSL) (#1301) Important bugfixes: security fixes in RAR5 reader (#1280 #1326) compression buffer fix in XAR writer (#1317) fix uname and gname longer than 32 characters in PAX writer (#1319) fix segfault when archiving hard links in ISO9660 and XAR writers (#1325) fix support for extracting 7z archive entries with Delta filter (#987) Libarchive 3.4.1 is a feature and security release. New features: Unicode filename support for reading lha/lzh archives New pax write option "xattrhdr" Important bugfixes: security fixes in wide string processing (#1276 #1298) security fixes in RAR5 reader (#1212 #1217 #1296) security fixes and optimizations to write filter logic (#351) security fix related to use of readlink(2) (1dae5a5) sparse file handling fixes (#1218 #1260) Thanks to all contributors and bug reporters. Special thanks to Christos Zoulas (@@zoulasc) from NetBSD for the atomic file extraction feature. @ text @d1 2 a2 2 d49 1 a49 1 pathname ’’TRAILER!!!’’. d53 5 a57 2 XXX Any documentation of the original PWB/UNIX 1.0 format? XXX d59 14 a72 5Old Binary
Format
The old binary cpio format stores numbers as 2-byte
and 4-byte binary values. Each entry begins with a header in
the following format:
struct
header_old_cpio {
unsigned short c_magic;
unsigned short c_dev;
unsigned short c_ino;
unsigned short c_mode;
unsigned short c_uid;
unsigned short c_gid;
unsigned short c_nlink;
unsigned short c_rdev;
|
unsigned short c_mtime[2]; |
unsigned short c_namesize;
|
unsigned short c_filesize[2]; |
};
The unsigned short fields here are 16-bit integer values; the unsigned int fields are 32-bit integer values. The fields are as follows
d84 1 a84 1magic
d86 2 a87 4The integer value octal 070707. This value can be used to determine whether this archive is written with little-endian or big-endian integers.
d89 1 a89 1dev, ino
d98 1 a98 1mode
d101 8 a108 2 specifies both the regular permissions and the file type. It consists of several bit fields as follows: d110 6 a115 10170000
d120 1 a120 10140000
d123 1 a123 1 for sockets. d125 1 a125 10120000
d128 1 a128 7 for symbolic links. For symbolic links, the link body is stored as file data.0100000
File type value for regular files.
a134 100040000
File type value for directories.
0020000
File type value for character special devices.
d137 2 a138 2File type value for named pipes or FIFOs.
d150 1 a150 3Sticky bit. On some systems, this modifies the behavior of executables and/or directories.
d158 1 a158 1uid, gid
d163 1 a163 1nlink
d165 11 a175 10The number of links to this file. Directories always have a value of at least two here. Note that hardlinked files include file data with every copy in the archive.
rdev
For block special and character special entries, this field contains the associated device number. For all other entry types, it d178 1 a178 1
mtime
d180 3 a182 7Modification time of the file, indicated as the number of seconds since the start of the epoch, 00:00:00 UTC January 1, 1970. The four-byte integer is stored with the most-significant 16 bits first followed by the least-significant 16 bits. Each of the two 16 bit values are stored in machine-native byte order.
d184 1 a184 1namesize
d190 1 a190 1filesize
d193 3 a195 3 this archive format is limited to four gigabyte file sizes. See mtime above for a description of the storage of four-byte integers. d198 1 a198 1 immediately follows the fixed header. If the namesize d200 3 a202 2 The file data is then appended, padded with NUL bytes to an even length. d208 101 d312 6 a317 7 (’’SUSv2’’) standardized an ASCII variant that is portable across all platforms. It is commonly known as the ’’old character’’ format or as the ’’odc’’ format. It stores the same numeric fields as the old binary format, but represents them as 6-character or 11-character octal values. d335 4 a338 4 identical to those in the old binary format. The name and file body follow the fixed header. Unlike the old binary format, there is no additional padding after the pathname or file contents. If the files being archived are themselves d369 1 a369 1 the old binary format above. d374 1 a374 1 ’’070701’’. d398 5 a402 5 field is set to ’’070702’’ and the check field is set to the sum of all bytes in the file data. This sum is computed treating all bytes as unsigned values and using unsigned arithmetic. Only the least-significant 32 bits of the sum are stored. d427 3 a429 4 (’’SUSv2’’). It has been supplanted in subsequent standards by pax(1). The portable ASCII format is currently part of the specification for the pax(1) utility. d436 7 a442 8 1.0, the ’’Programmer’s Work Bench’’ derived from Version 6 AT&T UNIX that was used internally at AT&T. Both the old binary and old character formats were in use by 1980, according to the System III source released by SCO under their ’’Ancient Unix’’ license. The character format was adopted as part of IEEE Std 1003.1-1988 (’’POSIX.1’’). XXX when did d449 9 a457 9The ’’CRC’’ format is mis-named, as it uses a simple checksum and not a cyclic redundancy check.
The old binary format is limited to 16 bits for user id, group id, device, and inode numbers. It is limited to 4 gigabyte file sizes.
@ 1.7 log @Update for libarchive-3.4.0: - improvements for Android APK and JAR archives - better support for non-recursive list and extract - tar --exclude-vcs support - fixes for file attributes and flags handling - zipx support - rar 5.0 reader @ text @d1 2 a2 2 d49 1 a49 1 pathname “TRAILER!!!”. d242 7 a248 6 (“SUSv2”) standardized an ASCII variant that is portable across all platforms. It is commonly known as the “old character” format or as the “odc” format. It stores the same numeric fields as the old binary format, but represents them as 6-character or 11-character octal values. d305 1 a305 1 “070701”. d322 1 a322 1 for each entry except the last one that appears in the d329 5 a333 5 field is set to “070702” and the check field is set to the sum of all bytes in the file data. This sum is computed treating all bytes as unsigned values and using unsigned arithmetic. Only the least-significant 32 bits of the sum are stored. d358 4 a361 3 (“SUSv2”). It has been supplanted in subsequent standards by pax(1). The portable ASCII format is currently part of the specification for the pax(1) utility. d368 8 a375 7 1.0, the “Programmer’s Work Bench” derived from Version 6 AT&T UNIX that was used internally at AT&T. Both the old binary and old character formats were in use by 1980, according to the System III source released by SCO under their “Ancient Unix” license. The character format was adopted as part of IEEE Std 1003.1-1988 (“POSIX.1”). XXX when did d382 4 a385 3The “CRC” format is mis-named, as it uses a simple checksum and not a cyclic redundancy check.
@ 1.6 log @libarchive: updated to 3.3.3 libarchive 3.3.3: Avoid super-linear slowdown on malformed mtree files Many fixes for building with Visual Studio NO_OVERWRITE doesn't change existing directory attributes New support for Zstandard read and write filters @ text @d1 2 a2 2 d49 1 a49 1 pathname ’’TRAILER!!!’’. d242 6 a247 7 (’’SUSv2’’) standardized an ASCII variant that is portable across all platforms. It is commonly known as the ’’old character’’ format or as the ’’odc’’ format. It stores the same numeric fields as the old binary format, but represents them as 6-character or 11-character octal values. d304 1 a304 1 ’’070701’’. d328 5 a332 5 field is set to ’’070702’’ and the check field is set to the sum of all bytes in the file data. This sum is computed treating all bytes as unsigned values and using unsigned arithmetic. Only the least-significant 32 bits of the sum are stored. d357 3 a359 4 (’’SUSv2’’). It has been supplanted in subsequent standards by pax(1). The portable ASCII format is currently part of the specification for the pax(1) utility. d366 7 a372 8 1.0, the ’’Programmer’s Work Bench’’ derived from Version 6 AT&T UNIX that was used internally at AT&T. Both the old binary and old character formats were in use by 1980, according to the System III source released by SCO under their ’’Ancient Unix’’ license. The character format was adopted as part of IEEE Std 1003.1-1988 (’’POSIX.1’’). XXX when did d379 3 a381 4The ’’CRC’’ format is mis-named, as it uses a simple checksum and not a cyclic redundancy check.
@ 1.5 log @Merge for libarchive-3.3.2. @ text @d2 1 a2 1 @ 1.4 log @Merge libarchive-3.3.1. @ text @d2 1 a2 1 d49 1 a49 1 pathname ‘‘TRAILER!!!’’. d242 1 a242 1 (‘‘SUSv2’’) standardized an ASCII d244 1 a244 1 commonly known as the ‘‘old d246 1 a246 1 ‘‘odc’’ format. It stores the same d305 1 a305 1 ‘‘070701’’. d329 1 a329 1 field is set to ‘‘070702’’ and the d358 1 a358 1 (‘‘SUSv2’’). It has been supplanted d368 1 a368 1 1.0, the ‘‘Programmer’s Work d373 1 a373 1 their ‘‘Ancient Unix’’ license. The d375 1 a375 1 (‘‘POSIX.1’’). XXX when did d383 1 a383 1 ‘‘CRC’’ format is mis-named, as it @ 1.3 log @Update for libarchive 3.2.1. @ text @d2 1 a2 1 d49 1 a49 1 pathname ’’TRAILER!!!’’. d242 1 a242 1 (’’SUSv2’’) standardized an ASCII d244 1 a244 1 commonly known as the ’’old d246 1 a246 1 ’’odc’’ format. It stores the same d305 1 a305 1 ’’070701’’. d329 1 a329 1 field is set to ’’070702’’ and the d358 1 a358 1 (’’SUSv2’’). It has been supplanted d368 1 a368 1 1.0, the ’’Programmer’s Work d373 1 a373 1 their ’’Ancient Unix’’ license. The d375 1 a375 1 (’’POSIX.1’’). XXX when did d383 1 a383 1 ’’CRC’’ format is mis-named, as it @ 1.2 log @Changes 3.1.2: This is a maintenance update to fix issues with the new RAR seeking feature. This new release also contains fixes for build failures when building libarchive using Visual Studio 2012 and MinGW. @ text @d1 2 a2 2 @ 1.1 log @Initial revision @ text @d1 2 a2 2 d11 4 a14 3 p { margin-top: 0; margin-bottom: 0; } pre { margin-top: 0; margin-bottom: 0; } table { margin-top: 0; margin-bottom: 0; } d23 1 a23 2CPIO(5) FreeBSD File Formats Manual CPIO(5)
d25 1 a25 1NAME
d27 1 a27 1cpio — format of d30 1 d32 1 a32 3
DESCRIPTION
The cpio archive format d37 1 a37 1
General d49 1 a49 1 pathname ‘‘TRAILER!!!’’.
d51 1 a51 1PWB d56 1 a56 1
Old Binary d62 1 a62 1
struct d73 1 a73 1
|
d80 3
a82 3
unsigned short c_mtime[2]; |
d85 1 a85 1 |
|
d94 1
a94 1
unsigned short c_filesize[2]; |
d97 1
a97 1
|
unsigned short c_mtime[2]; |
unsigned short c_namesize;
|
unsigned short c_filesize[2]; |
};
The unsigned short fields here are 16-bit integer values; the unsigned int fields are 32-bit integer values. The fields are as follows
magic
The integer value octal 070707. This value can be used to determine whether this archive is written with little-endian or big-endian integers.
dev, ino
The device and inode numbers from the disk. These are used by programs that read cpio archives to determine when two entries refer to the same file. Programs that synthesize cpio archives should be careful to set these to distinct values for each entry.
mode
The mode specifies both the regular permissions and the file type. It consists of several bit fields as follows:
0170000
This masks the file type bits.
0140000
File type value for sockets.
0120000
File type value for symbolic links. For symbolic links, the link body is stored as file data.
0100000
File type value for regular files.
0060000
File type value for block special devices.
0040000
File type value for directories.
0020000
File type value for character special devices.
0010000
File type value for named pipes or FIFOs.
0004000
SUID bit.
0002000
SGID bit.
0001000
Sticky bit. On some systems, this modifies the behavior of executables and/or directories.
0000777
The lower 9 bits specify read/write/execute permissions for world, group, and user following standard POSIX conventions.
uid, gid
The numeric user id and group id of the owner.
nlink
The number of links to this file. Directories always have a value of at least two here. Note that hardlinked files include file data with every copy in the archive.
rdev
For block special and character special entries, this field contains the associated device number. For all other entry types, it should be set to zero by writers and ignored by readers.
mtime
Modification time of the file, indicated as the number of seconds since the start of the epoch, 00:00:00 UTC January 1, 1970. The four-byte integer is stored with the most-significant 16 bits first followed by the least-significant 16 bits. Each of the two 16 bit values are stored in machine-native byte order.
namesize
The number of bytes in the pathname that follows the header. This count includes the trailing NUL byte.
filesize
The size of the file. Note that this archive format is limited to four gigabyte file sizes. See mtime above for a description of the storage of four-byte integers.
The pathname immediately follows the fixed header. If the namesize is odd, an additional NUL byte is added after the pathname. The file data is then appended, padded with NUL bytes to an even length.
Hardlinked files are not given special treatment; the full file contents are included with each copy of the file.
Portable
ASCII Format
Version 2 of the Single UNIX Specification
(‘‘SUSv2’’) standardized an ASCII
variant that is portable across all platforms. It is
commonly known as the ‘‘old
character’’ format or as the
‘‘odc’’ format. It stores the same
numeric fields as the old binary format, but represents them
as 6-character or 11-character octal values.
struct
cpio_odc_header {
char c_magic[6];
char c_dev[6];
char c_ino[6];
char c_mode[6];
char c_uid[6];
char c_gid[6];
char c_nlink[6];
char c_rdev[6];
char c_mtime[11];
char c_namesize[6];
char c_filesize[11];
};
The fields are identical to those in the old binary format. The name and file body follow the fixed header. Unlike the old binary format, there is no additional padding after the pathname or file contents. If the files being archived are themselves entirely ASCII, then the resulting archive will be entirely ASCII, except for the NUL byte that terminates the name field.
New ASCII
Format
The "new" ASCII format uses 8-byte hexadecimal
fields for all numbers and separates device numbers into
separate fields for major and minor numbers.
struct
cpio_newc_header {
char c_magic[6];
char c_ino[8];
char c_mode[8];
char c_uid[8];
char c_gid[8];
char c_nlink[8];
char c_mtime[8];
char c_filesize[8];
char c_devmajor[8];
char c_devminor[8];
char c_rdevmajor[8];
char c_rdevminor[8];
char c_namesize[8];
char c_check[8];
};
Except as specified below, the fields here match those specified for the old binary format above.
magic
The string ‘‘070701’’.
check
This field is always set to zero by writers and ignored by readers. See the next section for more details.
The pathname is followed by NUL bytes so that the total size of the fixed header plus pathname is a multiple of four. Likewise, the file data is padded to a multiple of four bytes. Note that this format supports only 4 gigabyte files (unlike the older ASCII format, which supports 8 gigabyte files).
In this format, hardlinked files are handled by setting the filesize to zero for each entry except the last one that appears in the archive.
New CRC
Format
The CRC format is identical to the new ASCII format
described in the previous section except that the magic
field is set to ‘‘070702’’ and the
check field is set to the sum of all bytes in the
file data. This sum is computed treating all bytes as
unsigned values and using unsigned arithmetic. Only the
least-significant 32 bits of the sum are stored.
HP
variants
The cpio implementation distributed with HPUX used
XXXX but stored device numbers differently XXX.
Other
Extensions and Variants
Sun Solaris uses additional file types to store extended
file data, including ACLs and extended attributes, as
special entries in cpio archives.
XXX Others? XXX
BUGS
The ‘‘CRC’’ format is mis-named, as it uses a simple checksum and not a cyclic redundancy check.
The old binary format is limited to 16 bits for user id, group id, device, and inode numbers. It is limited to 4 gigabyte file sizes.
The old ASCII format is limited to 18 bits for the user id, group id, device, and inode numbers. It is limited to 8 gigabyte file sizes.
The new ASCII format is limited to 4 gigabyte file sizes.
None of the cpio formats store user or group names, which are essential when moving files between systems with dissimilar user or group numbering.
Especially when writing older cpio variants, it may be necessary to map actual device/inode values to synthesized values that fit the available fields. With very large filesystems, this may be necessary even for the newer formats.
SEE ALSO
cpio(1), tar(5)
STANDARDS
The cpio utility is no longer a part of POSIX or the Single Unix Standard. It last appeared in Version 2 of the Single UNIX Specification (‘‘SUSv2’’). It has been supplanted in subsequent standards by pax(1). The portable ASCII format is currently part of the specification for the pax(1) utility.
HISTORY
The original cpio utility was written by Dick Haight while working in AT&T’s Unix Support Group. It appeared in 1977 as part of PWB/UNIX 1.0, the ‘‘Programmer’s Work Bench’’ derived from Version 6 AT&T UNIX that was used internally at AT&T. Both the old binary and old character formats were in use by 1980, according to the System III source released by SCO under their ‘‘Ancient Unix’’ license. The character format was adopted as part of IEEE Std 1003.1-1988 (‘‘POSIX.1’’). XXX when did "newc" appear? Who invented it? When did HP come out with their variant? When did Sun introduce ACLs and extended attributes? XXX
FreeBSD 9.0 October 5, 2007 FreeBSD 9.0
)0 60 Q()0
72 Q 0 Cg EP
%%Page: 12 12
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF(struct header_old_cpio {)0 12 Q(unsigned short)
20 24 Q(c_magic;)7.5 E(unsigned short)20 36 Q(c_de)7.5 E(v;)-.25 E
(unsigned short)20 48 Q(c_ino;)7.5 E(unsigned short)20 60 Q(c_mode;)7.5
E(unsigned short)20 72 Q(c_uid;)7.5 E 0 Cg EP
%%Page: 13 13
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF(unsigned short)20 12 Q(c_gid;)7.5 E
(unsigned short)20 24 Q(c_nlink;)7.5 E(unsigned short)20 36 Q(c_rde)7.5
E(v;)-.25 E(unsigned short)36 48 Q(c_mtime[2];)7.5 E(unsigned short)20
60 Q(c_namesize;)7.5 E(unsigned short)36 72 Q(c_\214lesize[2];)7.5 E 0
Cg EP
%%Page: 14 14
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF(};)0 12 Q()0 24 Q()0 36 Q()36 60 Q(The)0 72 Q 0 Cg EP %%Page: 15 15 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF(unsigned short)0 12 Q (\214elds here are 16-bit inte)0 24 Q(ger v)-.15 E(alues; the)-.25 E (unsigned int)0 36 Q(\214elds are 32-bit inte)0 48 Q(ger v)-.15 E(alues.)-.25 E(The \214elds are as follo)0 60 Q(ws)-.25 E (
)36 24 Q (The pathname immediately follo)0 36 Q(ws the \214x)-.25 E(ed header) -.15 E(.)-.55 E(If the)0 48 Q(namesize)0 60 Q (is odd, an additional NUL byte is added after the pathname.)0 72 Q 0 Cg EP %%Page: 29 29 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF (The \214le data is then appended, padded with NUL)0 12 Q(bytes to an e) 0 24 Q -.15(ve)-.25 G 2.5(nl).15 G(ength.)-2.5 E(
)36 36 Q(Hardlink)0 48 Q(ed \214les are not gi)-.1 E -.15(ve)-.25 G 2.5(ns).15 G (pecial treatment;)-2.5 E (the full \214le contents are included with each cop)0 60 Q 2.5(yo)-.1 G 2.5(ft)-2.5 G(he)-2.5 E(\214le.)0 72 Q 0 Cg EP %%Page: 30 30 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF(
)0 36 Q()0 48 Q
(struct cpio_odc_header {)0 60 Q 7.5(char c_magic[6];)20 72 R 0 Cg EP
%%Page: 33 33
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF 7.5(char c_de)20 12 R(v[6];)-.25 E 7.5
(char c_ino[6];)20 24 R 7.5(char c_mode[6];)20 36 R 7.5(char c_uid[6];)
20 48 R 7.5(char c_gid[6];)20 60 R 7.5(char c_nlink[6];)20 72 R 0 Cg EP
%%Page: 34 34
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF 7.5(char c_rde)20 12 R(v[6];)-.25 E 7.5
(char c_mtime[11];)20 24 R 7.5(char c_namesize[6];)20 36 R 7.5
(char c_\214lesize[11];)20 48 R(};)0 60 Q()0 72 Q 0 Cg EP
%%Page: 35 35
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF()0 12 Q()36 36 Q (The \214elds are identical to those in the old binary format.)0 48 Q (The name and \214le body follo)0 60 Q 2.5(wt)-.25 G(he \214x)-2.5 E (ed header)-.15 E(.)-.55 E(Unlik)0 72 Q 2.5(et)-.1 G (he old binary format, there is no additional padding)-2.5 E 0 Cg EP %%Page: 36 36 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF(after the pathname or \214le contents.)0 12 Q (If the \214les being archi)0 24 Q -.15(ve)-.25 G 2.5(da).15 G (re themselv)-2.5 E(es entirely ASCII, then)-.15 E(the resulting archi)0 36 Q .3 -.15(ve w)-.25 H(ill be entirely ASCII, e).15 E(xcept for the) -.15 E(NUL byte that terminates the name \214eld.)0 48 Q(
)
0 72 Q 0 Cg EP
%%Page: 38 38
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF()0 12 Q(struct cpio_ne)0 24 Q(wc_header {)
-.25 E 7.5(char c_magic[6];)20 36 R 7.5(char c_ino[8];)20 48 R 7.5
(char c_mode[8];)20 60 R 7.5(char c_uid[8];)20 72 R 0 Cg EP
%%Page: 39 39
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF 7.5(char c_gid[8];)20 12 R 7.5(char c_nlink[8];)
20 24 R 7.5(char c_mtime[8];)20 36 R 7.5(char c_\214lesize[8];)20 48 R
7.5(char c_de)20 60 R(vmajor[8];)-.25 E 7.5(char c_de)20 72 R
(vminor[8];)-.25 E 0 Cg EP
%%Page: 40 40
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF 7.5(char c_rde)20 12 R(vmajor[8];)-.25 E 7.5
(char c_rde)20 24 R(vminor[8];)-.25 E 7.5(char c_namesize[8];)20 36 R
7.5(char c_check[8];)20 48 R(};)0 60 Q()0 72 Q 0 Cg EP
%%Page: 41 41
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF()0 12 Q()36 36 Q (Except as speci\214ed belo)0 48 Q 1.3 -.65(w, t)-.25 H (he \214elds here match those speci\214ed).65 E (for the old binary format abo)0 60 Q -.15(ve)-.15 G(.).15 E (
)36 24 Q(The pathname is follo)0 36 Q(wed by NUL bytes so that the total size)-.25 E(of the \214x)0 48 Q (ed header plus pathname is a multiple of four)-.15 E(.)-.55 E(Lik)0 60 Q -.25(ew)-.1 G (ise, the \214le data is padded to a multiple of four bytes.).25 E (Note that this format supports only 4 gig)0 72 Q(abyte \214les \(unlik) -.05 E 2.5(et)-.1 G(he)-2.5 E 0 Cg EP %%Page: 44 44 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF(older ASCII format, which supports 8 gig)0 12 Q (abyte \214les\).)-.05 E(
)36 24 Q(In this format, hardlink)0 36 Q (ed \214les are handled by setting the)-.1 E (\214lesize to zero for each entry e)0 48 Q(xcept the last one that)-.15 E(appears in the archi)0 60 Q -.15(ve)-.25 G(.).15 E(
)36 24 Q(XXX Others? XXX)0 36 Q(
)36 60 Q (The old binary format is limited to 16 bits for user id,)0 72 Q 0 Cg EP %%Page: 51 51 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF(group id, de)0 12 Q(vice, and inode numbers.) -.25 E(It is limited to 4 gig)0 24 Q(abyte \214le sizes.)-.05 E(
)36 36 Q(The old ASCII format is limited to 18 bits for)0 48 Q (the user id, group id, de)0 60 Q(vice, and inode numbers.)-.25 E (It is limited to 8 gig)0 72 Q(abyte \214le sizes.)-.05 E 0 Cg EP %%Page: 52 52 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF(
)36 12 Q(The ne)0 24 Q 2.5(wA)-.25 G (SCII format is limited to 4 gig)-2.5 E(abyte \214le sizes.)-.05 E(
) 36 36 Q(None of the cpio formats store user or group names,)0 48 Q (which are essential when mo)0 60 Q(ving \214les between systems with) -.15 E(dissimilar user or group numbering.)0 72 Q 0 Cg EP %%Page: 53 53 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF(
)36 12 Q (Especially when writing older cpio v)0 24 Q (ariants, it may be necessary)-.25 E(to map actual de)0 36 Q (vice/inode v)-.25 E(alues to synthesized v)-.25 E(alues that)-.25 E (\214t the a)0 48 Q -.25(va)-.2 G(ilable \214elds.).25 E -.4(Wi)0 60 S (th v).4 E(ery lar)-.15 E(ge \214lesystems, this may be necessary e)-.18 E -.15(ve)-.25 G 2.5(nf).15 G(or)-2.5 E(the ne)0 72 Q(wer formats.)-.25 E 0 Cg EP %%Page: 54 54 %%BeginPageSetup BP %%EndPageSetup /F0 10/Times-Roman@@0 SF(
A)1.11 E(T&T UNIX)-1.11 E
(that w)0 48 Q(as used internally at A)-.1 E(T&T)-1.11 E(.)-.74 E
(Both the old binary and old character formats were in use)0 60 Q
(by 1980, according to the System III source released)0 72 Q 0 Cg EP
%%Page: 59 59
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF(by SCO under their)0 12 Q -.74(``)0 24 S
(Ancient Unix')-.06 E(')-.74 E(license.)0 36 Q(The character format w)0
48 Q(as adopted as part of)-.1 E(IEEE Std 1003.1-1988 \(`)2.5 60 Q
(`POSIX.1')-.74 E('\) .)-.74 E(XXX when did "ne)0
72 Q(wc" appear?)-.25 E(Who in)5 E -.15(ve)-.4 G(nted it?).15 E
(When did HP come out with their v)5 E 2.5(ariant? When)-.25 F
(did Sun introduce A)2.5 E(CLs and e)-.4 E(xtended attrib)-.15 E
(utes? XXX)-.2 E 0 Cg EP
%%Page: 60 60
%%BeginPageSetup
BP
%%EndPageSetup
/F0 10/Times-Roman@@0 SF()0 12 Q()0 24 Q 0 Cg EP
%%Trailer
end
%%EOF
@
1.1.1.5
log
@Import libarchive-3.2.1:
- security fixes and other bugfixes
- support for multhreading in xz 5.2+
@
text
@d1 769
a769 415
CPIO(5) BSD File Formats Manual CPIO(5)
NAME
cpio — format of cpio archive files
DESCRIPTION
The cpio archive format collects any number of files, directories, and other file system objects (symbolic links, device nodes, etc.) into a single stream of bytes.
General
Format
Each file system object in a cpio archive comprises a
header record with basic numeric metadata followed by the
full pathname of the entry and the file data. The header
record stores a series of integer values that generally
follow the fields in struct stat. (See stat(2) for
details.) The variants differ primarily in how they store
those integers (binary, octal, or hexadecimal). The header
is followed by the pathname of the entry (the length of the
pathname is stored in the header) and any file data. The end
of the archive is indicated by a special record with the
pathname ’’TRAILER!!!’’.
PWB
format
XXX Any documentation of the original PWB/UNIX 1.0 format?
XXX
Old Binary
Format
The old binary cpio format stores numbers as 2-byte
and 4-byte binary values. Each entry begins with a header in
the following format:
struct
header_old_cpio {
unsigned short c_magic;
unsigned short c_dev;
unsigned short c_ino;
unsigned short c_mode;
unsigned short c_uid;
unsigned short c_gid;
unsigned short c_nlink;
unsigned short c_rdev;
|
unsigned short c_mtime[2]; |
unsigned short c_namesize;
|
unsigned short c_filesize[2]; |
};
The unsigned short fields here are 16-bit integer values; the unsigned int fields are 32-bit integer values. The fields are as follows
magic
The integer value octal 070707. This value can be used to determine whether this archive is written with little-endian or big-endian integers.
dev, ino
The device and inode numbers from the disk. These are used by programs that read cpio archives to determine when two entries refer to the same file. Programs that synthesize cpio archives should be careful to set these to distinct values for each entry.
mode
The mode specifies both the regular permissions and the file type. It consists of several bit fields as follows:
0170000
This masks the file type bits.
0140000
File type value for sockets.
0120000
File type value for symbolic links. For symbolic links, the link body is stored as file data.
0100000
File type value for regular files.
0060000
File type value for block special devices.
0040000
File type value for directories.
0020000
File type value for character special devices.
0010000
File type value for named pipes or FIFOs.
0004000
SUID bit.
0002000
SGID bit.
0001000
Sticky bit. On some systems, this modifies the behavior of executables and/or directories.
0000777
The lower 9 bits specify read/write/execute permissions for world, group, and user following standard POSIX conventions.
uid, gid
The numeric user id and group id of the owner.
nlink
The number of links to this file. Directories always have a value of at least two here. Note that hardlinked files include file data with every copy in the archive.
rdev
For block special and character special entries, this field contains the associated device number. For all other entry types, it should be set to zero by writers and ignored by readers.
mtime
Modification time of the file, indicated as the number of seconds since the start of the epoch, 00:00:00 UTC January 1, 1970. The four-byte integer is stored with the most-significant 16 bits first followed by the least-significant 16 bits. Each of the two 16 bit values are stored in machine-native byte order.
namesize
The number of bytes in the pathname that follows the header. This count includes the trailing NUL byte.
filesize
The size of the file. Note that this archive format is limited to four gigabyte file sizes. See mtime above for a description of the storage of four-byte integers.
The pathname immediately follows the fixed header. If the namesize is odd, an additional NUL byte is added after the pathname. The file data is then appended, padded with NUL bytes to an even length.
Hardlinked files are not given special treatment; the full file contents are included with each copy of the file.
Portable
ASCII Format
Version 2 of the Single UNIX Specification
(’’SUSv2’’) standardized an ASCII
variant that is portable across all platforms. It is
commonly known as the ’’old
character’’ format or as the
’’odc’’ format. It stores the same
numeric fields as the old binary format, but represents them
as 6-character or 11-character octal values.
struct
cpio_odc_header {
char c_magic[6];
char c_dev[6];
char c_ino[6];
char c_mode[6];
char c_uid[6];
char c_gid[6];
char c_nlink[6];
char c_rdev[6];
char c_mtime[11];
char c_namesize[6];
char c_filesize[11];
};
The fields are identical to those in the old binary format. The name and file body follow the fixed header. Unlike the old binary format, there is no additional padding after the pathname or file contents. If the files being archived are themselves entirely ASCII, then the resulting archive will be entirely ASCII, except for the NUL byte that terminates the name field.
New ASCII
Format
The "new" ASCII format uses 8-byte hexadecimal
fields for all numbers and separates device numbers into
separate fields for major and minor numbers.
struct
cpio_newc_header {
char c_magic[6];
char c_ino[8];
char c_mode[8];
char c_uid[8];
char c_gid[8];
char c_nlink[8];
char c_mtime[8];
char c_filesize[8];
char c_devmajor[8];
char c_devminor[8];
char c_rdevmajor[8];
char c_rdevminor[8];
char c_namesize[8];
char c_check[8];
};
Except as specified below, the fields here match those specified for the old binary format above.
magic
The string ’’070701’’.
check
This field is always set to zero by writers and ignored by readers. See the next section for more details.
The pathname is followed by NUL bytes so that the total size of the fixed header plus pathname is a multiple of four. Likewise, the file data is padded to a multiple of four bytes. Note that this format supports only 4 gigabyte files (unlike the older ASCII format, which supports 8 gigabyte files).
In this format, hardlinked files are handled by setting the filesize to zero for each entry except the last one that appears in the archive.
New CRC
Format
The CRC format is identical to the new ASCII format
described in the previous section except that the magic
field is set to ’’070702’’ and the
check field is set to the sum of all bytes in the
file data. This sum is computed treating all bytes as
unsigned values and using unsigned arithmetic. Only the
least-significant 32 bits of the sum are stored.
HP
variants
The cpio implementation distributed with HPUX used
XXXX but stored device numbers differently XXX.
Other
Extensions and Variants
Sun Solaris uses additional file types to store extended
file data, including ACLs and extended attributes, as
special entries in cpio archives.
XXX Others? XXX
SEE ALSO
cpio(1), tar(5)
STANDARDS
The cpio utility is no longer a part of POSIX or the Single Unix Standard. It last appeared in Version 2 of the Single UNIX Specification (’’SUSv2’’). It has been supplanted in subsequent standards by pax(1). The portable ASCII format is currently part of the specification for the pax(1) utility.
HISTORY
The original cpio utility was written by Dick Haight while working in AT&T’s Unix Support Group. It appeared in 1977 as part of PWB/UNIX 1.0, the ’’Programmer’s Work Bench’’ derived from Version 6 AT&T UNIX that was used internally at AT&T. Both the old binary and old character formats were in use by 1980, according to the System III source released by SCO under their ’’Ancient Unix’’ license. The character format was adopted as part of IEEE Std 1003.1-1988 (’’POSIX.1’’). XXX when did "newc" appear? Who invented it? When did HP come out with their variant? When did Sun introduce ACLs and extended attributes? XXX
BUGS
The ’’CRC’’ format is mis-named, as it uses a simple checksum and not a cyclic redundancy check.
The old binary format is limited to 16 bits for user id, group id, device, and inode numbers. It is limited to 4 gigabyte file sizes.
The old ASCII format is limited to 18 bits for the user id, group id, device, and inode numbers. It is limited to 8 gigabyte file sizes.
The new ASCII format is limited to 4 gigabyte file sizes.
None of the cpio formats store user or group names, which are essential when moving files between systems with dissimilar user or group numbering.
Especially when writing older cpio variants, it may be necessary to map actual device/inode values to synthesized values that fit the available fields. With very large filesystems, this may be necessary even for the newer formats.
BSD December 23, 2011 BSD
The “CRC” format is mis-named, as it uses a simple checksum and not a cyclic redundancy check.
@ 1.1.1.10 log @Import libarchive 3.7.2 @ text @d2 1 a2 1 d53 8 a60 5 The PWB binary cpio format is the original format, when cpio was introduced as part of the Programmer’s Work Bench system, a variant of 6th Edition UNIX. It stores numbers as 2-byte and 4-byte binary values. Each entry begins with a header in the following format: d63 40 a102 13 header_pwb_cpio {The short fields here are 16-bit integer values, while the long fields are 32 bit integers. Since PWB UNIX, like the 6th Edition UNIX it was based on, only ran on PDP-11 computers, they are in PDP-endian format, which has little-endian shorts, and big-endian longs. That is, the long integer whose hexadecimal representation is 0x12345678 would be stored in four successive bytes as 0x34, 0x12, 0x78, 0x56. The fields are as follows:
h_magic
d106 4 a109 2The integer value octal 070707.
d111 1 a111 1h_dev, h_ino
d120 1 a120 1h_mode
d123 9 a131 8 specifies both the regular permissions and the file type, and it also holds a couple of bits that are irrelevant to the cpio format, because the field is actually a raw copy of the mode field in the inode representing the file. These are the IALLOC flag, which shows that the inode entry is in use, and the ILARG flag, which shows that the file it represents is large enough to have indirect blocks pointers in the inode. The mode is decoded as follows: d133 2 a134 10100000
d136 10 a145 2IALLOC flag - irrelevant to cpio.
d149 2 a150 2This masks the file type bits.
d162 1 a162 10060000
d165 1 a165 6 for block special devices.0010000
ILARG flag - irrelevant to cpio.
d177 3 a179 1Sticky bit.
d187 1 a187 1h_uid, h_gid
d192 1 a192 1h_nlink
d194 10 a203 11The number of links to this file. Directories always have a value of at least two here. Note that hardlinked files include file data with every copy in the archive.
h_majmin
For block special and character special entries, this field contains the associated device number, with the major number in the high byte, and the minor number in the low byte. For all other entry types, it d206 1 a206 1
h_mtime
d208 7 a214 3Modification time of the file, indicated as the number of seconds since the start of the epoch, 00:00:00 UTC January 1, 1970.
d216 1 a216 1h_namesize
d222 1 a222 1h_filesize
d225 3 a227 3 this archive format is limited to 16 megabyte file sizes, because PWB UNIX, like 6th Edition, only used an unsigned 24 bit integer for the file size internally. d230 1 a230 1 immediately follows the fixed header. If h_namesize d232 2 a233 3 The file data is then appended, again with an additional NUL appended if needed to get the next header at an even offset. a238 101New Binary
Format
The new binary cpio format showed up when cpio was
adopted into late 7th Edition UNIX. It is exactly like the
PWB binary format, described above, except for three
changes:
First, UNIX now ran on more than one hardware type, so the endianness of 16 bit integers must be determined by observing the magic number at the start of the header. The 32 bit integers are still always stored with the most significant word first, though, so each of those two, in the struct shown above, was stored as an array of two 16 bit integers, in the traditional order. Those 16 bit integers, like all the others in the struct, were accessed using a macro that byte swapped them if necessary.
Next, 7th Edition had more file types to store, and the IALLOC and ILARG flag bits were re-purposed to accommodate these. The revised use of the various bits is as follows:
0170000
This masks the file type bits.
0140000
File type value for sockets.
0120000
File type value for symbolic links. For symbolic links, the link body is stored as file data.
0100000
File type value for regular files.
0060000
File type value for block special devices.
0040000
File type value for directories.
0020000
File type value for character special devices.
0010000
File type value for named pipes or FIFOs.
0004000
SUID bit.
0002000
SGID bit.
0001000
Sticky bit.
0000777
The lower 9 bits specify read/write/execute permissions for world, group, and user following standard POSIX conventions.
Finally, the file size field now represents a signed 32 bit integer in the underlying file system, so the maximum file size has increased to 2 gigabytes.
Note that there is no obvious way to tell which of the two binary formats an archive uses, other than to see which one makes more sense. The typical error scenario is that a PWB format archive unpacked as if it were in the new format will create named sockets instead of directories, and then fail to unpack files that should go in those directories. Running bsdcpio -itv on an unknown archive will make it obvious which it is: if it’s PWB format, directories will be listed with an ’s’ instead of a ’d’ as the first character of the mode string, and the larger files will have a ’?’ in that position.
d265 4 a268 4 identical to those in the new binary format. The name and file body follow the fixed header. Unlike the binary formats, there is no additional padding after the pathname or file contents. If the files being archived are themselves d299 1 a299 1 the new binary format above. d321 1 a321 1 for each entry except the first one that appears in the d368 1 a368 1 at AT&T. Both the new binary and old character formats d383 4 a386 5The binary formats are limited to 16 bits for user id, group id, device, and inode numbers. They are limited to 16 megabyte and 2 gigabyte file sizes for the older and newer variants, respectively.
@ 1.1.1.11 log @libarchive: import version 3.7.3 @ text @d2 1 a2 1 @ 1.1.1.12 log @Libarchive 3.7.4 is a bugfix and security release Security fixes: rar: Fix OOB in rar e8 filter (CVE-2024-26256) zip: Fix out of boundary access Important bugfixes: 7zip: Limit amount of properties bsdtar: Fix error handling around strtol() usages passphrase: Improve newline handling on Windows passphrase: Never allow empty passwords rar: Fix "File CRC Error" when extracting specific rar4 archives xar: Avoid infinite link loop zip: Update AppleDouble support for directories zstd: Implement core detection @ text @d2 1 a2 1 @ 1.1.1.13 log @libarchive: imported version 3.7.5 Libarchive 3.7.5 Security fixes: fix multiple vulnerabilities identified by SAST cpio: ignore out-of-range gid/uid/size/ino and harden AFIO parsing lzop: prevent integer overflow rar4: protect copy_from_lzss_window_to_unp() rar4: fix CVE-2024-26256 rar4: fix OOB in delta and audio filter rar4: fix out of boundary access with large files rar4: add boundary checks to rgb filter rar4: fix OOB access with unicode filenames rar5: clear 'data ready' cache on window buffer reallocs rpm: calculate huge header sizes correctly unzip: unify EOF handling util: fix out of boundary access in mktemp functions uu: stop processing if lines are too long Important bugfixes: 7zip: fix issue when skipping first file in 7zip archive that is a multiple of 65536 bytes ar: fix archive entries having no type lha: do not allow negative file sizes lha: fix integer truncation on 32-bit systems shar: check strdup return value rar5: don't try to read rediculously long names xar: fix another infinite loop and expat error handling many Windows fixes, cleanups and improvements @ text @d2 1 a2 1 @ 1.1.1.14 log @libarchove: import version 3.7.7 @ text @d1 2 a2 2 d23 1 a23 1CPIO(5) File Formats Manual CPIO(5)
d27 2 a28 2cpio — format of cpio archive files
d32 1 a32 1The cpio archive format d37 21 a57 2
General Format
d59 1 a59 24Each file system object in a cpio archive comprises a header record with basic numeric metadata followed by the full pathname of the entry and the file data. The header record stores a series of integer values that generally follow the fields in struct stat. (See stat(2) for details.) The variants differ primarily in how they store those integers (binary, octal, or hexadecimal). The header is followed by the pathname of the entry (the length of the pathname is stored in the header) and any file data. The end of the archive is indicated by a special record with the pathname “TRAILER!!!”.
PWB format
The PWB binary cpio format is the original format, when cpio was introduced as part of the Programmer’s Work Bench system, a variant of 6th Edition UNIX. It stores numbers as 2-byte and 4-byte binary values. Each entry begins with a header in the following format:
struct d74 1 a74 1
The short d86 1 a86 1
The integer value octal d91 1 a91 1
The device and inode numbers d100 1 a100 1
The mode d112 1 a112 1
IALLOC flag - d117 1 a117 1
This masks the d122 1 a122 1
File type value d127 1 a127 1
File type value d132 1 a132 1
File type value d137 1 a137 1
ILARG flag - d142 1 a142 1
SUID bit.
d146 1 a146 1SGID bit.
d150 1 a150 1Sticky bit.
d154 1 a154 1The lower 9 d160 1 a160 1
The numeric user id and group d165 1 a165 1
The number of links to this d172 1 a172 1
For block special and character d180 1 a180 1
Modification time of the file, d186 1 a186 1
The number of bytes in the d192 1 a192 1
The size of the file. Note that d197 1 a197 1
The pathname d204 1 a204 1
Hardlinked files d208 6 a213 7
New Binary Format
The new binary cpio format showed up when cpio was adopted into late 7th Edition UNIX. It is exactly like the PWB binary format, described above, except for three changes:
d215 1 a215 1First, UNIX now d226 1 a226 1
Next, 7th d233 1 a233 1
This masks the d238 1 a238 1
File type value d243 1 a243 1
File type value d249 1 a249 1
File type value d254 1 a254 1
File type value d259 1 a259 1
File type value d264 1 a264 1
File type value d269 1 a269 1
File type value d274 1 a274 1
SUID bit.
d278 1 a278 1SGID bit.
d282 1 a282 1Sticky bit.
d286 1 a286 1The lower 9 d290 1 a290 1
Finally, the d295 1 a295 1
Note that there d309 9 a317 10
Portable ASCII Format
Version 2 of the Single UNIX Specification (“SUSv2”) standardized an ASCII variant that is portable across all platforms. It is commonly known as the “old character” format or as the “odc” format. It stores the same numeric fields as the old binary format, but represents them as 6-character or 11-character octal values.
d319 1 a319 1struct d334 1 a334 1
The fields are d343 5 a347 2
New ASCII Format
d349 1 a349 6The "new" ASCII format uses 8-byte hexadecimal fields for all numbers and separates device numbers into separate fields for major and minor numbers.
struct d367 1 a367 1
Except as d373 1 a373 1
The string d378 1 a378 1
This field is d382 1 a382 1
The pathname is d389 1 a389 1
In this format, d394 20 a413 25
New CRC Format
The CRC format is identical to the new ASCII format described in the previous section except that the magic field is set to “070702” and the check field is set to the sum of all bytes in the file data. This sum is computed treating all bytes as unsigned values and using unsigned arithmetic. Only the least-significant 32 bits of the sum are stored.
HP variants
The cpio implementation distributed with HPUX used XXXX but stored device numbers differently XXX.
Other Extensions and Variants
Sun Solaris uses additional file types to store extended file data, including ACLs and extended attributes, as special entries in cpio archives.
d415 1 a415 1XXX Others? d420 1 a420 2
cpio(1), tar(5)
d424 1 a424 1The cpio utility is no d428 2 a429 3 standards by pax(1). The portable ASCII format is currently part of the specification for the pax(1) utility.
d433 1 a433 1The original cpio utility was d449 1 a449 1
The “CRC” format is d453 1 a453 1
The binary d459 1 a459 1
The old ASCII d464 1 a464 1
The new ASCII d467 1 a467 1
None of the cpio d472 1 a472 1
Especially when d476 4 a479 2 be necessary even for the newer formats. Debian December 23, 2011 CPIO(5)
@ 1.1.1.15 log @libarchive: imported version 3.7.9 @ text @d2 1 a2 1 @ 1.1.1.16 log @libarchive: import version 3.8.0 Libarchive 3.8.0 is a feature and bugfix release. New features: bsdtar: support --mtime and --clamp-mtime lib: mbedtls 3.x compatibility 7-zip reader: improve self-extracting archive detection xar: xmllite support for the XAR reader and writer zip writer: added XZ, LZMA, ZSTD and BZIP2 support zip writer: added LZMA + RISCV BCJ filter Notable security fixes: rar: do not skip past EOF while reading rar: fix double free with over 4 billion nodes rar: fix heap-buffer-overflow warc: prevent signed integer overflow tar: fix overflow in build_ustar_entry Notable bugfixes: bsdtar: don't hardlink negative inode files together gz: allow setting the original filename for gzip compressed files lib: improve lseek handling lib: support @@-prefixed Unix epoch timestamps as date strings rar: support large headers on 32 bit systems tar reader: Improve LFS support on 32 bit systems @ text @d2 1 a2 1 @ 1.1.1.17 log @libarchive: import version 3.8.1 @ text @d2 1 a2 1 @ 1.1.1.18 log @libarchive: imported version 3.8.2 @ text @d2 1 a2 1 @ 1.1.1.19 log @libarchive: import version 3.8.3 @ text @d2 1 a2 1 @ 1.1.1.20 log @libarchive: import 3.8.4 @ text @d2 1 a2 1 @ 1.1.1.21 log @libarchive: import version 3.8.5 @ text @d2 1 a2 1 @ 1.1.1.22 log @libarchive: imported version 3.8.6 @ text @d2 1 a2 1 @ 1.1.1.23 log @libarchive: imported version 3.8.7 @ text @d2 1 a2 1 @ 1.1.1.24 log @libarchive: import 3.8.8 @ text @d2 1 a2 1 @