Spitwad
spitwad is a C library for reading, writing, and manipulating WAD archives. Include spitwad.h. A struct WAD holds an ordered list of named lumps and owns a copy of every lump's bytes. A struct WAD_View reads an EWAD in place without copying. The library prints nothing, and every int-returning function returns a WAD_Status. On-disk layouts are in WAD Format.
Usage#
Build a WAD with wad_init and wad_add_lump, then write it to a file or a buffer. wad_destroy releases it.
#include <stdio.h>
#include "spitwad.h"
int main(void) {
struct WAD wad;
wad_init(&wad, PWAD);
int rc = wad_add_lump(&wad, "GREETING", (const unsigned char *)"hello", 5);
if (rc == SPITWAD_OK)
rc = wad_write_file(&wad, "hello.wad");
if (rc != SPITWAD_OK)
puts(wad_status_name(rc));
wad_destroy(&wad);
return rc != SPITWAD_OK;
}Loading sets the WAD's type from the input's signature. wad_load_file and wad_load_data do not need wad_init first.
#include <stdio.h>
#include "spitwad.h"
int main(void) {
struct WAD wad;
const int rc = wad_load_file(&wad, "hello.wad");
if (rc != SPITWAD_OK) {
puts(wad_status_name(rc));
return 1;
}
const struct Lump *l = wad_find_lump(&wad, "GREETING");
if (l != NULL)
printf("%.*s\n", (int)l->size, (const char *)l->data);
wad_destroy(&wad);
return 0;
}An EWAD can also be read in place. wad_view_open validates the buffer once, including the header hash, and every lookup afterwards is a binary search that returns pointers into the buffer. Nothing is allocated, so the buffer may be a memory-mapped file.
struct WAD_View view;
if (wad_view_open(&view, mapped, mapped_size) != SPITWAD_OK)
return 1;
struct WAD_Entry e;
if (wad_view_find(&view, "textures/wall01.png", &e) == SPITWAD_OK
&& wad_view_verify(&e) == SPITWAD_OK)
upload_texture(e.data, e.size);A load overwrites the WAD without freeing what it held, so reloading a populated WAD without wad_destroy leaks its lumps. A failed load leaves the WAD exactly as it was.
The caller owns the struct WAD. Every other function requires a WAD that went through wad_init or a successful load. After wad_destroy the WAD is empty and can be used again. Pointers from wad_find_lump and wad_get_lump, and any pointer into lumps, are invalidated by adding a lump, removing a lump, or destroying the WAD. Concurrent modification of one WAD is not supported. Two WADs share no state.
Names may repeat in memory. Lookup and removal act on the first match, so walk with wad_get_lump to see duplicates. IWAD, PWAD, and PACK store 8 bytes of a name, WAD2 and WAD3 store 15, and EWAD stores 1 to SPITWAD_NAME_MAX. Types other than EWAD truncate long names on add, and truncate the query the same way on lookup and removal, so "LONGERNAME" finds the lump stored as "LONGERNA". EWAD compares whole names, and its writer refuses repeated names with SPITWAD_ERR_DUPLICATE_NAME. All comparisons are case-sensitive.
An EWAD on disk is sorted by name hash, so lumps loaded from one come back in that order rather than the order they were added. Writing the same set of lumps always produces the same bytes, whatever their order in memory.
Input can be untrusted. Every directory and lump range is checked against the input before it is read, and EWAD header hashes, keys, and content hashes are verified on load. The hashes catch corruption, not tampering: XXH64 is not cryptographic, so anyone can write a file whose hashes match. Lumps may share bytes, but each is copied on load, so a load fails with SPITWAD_ERR_MALFORMED if the lumps total more than 16 times the input size, or more than UINT32_MAX bytes for a classic type. The lump table adds about 290 bytes per entry, bounded by the directory's size.
PACK files are parsed with the classic layout, which is not the real Quake PAK layout, so real PAK files are misread.
Atomic file writes#
wad_write_file and wad_write_file_ex replace their target atomically. Atomic writes are POSIX-only: spitwad does not build on Windows. The writer creates a uniquely named temporary file in the target's directory, writes and fsyncs it, renames it over the target, then fsyncs the directory, so a crash or full disk leaves either the old file or the new one.
On any failure the function returns SPITWAD_ERR_IO, the target keeps its previous bytes, and the temporary file is removed. The one exception is a failed fsync of the directory: the rename has already happened and cannot be undone, so the target holds the new bytes and the error means their durability is unconfirmed. A process killed mid-write leaves a .spitwad-tmp-* file behind, which later writes ignore. On macOS, fsync does not flush the drive's own write cache.
An existing target keeps its permission bits. A new file gets 0666 minus the umask. Owner, extended attributes, and hard links are not carried over, since the target becomes a new file.
A symlink is followed: the file it points at is replaced, with the temporary file in that file's directory, and the link itself is left alone. A dangling symlink returns SPITWAD_ERR_IO and creates nothing. Only regular files can be replaced, so a target that is a directory, device, or FIFO also returns SPITWAD_ERR_IO, so /dev/stdout and the like cannot be written to.
Integrity#
Three layers catch corruption, and they differ in what they read. wad_view_open checks the header hash, which covers the header, directory, and string table, and validates the structure. It reads no lump data, so opening stays cheap on a memory-mapped file. wad_view_verify hashes one lump's stored bytes. wad_load_data and wad_load_file do both, for every lump.
A view that opened successfully therefore has trustworthy metadata, but each lump is trustworthy only after wad_view_verify passes on it. Padding between the string table and the lumps, or between lumps, belongs to nothing and is covered by no hash, so damage there is harmless and undetected.
archive_hash identifies an archive: record it in a save game and compare it on load to learn whether the archive is the same one. Only format version 2 can be read. Other versions fail with SPITWAD_ERR_MALFORMED.
Mounting several archives#
spitwad does not implement mount order. To overlay archives the Doom way, keep an array of views and search it from last to first, taking the first hit. Each archive has its own seed, so hash the name per view by calling wad_view_find, or cache wad_hash per distinct seed and call wad_view_find_key. Override is by name, since the same name has different keys under different seeds.
Compression#
spitwad links no codec. To store a compressed lump, compress it yourself, add the compressed bytes, then set flags to EWAD_FLAG_LZ4 and raw_size to the decompressed size on the new Lump. Readers get the stored bytes, the flag, and raw_size, and decompress themselves. The content hash covers the stored bytes, so a lump can be verified without decompressing it.
Structs#
WAD_Type#
enum WAD_Type {
IWAD,
PWAD,
WAD2,
WAD3,
PACK,
EWAD
};The archive type, which selects the on-disk layout and the name limit. See WAD Format.
WAD_Status#
enum WAD_Status {
SPITWAD_OK = 0,
SPITWAD_ERR_MALFORMED,
SPITWAD_ERR_NO_MEMORY,
SPITWAD_ERR_NAME_TOO_LONG,
SPITWAD_ERR_INVALID_ARGUMENT,
SPITWAD_ERR_IO,
SPITWAD_ERR_NOT_FOUND,
SPITWAD_ERR_TOO_LARGE,
SPITWAD_ERR_UNSUPPORTED,
SPITWAD_ERR_DUPLICATE_NAME,
SPITWAD_ERR_HASH_COLLISION
};| Status | Meaning |
|---|---|
SPITWAD_OK | Success. Always 0 |
SPITWAD_ERR_MALFORMED | Input is not a valid WAD, including an empty file, its lumps overlap too heavily to copy, or an EWAD header hash or content hash does not match |
SPITWAD_ERR_NO_MEMORY | Allocation failed, or the lump table cannot grow |
SPITWAD_ERR_NAME_TOO_LONG | Name does not fit the WAD's type |
SPITWAD_ERR_INVALID_ARGUMENT | NULL or out-of-range argument, unknown type, empty EWAD name, unknown EWAD flag, raw_size that disagrees with size without the LZ4 flag, alignment that is not a power of two, or a lump with a size and no data |
SPITWAD_ERR_IO | File could not be opened, sized, read, or written, or a write target is not a regular file |
SPITWAD_ERR_NOT_FOUND | No lump has that name or key |
SPITWAD_ERR_TOO_LARGE | Output would go past the 32-bit offset range of a classic type |
SPITWAD_ERR_UNSUPPORTED | Valid input using a feature spitwad lacks: a compressed WAD2 or WAD3 lump, an EWAD lump with a reserved flag bit, or, for wad_load_data, a stripped EWAD or a lump larger than UINT32_MAX |
SPITWAD_ERR_DUPLICATE_NAME | Two lumps in an EWAD being written share a name |
SPITWAD_ERR_HASH_COLLISION | Two different EWAD names hash to the same key under 64 consecutive seeds. Practically unreachable |
Lump#
struct Lump {
char name[SPITWAD_NAME_MAX + 1];
unsigned char *data;
uint32_t size;
uint8_t kind;
uint32_t type;
uint16_t schema;
uint16_t flags;
uint64_t raw_size;
};name is always NUL-terminated. data is owned by the WAD and is NULL when size is 0. kind is the WAD2 or WAD3 lump type byte, such as 0x43 for a miptex. type, schema, flags, and raw_size are the EWAD entry fields: an application-chosen fourcc and schema version, EWAD_FLAG_* bits, and the decompressed size. wad_add_lump sets kind, type, schema, and flags to 0 and raw_size to size. All five are set by assigning through lumps, and none is stored by a type that lacks the field.
#define SPITWAD_NAME_MAX 255
#define EWAD_FLAG_LZ4 0x0001uWAD#
struct WAD {
enum WAD_Type type;
uint32_t num_lumps;
uint32_t capacity;
struct Lump *lumps;
};lumps holds num_lumps entries in the order they were added. capacity is the allocated entry count. Fields are public, and type can be changed to convert a WAD. The writer re-checks every lump against the type it is writing.
WAD_WriteOptions#
struct WAD_WriteOptions {
uint32_t align;
uint64_t seed;
int strip_names;
};Options for writing an EWAD, ignored for other types. align is the lump data alignment, a power of two up to 2^30; 0 means 16. seed is the first hash seed tried. strip_names writes an empty string table, so the archive can be read only through a WAD_View.
WAD_View#
struct WAD_View {
const unsigned char *data;
size_t size;
uint64_t seed;
uint32_t count;
uint32_t align;
const unsigned char *dir;
const unsigned char *names;
uint64_t names_size;
uint64_t archive_hash;
};A validated EWAD in the caller's memory. Every pointer points into the buffer passed to wad_view_open, so the view is valid only while that buffer is, and it owns nothing. names is NULL when the archive has no string table. Fields are read-only.
archive_hash is the header hash, usable as the archive's identity, for example recorded in a save game. It covers every key and content hash, so it changes when any lump's bytes change. It is read from the header and checked against the header, directory, and string table only, so a damaged lump keeps the same value until wad_view_verify catches it. It also changes with anything else in those ranges, such as the seed, alignment, or stripping names.
WAD_Entry#
struct WAD_Entry {
uint64_t key;
const unsigned char *data;
uint64_t size;
uint64_t raw_size;
uint64_t content_hash;
const char *name;
uint32_t type;
uint16_t schema;
uint16_t flags;
};One directory entry as read through a view. data points into the view's buffer and is NULL when size is 0. name points into the string table and is NULL when names were stripped. If the buffer is a file mapped at a page boundary, data has the archive's alignment; a malloc buffer only guarantees what malloc does.
Functions#
wad_status_name#
Returns the identifier of a status code, such as "SPITWAD_ERR_IO", or "SPITWAD_ERR_UNKNOWN" for any other value. Never returns NULL. The string is static.
const char *wad_status_name(int status);wad_hash#
XXH64 of len bytes under seed. EWAD keys are the hash of the name bytes, without the NUL, under the archive's seed; content hashes use seed 0. bytes may be NULL when len is 0. Exposed so that tools can precompute keys for wad_view_find_key.
uint64_t wad_hash(const void *bytes, size_t len, uint64_t seed);wad_init#
Makes wad an empty WAD of the given type. Does not allocate. Returns SPITWAD_OK, or SPITWAD_ERR_INVALID_ARGUMENT if wad is NULL. type is not validated here. An unknown type fails when written.
int wad_init(struct WAD *wad, enum WAD_Type type);wad_destroy#
Frees every lump and the lump table, and leaves the WAD empty with its type unchanged. A NULL wad returns without doing anything. Calling it twice is safe. The WAD must have been through wad_init or a successful load.
void wad_destroy(struct WAD *wad);wad_load_data / wad_load_file#
Parse a WAD from memory or from a file. wad_load_data copies what it needs, so data can be freed afterwards. On success the WAD's fields are overwritten and nothing it previously held is freed. On failure the WAD is unchanged. An EWAD is validated as wad_view_open does, every content hash is checked, and its lumps arrive in key order.
Return SPITWAD_OK, SPITWAD_ERR_MALFORMED, SPITWAD_ERR_UNSUPPORTED, SPITWAD_ERR_NO_MEMORY, or SPITWAD_ERR_INVALID_ARGUMENT for NULL arguments. wad_load_file also returns SPITWAD_ERR_IO, including for directories and unseekable streams. An empty file is SPITWAD_ERR_MALFORMED.
int wad_load_data(struct WAD *wad, const unsigned char *data, size_t size);
int wad_load_file(struct WAD *wad, const char *path);wad_write_data / wad_write_file#
Serialize a WAD. Output is deterministic: identical WADs produce identical bytes. wad_write_data allocates the buffer and stores it in out, and the caller releases it with free. On failure out and out_size are not written. The _ex forms take WAD_WriteOptions, and the plain forms are the same as passing NULL.
Return SPITWAD_OK, SPITWAD_ERR_NAME_TOO_LONG if a name does not fit the current type, SPITWAD_ERR_TOO_LARGE, SPITWAD_ERR_NO_MEMORY, SPITWAD_ERR_DUPLICATE_NAME, SPITWAD_ERR_HASH_COLLISION, or SPITWAD_ERR_INVALID_ARGUMENT for NULL arguments, an unknown type, an unterminated name, an empty EWAD name, a lump with a size and no data, an unknown flag, a raw_size that disagrees with size without the LZ4 flag, or an alignment that is not a power of two. wad_write_file also returns SPITWAD_ERR_IO. The file forms replace their target atomically; see Atomic file writes.
int wad_write_data(const struct WAD *wad, unsigned char **out, size_t *out_size);
int wad_write_file(const struct WAD *wad, const char *path);
int wad_write_data_ex(const struct WAD *wad, const struct WAD_WriteOptions *opts,
unsigned char **out, size_t *out_size);
int wad_write_file_ex(const struct WAD *wad, const struct WAD_WriteOptions *opts, const char *path);wad_add_lump#
Appends a lump holding a copy of data. data may be NULL only when size is 0. name and data may point into the WAD's existing lumps. Types other than EWAD truncate name to what they can store. EWAD returns SPITWAD_ERR_INVALID_ARGUMENT for an empty name and SPITWAD_ERR_NAME_TOO_LONG for one past SPITWAD_NAME_MAX. Also returns SPITWAD_ERR_NO_MEMORY. On failure the WAD is unchanged. The new lump's kind, type, schema, and flags are 0 and its raw_size is size.
int wad_add_lump(struct WAD *wad, const char *name, const unsigned char *data, uint32_t size);wad_remove_lump / wad_remove_lump_at#
Remove the first lump matching name, or the lump at index, and free its data. Later lumps move down one index, and capacity is kept. wad_remove_lump returns SPITWAD_ERR_NOT_FOUND if nothing matches. wad_remove_lump_at returns SPITWAD_ERR_INVALID_ARGUMENT for an out-of-range index.
int wad_remove_lump(struct WAD *wad, const char *name);
int wad_remove_lump_at(struct WAD *wad, uint32_t index);wad_find_lump / wad_get_lump#
Return the first lump matching name, or the lump at index. Return NULL if there is no such lump or an argument is NULL. The WAD keeps ownership. wad_find_lump is a linear scan; for hashed lookup, use a view.
const struct Lump *wad_find_lump(const struct WAD *wad, const char *name);
const struct Lump *wad_get_lump(const struct WAD *wad, uint32_t index);wad_num_lumps#
Returns the number of lumps. A NULL wad returns 0.
uint32_t wad_num_lumps(const struct WAD *wad);wad_view_open#
Validates an EWAD in memory and fills view with pointers into it. The directory and string table ranges are bounds-checked, then the header hash is checked over them, and then every directory entry, string table name and lump range is checked once, in one pass over the directory, including that each key is the seeded hash of its name. Lump data is not read, so a damaged lump is found only by wad_view_verify. Returns SPITWAD_OK, SPITWAD_ERR_MALFORMED for anything that is not a well-formed EWAD of format version 2 (including any other WAD type or a header hash that does not match), SPITWAD_ERR_UNSUPPORTED for a reserved flag bit, or SPITWAD_ERR_INVALID_ARGUMENT for NULL arguments. On failure view is unchanged.
int wad_view_open(struct WAD_View *view, const unsigned char *data, size_t size);wad_view_get / wad_view_find / wad_view_find_key#
Fill out with the entry at index, the entry whose name hashes to a key under the view's seed, or the entry with that key. wad_view_find works on stripped archives, since it never reads the string table. Lookups are a binary search over the directory. Return SPITWAD_OK, SPITWAD_ERR_NOT_FOUND when no key matches, or SPITWAD_ERR_INVALID_ARGUMENT for a NULL argument or an out-of-range index.
int wad_view_get(const struct WAD_View *view, uint32_t index, struct WAD_Entry *out);
int wad_view_find(const struct WAD_View *view, const char *name, struct WAD_Entry *out);
int wad_view_find_key(const struct WAD_View *view, uint64_t key, struct WAD_Entry *out);wad_view_verify#
Hashes the entry's stored bytes and compares the result with content_hash. Returns SPITWAD_OK, SPITWAD_ERR_MALFORMED on a mismatch, or SPITWAD_ERR_INVALID_ARGUMENT for NULL. This is the only view call that reads lump data.
int wad_view_verify(const struct WAD_Entry *entry);