[pcsx_rearmed.git] / deps / libchdr / deps / zstd-1.5.5 / contrib / seekable_format / README.md

# Zstandard Seekable Format

The seekable format splits compressed data into a series of independent "frames",
each compressed individually,
so that decompression of a section in the middle of an archive
only requires zstd to decompress at most a frame's worth of extra data,
instead of the entire archive.

The frames are appended, so that the decompression of the entire payload
still regenerates the original content, using any compliant zstd decoder.

On top of that, the seekable format generates a jump table,
which makes it possible to jump directly to the position of the relevant frame
when requesting only a segment of the data.
The jump table is simply ignored by zstd decoders unaware of the seekable format.

The format is delivered with an API to create seekable archives
and to retrieve arbitrary segments inside the archive.

### Maximum Frame Size parameter

When creating a seekable archive, the main parameter is the maximum frame size.

At compression time, user can manually select the boundaries between segments,
but they don't have to: long segments will be automatically split
when larger than selected maximum frame size.

Small frame sizes reduce decompression cost when requesting small segments,
because the decoder will nonetheless have to decompress an entire frame
to recover just a single byte from it.

A good rule of thumb is to select a maximum frame size roughly equivalent
to the access pattern when it's known.
For example, if the application tends to request 4KB blocks,
then it's a good idea to set a maximum frame size in the vicinity of 4 KB.

But small frame sizes also reduce compression ratio,
and increase the cost for the jump table,
so there is a balance to find.

In general, try to avoid really tiny frame sizes (<1 KB),
which would have a large negative impact on compression ratio.
Commit	Line	Data
648db22b	1	# Zstandard Seekable Format
	2
	3	The seekable format splits compressed data into a series of independent "frames",
	4	each compressed individually,
	5	so that decompression of a section in the middle of an archive
	6	only requires zstd to decompress at most a frame's worth of extra data,
	7	instead of the entire archive.
	8
	9	The frames are appended, so that the decompression of the entire payload
	10	still regenerates the original content, using any compliant zstd decoder.
	11
	12	On top of that, the seekable format generates a jump table,
	13	which makes it possible to jump directly to the position of the relevant frame
	14	when requesting only a segment of the data.
	15	The jump table is simply ignored by zstd decoders unaware of the seekable format.
	16
	17	The format is delivered with an API to create seekable archives
	18	and to retrieve arbitrary segments inside the archive.
	19
	20	### Maximum Frame Size parameter
	21
	22	When creating a seekable archive, the main parameter is the maximum frame size.
	23
	24	At compression time, user can manually select the boundaries between segments,
	25	but they don't have to: long segments will be automatically split
	26	when larger than selected maximum frame size.
	27
	28	Small frame sizes reduce decompression cost when requesting small segments,
	29	because the decoder will nonetheless have to decompress an entire frame
	30	to recover just a single byte from it.
	31
	32	A good rule of thumb is to select a maximum frame size roughly equivalent
	33	to the access pattern when it's known.
	34	For example, if the application tends to request 4KB blocks,
	35	then it's a good idea to set a maximum frame size in the vicinity of 4 KB.
	36
	37	But small frame sizes also reduce compression ratio,
	38	and increase the cost for the jump table,
	39	so there is a balance to find.
	40
	41	In general, try to avoid really tiny frame sizes (<1 KB),
	42	which would have a large negative impact on compression ratio.