git subrepo pull --force deps/lightrec
[pcsx_rearmed.git] / deps / libchdr / deps / lzma-19.00 / lzma.txt
CommitLineData
b24e7fce 1LZMA compression
2----------------
3Version: 9.35
4
5This file describes LZMA encoding and decoding functions written in C language.
6
7LZMA is an improved version of famous LZ77 compression algorithm.
8It was improved in way of maximum increasing of compression ratio,
9keeping high decompression speed and low memory requirements for
10decompressing.
11
12Note: you can read also LZMA Specification (lzma-specification.txt from LZMA SDK)
13
14Also you can look source code for LZMA encoding and decoding:
15 C/Util/Lzma/LzmaUtil.c
16
17
18LZMA compressed file format
19---------------------------
20Offset Size Description
21 0 1 Special LZMA properties (lc,lp, pb in encoded form)
22 1 4 Dictionary size (little endian)
23 5 8 Uncompressed size (little endian). -1 means unknown size
24 13 Compressed data
25
26
27
28ANSI-C LZMA Decoder
29~~~~~~~~~~~~~~~~~~~
30
31Please note that interfaces for ANSI-C code were changed in LZMA SDK 4.58.
32If you want to use old interfaces you can download previous version of LZMA SDK
33from sourceforge.net site.
34
35To use ANSI-C LZMA Decoder you need the following files:
361) LzmaDec.h + LzmaDec.c + 7zTypes.h + Precomp.h + Compiler.h
37
38Look example code:
39 C/Util/Lzma/LzmaUtil.c
40
41
42Memory requirements for LZMA decoding
43-------------------------------------
44
45Stack usage of LZMA decoding function for local variables is not
46larger than 200-400 bytes.
47
48LZMA Decoder uses dictionary buffer and internal state structure.
49Internal state structure consumes
50 state_size = (4 + (1.5 << (lc + lp))) KB
51by default (lc=3, lp=0), state_size = 16 KB.
52
53
54How To decompress data
55----------------------
56
57LZMA Decoder (ANSI-C version) now supports 2 interfaces:
581) Single-call Decompressing
592) Multi-call State Decompressing (zlib-like interface)
60
61You must use external allocator:
62Example:
63void *SzAlloc(void *p, size_t size) { p = p; return malloc(size); }
64void SzFree(void *p, void *address) { p = p; free(address); }
65ISzAlloc alloc = { SzAlloc, SzFree };
66
67You can use p = p; operator to disable compiler warnings.
68
69
70Single-call Decompressing
71-------------------------
72When to use: RAM->RAM decompressing
73Compile files: LzmaDec.h + LzmaDec.c + 7zTypes.h
74Compile defines: no defines
75Memory Requirements:
76 - Input buffer: compressed size
77 - Output buffer: uncompressed size
78 - LZMA Internal Structures: state_size (16 KB for default settings)
79
80Interface:
81 int LzmaDecode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen,
82 const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode,
83 ELzmaStatus *status, ISzAlloc *alloc);
84 In:
85 dest - output data
86 destLen - output data size
87 src - input data
88 srcLen - input data size
89 propData - LZMA properties (5 bytes)
90 propSize - size of propData buffer (5 bytes)
91 finishMode - It has meaning only if the decoding reaches output limit (*destLen).
92 LZMA_FINISH_ANY - Decode just destLen bytes.
93 LZMA_FINISH_END - Stream must be finished after (*destLen).
94 You can use LZMA_FINISH_END, when you know that
95 current output buffer covers last bytes of stream.
96 alloc - Memory allocator.
97
98 Out:
99 destLen - processed output size
100 srcLen - processed input size
101
102 Output:
103 SZ_OK
104 status:
105 LZMA_STATUS_FINISHED_WITH_MARK
106 LZMA_STATUS_NOT_FINISHED
107 LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK
108 SZ_ERROR_DATA - Data error
109 SZ_ERROR_MEM - Memory allocation error
110 SZ_ERROR_UNSUPPORTED - Unsupported properties
111 SZ_ERROR_INPUT_EOF - It needs more bytes in input buffer (src).
112
113 If LZMA decoder sees end_marker before reaching output limit, it returns OK result,
114 and output value of destLen will be less than output buffer size limit.
115
116 You can use multiple checks to test data integrity after full decompression:
117 1) Check Result and "status" variable.
118 2) Check that output(destLen) = uncompressedSize, if you know real uncompressedSize.
119 3) Check that output(srcLen) = compressedSize, if you know real compressedSize.
120 You must use correct finish mode in that case. */
121
122
123Multi-call State Decompressing (zlib-like interface)
124----------------------------------------------------
125
126When to use: file->file decompressing
127Compile files: LzmaDec.h + LzmaDec.c + 7zTypes.h
128
129Memory Requirements:
130 - Buffer for input stream: any size (for example, 16 KB)
131 - Buffer for output stream: any size (for example, 16 KB)
132 - LZMA Internal Structures: state_size (16 KB for default settings)
133 - LZMA dictionary (dictionary size is encoded in LZMA properties header)
134
1351) read LZMA properties (5 bytes) and uncompressed size (8 bytes, little-endian) to header:
136 unsigned char header[LZMA_PROPS_SIZE + 8];
137 ReadFile(inFile, header, sizeof(header)
138
1392) Allocate CLzmaDec structures (state + dictionary) using LZMA properties
140
141 CLzmaDec state;
142 LzmaDec_Constr(&state);
143 res = LzmaDec_Allocate(&state, header, LZMA_PROPS_SIZE, &g_Alloc);
144 if (res != SZ_OK)
145 return res;
146
1473) Init LzmaDec structure before any new LZMA stream. And call LzmaDec_DecodeToBuf in loop
148
149 LzmaDec_Init(&state);
150 for (;;)
151 {
152 ...
153 int res = LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, SizeT *destLen,
154 const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode);
155 ...
156 }
157
158
1594) Free all allocated structures
160 LzmaDec_Free(&state, &g_Alloc);
161
162Look example code:
163 C/Util/Lzma/LzmaUtil.c
164
165
166How To compress data
167--------------------
168
169Compile files:
170 7zTypes.h
171 Threads.h
172 LzmaEnc.h
173 LzmaEnc.c
174 LzFind.h
175 LzFind.c
176 LzFindMt.h
177 LzFindMt.c
178 LzHash.h
179
180Memory Requirements:
181 - (dictSize * 11.5 + 6 MB) + state_size
182
183Lzma Encoder can use two memory allocators:
1841) alloc - for small arrays.
1852) allocBig - for big arrays.
186
187For example, you can use Large RAM Pages (2 MB) in allocBig allocator for
188better compression speed. Note that Windows has bad implementation for
189Large RAM Pages.
190It's OK to use same allocator for alloc and allocBig.
191
192
193Single-call Compression with callbacks
194--------------------------------------
195
196Look example code:
197 C/Util/Lzma/LzmaUtil.c
198
199When to use: file->file compressing
200
2011) you must implement callback structures for interfaces:
202ISeqInStream
203ISeqOutStream
204ICompressProgress
205ISzAlloc
206
207static void *SzAlloc(void *p, size_t size) { p = p; return MyAlloc(size); }
208static void SzFree(void *p, void *address) { p = p; MyFree(address); }
209static ISzAlloc g_Alloc = { SzAlloc, SzFree };
210
211 CFileSeqInStream inStream;
212 CFileSeqOutStream outStream;
213
214 inStream.funcTable.Read = MyRead;
215 inStream.file = inFile;
216 outStream.funcTable.Write = MyWrite;
217 outStream.file = outFile;
218
219
2202) Create CLzmaEncHandle object;
221
222 CLzmaEncHandle enc;
223
224 enc = LzmaEnc_Create(&g_Alloc);
225 if (enc == 0)
226 return SZ_ERROR_MEM;
227
228
2293) initialize CLzmaEncProps properties;
230
231 LzmaEncProps_Init(&props);
232
233 Then you can change some properties in that structure.
234
2354) Send LZMA properties to LZMA Encoder
236
237 res = LzmaEnc_SetProps(enc, &props);
238
2395) Write encoded properties to header
240
241 Byte header[LZMA_PROPS_SIZE + 8];
242 size_t headerSize = LZMA_PROPS_SIZE;
243 UInt64 fileSize;
244 int i;
245
246 res = LzmaEnc_WriteProperties(enc, header, &headerSize);
247 fileSize = MyGetFileLength(inFile);
248 for (i = 0; i < 8; i++)
249 header[headerSize++] = (Byte)(fileSize >> (8 * i));
250 MyWriteFileAndCheck(outFile, header, headerSize)
251
2526) Call encoding function:
253 res = LzmaEnc_Encode(enc, &outStream.funcTable, &inStream.funcTable,
254 NULL, &g_Alloc, &g_Alloc);
255
2567) Destroy LZMA Encoder Object
257 LzmaEnc_Destroy(enc, &g_Alloc, &g_Alloc);
258
259
260If callback function return some error code, LzmaEnc_Encode also returns that code
261or it can return the code like SZ_ERROR_READ, SZ_ERROR_WRITE or SZ_ERROR_PROGRESS.
262
263
264Single-call RAM->RAM Compression
265--------------------------------
266
267Single-call RAM->RAM Compression is similar to Compression with callbacks,
268but you provide pointers to buffers instead of pointers to stream callbacks:
269
270SRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen,
271 const CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int writeEndMark,
272 ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig);
273
274Return code:
275 SZ_OK - OK
276 SZ_ERROR_MEM - Memory allocation error
277 SZ_ERROR_PARAM - Incorrect paramater
278 SZ_ERROR_OUTPUT_EOF - output buffer overflow
279 SZ_ERROR_THREAD - errors in multithreading functions (only for Mt version)
280
281
282
283Defines
284-------
285
286_LZMA_SIZE_OPT - Enable some optimizations in LZMA Decoder to get smaller executable code.
287
288_LZMA_PROB32 - It can increase the speed on some 32-bit CPUs, but memory usage for
289 some structures will be doubled in that case.
290
291_LZMA_UINT32_IS_ULONG - Define it if int is 16-bit on your compiler and long is 32-bit.
292
293_LZMA_NO_SYSTEM_SIZE_T - Define it if you don't want to use size_t type.
294
295
296_7ZIP_PPMD_SUPPPORT - Define it if you don't want to support PPMD method in AMSI-C .7z decoder.
297
298
299C++ LZMA Encoder/Decoder
300~~~~~~~~~~~~~~~~~~~~~~~~
301C++ LZMA code use COM-like interfaces. So if you want to use it,
302you can study basics of COM/OLE.
303C++ LZMA code is just wrapper over ANSI-C code.
304
305
306C++ Notes
307~~~~~~~~~~~~~~~~~~~~~~~~
308If you use some C++ code folders in 7-Zip (for example, C++ code for .7z handling),
309you must check that you correctly work with "new" operator.
3107-Zip can be compiled with MSVC 6.0 that doesn't throw "exception" from "new" operator.
311So 7-Zip uses "CPP\Common\NewHandler.cpp" that redefines "new" operator:
312operator new(size_t size)
313{
314 void *p = ::malloc(size);
315 if (p == 0)
316 throw CNewException();
317 return p;
318}
319If you use MSCV that throws exception for "new" operator, you can compile without
320"NewHandler.cpp". So standard exception will be used. Actually some code of
3217-Zip catches any exception in internal code and converts it to HRESULT code.
322So you don't need to catch CNewException, if you call COM interfaces of 7-Zip.
323
324---
325
326http://www.7-zip.org
327http://www.7-zip.org/sdk.html
328http://www.7-zip.org/support.html