LibDeflate

• Index

Contents

• Functions
• Tables

Source

• LibDeflate.lua

Topics

• readme

Examples

• example.lua

File LibDeflate.lua

LibDeflate v0.9-alpha1 Pure Lua DEFLATE/zlib compressors and decompressors.

Info:

• Copyright: LibDeflate <2018> Haoqian He
• License: GNU General Public License Version 3 or later

  This library is implemented according to the following specifications.
  Report a bug if LibDeflate is not fully compliant with those specs.
  Both compressors and decompressors have been implemented in the library.
  1. RFC1950: DEFLATE Compressed Data Format Specification version 1.3
  https://tools.ietf.org/html/rfc1951
  2. RFC1951: ZLIB Compressed Data Format Specification version 3.3
  https://tools.ietf.org/html/rfc1950
  

  This library requires Lua 5.1/5.2/5.3 interpreter or LuaJIT v2.0+.
  This library does not have any external library dependencies.
  (Exception: will register in the World of Warcraft library "LibStub", if detected).
  This file "LibDeflate.lua" is the only source file of the library.
  Submit suggestions or report bugs to https://github.com/safeteeWow/LibDeflate/issues

• Author: Haoqian He (Github: SafeteeWoW; World of Warcraft: Safetyy-Illidan(US))

Functions

LibDeflate:Adler32 (str)	Calculate the Adler-32 checksum of the string.
LibDeflate:CompressDeflate (str, configs)	Compress using the raw deflate format.
LibDeflate:CompressDeflateWithDict (str, dictionary, configs)	Compress using the raw deflate format with a preset dictionary.
LibDeflate:CompressZlib (str, configs)	Compress using the zlib format.
LibDeflate:CompressZlibWithDict (str, dictionary, configs)	Compress using the zlib format with a preset dictionary
LibDeflate:CreateCodec (reserved_chars, escape_chars, map_chars)	Create a custom codec with encoder and decoder.
LibDeflate:CreateDictionary (str)	Create a preset dictionary.
LibDeflate:Decode6Bit (str)	Decode the string produced by LibDeflate:Encode6Bit
LibDeflate:DecodeForWoWAddonChannel (str)	Decode the string produced by LibDeflate:EncodeForWoWAddonChannel
LibDeflate:DecodeForWoWChatChannel (str)	Decode the string produced by LibDeflate:EncodeForWoWChatChannel.
LibDeflate:DecompressDeflate (str)	Decompress a raw deflate compressed data.
LibDeflate:DecompressDeflateWithDict (str, dictionary)	Decompress a raw deflate compressed data with a preset dictionary.
LibDeflate:DecompressZlib (str)	Decompress a zlib compressed data.
LibDeflate:DecompressZlibWithDict (str, dictionary)	Decompress a zlib compressed data with a preset dictionary.
LibDeflate:Encode6Bit (str)	Encode the string so it does not only contains characters other than 6bit = 64 printable ASCII characters.
LibDeflate:EncodeForWoWAddonChannel (str)	Encode the string to make it ready to be transmitted in World of Warcraft addon channel.
LibDeflate:EncodeForWoWChatChannel (str)	Encode the string to make it ready to be transmitted in World of Warcraft chat channel.
LibDeflate:GetDictForWoW ()	(WILL BE CHANGE IN v1.0) Get the dictionary designed for world of warcraft shipped with LibDeflate.
LibDeflate:VerifyDictionary (str, dictionary, adler32, strlen)	Verify newly created dictionary so it is ready to use.

Tables

CommandlineOptions	Commandline options
compression_configs	The description to compression configuration table.

Functions

LibDeflate:Adler32 (str)

Calculate the Adler-32 checksum of the string.
See RFC1950 Page 9 https://tools.ietf.org/html/rfc1950 for the definition of Adler-32 checksum.

Parameters:

• str [string] the input string to calcuate its Adler-32 checksum.

Returns:

[integer] The Adler-32 checksum.

LibDeflate:CompressDeflate (str, configs)

Compress using the raw deflate format.

Parameters:

• str [string] The data to be compressed.
• configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.

Returns:

1. [string] The compressed data
2. [integer] The number of bits padded at the end of output. 0 <= bits < 8
   You don't need to use this value unless you want to do some postprocessing to the compressed data.

See also:

• compression_configs
• LibDeflate:DecompressDeflate

LibDeflate:CompressDeflateWithDict (str, dictionary, configs)

Compress using the raw deflate format with a preset dictionary.

Parameters:

• str [string] The data to be compressed.
• dictionary

  [table] The preset dictionary produced by

                  LibDeflate:CreateDictionary
  

• configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.

Returns:

1. [string] The compressed data
2. [integer] The number of bits padded at the end of output. 0 <= bits < 8
   You don't need to use this value unless you want to do some postprocessing to the compressed data.

See also:

• compression_configs
• LibDeflate:CreateDictionary
• LibDeflate:VerifyDictionary
• LibDeflate:DecompressDeflateWithDict

LibDeflate:CompressZlib (str, configs)

Compress using the zlib format.

Parameters:

• str [string] the data to be compressed.
• configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.

Returns:

1. [string] The compressed data
2. [integer] The number of bits padded at the end of output. Should always be 0.

See also:

• compression_configs
• LibDeflate:DecompressZlib

LibDeflate:CompressZlibWithDict (str, dictionary, configs)

Compress using the zlib format with a preset dictionary

Parameters:

• str [string] the data to be compressed.
• dictionary

  [table] A preset dictionary produced

          by LibDeflate:CreateDictionary()
  

• configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.

Returns:

1. [string] The compressed data
2. [integer] The number of bits padded at the end of output. Should always be 0.

See also:

• compression_configs
• LibDeflate:CreateDictionary
• LibDeflate:VerifyDictionary
• LibDeflate:DecompressZlibWithDict

LibDeflate:CreateCodec (reserved_chars, escape_chars, map_chars)

Create a custom codec with encoder and decoder.
Credits to LibCompress.

Parameters:

• reserved_chars [string] The created encoder will ensure no encoded data will contain any characters in reserved_chars.
• escape_chars [string] The escape character(s) used to escape reserved_chars. The length of param is usally 1 character, but you need to provide one more character if there are more than 127 reseverd chars.
• map_chars [string] The created encoder will encode the input string with mapping every reservedchars[i] (1 <= i <= #mapchars) to map_chars[i]

Returns:

[table] The encode/decode table. The table contains two functions t:Encode(str) returns the encoded string.
t:Decode(str) returns the decoded string if succeeds. nil if fails.

Raises:

error if encode/decoder cannot be produced.

Usage:

-- Create an encoder/decoder that maps all "\000" to "\003",
-- and escape "\001" (and "\002" and "\003") properly
local codec = LibDeflate:CreateCodec("\000\001",
       "\002", "\003")

local encoded = codec:Encode(SOME_STRING)
local decoded = codec:Decode(encoded)
-- decoded == SOME_STRING

LibDeflate:CreateDictionary (str)

Create a preset dictionary.
You must call LibDeflate:VerifyDictionary(str, dictionary, adler32, strlen) after this function. Otherwise, the dictionary is not ready to use.

This function is not fast, and the memory consumption of the produced dictionary is about 50 times of the input string. Therefore, it is suggestted to run this function only once in your program.

Parameters:

• str [string] The string used as the preset dictionary.
  You should put stuffs that frequently appears in the dictionary string and preferablely put more frequently appeared stuffs toward the end of the string.
  Empty string and string longer than 32768 bytes are not allowed.

Returns:

[table] The dictionary used for preset dictionary compression and decompression.

See also:

LibDeflate:VerifyDictionary

Usage:

local dict_str = "1234567890"
local dict = LibDeflate:CreateDictionary(dict_str)

-- print(LibDeflate:Adler32(dict_str), dict_str:len())

-- Hardcode the result below to verify it to avoid acciently modification
-- during the program development.

-- Adler-32: 187433486, string length: 10

LibDeflate:VerifyDictionary(dict_str, dict, 187433486, 10)

LibDeflate:Decode6Bit (str)

Decode the string produced by LibDeflate:Encode6Bit

Parameters:

• str [string] The string to be decoded

Returns:

[string/nil] The decoded string if succeeds. nil if fails.

LibDeflate:DecodeForWoWAddonChannel (str)

Decode the string produced by LibDeflate:EncodeForWoWAddonChannel

Parameters:

• str [string] The string to be decoded.

Returns:

[string/nil] The decoded string if succeeds. nil if fails.

See also:

LibDeflate:EncodeForWoWAddonChannel

LibDeflate:DecodeForWoWChatChannel (str)

Decode the string produced by LibDeflate:EncodeForWoWChatChannel.

Parameters:

• str [string] The string to be decoded.

Returns:

[string/nil] The decoded string if succeeds. nil if fails.

See also:

LibDeflate:EncodeForWoWChatChannel

LibDeflate:DecompressDeflate (str)

Decompress a raw deflate compressed data.

Parameters:

• str [string] The data to be decompressed

Returns:

1. [string/nil] The decompressed data if succeeds. nil if fails.
2. [integer] The number of unprocessed bytes if succeeds. Positive integer if any extra bytes are appended after the compressed data. If you consider this as an error, check if this is zero.
   UNDEFINED value if fails.

See also:

LibDeflate:CompressDeflate

LibDeflate:DecompressDeflateWithDict (str, dictionary)

Decompress a raw deflate compressed data with a preset dictionary.

Parameters:

• str [string] The data to be decompressed
• dictionary [table] The preset dictionary used by LibDeflate:CompressDeflateWithDict when the compressed data is produced. Decompression and compression must use the same dictionary. Otherwise wrong decompressed data could be produced without generating any error.

Returns:

1. [string/nil] The decompressed data if succeeds. nil if fails.
2. [integer] The number of unprocessed bytes if succeeds. Positive integer if any extra bytes are appended after the compressed data. If you consider this as an error, check if this is zero.
   UNDEFINED value if fails.

See also:

LibDeflate:CompressDeflateWithDict

LibDeflate:DecompressZlib (str)

Decompress a zlib compressed data.

Parameters:

• str [string] The data to be decompressed

Returns:

1. [string/nil] The decompressed data if succeeds. nil if fails.
2. [integer] The number of unprocessed bytes if succeeds. Positive integer if any extra bytes are appended after the compressed data. If you consider this as an error, check if this is zero.
   UNDEFINED value if fails.

See also:

LibDeflate:CompressZlib

LibDeflate:DecompressZlibWithDict (str, dictionary)

Decompress a zlib compressed data with a preset dictionary.

Parameters:

• str [string] The data to be decompressed
• dictionary [table] The preset dictionary used by LibDeflate:CompressDeflateWithDict when the compressed data is produced. Decompression and compression must use the same dictionary. Otherwise wrong decompressed data could be produced without generating any error.

Returns:

1. [string/nil] The decompressed data if succeeds. nil if fails.
2. [integer] The number of unprocessed bytes if succeeds. Positive integer if any extra bytes are appended after the compressed data. If you consider this as an error, check if this is zero.
   UNDEFINED value if fails.

See also:

LibDeflate:CompressZlibWithDict

LibDeflate:Encode6Bit (str)

Encode the string so it does not only contains characters other than 6bit = 64 printable ASCII characters.
Credis to WeakAuras2, this function is equivalant to the implementation it is using right now.
The encoded string will be 25% larger than the origin string. However, every single byte of the encoded string will be one of 64 printable ASCII characters, which are can be easier copied, pasted and displayed. (26 lowercase letters, 26 uppercase letters, 10 numbers digits, left parenthese, or right parenthese)

Parameters:

• str [string] The string to be encoded.

Returns:

[string] The encoded string.

LibDeflate:EncodeForWoWAddonChannel (str)

Encode the string to make it ready to be transmitted in World of Warcraft addon channel.
The encoded string is guaranteed to contain no NULL ("\000") character.

Parameters:

• str [string] The string to be encoded.

Returns:

The encoded string.

See also:

LibDeflate:DecodeForWoWAddonChannel

LibDeflate:EncodeForWoWChatChannel (str)

Encode the string to make it ready to be transmitted in World of Warcraft chat channel.
See also https://wow.gamepedia.com/ValidChatMessageCharacters

Parameters:

• str [string] The string to be encoded.

Returns:

[string] The encoded string.

See also:

LibDeflate:DecodeForWoWChatChannel

LibDeflate:GetDictForWoW ()

(WILL BE CHANGE IN v1.0) Get the dictionary designed for world of warcraft shipped with LibDeflate.
LibDeflate caches the result of the function, so the cost to repeatedly run this function is tiny.
WARNING: This function is scheduled to be changed without backward compatibilty in v1.0, which will be released before BFA. Just do not use this function in production right now.

Returns:

[table] The dictionary for WoW shipped with LibDeflate which can be directly used as the dictionary for compression and decompression.
There is NO need to call LibDeflate:VerifyDictionary for the dictionary returned by this function.

LibDeflate:VerifyDictionary (str, dictionary, adler32, strlen)

Verify newly created dictionary so it is ready to use.
This is to make sure dictionary string is not accidentally corrupted in the programming development.
Dictionary created by LibDeflate:CreateDictionary won't be usable by compression or decompression until it is verified by this function.

Parameters:

• str [string] the dictionary string.
• dictionary [table] The dictionary produced by LibDeflate:CreateDictionary(str)
• adler32 [integer] The adler32 value of str. You should pass in this argument as a hardcoded constant.
• strlen [integer] The length of str. You should pass in this argument as a hardcoded constant.

See also:

LibDeflate:CreateDictionary

Tables

CommandlineOptions

Commandline options

Usage:

lua LibDeflate.lua [OPTION] [INPUT] [OUTPUT]
-0    store only. no compression.
-1    fastest compression.
-9    slowest and best compression.
-d    do decompression instead of compression.
--dict <filename> specify the file that contains
he entire preset dictionary.
-h    give this help.
--strategy <fixed/huffman_only/dynamic> specify a special compression strategy.
-v    print the version and copyright info.
--wowdict Use the preset dictionary designed for WoW shipped with LibDeflate.
--zlib  use zlib format instead of raw deflate.

compression_configs

The description to compression configuration table.
Any field can be nil to use its default.
Table with keys other than those below is an invalid table.

Fields:

• level The compression level ranged from 0 to 9. 0 is no compression. 9 is the slowest but best compression. Use nil for default level.
• strategy The compression strategy. "fixed" to only use fixed deflate compression block. "dynamic" to only use dynamic block. "huffman_only" to do no LZ77 compression. Only do huffman compression.