Skip to content

Commit

Permalink
add zstdencode and zstddecode via new version of zmat
Browse files Browse the repository at this point in the history
  • Loading branch information
fangq committed Jun 10, 2023
1 parent 837c8b5 commit a3c807f
Show file tree
Hide file tree
Showing 20 changed files with 117 additions and 31 deletions.
2 changes: 1 addition & 1 deletion ChangeLog.txt
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ JSONlab ChangeLog (key features marked by *):
2019-06-01 [f421b14] add a modified msgpack decoder based on Bastian Bechtold's transplant project
2019-05-31 [428b47f] fix savemsgpack input test
2019-05-31 [0ff5d47] support table in ubjson,add demo for handles,containers.Map and datetime and tables
2019-05-31*[0c467ee] support lzma and lzip compression decompression via zmat toolbox (http://github.com/fangq/zmat)
2019-05-31*[0c467ee] support lzma and lzip compression decompression via zmat toolbox (http://github.com/NeuroJSON/zmat)
2019-05-31 [599ee4c] support categorical data
2019-05-31 [15c4b8d] completely remove global var using nested functions
2019-05-30*[d47be45] fast bracket matching
Expand Down
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ MATLAB will execute this file every time it starts. For Octave, the file
you need to edit is ``~/.octaverc``, where ``~`` is your home directory.

To use the data compression features, please download the ZMat toolbox from
https://github.com/fangq/zmat/releases/latest and follow the instruction to
https://github.com/NeuroJSON/zmat/releases/latest and follow the instruction to
install ZMat first. The ZMat toolbox is required when compression is used on
MATLAB running in the ``-nojvm`` mode or GNU Octave, or 'lzma/lzip/lz4/lz4hc'
compression methods are specified. ZMat can also compress large arrays that
Expand Down
4 changes: 2 additions & 2 deletions base64decode.m
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
%
% Decoding a Base64-encoded byte-stream to recover the original data
% This function depends on JVM in MATLAB or, can optionally use the ZMat
% toolbox (http://github.com/fangq/zmat)
% toolbox (http://github.com/NeuroJSON/zmat)
%
% Copyright (c) 2012, Kota Yamaguchi
% URL: https://www.mathworks.com/matlabcentral/fileexchange/39526-byte-encoding-utilities
Expand Down Expand Up @@ -34,7 +34,7 @@
output=zmat(varargin{1},0,'base64');
return;
elseif(isoctavemesh)
error('You must install the ZMat toolbox (http://github.com/fangq/zmat) to use this function in Octave');
error('You must install the ZMat toolbox (http://github.com/NeuroJSON/zmat) to use this function in Octave');
end

error(javachk('jvm'));
Expand Down
2 changes: 1 addition & 1 deletion base64encode.m
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
% Encoding a binary vector or array using Base64
%
% This function depends on JVM in MATLAB or, can optionally use the ZMat
% toolbox (http://github.com/fangq/zmat)
% toolbox (http://github.com/NeuroJSON/zmat)
%
% Copyright (c) 2012, Kota Yamaguchi
% URL: https://www.mathworks.com/matlabcentral/fileexchange/39526-byte-encoding-utilities
Expand Down
2 changes: 1 addition & 1 deletion blosc2decode.m
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
% output = blosc2decode(input,info)
%
% Decompressing an blosc2-compressed byte-stream to recover the original data
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down
2 changes: 1 addition & 1 deletion blosc2encode.m
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
%
% Compress a string or a numerical array using LZ4-compression
%
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down
4 changes: 2 additions & 2 deletions gzipdecode.m
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
%
% Decompressing a GZIP-compressed byte-stream to recover the original data
% This function depends on JVM in MATLAB or, can optionally use the ZMat
% toolbox (http://github.com/fangq/zmat)
% toolbox (http://github.com/NeuroJSON/zmat)
%
% Copyright (c) 2012, Kota Yamaguchi
% URL: https://www.mathworks.com/matlabcentral/fileexchange/39526-byte-encoding-utilities
Expand Down Expand Up @@ -44,7 +44,7 @@
end
return;
elseif(isoctavemesh)
error('You must install the ZMat toolbox (http://github.com/fangq/zmat) to use this function in Octave');
error('You must install the ZMat toolbox (http://github.com/NeuroJSON/zmat) to use this function in Octave');
end
error(javachk('jvm'));

Expand Down
4 changes: 2 additions & 2 deletions gzipencode.m
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
% Compress a string or numerical array using the GZIP-compression
%
% This function depends on JVM in MATLAB or, can optionally use the ZMat
% toolbox (http://github.com/fangq/zmat)
% toolbox (http://github.com/NeuroJSON/zmat)
%
% Copyright (c) 2012, Kota Yamaguchi
% URL: https://www.mathworks.com/matlabcentral/fileexchange/39526-byte-encoding-utilities
Expand Down Expand Up @@ -40,7 +40,7 @@
[varargout{1:nargout}]=zmat(varargin{1},1,'gzip');
return;
elseif(isoctavemesh)
error('You must install the ZMat toolbox (http://github.com/fangq/zmat) to use this function in Octave');
error('You must install the ZMat toolbox (http://github.com/NeuroJSON/zmat) to use this function in Octave');
end

error(javachk('jvm'));
Expand Down
4 changes: 2 additions & 2 deletions lz4decode.m
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
% output = lz4decode(input,info)
%
% Decompressing an LZ4-compressed byte-stream to recover the original data
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down Expand Up @@ -39,5 +39,5 @@
[varargout{1:nargout}]=zmat(varargin{1},0,'lz4',varargin{2:end});
end
else
error('you must install ZMat toolbox to use this feature: http://github.com/fangq/zmat')
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
4 changes: 2 additions & 2 deletions lz4encode.m
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
%
% Compress a string or a numerical array using LZ4-compression
%
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down Expand Up @@ -35,5 +35,5 @@
[varargout{1:nargout}]=zmat(varargin{1}, 1,'lz4',varargin{2:end});
return;
else
error('you must install ZMat toolbox to use this feature: http://github.com/fangq/zmat')
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
4 changes: 2 additions & 2 deletions lz4hcdecode.m
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
% output = lz4hcdecode(input,info)
%
% Decompressing an LZ4HC-compressed byte-stream to recover the original data
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down Expand Up @@ -39,5 +39,5 @@
[varargout{1:nargout}]=zmat(varargin{1},0,'lz4hc',varargin{2:end});
end
else
error('you must install ZMat toolbox to use this feature: http://github.com/fangq/zmat')
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
4 changes: 2 additions & 2 deletions lz4hcencode.m
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
%
% Compress a string or a numerical array using LZ4HC-compression
%
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down Expand Up @@ -35,5 +35,5 @@
[varargout{1:nargout}]=zmat(varargin{1}, 1,'lz4hc',varargin{2:end});
return;
else
error('you must install ZMat toolbox to use this feature: http://github.com/fangq/zmat')
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
4 changes: 2 additions & 2 deletions lzipdecode.m
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
% output = lzipdecode(input,info)
%
% Decompressing an Lzip-compressed byte-stream to recover the original data
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down Expand Up @@ -39,5 +39,5 @@
[varargout{1:nargout}]=zmat(varargin{1},0,'lzip',varargin{2:end});
end
else
error('you must install ZMat toolbox to use this feature: http://github.com/fangq/zmat')
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
4 changes: 2 additions & 2 deletions lzipencode.m
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
%
% Compress a string or a numerical array using LZip-compression
%
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down Expand Up @@ -35,5 +35,5 @@
[varargout{1:nargout}]=zmat(varargin{1}, 1,'lzip',varargin{2:end});
return;
else
error('you must install ZMat toolbox to use this feature: http://github.com/fangq/zmat')
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
4 changes: 2 additions & 2 deletions lzmadecode.m
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
% output = lzmadecode(input,info)
%
% Decompressing an LZMA-compressed byte-stream to recover the original data
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down Expand Up @@ -39,5 +39,5 @@
[varargout{1:nargout}]=zmat(varargin{1},0,'lzma',varargin{2:end});
end
else
error('you must install ZMat toolbox to use this feature: http://github.com/fangq/zmat')
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
4 changes: 2 additions & 2 deletions lzmaencode.m
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
%
% Compress a string or a numerical array using LZMA-compression
%
% This function depends on the ZMat toolbox (http://github.com/fangq/zmat)
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
Expand Down Expand Up @@ -35,5 +35,5 @@
[varargout{1:nargout}]=zmat(varargin{1}, 1,'lzma',varargin{2:end});
return;
else
error('you must install ZMat toolbox to use this feature: http://github.com/fangq/zmat')
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
4 changes: 2 additions & 2 deletions zlibdecode.m
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
%
% Decompressing a ZLIB-compressed byte-stream to recover the original data
% This function depends on JVM in MATLAB or, can optionally use the ZMat
% toolbox (http://github.com/fangq/zmat)
% toolbox (http://github.com/NeuroJSON/zmat)
%
% Copyright (c) 2012, Kota Yamaguchi
% URL: https://www.mathworks.com/matlabcentral/fileexchange/39526-byte-encoding-utilities
Expand Down Expand Up @@ -45,7 +45,7 @@
end
return;
elseif(isoctavemesh)
error('You must install the ZMat toolbox (http://github.com/fangq/zmat) to use this function in Octave');
error('You must install the ZMat toolbox (http://github.com/NeuroJSON/zmat) to use this function in Octave');
end
error(javachk('jvm'));

Expand Down
4 changes: 2 additions & 2 deletions zlibencode.m
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
% Compress a string or numerical array using the ZLIB-compression
%
% This function depends on JVM in MATLAB or, can optionally use the ZMat
% toolbox (http://github.com/fangq/zmat)
% toolbox (http://github.com/NeuroJSON/zmat)
%
% Copyright (c) 2012, Kota Yamaguchi
% URL: https://www.mathworks.com/matlabcentral/fileexchange/39526-byte-encoding-utilities
Expand Down Expand Up @@ -39,7 +39,7 @@
[varargout{1:nargout}]=zmat(varargin{1},1,'zlib');
return;
elseif(isoctavemesh)
error('You must install the ZMat toolbox (http://github.com/fangq/zmat) to use this function in Octave');
error('You must install the ZMat toolbox (http://github.com/NeuroJSON/zmat) to use this function in Octave');
end

error(javachk('jvm'));
Expand Down
43 changes: 43 additions & 0 deletions zstddecode.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
function varargout = zstddecode(varargin)
%
% output = zstddecode(input)
% or
% output = zstddecode(input,info)
%
% Decompressing an Zstd-compressed byte-stream to recover the original data
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
% input:
% input: a string, int8/uint8 vector or numerical array to store Zstd-compressed data
% info (optional): a struct produced by the zmat/zstdencode function during
% compression; if not given, the inputs/outputs will be treated as a
% 1-D vector
%
% output:
% output: the decompressed byte stream stored in a uint8 vector; if info is
% given, output will restore the original data's type and dimensions
%
% examples:
% [bytes, info]=zstdencode(eye(10));
% orig=zstddecode(bytes,info);
%
% license:
% BSD or GPL version 3, see LICENSE_{BSD,GPLv3}.txt files for details
%
% -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab)
%

if(nargin==0)
error('you must provide at least 1 input');
end
if(exist('zmat','file')==2 || exist('zmat','file')==3)
if(nargin>1)
[varargout{1:nargout}]=zmat(varargin{1},varargin{2:end});
else
[varargout{1:nargout}]=zmat(varargin{1},0,'zstd',varargin{2:end});
end
else
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end
43 changes: 43 additions & 0 deletions zstdencode.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
function varargout = zstddecode(varargin)
%
% output = zstddecode(input)
% or
% output = zstddecode(input,info)
%
% Decompressing an Zstd-compressed byte-stream to recover the original data
% This function depends on the ZMat toolbox (http://github.com/NeuroJSON/zmat)
%
% authors:Qianqian Fang (q.fang <at> neu.edu)
%
% input:
% input: a string, int8/uint8 vector or numerical array to store Zstd-compressed data
% info (optional): a struct produced by the zmat/zstdencode function during
% compression; if not given, the inputs/outputs will be treated as a
% 1-D vector
%
% output:
% output: the decompressed byte stream stored in a uint8 vector; if info is
% given, output will restore the original data's type and dimensions
%
% examples:
% [bytes, info]=zstdencode(eye(10));
% orig=zstddecode(bytes,info);
%
% license:
% BSD or GPL version 3, see LICENSE_{BSD,GPLv3}.txt files for details
%
% -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab)
%

if(nargin==0)
error('you must provide at least 1 input');
end
if(exist('zmat','file')==2 || exist('zmat','file')==3)
if(nargin>1)
[varargout{1:nargout}]=zmat(varargin{1},varargin{2:end});
else
[varargout{1:nargout}]=zmat(varargin{1},0,'zstd',varargin{2:end});
end
else
error('you must install ZMat toolbox to use this feature: http://github.com/NeuroJSON/zmat')
end

0 comments on commit a3c807f

Please sign in to comment.