Skip to content

JSONLab 1.9.8 (codename: Magnus - beta)
Compare
Choose a tag to compare
@fangq fangq released this 02 Nov 03:19
· 231 commits to master since this release
v1.9.8
f2bfb65

What's New

JSONLab v1.9.8 is the beta release of the next milestone - code named "Magnus".

Starting from this release, JSONLab supports encoding/decoding MessagePack,
a widely-used binary JSON-like data format. Via ZMat v0.9, JSONLab v1.9.8
also supports LZMA/LZ4/LZ4HC data compression/decompression. More importantly,
JSONLab is now the official reference implementation for JData Specification (Draft 2)
as defined in http://github.com/fangq/jdata, the foundation for the OpenJData
Project (http://openjdata.org).

There have been numerous major updates to this toolbox since the previous
release v1.9 in May 2019. A list of the major changes are summarized below
with key features marked by *:

  • 2019-10-22*[650b5ec] enable preencode by default for savejson and saveubjson
  • 2019-10-21*[874945f] decode graph data, encode non-char-keyed map data
  • 2019-10-18 [11712b7] add any2jd, pass opt to name check, add more options
  • 2019-10-18*[f97de9b] extract name encoding/decoding to separate function, like in easyh5
  • 2019-10-17*[9d0fd4a] rewrite jdataencode
  • 2019-10-15 [23f14d6] minor updates to make msgpack to work on octave
  • 2019-09-16*[689cb40] support lz4 and lz4hc compression via zmat v0.9
  • 2019-07-11*[06d33aa] update zmat test to support zmat v0.8 mox-the-fox
  • 2019-06-24*[eba4078] saving table objects with new syntax
  • 2019-06-12 [3eb6d56] change ArrayCompression keywords to ArrayZip to be short
  • 2019-06-12*[e5f2ffb] complete saveubjson debug mode, add compression example
  • 2019-06-11 [ebbcfd2] pass formatversion tag to jdatadecode
  • 2019-06-10*[95b2eb0] add jdataencode and jdatadecode
  • 2019-06-10*[f86219d] major update: use row-major for N-D array, incompatible with old JSONLab
  • 2019-05-31*[0c467ee] support lzma and lzip compression decompression via ZMat Toolbox
  • 2019-05-31 [599ee4c] support categorical data
  • 2019-05-30*[d47be45] fast bracket matching
  • 2019-05-24*[0ec2d01] rewriting fastarrayparser, 10x faster for Octave, close #4
  • 2019-05-22*[d8c19b8] add support to MessagePack, close #53, add NestArray option, close #6
  • 2019-05-19*[c87e7d2] support containers.Map

Please note that JSONLab v1.9.8 is compliant with JData Spec Draft 2, while
v1.9 and previous releases are compatible with Draft 1. The main differences are

  • _ArrayCompressionMethod_, _ArrayCompressionSize_ and _ArrayCompressedData_
    were replaced by _ArrayZipType_, _ArrayZipSize_ and _ArrayZipData_, respectively
  • The serialization of N-D array data stored in _ArrayData_ was changed from column-major to
    row-major

To read data files generated by JSONLab v1.9 or older versions, you need to attach
option 'FormatVersion', 1.9 in all loadjson/savejson function calls.
To convert an older file (JSON/UBJSON) to the new format, you should run

   data=loadjson('my_old_data_file.json','FormatVersion',1.9)
   savejson('',data,'FileName','new_file.json')

You are strongly encouraged to convert all previously generated data files using the new
format.

Introduction

JSONLab is a free and open-source implementation of a JSON/UBJSON/MessagePack
encoder and a decoder in the native MATLAB language. It can be used to convert a
MATLAB data structure (array, struct, cell, struct array, cell array, and objects) into
JSON/UBJSON/MessagePack formatted strings, or to decode a JSON/UBJSON/MessagePack
file into MATLAB data structure. JSONLab supports both MATLAB and GNU Octave
(a free MATLAB clone).

JSON (JavaScript Object Notation, http://json.org/ ) is a highly portable, human-readable
and "fat-free" text format to represent complex and hierarchical data. It is as powerful as
XML, but less verbose. JSON format is widely used for data-exchange in applications.

UBJSON (Universal Binary JSON, http://ubjson.org/) is a binary JSON format,
specifically optimized for compact file size and better performance while keeping
the semantics as simple as the text-based JSON format. Using the UBJSON
format allows to wrap complex binary data in a flexible and extensible
structure, making it possible to process complex and large dataset without accuracy
loss due to text conversions. MessagePack is another binary JSON-like data
format widely used in data exchange in web/native applications. It is slightly more
compact than UBJSON, but is not directly readable compared to UBJSON.

We envision that both JSON and its binary counterparts will play important roles as
mainstream data-exchange formats for scientific research. It has both the flexibility
and generality as offered by other popular general-purpose file specifications, such
as HDF5 but with significantly reduced complexity and excellent readability.

Towards this goal, we have developed the JData Specification (http://github.com/fangq/jdata)
to standardize serializations of complex scientific data structures, such as
N-D arrays, sparse/complex-valued arrays, trees, maps, tables and graphs using
JSON/binary JSON constructs. The text and binary formatted JData files are
syntactically compatible with JSON/UBJSON formats, and can be readily parsed
using existing JSON and UBJSON parsers.

Please note that data files produced by saveubjson may utilize a special
"optimized header" to store N-D (N>=2) arrays, as defined in the JData Specification Draft 2.
This feature is not supported by UBJSON Specification Draft 12. To produce
UBJSON files that can be parsed by UBJSON-Draft-12 compliant parsers, you must
add the option 'NestArray',1 in the call to saveubjson.

Installation

The installation of JSONLab is no different from installing any other
MATLAB toolbox. You only need to download/unzip the JSONLab package
to a folder, and add the folder's path to MATLAB/Octave's path list
by using the following command:

    addpath('/path/to/jsonlab');

If you want to add this path permanently, you can type "pathtool",
browse to the JSONLab root folder and add to the list, then click "Save".
Then, run "rehash" in MATLAB, and type "which savejson", if you see an
output, that means JSONLab is installed for MATLAB/Octave.

If you use MATLAB in a shared environment such as a Linux server, the
best way to add path is to type

   mkdir ~/matlab/
   nano ~/matlab/startup.m

and type addpath('/path/to/jsonlab') in this file, save and quit the editor.
MATLAB will execute this file every time it starts. For Octave, the file
you need to edit is ~/.octaverc , where "~" is your home directory.

Install JSONLab on Fedora 24 or later

JSONLab has been available as an official Fedora package since 2015. You may
install it directly using the below command

   sudo dnf install octave-jsonlab

To enable data compression/decompression, you are encouraged to install octave-zmat using

   sudo dnf install octave-zmat

Install JSONLab on Arch Linux

JSONLab is also available on Arch Linux. You may install it using the below command

   sudo pacman -S jsonlab
Assets 2
Loading