Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for multiple root elements #529

Closed
alexculea opened this issue Mar 23, 2017 · 7 comments
Closed

Support for multiple root elements #529

alexculea opened this issue Mar 23, 2017 · 7 comments
Labels
kind: enhancement/improvement solution: duplicate the issue is a duplicate; refer to the linked issue instead

Comments

@alexculea
Copy link

I've been using the library for a very short time, so please excuse me if I'm coming out of the context of previous discussions (or might've missed some important aspect).

Taking note of the contribution guidelines in regards to the JSON standard conformation, this is a proposition to start discussing if supporting multiple root elements could be considered for this great project.

The problem:

The constraint of having a single root element per document adds the necessity for JSON-generator programs to consider previously generated data when adding new entries. Consider the following scenario: a program takes performance samples every n-secs and dumps them to the filesystem. To provide valid JSON output, the program has look at the previously generated data, adding commas, re-align '}'s or ']'s and so forth. Imagine how this scales when a short time, say 1 second, is used to store the dumps on a relatively busy system. Not good. The alternative: generate a file per sample but that doesn't scale either.

What do the standards say?

rfc4627, says that "A JSON text is a serialized object or array." (paragraph 2) which enforces the current implementations to look at everything being enclosed in root object {} or array [].

How about the real world?

Although, the same rfc4627 says that generators MUST conform to the grammar (see chapter 5 Generators) I've seen implementations that do not respect this, understandably IMHO because of the trouble of parsing back the old data. Two examples I can think of right now are: GlusterFS 3.8 (diagnostics.stats-dump-interval) and WinstonJS.

What could be done?

I think some sort of flag could be implemented in the library to allow multiple root elements, thus solving the said dilemma. I'm not sure of the implementation details at the this time but would appreciate the contributors feedback on the matter.

@nlohmann
Copy link
Owner

nlohmann commented Mar 23, 2017

I have not yet understood the problem you try to solve. Could you please provide a concrete example and where the library currently falls short?

Note RFC4627 has been obsoleted by RFC7159. There, a JSON value can also be a string, number, boolean, or null.

Changed the definition of "JSON text" so that it can be any JSON
value, removing the constraint that it be an object or array.

(https://tools.ietf.org/html/rfc7159#appendix-A)

@alexculea
Copy link
Author

alexculea commented Mar 23, 2017

This happens when trying to parse JSON from files having multiple root objects.

The code (used with the attached dump):

json data;
ifstream dump_file("glusterfs_storage0.dump.txt");

data.parse(dump_file);

glusterfs_storage0.dump.txt

Throws "[json.exception.parse_error.101] parse error at 15906: parse error - unexpected '{'; expected end of input" because of line 289. I've implemented a workaround in this case because there aren't nested objects, but I think it would help a lot if the library could be instructed accordingly.

@nlohmann
Copy link
Owner

This seems to be related to #367. There, the idea is to allow something like:

json data1;
json data2;

data1 << dumpfile;
data2 << dumpfile;

That is, there would not be an error when a complete JSON value has been parsed, yet the file has not yet reached EOF.

@nlohmann
Copy link
Owner

Your original problem seems to be very related to JSON Lines (http://jsonlines.org/).

@nlohmann nlohmann added the solution: duplicate the issue is a duplicate; refer to the linked issue instead label Mar 28, 2017
@nlohmann
Copy link
Owner

This issue is tracked in #367.

@ceztko
Copy link

ceztko commented May 16, 2017

I think that tracking this feature in another issue regarding very specific behavior while paring a stream is not gonna help to add this feature. The user that opened the issue should reopen it and post a sample code that shows the expected behavior. I am posting what I was trying to do (without success) but please note I am new to json and nlohmann/json:

#include <sstream>
#include <json.hpp>

using namespace std;
using namespace nlohmann;

char* test =
"{\
    \"one\"   : 1,\
    \"two\"   : 2\
}\
\
{\
    \"three\" :3\
}";

int main()
{
    string str(test);
    stringstream sstream(str);
    json val;
    try
    {
        size_t i = 0;
        while (!sstream.eof())
        {
            sstream >> val;
            cerr << "    result[" << i++ << "]=" << val
                << "; streampos=" << sstream.tellg() << std::endl;
        }
    }
    catch (std::exception& e)
    {
        cerr << "    std::exception:" << e.what() << std::endl;
    }
    return 0;
}

What I am expecting here (or what I am trying to achieve) is being able to parse the first two full roots elements in the first two iterations, parse an empty element at the third iteration and read EOF on the stream so there won't be a fourth iteration with an exception.

@nlohmann
Copy link
Owner

This seems to be related to #367.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: enhancement/improvement solution: duplicate the issue is a duplicate; refer to the linked issue instead
Projects
None yet
Development

No branches or pull requests

3 participants