Exception while linting

[charpour]

I'm getting the following exception while sonar is linting TS code:

ERROR: Error during SonarQube Scanner execution
com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Unterminated string at line 1 column 118932 path $[0][0].name
at com.google.gson.Gson.fromJson(Gson.java:820)
at com.google.gson.Gson.fromJson(Gson.java:770)
at com.google.gson.Gson.fromJson(Gson.java:719)
at com.google.gson.Gson.fromJson(Gson.java:691)
at com.pablissimo.sonar.TsLintParserImpl.parse(TsLintParserImpl.java:12)
at com.pablissimo.sonar.TsLintSensor.execute(TsLintSensor.java:101)
at org.sonar.scanner.sensor.SensorWrapper.analyse(SensorWrapper.java:53)
at org.sonar.scanner.phases.SensorsExecutor.executeSensor(SensorsExecutor.java:57)
at org.sonar.scanner.phases.SensorsExecutor.execute(SensorsExecutor.java:49)
at org.sonar.scanner.phases.AbstractPhaseExecutor.execute(AbstractPhaseExecutor.java:78)
at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:184)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:241)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:236)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:234)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:234)
at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:226)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.scanner.task.ScanTask.execute(ScanTask.java:47)
at org.sonar.scanner.task.TaskContainer.doAfterStart(TaskContainer.java:86)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.scanner.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:115)
at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:118)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy0.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:244)
at org.sonarsource.scanner.api.EmbeddedScanner.runAnalysis(EmbeddedScanner.java:154)
at org.sonarsource.scanner.cli.Main.runAnalysis(Main.java:110)
at org.sonarsource.scanner.cli.Main.execute(Main.java:72)
at org.sonarsource.scanner.cli.Main.main(Main.java:60)
Caused by: com.google.gson.stream.MalformedJsonException: Unterminated string at line 1 column 118932 path $[0][0].name
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1572)
at com.google.gson.stream.JsonReader.nextQuotedValue(JsonReader.java:1029)
at com.google.gson.stream.JsonReader.nextString(JsonReader.java:825)
at com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:358)
at com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:346)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:95)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:183)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
at com.google.gson.internal.bind.ArrayTypeAdapter.read(ArrayTypeAdapter.java:72)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
at com.google.gson.internal.bind.ArrayTypeAdapter.read(ArrayTypeAdapter.java:72)
at com.google.gson.Gson.fromJson(Gson.java:805)
... 37 more

Any ideas what the problem might be?

I'm using SonarQube 6.0 and the latest released version of the plugin (0.94)

[Pablissimo]

Not fun!

• What version of tslint are you running? (tslint -v on the command line to find out)
• What OS are you running?
• Can you run the sonar-runner with the -X -e arguments and attach/email me the log please? It'll give you the exact command line that was used to invoke tslint, which you can then run yourself - the output of that would also be handy if you can?

[charpour]

I'll try to debug first and then I'll let you know of the outcome. Looks like that even though the tslint command generates a correct JSON, not the whole thing is passed to the parser.

ts lint version: 3.15.1
OS: debian linux

-X -e didn't provide any additional useful information

[charpour]

out of curiosity, in order to debug the plugin, where do you configure the debug JVM flags? the scanner itself or one of the sonar server processes (web, ce, search)?

Thanks

[charpour]

I've noticed that out of the 349066 characters of output that the actual tslint command produces, only 117603 make it to the stream when consumed via the CommandExecutor, for some reason.

Thus, to reproduce you can create a file that has a lot of lint errors.

[Pablissimo]

Cracking, thanks - should be easy enough to repro from that investigation!

[Pablissimo]

Seems to be Linux-specific, and am wondering if it's a bug in the Sonar API's CommandExecutor - on Ubuntu 14.04 LTS the command executes fine but only generates a maximum of 79,278 characters of output and just seems to bin the rest silently, oddly. Whole thing's weird.

Am curious to see if a different mechanism for executing the command might help. Shall have a nose.

[charpour]

Quite weird indeed.
perhaps apache commons exec could help

[charpour]

As a workaround to the problem on linux, '-o /dev/stdout' argument has to be passed to the tslint command. For some reason, when it outputs to stdout some data gets discarded. If it's a file, everything works as expected. Some flushing node issue perhaps?

[charpour]

Finally.. found the issue. It's in tslint:

outputStream.write(lintResult.output, () => {
        if (lintResult.failureCount > 0) {
            process.exit(argv.force ? 0 : 2);
        }
    });

process.exit is the culprit. It terminates the process before all of the data is flushed.

[Pablissimo]

Nice debugging - hooray for cross-platform development I guess! I notice that there are Node issues around this specific case (e.g. nodejs/node#2972, nodejs/node#6456). Seems the guidance on the node side is not to use process.exit unless you just want to tear the whole thing down even without draining output streams.

The behaviour also seems to vary by Node version, excitingly. Wondering if it's safer to just skip streams entirely and always on both platforms output to a temporary file and re-read it (slower, and generates a little IO but I wouldn't have thought noticeable).

I'll try a patch along the lines you're suggesting - detect Linux, be explicit about the output stream. I'm already special-casing Windows to get quoted paths to work properly so that ship's sailed

[charpour]

I'd think that writing the output to an intermediate file on all platforms (i.e. using the -o argument of tslint) and consuming it from there is the better approach, as personally I'm not fond of using OS based branches in the code. It'd be harder to remove it in the future if for some reason you decide to do so.

[Pablissimo]

Got a local build of this running against a temporary file that gets re-read, seems performant enough and the issue goes away on Ubuntu 14.04 LTS. Got to check Windows yet, and fix a few tests but seems to be a go-er.

[Pablissimo]

Ok, CI build for the issue69 branch finished and should hopefully be an improvement for you (appears to continue to work fine on Windows and the couple of Linux variants I've got access to). You can either pull and build or give the binary a bash:

https://s3-us-west-2.amazonaws.com/com.pablissimo.github.sonartsplugin/builds/issue69/128/target/sonar-typescript-plugin-0.95-SNAPSHOT.jar

Will make it a proper release once you confirm it's done the job!

[charpour]

I've tried 0.95 against [email protected], but the run failed with the following error:

01:55:45 INFO: Sensor Linting sensor for TypeScript files
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 INFO: EXECUTION FAILURE
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 INFO: Total time: 1:32.425s
01:55:47 INFO: Final Memory: 59M/383M
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 ERROR: Error during SonarQube Scanner execution
01:55:47 ERROR: 54 is not a valid line for pointer. File [moduleKey=<some_ts_file>, basedir=<some_directory>] has 24 line(s)

However, it works great against [email protected]. I'll stick to that version for now.

Thanks for the quick fix!

[Pablissimo]

Never rains but it pours! Thanks for feedback, will raise a new issue for that then!

-------- Original message --------
From: charpour [email protected]
Date: 20/11/2016 12:18 (GMT+00:00)
To: Pablissimo/SonarTsPlugin [email protected]
Cc: Pablissimo [email protected], Comment [email protected]
Subject: Re: [Pablissimo/SonarTsPlugin] Exception while linting (#69)

I've tried 0.95 against [email protected], but the run failed with the following error:

01:55:45 INFO: Sensor Linting sensor for TypeScript files
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 INFO: EXECUTION FAILURE
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 INFO: Total time: 1:32.425s
01:55:47 INFO: Final Memory: 59M/383M
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 ERROR: Error during SonarQube Scanner execution
01:55:47 ERROR: 54 is not a valid line for pointer. File [moduleKey=<some_ts_file>, basedir=<some_directory>] has 24 line(s)

However, it works great against [email protected]:[email protected]. I'll stick to that version for mow.

Thanks for the quick fix!

You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/69#issuecomment-261774808, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABZvgB3cTp2v-tPPDatOFapUXir59jzNks5rADqOgaJpZM4K2bzk.

[Pablissimo]

Fixed in the above-linked release but also in 0.96, since 0.95 never made it to the Releases page.