Regression when calling macros and using haxe.io.Bytes

[hklindworth]

I encountered a regression that is most likely related to the new macro handling introduced with Haxe 4. I can reproduce it when calling a macro function on a class which uses haxe.io.Bytes. With Haxe3 compilation is possible.

This is a simple OpenFL project that triggers the wrong behavior and compiler fails:
https://github.com/hklindworth/eval-issue-openfl

openfl build html5
/Dev/haxe/lime/lime/utils/Bytes.hx:23: characters 10-43 : haxe.io.Bytes does not have a constructor
/Dev/haxe/openfl/openfl/utils/ByteArray.hx:286: characters 3-26 : haxe.io.Bytes does not have a constructor
/Dev/haxe/openfl/openfl/utils/ByteArray.hx:786: characters 3-4 : Unknown identifier : b
Field index for length not found on prototype haxe.io.Bytes

This is a minimal project that show also some error. No internal compiler error, but constructor cannot be accessed, even though it is made available via @:access:
https://github.com/hklindworth/eval-issue

haxe release.hxml 
Main.hx:22: characters 10-43 : haxe.io.Bytes does not have a constructor

I also tried to add the private constructor to

haxe/std/eval/_std/haxe/io/Bytes.hx

Line 24 in 74fe2ca

extern class Bytes {

This fixes the "does not have a constructor" error message, but not the compiler crash.

[Simn]

Yes, haxe.io.Bytes does in fact not have a constructor on this target. It might be possible to add it for backwards compatibility, but I'm not sure if that's worth it. I think the best solution is to use the public API (Bytes.ofData) instead of trying to access implementation details...

[hklindworth]

Currently lime is accessing the private constructor:
https://github.com/openfl/lime/blob/9914b2498fc01814a45dffb5fc30e88f781ed267/lime/utils/Bytes.hx#L18
Also openfl is accessing private members. These can be fixed there. But it still seems strange to me that these errors only occour when you call a macro function.

And even when I fix these issues (just commented them out), the compilation still fails with
Field index for length not found on prototype haxe.io.Bytes
This is the most problematic issue, especially since no filename or linenumber is given that is hinting at the problem.

[hklindworth]

PS: I just created another test project which is selfcontaining (unfortuntely quite big) and triggers a compiler assertion when you call the macro function and compiles successfully when not calling the macro function:

https://github.com/hklindworth/eval-issue-lime

Hope this helps

[Simn]

I'll take a look at the assertion, that clearly shouldn't happen. Any idea how to reproduce the Field index for length not found on prototype haxe.io.Bytes error?

As for the constructor call, that should really be fixed in lime and it shouldn't access any private members in general.

[Simn]

The assertion comes from this code: https://github.com/hklindworth/eval-issue-lime/blob/master/lime/lime/app/Application.hx#L94

There's probably an assumption that a var expression is not the only expression in a block, as that is rather pointless. Still, it obviously shouldn't cause an assertion failure.

[Simn]

Fixed the var issue, and added that constructor after all because I guess it doesn't hurt.

Still curious about that length issue though!

[hklindworth]

I adjusted the test project:
https://github.com/hklindworth/eval-issue-openfl

When you compile it now via haxe release.hxml is fails with
Field index for length not found on prototype haxe.io.Bytes

With Haxe3 it fails with
openfl/utils/ByteArray.hx:278: lines 278-319 : Missing super constructor call lime/lime/utils/Bytes.hx:67: characters 16-41 : Unknown<0> cannot be constructed lime/lime/utils/AssetLibrary.hx:659: characters 17-43 : Unknown<0> cannot be constructed openfl/utils/ByteArray.hx:278: lines 278-319 : Missing super constructor call lime/lime/media/AudioSource.hx:18: characters 25-49 : Unknown<0> cannot be constructed

[Simn]

That just gives me Type not found : ApplicationMain

[hklindworth]

That seems to a bug in my test example. Just pushed a fix.

On my machine it is now:
hendrik:eval-issue-openfl hendrik$ haxe release.hxml openfl/utils/ByteArray.hx:278: lines 278-319 : Missing super constructor call Field index for length not found on prototype haxe.io.Bytes

If you do not have the Field index length message, this seems like a good sign to me. I will recheck my examples ones the new nightly build of haxe has been released.

Thank you very much for your great help!

[Simn]

I changed it so the error message gives a position now: openfl/utils/ByteArray.hx:324: characters 3-9 : Field index for length not found on prototype haxe.io.Bytes. The line in question tries to set the length of a Bytes object.

[Simn]

Try now. If it complains about some b field, just use the object itself instead of the field (obj instead of obj.b).

[hklindworth]

Thanks, the compiler behavior greatly improved. No assertions and no compiler stops anymore.

Openfl still has one access to the private variable b that is not easy to remove
https://github.com/openfl/openfl/blob/31f6f6437ea37652b3fe69dea393c88f8a79b8b1/openfl/utils/ByteArray.hx#L814

Question is, if this should rather be fixed in openfl or in Haxe4? I still find a bit confusing that this error only appears when using macros in Main.hx. When you uncomment following line, the project compiles:
https://github.com/hklindworth/eval-issue-openfl/blob/180ba3d7836b1de7303d26e020a50e72255aa804/Main.hx#L29

I could try to change OpenFL ByteArrayData not to extend haxe.io.Bytes but to embed it.

[Simn]

The correct usage is b.getData(), i.e. the public API.

[hklindworth]

Alright, I agree that it is better to fix openfl.

[jgranick]

I'm trying to understand this, the issue is that b does not exist in the macro context? Can we just #if !macro around any references to the b object, or are there other targets that do not have a b property?

Are there any recommendations for being able to test Haxe 4 development builds alongside using Haxe 3 stable builds on the same system?

[hklindworth]

One way to test Haxe3 / Haxe4 at the same time (on MacOS):

1. copy existing /usr/local/lib/haxe to /usr/local/lib/haxe3. This way all the haxelibs are initially the same between Haxe3 & Haxe4
2. overwrite all files in /usr/local/lib/haxe with the latest nightly build from http://hxbuilds.s3-website-us-east-1.amazonaws.com/builds/haxe/mac/
3. Now you can just rename the directories to switch between Haxe3 and Haxe4. I wrote a small shell-script to switch between versions:

#!/bin/sh

if [ -d /usr/local/lib/haxe4 ]; then
	mv /usr/local/lib/haxe /usr/local/lib/haxe3
	mv /usr/local/lib/haxe4 /usr/local/lib/haxe
else
	mv /usr/local/lib/haxe /usr/local/lib/haxe4
	mv /usr/local/lib/haxe3 /usr/local/lib/haxe
fi