Outstanding issues for the interpreter on Windows

[HertzDevil]

These are the biggest issues I could identify after successfully building the interpreter on Windows, in case I forget about them.

EDIT: All the issues in this comment have been resolved, but there are more to follow below.

• The entire family of printf functions in LibC are defined inline in legacy_stdio_definitions.lib, which means they are inaccessible via the dynamic loader. It seems there is no way to reimplement them in Crystal, since their definitions rely on other internal state that is not exported. Solving this might require defining printf as an interpreter primitive so that the compiler's own statically linked versions may be used. However, not using those LibC functions at all is also a possible end goal for Functions that depend on the current C locale #11952.
• Because exit calls the at_exit handlers, which bail out with Crystal::System.print_error in case of an unhandled exception, calling exit produces an undefined reference to `snprintf' error. Subsequent calls to exit produce Unhandled case: enum value outside of defined enum members. String#% is also subject to this, and the only way to exit a REPL session is by pressing Ctrl+C.
• Running a file with the interpreter (bin\crystal i file.cr) always ends with calling Crystal.exit, so again the above error appears all the time.
• Commenting out the above print_error does not help either, as now a different error shows up:

  Invalid memory access (C0000005) at address 0x8
  [0x7ff6c31068da] memcpy at D:\a\_work\1\s\src\vctools\crt\vcruntime\src\string\amd64\memcpy.asm:207
  [0x7ff6c01d34c8] copy_from_impl at C:\Users\Quinton\crystal-win\src\pointer.cr:248
  [0x7ff6c01d3456] copy_to at C:\Users\Quinton\crystal-win\src\pointer.cr:185
  [0x7ff6c01d3436] copy_from at C:\Users\Quinton\crystal-win\src\pointer.cr:154
  [0x7ff6c0feca5f] interpret at C:\Users\Quinton\crystal-win\src\compiler\crystal\interpreter\interpreter.cr:337
  [0x7ff6c0f9dd95] interpret at C:\Users\Quinton\crystal-win\src\compiler\crystal\interpreter\interpreter.cr:235
  [0x7ff6c1519dbd] interpret at C:\Users\Quinton\crystal-win\src\compiler\crystal\interpreter\repl.cr:100
  [0x7ff6c1519b41] interpret_and_exit_on_error at C:\Users\Quinton\crystal-win\src\compiler\crystal\interpreter\repl.cr:104
  [0x7ff6c151a54b] run_file at C:\Users\Quinton\crystal-win\src\compiler\crystal\interpreter\repl.cr:75
  [0x7ff6c14a5aa9] repl at C:\Users\Quinton\crystal-win\src\compiler\crystal\command\repl.cr:44
  [0x7ff6c14a0379] run at C:\Users\Quinton\crystal-win\src\compiler\crystal\command.cr:100
  [0x7ff6c149fd4d] run at C:\Users\Quinton\crystal-win\src\compiler\crystal\command.cr:51
  [0x7ff6c149fcbc] run at C:\Users\Quinton\crystal-win\src\compiler\crystal\command.cr:50
  [0x7ff6c00f1ab5] __crystal_main at C:\Users\Quinton\crystal-win\src\compiler\crystal.cr:11
  [0x7ff6c030afd3] main_user_code at C:\Users\Quinton\crystal-win\src\crystal\main.cr:115
  [0x7ff6c030aeea] main at C:\Users\Quinton\crystal-win\src\crystal\main.cr:101
  [0x7ff6c01077c9] main at C:\Users\Quinton\crystal-win\src\crystal\main.cr:127
  [0x7ff6c01078e3] wmain at C:\Users\Quinton\crystal-win\src\crystal\system\win32\wmain.cr:37
  [0x7ff6c305f7f4] __scrt_common_main_seh at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
  [0x7ffe58897034] BaseThreadInitThunk +20 in C:\WINDOWS\System32\KERNEL32.DLL
  [0x7ffe5a002651] RtlUserThreadStart +33 in C:\WINDOWS\SYSTEM32\ntdll.dll
  

  This is caused by the at_exit handler in Crystal::System::FileDescriptor.from_stdio. It suggests there might be some other bytecode generation issue despite the workaround in the given to-do.
• No specs can be run yet, due to a missing interpreter feature used by Crystal::System::Path.home:

  In src\crystal\system\win32\path.cr:9:37
  
   9 | elsif LibC.SHGetKnownFolderPath(pointerof(LibC::FOLDERID_Profile), 0, nil, out path_ptr) == 0
                                       ^
  Error: BUG: missing interpret for PointerOf with exp Crystal::Path
  

• The macro run fails with 0xC0000135 STATUS_DLL_NOT_FOUND. (This might be related to -Dpreview_dll rather than the interpreter per se)

[HertzDevil]

Crystal::System.print_error cannot be easily replaced with our sprintf because it must continue to understand C's version of format strings which include argument length modifiers, such as %llu. In particular, the GC may pass format strings beyond our control to print_error due to LibGC.set_warn_proc (although they limit the possible format specifiers and arities). Besides, using a full String::Formatter seems to be overkill for this purpose.

Perhaps we need a custom low-level implementation that supports only a subset of C's printf...? For our own uses it needs to cover %s, %d, %x, %lx, %lld, and %llx. If we trust Boehm GC then we need to also handle %p, %ld, and %lu from there.

[HertzDevil]

This snippet keeps spawning fibers that do nothing:

while true
  spawn do
  end
  p GC.stats.heap_size
  sleep 0.05
end

On Windows only, sometimes this would commit multiple GBs of memory at once:

157192192
157192192
173977600
173977600
4135460864
4169031680
4185817088
4303314944
4336885760
...
5394366464
5411151872
5427937280
9725001728
9808928768
9909641216
9943212032

This would continue until the GC heap reaches around 32 GB and fails to expand any further, even though the actual memory usage is only slightly above 2 GB. This is currently breaking spec/std/channel_spec.cr. Linux doesn't have the same issue.

---

Interpreted code currently crashes on Crystal::System::Socket.accept_ex.call. This is affecting networking-related specs. This might be related to #12495, except here our Proc is explicitly constructed like LibC::AcceptEx.new(function_pointer, Pointer(Void).null).

[crysbot]

This issue has been mentioned on Crystal Forum. There might be relevant details there:

https://forum.crystal-lang.org/t/dl-on-windows/7152/2