On Windows, REPL fails to start if there's a non-ASCII character in .hy-history

[gilch]

At least on Windows, if you paste a Δ into the repl, you can get in a state where the repl refuses to start.
It gets saved into the .hy-history file properly. But if you re-run the repl while a Δ is in the history file, it gets corrupted as something like Δ upon exit.

Seems like it could be an encoding issue in pyreadline, which completely chokes on it. I think that's what we're using to save the repl history. Maybe we've set some options improperly, or maybe pyreadline itself has a bug. Not sure.

This may be just on Windows though, since it uses its own encoding in the console. If you're running a different OS, can you confirm?

Deleting the offending characters from .hy-history allows the repl to start again.

I'm wondering if we should mangle with X's instead of Δ's even on Python3 to avoid this kind of issue. Pasting these things in the repl would be very tempting.

[jgkamat]

On my computer (debian 9), I don't see this issue. The Δ is saved properly in history, even after opening the hy repl up multiple times.

I'm using hy3 though, maybe this is only an issue on hy2 (since python string encoding was improved in python3)

[vodik]

Might not be a bad idea to add appveyor CI

[Kodiologist]

I'm wondering if we should mangle with X's instead of Δ's even on Python3

That was my original suggestion, and I still support it, although it presumably wouldn't fix the underlying problem here.

[Kodiologist]

@gilch, would you veto a PR to change the delimiter to X on Python 3?

[gilch]

@Kodiologist, no. I might even approve it. But that doesn't solve this underlying problem.

[Kodiologist]

Right, that's probably something to do with pyreadline, either a bug in pyreadline or a misconfiguration on our part.

[lafrenierejm]

It looks like the current HEAD of master requires Python 3.7+. @gilch are you able to confirm whether this issue is still present? I don't have a Windows machine on-hand, so I'm not able to test it myself.

[Kodiologist]

We actually run the test suite on Windows now. I think it would be pretty easy to add a test for this, if you'd like to take a shot.

[Kodiologist]

In particular, GitHub Actions will run on your personal fork of Hy, which I've found to be a handy way to check that stuff works on Windows before opening a PR.

[Kodiologist]

I looked into this. It turns out that we no longer use a history file on Windows, anyway, since we dropped support for the unmaintained and permanently broken pyreadline. In fact, chances are decent that this was a pyreadline bug.