Pattern Matching

[peaceamongworlds]

Would there be interest in adding a pattern matching macro to contrib, or would that have to go in an external library?

I have a working proof of concept right now, and it seems pretty similar to destructuring #1940, so I thought they might go well together.

[Kodiologist]

If it's just one macro that provides a destructuring equivalent of cond, you might as well put it in hy.contrib.destructure. Otherwise, perhaps I'm not clear what you're proposing.

[allison-casey]

Assuming you're talking about what I think you're talking about I would be very interested in a pattern matching macro in contrib. What were you thinking about in terms of syntax?

[jams2]

Something like pmatch would be cool. Structural pattern matching looks to be making its way into Python 3.10, that may be relevant.

[peaceamongworlds]

If it's just one macro that provides a destructuring equivalent of cond, you might as well put it in hy.contrib.destructure. Otherwise, perhaps I'm not clear what you're proposing.

I think that it'll require more than just one extra macro, so should probably go in a new module/library.

I would want the syntax to be pretty similar to Fennel's match macro or core.match, so that you could do something like:

(match [1 [2 1] 1]
  [a b] :not-enough-items
  [a [b a] b] [a b]
  [a a & _] a
  _ :no-match) ; => [1 2]

(match {:a 1 :b 2}
  {:a a :b a} a
  {:a 1 :b 2 :c _} :has-key-c
  {:a 1 :b 2} :no-key-c) ; => :no-key-c

So HyLists would match sequences and HyDict would match dictionaries (or anything supporting __getitem__). HyExpressions would be reserved for extra patterns, such as or, guard, as etc. These could also be made extensible with some sort of defpattern macro to allow for user defined patterns.

By default symbols would be bound to new variables in the resulting expression with an implicit let. Prefixing the symbol with $ could be used to refer to an actual variable in scope instead.

[Kodiologist]

You could try implementing a Hy equivalent of PEP 634 so that once that's in Python, we can produce the corresponding Python AST.

[asemic-horizon]

Questions:

1. Assume that I could write a set of macros that would emulate the PEP specification by generating Hy code. Would this be significantly different (slower, edge cases that can't be addressed even if discovered) from a "proper" implementation by the Hy dev team?

2. Assume that I could write a set of macros that would emulate the functionality of this specification by manipulating the Hy AST ("Hy models", which are accessible from Hy). Would be significantly different from a "proper" implementation into core Hy?

3. If let syntax (an ancient venerable Lisp utility and quite alive since many kids are introduced to Scheme/Racket) can be implemented as a contributors library, can pattern-matching? I don't mean to rekindle the debate about let (which I imagine must have been heated), but wonder how Hy should react to the increasingly wacky (cf. "walrus operator") ideas coming into Python.

If there was a vote, I'd say pattern matching is a worthy feature and the goal should be to implement whatever Clojure or even Scheme does rather than try to automatically mirror new Pythonisms. There's already the (pys "new_syntax being: whatever") escape hatch.

[Kodiologist]

I don't understand the difference between (1) and (2). An implementation that works purely at the Hy level, instead of producing the same ast nodes that Python does for this construct, would indeed be different and not as nice to have, but it's better than nothing. contrib might be a logical place to put such an implementation.

With few exceptions, we want every Python feature to be accessible from Hy, no matter how stupid it is, even though one can also use py or pys. It's still in scope for Hy to provide extensions, additional features, and even alternative approaches to the same problem.

[sheganinans]

So since PEP 634 has been officially blessed, my current thinking about the problem is that ideally Hy would output the match expressions directly. This of course means compiler changes.

I like the idea that "Hy is more like Coffeescript than Clojure", keep the interface as thin as possible, ignore the bad parts (walrus operator), and keep up with developments in the underlying lang.

Aside from questions about syntactic bikeshedding, I think PEP 634 is "good enough". Sure it's not as fancy as Prolog-style pattern matching, but its such a huge improvement in the Python language story I think it would be a shame to not leverage the change. The idea is that match would just be exposed as thinly as possible in Hy, with fancier match libraries built on-top of the raw functionality.

I think it would be a mistake to mainline a match functionality in Hy that just desugars to a whole bunch of if/elses, it would just be a terrible user story to have 5 years from now..

So what to do now? I suppose this would involve:

• Setting up the current Python betas for testing.
• Make the Hy compiler play nice with the new syntax.
• Backport match for <3.10 versions, expanding out to if/elses.
• Finally release a new version of Hy with the functionality exposed when 3.10 lands. If not before, given a backport.

I'd really like to see this happen and I'd be willing to put in what effort I can to make it a reality.

[Kodiologist]

ignore the bad parts (walrus operator)

You know that it's been implemented in Hy since #1772, right? More generally, we might procrastinate about implementing Python features that none of us care about (and by "us", I mean the people working on Hy at any given time), but we're not going to purposefully forgo them.

I think it would be a mistake to mainline a match functionality in Hy that just desugars to a whole bunch of if/elses, it would just be a terrible user story to have 5 years from now.

There's no reason to refuse such an implementation if the design is such that it can be changed to output the Python match AST later.

I'd really like to see this happen and I'd be willing to put in what effort I can to make it a reality.

I'd appreciate that. The best way to start is probably to run the test suite on your machine with Python 3.10 and start fixing test failures.

[sheganinans]

@Kodiologist Thanks for the quick reply!

On all of my comments: I'm obviously more of a newcomer to Hy, so I admit my views are going to come from my knowledge of the surface language.

Otherwise on the walrus operator, thanks for clearing up my misconceptions and pointing me to concrete implementation details. I was more appealing to some of its minor semantic quirks, but it being exposed as-is with setx is great, forgot that was the case.

As well as about new features, I fully understand the limited time and interest Hy developers have. So I'm not trying to push anyone one way or another, just give my 2c on my ideal.

Sure, thanks for the guidance, I've been meaning to dive into the Hy codebase more seriously, I suppose fixing tests for 3.10 sounds like a good place to start!

[allison-casey]

Hi Sheganinans!

as kodi already mentioned Hy tries to be more "lipstick on python" than an entirely decoupled language. So we'd like to make sure pythoners that try hy have everything they would expect from from python available to them. But as with the destructuring contrib module that was just added, there's nothing stopping anyone from adding a match+ module with more lispy semantics either. There's plenty of work to do to help push Hy over the 1.0 finish line and if you're looking to help we'd love to have you! Kodi mentioned getting 3.10 tests working as a good place to start if that's what you're interested in, and I'm going through and cleaning up the issue tracker and marking some good first issue stuff if you wanna get your hands dirty squashing some bugs.

[sheganinans]

Hi @allison-casey!

I've found that currently 14 tests break on Win10, all seemingly simple stuff like missing/incorrect paths and permissions. I think I'll focus on those first, since for me, the entire test suite under py3.9 passes on Linux, have yet to run the test suite under py3.10. To keep this thread focused on the Pattern Matching, I'll be filling separate issues/PRs.

That said, one last question: Is the IRC still the best place to ask "Hy Hacking" related questions? Since I'm sure I'll have some questions here and there, since I'm assuming it's preferable to keep the Github issues unclogged with my sporadic questions.

[allison-casey]

Awesome! Most of us don't do our work on linux all that much so if you can tackle some of that that would be fantastic! Looking forward to seeing what you do.

This has kind of been an open question (#1778) one that i was going to reopen, but for now there's stackoverflow under the [hy] tag or you can feel free to email me. I'm most active on discord so if you're on there you can also email me your account id.

[Kodiologist]

You mean "don't do our work on Windows", right? I do everything on Ubuntu.

[allison-casey]

yup lol, I'm tired. I'm on Fedora myself

[sheganinans]

Thanks I'm on Discord as well, emailing now.

I have intentions on maintaining 1st class support of Hy on Win10, alongside Linux. Even though I'm primarily a Linux user, out of necessity I'm using Win10 with Hy for some projects, so I suppose I can take up the mantle of Win10 compat.

Otherwise, so far I've managed to patch 3 out of the 14 failing tests I've found. I'll open a PR to track my progress, so we can bikeshed over there.