Idea: lib.wiring and the Transactron library of Coreblocks

[tilk]

I have a design problem I would like to discuss. This doesn't seem right to open issue for, and that's why I'm writing here. I hope this is a correct channel for things like this.

The problem is as follows. As a part of the Coreblocks project, we developed the Transactron framework, which allows to easily construct complex, latency-insensitive circuits. The connection between modules inside Transactron are mostly handled not by using interfaces (in sense of lib.wiring), but a Transactron concept called a method. Methods have some input and output signals associated with them, but they also have some important, generation-level metadata which make them tick.

In our modules, exported methods are currently defined as attributes on an Elaboratable, while imported ones are typically passed by parameters, but another possibility would be to define them as attributes too, and connect them later by proxying. The situation is thus extremely similar to the one in Amaranth before the introduction of lib.wiring, and it has the same drawbacks. Therefore I had an idea that signatures, interfaces and components of lib.wiring could possibly be used for methods, in this way leveraging an existing solution instead of creating from scratch a very similar one. This would also have the advantage that the modules which use both methods and plain Amaranth signals could use lib.wiring for both.

Unfortunately, I hit a snag. The way connect in lib.wiring works is by .eq()-ing in the right direction every Value-castable thing in the interface. This is not enough to connect Transactron methods, as the metadata needs to be taken care of, too. Therefore, it turned out that I can't use lib.wiring to solve my issue after all.

So, the question/idea is as follows. This plan could work, if there was a way to hook into connect, performing some other action instead of (or together with) the default one (using .eq()), for certain kinds of interfaces/ports. Does this idea fit with your plans for Amaranth's development? Or, possibly, you have some other idea which also solves the problem we have? Of course, I'm happy to develop the implementation if we agree on something. I sincerely appreciate your consideration in this matter.

[whitequark]

How exactly does the metadata work? The desire or need for adding some kind of handling for some kind of metadata has been expressed before, but the most I've actually been planning for is additional checks, similar to how you would check that the parameters (and not just structure) of two interfaces are compatible, for things such as latency handling.

Depending on what exactly you need, it might make sense to accelerate those plans somewhat, or perhaps not.

Also, even if you can't use wiring.connect you should still be able to use interfaces per se--and perhaps the check discussed would only just prevent any connection to be made at all with amaranth.lib.wiring.connect rather than transactron.connect or some other method you provide.

[tilk]

The metadata is collected during elaboration, and they describe the ways the methods are used inside a Transactron project. This data is then used at the final stage of elaboration of the entire project, to guide the construction of a coordinator circuit for the project. (If you want some details, see https://github.com/kuznia-rdzeni/coreblocks/blob/master/transactron/core/method.py#L302 and the method_calls/method_uses attributes. These are utilized in the transaction manager of Transactron, see for example https://github.com/kuznia-rdzeni/coreblocks/blob/master/transactron/core/manager.py#L185.)

Also, even if you can't use wiring.connect you should still be able to use interfaces per se--and perhaps the check discussed would only just prevent any connection to be made at all with amaranth.lib.wiring.connect rather than transactron.connect or some other method you provide.

I thought about implementing Transactron's version of connect, which would handle my use case. I would like to use lib.wiring features with that, for example arrays: an array of methods makes sense for implementing "multi-port" or "superscalar" modules. The question I have is that if this can be cleanly done without reimplementing a large part of the logic in Amaranth's connect.

[whitequark]

OK, so the metadata is essentially read-only, i.e. does not change what gets appended to m.d.comb, right? If so, I am not opposed to this on semantic grounds (I do not wish for lib.wiring.connect() to be extendable in a way that changes the netlist produced by it because this makes reasoning about it too difficult). Can you elaborate on what contextual information would the callback need?

I thought about implementing Transactron's version of connect, which would handle my use case.

If the final netlist (within the callee-provided module) is the same, could you not call lib.wiring.connect() and then add the metadata? In fact, this is something you could do to have your needs fulfilled right now and not when the upstream metadata handling is implemented.

[tilk]

OK, so the metadata is essentially read-only, i.e. does not change what gets appended to m.d.comb, right?

Well, not really. The metadata is collected during elaboration of modules, and the addition of new metadata might happen both before and after the connection is made. But I think this could be handled via reshaping the metadata somewhat, so for now I wouldn't think about this as an obstacle.

If the final netlist (within the callee-provided module) is the same, could you not call lib.wiring.connect() and then add the metadata? In fact, this is something you could do to have your needs fulfilled right now and not when the upstream metadata handling is implemented.

That's a pretty nice idea, it's possible this could do the trick. I'll spend some time experimenting with this and return to this discussion later.