Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bikeshed: design for .get_fragment() #9

Closed
whitequark opened this issue Dec 17, 2018 · 24 comments
Closed

Bikeshed: design for .get_fragment() #9

whitequark opened this issue Dec 17, 2018 · 24 comments
Labels

Comments

@whitequark
Copy link
Contributor

whitequark commented Dec 17, 2018

I know @jordens expressed in the past a desire to simplify the boilerplate around .get_fragment. Also, there is currently both Module.lower and Module.get_fragment (aliased to one another), which is confusing and unnecessary. Also#2, Migen has a Module.get_fragment() and combining Migen modules with nMigen modules gets even more confusing with that.

The design I see for the function currently named .get_fragment() is as follows:

  • There are 2 kinds of synthesizable entities in nMigen: fragments (which are directly synthesizable) and anything that has .get_fragment(platform) (which is iterated to a fixed point, i.e. a fragment, when a fragment is required).
  • Instances, memory ports, etc, are synthesizable. Which class they actually are is an implementation detail that is subject to change. (Instances are currently just fragments, memory ports are significantly more complex due to some mismatch between Yosys and Migen representation of ports, etc).
  • There is a Fragment.get(obj, platform) function that iterates an object to a fixed point with .get_fragment(platform) or fails in a nice way.

Modules are in a somewhat weird space here. Modules should be synthesizable so that you can replace:

sm = Module()
# ... operate on sm
m = Module()
m.submodules += sm

with:

class SubCircuit:
    def get_fragment(self, platform):
        m = Module()
        return m.lower(platform)
sm = SubCircuit()
m = Module()
m.submodules += sm

that is without having to do downstream changes after replacing a Module with a user class. This means that in principle you can directly return a module, like return m.

But this means two things:

  1. The actual get_fragment call for that module is delayed. The conversion of a module to a fragment is mostly (but not entirely) error-free, so delaying this call means some errors lose context. I think it might be more useful to make Module.get_fragment actually never fail, and permit return m.
  2. If a parent module directly calls get_fragment on a submodule, it gets a Module object with the DSL enabled, as opposed to a Fragment object. So it can potentially abuse the DSL that way. This isn't very likely so maybe we should just ignore it.

And finally, there is an important question of what to rename .get_fragment too. I don't want something like .build because it is ambiguous and prone to collisions. What about .synthesize?

@whitequark
Copy link
Contributor Author

@jordens Any ideas on naming here?

@jordens
Copy link
Member

jordens commented Jan 15, 2019

IMO .synthesize() would also colide with what the lower toolchain layers claim to do.
.elaborate() maybe?

One other thing that I noticed when trying to port the CORDIC in misoc to nmigen, that may tie into this refactoring: Subclassing a core in nmigen seems hard or impossible. Previously in migen, I'd just implement a core wrapping some functionality of another core by inheriting and then intercepting Signal()s as needed (see TwoQuadrantCORDIC vs the full CORDIC that remaps the quadrants).
This is now hard because I was unable to figure out a workable way to intercept/override/remap Signals in __init__ and at the same time properly overload and handle get_fragment() from the two classes. If this explanation is too convoluted I'll come up with a minimal example that shows what I'd like to do.

@whitequark
Copy link
Contributor Author

I don't like .elaborate() but I don't have any options I like so it's probably fine.

If this explanation is too convoluted I'll come up with a minimal example that shows what I'd like to do.

I think I understand what you mean but I'd appreciate an example as it would let me see a solution more clearly. I definitely acknowledge that subclassing may be more difficult in nmigen.

@jordens
Copy link
Member

jordens commented Jan 15, 2019

class And(Module):
    def __init__(self):
        self.i0 = Signal()
        self.i1 = SIgnal()
        self.o = Signal()
        ###
        self.sync += self.o.eq(self.a & self.b)

class Nand(And):
    def __init__(self):
        super().__init__()
        ###
        self.o, o_inner = Signal(), self.o
        self.comb += self.o.eq(~o_inner)

This could be done in a different way here. But the basic pattern of providing modified functionality through a similar interface by intercepting inputs and outputs is what I am looking for.
In nmigen I would have to own the inner And in submodules and then add i0, i1 as attributes to Nand (or forward them with jet more signals).

@whitequark
Copy link
Contributor Author

(I assume you meant to use super() above.)

Can you explain why this pattern is preferable to having a submodule? Is it just because you have to forward some of the signals in the wrapper, or is there a deeper reason?

@jordens
Copy link
Member

jordens commented Jan 15, 2019

No need to forward. It's less code, shorter and easier to read verilog. But more importantly Nand doesn't need to know anything about the input interface (how many and where the inputs are and how they are named). All those also apply to the original CORDIC case.

@whitequark
Copy link
Contributor Author

shorter and easier to read verilog

You can set f.flatten on a fragment to get the same Verilog, at least.

But more importantly Nand doesn't need to know anything about the input interface (how many and where the inputs are and how they are named). All those also apply to the original CORDIC case.

Is subclassing like that really the most appropriate way to solve this? What about having a single base class that computes an intermediate result, and two subclasses that bring out different final results? (Or a single class with an argument to the constructor, which is effectively equivalent, but arugably more unwieldy, especially with a larger amount of variants than two.)

That's how I would write this code in the first place in oMigen as well--using this pattern would not really occur to me, as it is IME not common in Python and it violates LSP.

@jordens
Copy link
Member

jordens commented Jan 15, 2019

If one of the subclasses is the identity (as in these two examples) that's a lot of low density mapping code again. And while LSP is maybe violated by the example above, it isn't in the CORDIC example where the four quadrant cordic strictly extends the applicability of the two quadrant cordic. But anyway LSP violation doesn't seem relevant to the question how to subclass here.
In addition to the signal interception question: How do you overload get_fragment()? Can I add more logic to whatever (Fragment or Module) I obtain from super().get_fragment()? Is .lower() idempotent?
I think there is a functional problem with the semantics of the Fragment/Module split. Can I add a fragment as a submodule?

def get_fragment(self, platform):
    m = Module()
    m.submodules += super().get_fragment(platform)
    return m.lower()

@whitequark
Copy link
Contributor Author

that's a lot of low density mapping code again

Is it? You have something like:

class And(Common):
    def __init__(self):
        super().__init__(self)
        self.o = self.intermediate

which does not seem to me like a lot of code. I.e. only the things that would be changed in the other subclass would need to be brought out.

I think there is a functional problem with the semantics of the Fragment/Module split. Can I add a fragment as a submodule?

Yes, fragments can be added as submodules.

In addition to the signal interception question: How do you overload get_fragment()? Can I add more logic to whatever (Fragment or Module) I obtain from super().get_fragment()? Is .lower() idempotent?

After this discussion I think that get_fragment() should return a Module i.e. not call lower. (This is actually easier than the current behavior.) So yes, you could extend the returned module in a subclass.

@jordens
Copy link
Member

jordens commented Jan 15, 2019

On the name, there are also .emit(), .refine(), .specialize(), .generate().

@whitequark
Copy link
Contributor Author

I think I like elaborate the most here, because it is clearly HDL-specific, even if the meaning is somewhat different from what Verilog has.

@jordens
Copy link
Member

jordens commented Jan 15, 2019

that's a lot of low density mapping code again

Is it? You have something like:
[...]
which does not seem to me like a lot of code. I.e. only the things that would be changed in the other subclass would need to be brought out.

I think it's super().__init__() without the self.
Yes. I meant those four lines of extra code. That could arguably be made smaller for complicated interfaces with many Signals by the use of Records. But to force the use of a Record just because it makes implementing an identity wrapper easier seems backwards.

I think there is a functional problem with the semantics of the Fragment/Module split. Can I add a fragment as a submodule?

Yes, fragments can be added as submodules.

That begs the question: Are fragments also modules? Functionally and/or semantically?

After this discussion I think that get_fragment() should return a Module i.e. not call lower. (This is actually easier than the current behavior.) So yes, you could extend the returned module in a subclass.

Yes. Getting access to the base class Module() in subclasses generally seems desirable.
Then if get_fragment() returns a Module, who should call lower()?

.modularize() or even .get_module()?

@whitequark
Copy link
Contributor Author

Yes. I meant those four lines of extra code.

I'm not convinced it's actually bad. If nothing else, it makes the signals that are being overridden explicit. I would like to see that written out when working on such code.

That begs the question: Are fragments also modules? Functionally and/or semantically?

Anything that is "fragment-compatible" may be added as a submodule. That is, anything with get_fragment() function. The only constraint on get_fragment() is that it should return another fragment-compatible object; during lowering, this is iterated to fixed point. That is how a get_fragment() function can delegate to platform.get_primitive() which might want to return a Module or even another fragment-compatible object.

This answers your other question as well, I believe.

@jordens
Copy link
Member

jordens commented Jan 15, 2019

Ok. Modules are fragment-compatible. But the naming of submodule is at least confusing as it indicates the reverse is also true (Fragment being added as a submodule).

@whitequark
Copy link
Contributor Author

Point. Suggestions for better naming?

@adamgreig
Copy link
Contributor

Why do we need both Module and Fragment? Given as you can always turn a Module into a Fragment, and you can use a Fragment anywhere you currently use a Module, could we just combine the two classes?

This would also solve the nuisance of currently being able to add ports to a Fragment but not a Module (so you need to frag = m.lower(); frag.add_ports(...); return frag, which would break just always returning a module from get_fragment() proposed above), and being able to set .flatten on Fragment but not Module, etc.

@jordens
Copy link
Member

jordens commented Jan 15, 2019

The naive .subfragments, .parts, .children. But let's take a step back.
Now we have all three (bare fragment-compatible "cores", Fragment, Module) being fragment-compatible but get_fragment() not returning Fragment in most cases. Yet, the return value is always submodule-compatible.

@whitequark
Copy link
Contributor Author

Why do we need both Module and Fragment? Given as you can always turn a Module into a Fragment, and you can use a Fragment anywhere you currently use a Module, could we just combine the two classes?

Module holds additional state that Fragment does not need and that cannot be correctly updated e.g. when merging Fragments during hierarchy flattening.

This would also solve the nuisance of currently being able to add ports to a Fragment but not a Module (so you need to frag = m.lower(); frag.add_ports(...); return frag, which would break just always returning a module from get_fragment() proposed above), and being able to set .flatten on Fragment but not Module, etc.

You should never manually call add_ports(). If you need that it is a bug elsewhere.

Now we have all three (bare fragment-compatible "cores", Fragment, Module) being fragment-compatible but get_fragment() not returning Fragment in most cases. Yet, the return value is always submodule-compatible.

We agreed to rename get_fragment() to elaborate(), which works well with this context.

@jordens
Copy link
Member

jordens commented Jan 15, 2019

We agreed to rename get_fragment() to elaborate(), which works well with this context.

Ok. The new terminology is then: Implementing elaborate(platform) makes something elaborate-compatible. Fragment, Module and typical "cores" are elaborate-compatible. Everything elaborate-compatible can be added to Module().xxx, where xxx is TBD.

@whitequark
Copy link
Contributor Author

SGTM.

@jordens
Copy link
Member

jordens commented Jan 15, 2019

Then I have the usual components, parts, children, pieces as group noun for the elaborate-compatible things, i.e. xxx.

@whitequark
Copy link
Contributor Author

components sounds best to me of all these options.

@adamgreig
Copy link
Contributor

It's a bit longer, but perhaps subcomponents?

@jordens
Copy link
Member

jordens commented Jan 15, 2019

Or just the shorter and slightly idiosyncratic sub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants