The Persnickety Order of Self Serving Modules

This one caught me off guard.

  irb(main):001:0> module M
  orb(main):002:1>   def x; "x"; end
  orb(main):003:1> end
  => nil
  orb(main):005:0> module Q
  orb(main):006:1>   extend self
  orb(main):007:1>   include M
  orb(main):008:1> end
  => Q
  orb(main):009:0> Q.x
  NoMethodError: undefined method `x' for Q:Module
          from (orb):9
          from :0
  

Why isn't the included #x coming along for the ride in the self extension of Q? This kind of dyanmicism is vital when dynamically loading behaviors. And so I suspect it must do with the dread Dynamic Module Inclusion Problem? And it would appear that I am right:

  orb(main):012:0> module M
  orb(main):013:1>   def x; "x"; end
  orb(main):014:1> end
  => nil
  orb(main):015:0> module Q
  orb(main):016:1>   include M
  orb(main):017:1>   extend self
  orb(main):018:1> end
  => Q
  orb(main):019:0> Q.x
  => "x"
  

Yes, another of the edge cases. But the preponderance weighs heavy on the Coding Spirit. The Dynamic Module Inclusion Problem is getting old.

Main Campaign "Out of Object Now!"

The word from Matz on Kernel as toplevel object:

  I don't feel that making Kernel as toplevel self is not a good idea,
  because:

    * toplevel def does not define methods on Kernel, but Object.
    * toplevel include does not include modules into Kernel, but Object.
    * toplevel private etc. do not work on Kernel, but Object.

I wouldn't call that an explanation exactly, more an explicative of the current behavior. I'm sure Matz has his reasons, and we can just assume that he wants to keep the namespace distinct. Fair enough, and of course he can do that. (It's not REALLY a democracy after all!) So Kernel drops out of the race. But the Kernel's running mate, Main, is still here campaigning.

You might be surprised to learn (as I was when I first discovered it):

  Object.public_instance_methods(false) +
  Object.private_instance_methods(false) +
  Object.protected_instance_methods(false)
  => []

That's right. There's not a single method defined in Object. All the methods ri tells you belong to Object actually are inherited from Kernel. But from there, any toplevel method we define does end-up in Object. Hence the clear separation of namespace I mentioned above.

Now a separate module, eg. Main, would do just as well. Kernel need not be used. And as I've expressed before, Main could be induced into Object for all the same effects. The base hierarchy then being Object < Main < Kernel.

But wait a second! Why are all these toplevel methods sneaking into all my Object's anyway? I can just as easily add them to Object myself if that's what I want. I don't need some cheap toplevel proxy to do it for me. In fact, that can be a problem too.

  module Foo
    def self.method_missing( name, *args )
      super unless require "foo/#{name}" rescue nil
      send( name,*args )
    end
  end

Then some unsuspecting nuby comes along (okay I admit, it was I and it happened to me today!) and innocently adds to the top level:

  def check( name )
    name == "foo"
  end

Well, so much for my lazily required Foo.check routine. It's been whacked from the top down!

You see where I'm going now? Primaries are over Main has taken Kernel out of the running with a new divisive platform. "Out of Object Now!"

Vote Kernel for Toplevel Object

As a follow up to my last post on the "pain that is main", I want to offer a potential improvement for Ruby 2.0. I approached the topic on ruby-talk this week and while Matz initially took some interest, he hasn't followed up since his last comment:

  matz: Why?  If it is really required it's fairly
  easy to add toplevel methods like we did for #include.

But it isn't always easy. In order get my Taskable module to work, for instance, I had to make exceptions for the toplevel case, which is far from ideal and is fragile [Ed- in fact I'm still getting bugs that I haven't yet pinned down]. These subtile difficulties arise becuase main acts as a partial proxy for the Object class. Anyone who has created a proxy object before knows the subtile issues that can come into play. In this case, only the bare minimal interface is supported --essentially the method #include. Yet, even if we take matz' advice and add in all the missing proxy methods, we still won't be 100% out of the woods. The Object class and main are fundamentally two distinct objects --self is not the same, nor are their singleton classes, &c. In the vast majority of cases this will never present an issue, but the distinction can creep in. Here's an highlight of one way it can:

    module Q
      define_method :q do
        base = self
        define_method :r do
          base == self.class
        end
      end
    end

    class C
      extend Q
      q
    end

    c = C.new
    p c.r

    # per matz' direction

    def define_method( name, &act )
      Object.class_eval {
        define_method( name, &act )
        private name
      }
    end

    extend Q
    q
    p r

 produces

    true
    false
  

So in effect Ruby is mildly schizophrenic. The false reading is because main != Object. So, you can't neccessarily create a DSL for Object to be used in main, and you can't neccessairly create a DSL for main to be used in any Object. Hence the devolution to DRYless code.

There is a potentially elegant solution however, and I'd really like to understand others insights into this (esspecially Matz' of course): Instead of main being a special proxy object, just let it be a self extended module.

    module Main
      extend self
      # programs are written as if in here
    end
  

This would provide all the facilities required of the toplevel without all the proxy troubles. Also, while I'm not so convinced of the merits of every toplevel method becoming a private method of all objects (and with Main that can be easily prevented), it has proven workable in practice so it's not a significant factor of consideration here. Main can simply be include in Object to achieve that effect.

    class Object
      include Main
    end
  

But when we do that it becomes very clear what Main appears to be: Kernel. That strikes me as esspecially interesting. Then again, there may be good reasons to keep the Kernel as a separate module, in which case we'd just have a class hierachy:

    Object.ancestors
    => [Object, Main, Kernel]
  

Nevertheless, it is clear the Kernel could just as well serve as the toplevel object, which, IMHO, makes this an elegant proposition to consider. Perhaps I'll start a campaign as November elections roll around: "Vote Kernel for Toplevel Object!" ;-)

Main is a DRY Pain

I have a module that depends on define_method and ancestors. It works great when I include it in other modules or classes. But if I try including it into the toplevel it fails miserably.

    NameError: undefined local variable or method `define_method' for main:Object
  

That seems fairly peculiar when you consider that defining methods at toplevel is perfectly acceptable. One must then wonder, what is this toplevel thing anyway?

    self  #=> main
  

Okay, it calls itself "main". Great. But that doesn't really tell us anything. Let's check it's class:

    self.class  #=> Object
  

Ah. So it's an instance of Object. An instance of Object!? How is that possible? A normal instance of Object and main can't be exactly the same. Indeed, they are not.

    (public_methods - Object.public_instance_methods).sort
    => ["include", "private", "public"]
    singleton_methods
    => ["include", "private", "public", "to_s"]
  

Notice include has been defined here specifically for main. So something special's going on when you include a module into the toplevel. Hmm... What about methods defined at the toplevel? If main is an instance of Object then are they singleton methods? Well, no. Turns out they get "magically" defined as private methods of the Object class itself, and main's singleton class space is actually something else entirely.

    class << self
      def x; "x"; end
    end

    class Q
      def q; x; end
    end

    Q.new.q
    => NameError: undefined local variable or method `s' for #<Q:0xb7ce9a3c>
  

Which means the sure solution for my problem...

    module Kernel
      include MyModule
    end
  

Can you guess?

    module M
      def m; "m"; end
    end

    module Kernel
      include M
    end

    m
    => NameError: undefined local variable or method `m' for main:Object

    class Q
      def q; m; end
    end

    Q.new.q
    => NameError: undefined local variable or method `m' for #<Q:0xb7cbb324>
  

Lands me squre in the face of the Module Inclusion Problem.

All this leads me to two points. First, I'm stuck! There seems to be no solution to my problem other than rewritting a second version of my module specifically for the toplevel. Talk about lack of DRY! And 2) Why in the world isn't main a self extended module?