Background

Idiomatic Fan makes heavy use of method chaining. We've designed most of the APIs to return this to enable this style of programming. The problem is method chaining with subclasses returns the base class type. For example consider this use case:

fansh> out := web::WebOutStream.make(Sys.out)
fan.web.WebOutStream@d6c16c
fansh> out.p.print("hello").pEnd
ERROR(22): Unknown slot 'sys::OutStream.pEnd'
fansh> out.p.print("hello")->pEnd
<p>hello</p>

The problem is that OutStream.print declares that it returns a OutStream, not a WebOutStream. But since the definition of the method is to return this we actually know that it is a WebOutStream. Another use cases which I've postponed is enhancing the Obj class:

class Obj
{
  Obj dup()
  Obj toImmutable()
}

These methods are declared on List and Map, but not for all objects. One solution is to override all those methods with a covariant return type - but that is a ton of useless boiler plate work, not to mention it isn't very DRY. You can also use the "->" operator to drop into dynamic calls but that is a fairly big performance hit.

This Type Syntax

So I propose enhancing the type system with this types:

// old way
Obj dup()
OutStream print(Obj x)

// new way
this dup()
this print(Obj x)

The keyword this would be used as the return type. It is a little confusing because we aren't necessarily saying we are returning this, only that we are returning "this type". So I'm not thrilled with the syntax - originally I was thinking of using T similiar to the generic parameters used by List, Map, and Func. But this is a bit different since it spills out of the sys pod into all the pods. Any other syntax ideas?

The this type will only be usable in method return types which keeps things covariant. It cannot be used in fields and method parameters, which would introduce contravariance.

Interestingly enough the top google hit for "self types" is Stephan's blog :-)

Reflection

The Method reflection API will report the declared return type, but will be enhanced with a new returnsThis method:

m := WebOutStream.type.method("print")
m.qname       => "sys::OutStream.print" // don't use new qnames for covariant methods
m.returns     => OutStream.type         // returns declaring type
m.returnsThis => true                   // tells us that 'this' was used as return

FCode

We have to enhance the fcode to include the new meta-data about this types. This will be done via a new flag used only by method definitions. Consider the existing fcode method format:

method
{
  u2 name  (names.def)
  u4 flags   
  u2 returnType (typeRefs.def)
  u2 inheritReturnType (typeRefs.def) // for covariance
  ...
}

Unlike normal covariant methods, we won't actually be generating a new implementation for every subclass. So this typed methods will declare a normal returnType, and have a null inheritReturnType. If a subclass actually overrides a this typed method, then normal covariance will apply.

Since the CLR doesn't support covariance, the Fan compiler already implements covariance by inserting an implicit cast operator. This types will work the same way - the compiler will detect use of a this typed method and insert a cast opcode as necessary. So there shouldn't be any changes needed to the Java and C# runtimes (other than support for the new reflection method).

I like that - it will make designing and using API's much easier. I'm not 100% crazy about the syntax either - but its fairly intuitive, reads well, and not worth taking a new keyword over IMO.

What about capitalizing the word?

"This" could be more easily recognizable as the Type (in contrast with "this", an instance of the type).

In addition:

• It would be harder to misunderstand your intention (you don't need to return "this" but some object of the same type).
• You wouldn't overload the meaning of the keyword "this".

The problem with "This" is that it is just an identifier (which is a problem for anything which isn't a keyword or symbolic). So the question then becomes: is that an identifier with special restrictions? If I can't use anywhere, then it effectively becomes a keyword - but it seems a bit weird to have a keyword which differs from all the others by only capitalization. Then again if it is an identifier with special rules, that has its own complications. Between the two I would probably go for "This" as a keyword versus an identifier with special semantics. Any other votes for This versus this? Any other ideas?

What about something like this.type - its still a little weird though. But I think I still lean towards simply this.

I don't particularly like the idea of reusing the this keyword. I think Type would be fine, except that I agree that it is a little weird to have a capitalized keyword. So how about just type?

I agree that this is a very useful language feature.

At Javapolis 2007, this came up, and it is possible that it may make it in Java 7. The favoured syntax discussed there (with Neal Gafter) was This. The argument was that the capital allowed it to be thought of as a type. I believe that it would be modelled in Java in the same way as the Void class - a singleton - rather than a keyword.

Having said that, I quite liked the original idea of using T. I've always thought of self types as a form of generics (in broad terms, they substitute the for another type).

In addition, has there been any thought of self-types as method parameters? They come in useful if you want to write an abstract superclass for lots of immutable subclasses - something that may be common in fan. The rules involved are complicated however. It would have to be that the method definition on the superclass can't be invoked - it would simply be a template that is actually callable in each final subclass.

Actually I think that makes a lot of sense to treat sys::This just like sys::Void - just a marker class that never gets instantiated. Then it is just an normal identifier which follows standard type resolution rules.

With that approach then reflection is a bit different:

m := WebOutStream.type.method("print")
m.qname   => "sys::OutStream.print" // don't use new qnames for covariant methods
m.returns => sys::This.type         // works like void

And I wouldn't even create the Method.returnsThis method.

I like that - it is cleaner and more elegant than using a keyword.

The problem with self-types for method parameters is that they are contravariant in which case subclasses don't fulfill the contract of their base class. I agree this happens in the real world sometimes, but in my experience not near as often as the covariant case. The prototypical cases are Obj.equals and Obj.compare (assuming we don't want to allow those methods to be used with unmatched types). Not to mention it is a pain to always check instanceof. I've tried to think about how I would do that and keep the Java verifier happy - most likely generating a wrapper method which calls a strongly typed version with appropriate casts. So I think that would be cool, but it needs a lot more thought.

A covariant type in a contravariant position would be a serious bug in the type system.

class Base {
   void method(This arg)
   {}
}

class Derived : Base {
   void method(This arg)  // using covariant type in contravariant position
   {
      arg.otherMethod();  // allowed, but shouldn't
   }

   void otherMethod()
   {}
}


Base foo = new Base();
Derived bar = new Derived();
bar.method(foo);             // Error! Base.otherMethod does not exists

Method argument types are contravariant Method return types are covariant Assignable fields are invariant Read-only fields are covariant

If there would be something like write-only fiels (doesn't make sense IMO) they would be contravariant.

Regarding equals and compare: The only solution I can think of is removing them from Obj and rather define generic interfaces.

class MyComparableClass : IComparable[MyComparableClass] {  
    int compare (MyComparableClass other)
    {
        ...
    }
}

So you cound define with which types you can compare:

class ComparableWithStringFooAndItself 
    : IComparable[String]
    , IComparable[Foo]
    , IComparable[ComparableWithStringFooAndItself]
{  
    int compare (String other)
    { ... }
    int compare (Foo other)
    { ... }
    int compare (ComparableWithStringFooAndItself other)
    { ... }
}

As you don't like parametric polymorphism I'm not sure whether this is an option. But you could make IComparable and IEqual special cases just like Map, etc.

wrt

Base foo = new Base();
Derived bar = new Derived();
bar.method(foo);             // Error! Base.otherMethod does not exists

The point is to setup the system such that bar.method(foo) cannot compile. THis is what I meant by my comment that This method arguments should only be callable on non-subclassable subclasses. In your example, you could write method() in Base together with an abstract otherMethod(). But you could never call method() on anything other than a non-subclassable subclass - its just a template to be copied to the subclasses.

I think we are basically in agreement that covariant This returns are a good thing and belong in the language. I'm going to proceed with the sys::This design and see how it works out.

I don't think we are ready to tackle This for method parameters unless we clearly understand how we make them work without being contravariant. Stephen - I'm not sure I understand what you are proposing by "templates". Maybe you are talking about some kind of mixin which provides implementation reuse, but isn't a polymorphic type? That sounds like a bigger feature (maybe capture in another thread)?

This feature is complete. It was really only one extra step in CallResolver, plus a couple dozen lines of new error checking. Use of sys::This is restricted to non-static method return types.

Tomorrow night I will update the docs and go thru the libraries and make appropriate changes to the APIs. Then I'll probably post another build.

"The point is to setup the system such that bar.method(foo) cannot compile. THis is what I meant by my comment that This method arguments should only be callable on non-subclassable subclasses. In your example, you could write method() in Base together with an abstract otherMethod(). But you could never call method() on anything other than a non-subclassable subclass - its just a template to be copied to the subclasses."

OK, I'm not sure if I get you right but assuming I did it seems like you could do something similar by self type annotations in Scala.

But the way you describe your feature seems rather limiting.

interface Foo {
    void method (This that);
}
class Bar : Foo {             # class now sealed just because it implements Foo
    void method (This that)
    { ... }
}

I still think using a covariant type in a contravariant position is error. You can design strange very specialised rules like you did to make it sound. But that would be rather confusing. Or you could make it sound by defering type checking until runtime like Java's covariant arrays. (AHRG! Such a big design flaw especially in combination with covariant generic types plus type erasure; OK, sorry I have to stop ranting about Java.)