#46 Peer Design for Native Integration

brian Thu 2 Mar 2006

Let's layout the concrete design and plan for how to approach native integration using peers. My ASCII rendition of how Java native integration would work:

       fanc
.fan ------>  .pod/.fcode <---\
                |             |
                | jstub       |
                |             |
                V             |
             .jar/.class ---->| makenative.rb
                |             |
                | javac -cp   |
                |             |
       javac    V             |          
.java ------> .class ---------/       

Hopefully C# would work very similar, except we don't know what serves for the .class file - if we can break individual IL types out of the assembly. Now let's look at some example code for a type which uses native methods:

// Fan class
class Graphics 
{
  native Void fillRect(Int x, Int y, Int w, Int h)
} 

// Result of jstub tool (this would be generated directly to a .class file)
class Graphics
{
  Graphics() { peer = GraphicsPeer.make(this) }
  Void fillRect(Int x, Int y, Int w, Int h) { peer.fillRect(this, x, y, y, z) }  
  public final GraphicsPeer peer;
}

// Hand-coded native peer class
class GraphicsPeer
{
  static GraphicsPeer make(Graphics g) { return new GraphicsPeer(); }
  void fillRect(Graphics g, Int x, Int y, Int w, Int h) { awt.fillRect(x.val, x.val, w.val, h.val); }
  java.awt.Graphics awt;
}

Rules for native peers:

  • Must have static make method; you can return a singleton for all Fan instances, or create a peer instance per Fan instance. If you don't have any static methods you can just return null.
  • Instance methods in Fan map to an instance method on the peer with an explicit self parameter
  • Static methods in Fan map to a static method in the peer using the exact same signature

The proposed directory structure for modules with native code:

moduleName/
   fan/
     Foo.fan
     Bar.fan
   java/
     FooPeer.java
   net/
     FooPeer.cs
   temp/
     working files

If we do that for modules that have native code, does that mean we should always have a fan subdirectory for modules that are just pure Fan? What do you guys think?

Tasks to implement this feature:

  1. Need jstub and nstub tools. For now we can just roll them as new mains into fan.jar and fan.exe.
  2. Need support for native in compiler/fcode
  3. Need support for native callouts in Java and CLR runtime emitters
  4. Need native compiler tool which generates appropriate stubs, calls compiler with correct stub dependencies, rolls native code back into pods. For now I think these tools will just be Ruby scripts (until we get our Fan DSL build tools flushed out).
  5. Need support in emitters to use pre-compiled code if available

I think the implementation is pretty straight forward - just 4. is a bit tricky. John needs to figure out the big issue with how C# is going to work regarding partial stubbing, compiling, and rolling back into the pod. I think I should be able to implement the Java side of things pretty quickly.

andy Thu 2 Mar 2006

I think that looks good (though there is some ambiguity in your native fillRect method with the g variable). I like the directory structure - I think its always the same even if its pure fan.

brian Sat 4 Mar 2006

Native integration for Java is implemented and checked into StarTeam as designed. Couple of notes:

  • Use the makenative Ruby script to make a module's native code. It calls jstub to generate the stubs, calls javac, and rolls the changes back into the pod using the JDK jar tool. I left all the hooks for John to add C# integration into the same script
  • Since we don't capture depends, I just put sys and your own module into javac classpath for now
  • You are guaranteed that the Fan constructor has completely run before Peer.make gets called

Checkout the sysTest::NativeTest code and it's respective peer to see how it all works. It illustrates making a peer, instance methods, static methods, and how you can access the peer off the Fan class in your native code:

makej
fan sys\sysTest
makenative -j sysTest
fant sysTest::NativeTest

We can make the build more elegant once we switch the build system over to Fan itself.

So Andy you can start writing Fan code with native Java integration!

brian Sat 11 Mar 2006

I originally had the peer factory being called after the constructor, but for performance reasons related to subclassing I moved peer creation as the very first thing that occurs in object initialization (before field init and constructor code). After some thought, that's probably also the design that affords the least surprise.

Although in adding that scenerio to the test suite, HotSpot crashes during classfile verification - I can't for the life of me figure out why since that same code works else where (I even took out all the native methods).

Login or Signup to reply.