c2ffi useful?

[snunez1]

c2ffi has the ability to output SEXPs, and to add new drivers. Perhaps the path of least resistance lies in modifying c2ffi for our purposes? The clang and parsing work is already done and it seems that the output driver updates may be all that's required.

Edit: c2ffi is currently LGPL, but the author has suggested on the README that he's open to changing this.

[xrme]

I completely respect the work done in c2ffi.

We did discuss trying it out it in Clozure/ccl#13.

I admit that it's possible that I have a little bit of "not-invented-here" syndrome. But I genuinely do place a high value on libclang's stable interface. I also feel that the process of generating the interface databases is an important part of building CCL (and one that has been neglected), so I want to be able to keep it under control.

This code generated the FFI files in the current release of CCL for FreeBSD, so we're already past the proof-of-concept stage. I'm confident that other platforms will be able to follow.

[snunez1]

What exactly is the 'high value' on libclang's stable interface? Have there been any known failures? I agree that stability and of the importance of the interface database, but this definitely sounds like an NIH problem.

I'm not trying to disparaging the opinions expressed on this topic, but if I were wearing a Product Manager hat here, you'd have to do better than that to convince me. Some initial thoughts:

• I can't see any specific demonstrated advantage in using libclang vs c2ffi's approach
• Do we think that c2ffi is going to leave bugs unfixed? At the point of writing an output driver, we're in control
• Do we have the resources to maintain a duplicate wheel of C/C++ parsing? c2ffi is keeping up with all the latest updates in clang already.

I can totally see from a developer perspective why the current path is desirable. But, from an overall project perspective, long term, is this really the best design choice? To me, write a c2ffi output driver and be done with it. Outsource support and development of parsing to someone that's already volunteered and is maintaining that functionality.

[xrme]

https://clang.llvm.org/docs/Tooling.html talks about the various interfaces one might use when looking for syntactic and semantic information about a program. They basically say, "use libclang unless there's some reason that you can't".

c2ffi is willing to keep up-to-date with changes to llvm/clang interfaces written in C++. This means that it is tied strongly to llvm major versions. That's why you have to use the correct branch of c2ffi for the version of llvm that you have installed.

By contrast, because libclang's interface is stable, the version of the installed llvm doesn't matter much.

Dealing with Objective-C header files will be important for CCL. The README file for c2ffi says that basic Objective-C support exists, but that the c2ffi developer doesn't really use or test it. I'm sure he'd accept patches, but it's not clear to me who would learn the necessary C++ programming interfaces in order to write such patches if they should be needed.

Like c2ffi, we are already outsourcing the parsing to Clang (just via a different interface), so there's no question of keeping up with anything in that area.

I believe that the approach we're taking in the code in this repository is reasonable and suitable for CCL.

[snunez1]

I think we're talking at cross-purposes here. I'm not questioning the choice of libblang, although that choice is debatable. My point is one strictly of resource allocation , which is the key to successfully driving a product forward.

With CCL barely surviving on it's newly established life-support (your new availability), I question whether reinventing this wheel is worth the cost when an alternative exists? Maybe libclang is better, but c2ffi is here now, supported and needs only ObjC, which the author will merge.

If there's some magic in a libclang implementation that is required to make ObjC work, fine, we'll have to deal with it. But if c2ffi can be made to work, at lower cost, we really need to be asking ourselves: "what do we get for all this extra work when there's a long list of other high-value things we could be doing". From that perspective, it's not clear that this approach is 'reasonable and suitable for CCL' at the present moment.