Wrapping macro and conditional compilation semantics

[haxscramper]

C macros can do anything - create types, procs, conditionally enable parts of the code or define simple constants. All of this has to be somehow converted to nim macros if possible. Boost wave should be pretty useful for solving this problem.

• https://www.reddit.com/r/nim/comments/pti32g/comment/hdwxzaq/?utm_source=share&utm_medium=web2x&context=3
• https://irclogs.nim-lang.org/29-09-2021.html#13:11:16
• https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=1494&context=sis_research
• https://www.cs.cmu.edu/~ckaestne/pdf/tse17-refactoringifdef.pdf
• https://ruche-home.net/program/boost/wave#hooks-member-expanded-macro
• http://mbeddr.com/files/tomassettiratiu-extractingvariabilityfromcandliftingittombeddr.pdf

Many macro definitions are relatively simple, which means there are multiple steps of different complexity:

• Wrap standalone macro constants like #define GIT_CLONE_OPTIONS_VERSION 1
• Wrap group of macros that should've been an enum - Automatically generate boilerplate code that makes wrappers easier to use #10
• Convert macros to templates #define get_field(arg) (arg)->name
• Map macro semantics to the nim macro - need to find concrete examples of where this is done and how to meaningfully port this to nim. #define MAKE_STRUCT(name) struct name {};. Maybe Qt macros from Qt support #25 would be a good starting point?
• Macro can be hidden inside of the ifdef section, this should be accounted for as well. Need to get the AST of the macro preprocessor

[haxscramper]

https://code.woboq.org/gtk/include/glib-2.0/gobject/gtype.h.html#_M/G_DECLARE_INTERFACE

[haxscramper]

Algorithm description from the IRC discussion -

I think the only way to properly map conditional compilation and related logic is to try and get the proper /AST/ of the macro, or #ifdef/elif/ifndef/if check. Parsing preprocessor statements is easier - I already have a boost-wave based lexer that can report all the tokens
encountered, assembling elements to the tree structure is not that difficult. The main problem is a macro defines themselves - if the replacement list does not contain any special metaprogramming tokens (## and # for concatenation and stringification respectively) it could be parsed directly using tree-sitter. This functionality is related to the #12

If the macro contains concatenation tokens it becomes somewhat more problematic, but those case be handled by replacing <tok> ## <tok> with specially constructed token names like TOK_INDEX_1_CONCAT_TOK_INDEX_2`.

But we still to operate under the assumption that the token body forms a valid C code, which is of course not the case.

• If the expanded body contains a completely invalid entry, like #define unless(epxr) if (!(expr)) I don't think we can (or should) try to port it properly.
• If an expanded body contains a call to another macro, we might treat it differently.
  • If the expansion is currently not valid, but macro expansion would make it correct, we try to analyze a semi-expanded version (do one step, try expansion again).
  • If the expansion is valid, we wrap things as regular macro - SOME_MACRO_CALL() is translated as-is.

[haxscramper]

It is necessary to overlay conditional compilation on top of the API description. That is - CxxEntry should itself might information about conditional compilation. This would make type graph construction a lot more complex, since various groupings must be considered at once in order to deal with mutually recursive type uses wrapped in conditional compilation logic.