Add_custom_command - list of update dependencies by rebuilds
See latest status update
Initial conditions
- a code generator that generates a set of C ++ sources taking a single input file as a parameter
- the input file may contain other input files
-
already solved the problem of getting a list of output files, parsing the codegen input files to get a complete list of codegen input files. That is, add_custom_command is provided with the correct set of dependencies the first time:
add_custom_command(OUTPUT ${generatedSources} COMMAND ${codegenCommand} ARGS ${codegenArgs} DEPENDS ${codegenInputFiles})
Problematic scenario
- The current system works well, unless someone modifies one of the codegen input files to include a new input file or delete an existing one. In this case it is required to update the list of codegen input files provided by add_custom_command as dependencies, but I have no idea how
What is missing
- ability to update add_custom_command dependencies from project rebuilds
Is there a way to solve this problem without completely rebuilding the project?
UPDATE - description of an alternative (better?)
I found a similar unanswered question on the cmake mailing list, post it here for better clarity: http://article.gmane.org/gmane.comp.programming.tools.cmake.user/52279
I am trying to get the code generation tool to behave "exactly like" the C source file regarding dependencies. By that I mean, suppose you have a C file "ac". Since it can # include files, every time the content a.c
changes, its dependencies can also be changed. Dependencies get rescanned with -MMD. I would like to emulate this in some way for my code generator. I first tried add_custom_command, which takes a fixed list of DEPENDS defined during the custom command definition. Specifically, I mean something like this:
function(add_generated_library)
figure_out_dependencies(deps ${ARGN})
add_custom_command(... DEPENDS ${deps})
endfunction()
But this only fixes the dependencies in the generation time of the system. Each time a custom command is run, the DEPENDS list may need to be changed as changes may mean new dependencies. How should I do it?
UPDATE 2 - Possible solution
After I review the facts - there are voices on the internet regarding dynamic cmake support for dependencies that are needed to seamlessly integrate many non-trivial code generation tools - there is no out-of-the-box optimal solution as we really need to hook up custom DSL support to IMPLICIT_DEPENDS
From the cmake manual:
The IMPLICIT_DEPENDS option requests a scan for the implicit dependencies of the input file. The specified language specifies the programming language in which the corresponding dependency scanner should be used. Currently only C and CXX scanners are supported. The language must be listed for each file in the IMPLICIT_DEPENDS list. Dependencies found during scanning are added to the custom command settings during build .
The solution below adheres to (hopefully) the following criteria:
- avoid having to scan for dependencies when rebuilding
- avoid unnecessarily running the code generator when rebuilding
- allows you to provide clients with cmake functions to register their models and generate code / build libraries from that code without imposing any requirements on the project structure (i.e. the subproject responsible for generating the code, the models are distributed across the project hierarchy using a policy, specific to the project).
Solution idea
You cannot register your own language scanner, but you can reuse an existing one. The idea is that the dependencies / hierarchy of custom model files is reflected as a hierarchy of "C" header files. Each node hierarchy is added when the model file is registered, and the C file includes a matching model file. If the model file includes a change, the C file includes the change. Thus, each call-code will depend on only one generated C-header, reflecting the model traversed. Each reflected file will have a dependency on the model file and will be affected by modifying the model file.
To summarize: maybe my wording is not so clear at the moment, but regarding the needs of other people and the community helped me research this issue, I will post a generic solution (+ link to github or new cmake wiki) without my project spec when it will be ready (in 1-3 days).
source to share
Can you show how you initialize a variable codegenInputFiles
? Perhaps you can use the command file(GLOB ... )
or file(GLOB_RECURSE ... )
. See documentation .
Note that you will need to rerun cmake for your command to be generated. Are you working with git? Then you can have a hook that forces cmake to be called every time you pull (so that if someone changes codegenInputFiles
, your auto-generated files will be updated).
After clearing the problem, you should be able to find a workaround using IMPLICIT_DEPENDS
instead DEPENDS
. Limitations:
- It can only work if your input file is C / C ++ (check the syntax as you have to specify the language for each file specified)
- You may need to check if your version of cmake supports this command, although it looks like it has been around for a while.
- It's only supported for the Makefile generator, which sounds pretty bad ...
EDIT
After some iterations, I finally figured out what the problem was. I suggest the following solution: separate the file creation in a separate cmake subproject. When you create your main project (by calling make), you will call both cmake and do for your subproject. A call to cmake is required to update the updated dependencies, while calling make to actually build your auto-generated sources.
Here I am showing an example of a project and sub-project with a project calling cmake and building for the sub-project.
Composition:
.
├── CMakeLists.txt
├── a.cpp
├── build
└── subProject
└── CMakeLists.txt
File contents
./CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
add_custom_target(subProjectTarget ALL)
add_custom_command(TARGET subProjectTarget PRE_BUILD COMMAND mkdir -p ${CMAKE_BINARY_DIR}/subProject && cd ${CMAKE_BINARY_DIR}/subProject && ${CMAKE_COMMAND} ${CMAKE_SOURCE_DIR}/subProject && make)
include_directories(${CMAKE_BINARY_DIR}/subProject)
add_executable (dummy a.cpp)
add_dependencies (dummy subProjectTarget)
./a.cpp (note that bh doesn't exist yet)
#include "b.h"
int main () {
}
./subproject/CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
file(WRITE ${CMAKE_BINARY_DIR}/b.h "//I am a dummy file\n")
Create a project (using the default make
)
me@here:~/example/build$ cmake ..
-- The C compiler identification is GNU 4.8.2
-- The CXX compiler identification is GNU 4.8.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/me/example/build
me@here:~/example/build$ make
Scanning dependencies of target subProjectTarget
-- The C compiler identification is GNU 4.8.2
-- The CXX compiler identification is GNU 4.8.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/me/example/build/subProject
[ 0%] Built target subProjectTarget
Scanning dependencies of target dummy
[100%] Building CXX object CMakeFiles/dummy.dir/a.cpp.o
Linking CXX executable dummy
[100%] Built target dummy
Note that the second time the cmake calls are in the subproject.
Still faster on next call:
me@here:~/example/build$ make
-- Configuring done
-- Generating done
-- Build files have been written to: /home/me/example/build/subProject
[ 0%] Built target subProjectTarget
Scanning dependencies of target dummy
[100%] Building CXX object CMakeFiles/dummy.dir/a.cpp.o
Linking CXX executable dummy
[100%] Built target dummy
(Although here the bh file is written every time causing a.cpp to recompile)
This stub can be greatly improved by using cmake commands to generate output directories (not mkdir) and cascade the generator chosen for the main project (I'm assuming everything uses make here)
Let me know if you need further clarification.
source to share
I think I ${codegenInputFiles}
should contain a list of hard-coded source files and include the files needed for the custom command. The file (GLOB ...) documentation says:
We do not recommend using GLOB to collect a list of source files from your source tree. If the CMakeLists.txt file does not change when a source is added or removed, then the generated build system cannot know when to ask CMake to regenerate.
The hard work (which we pay for) is keeping it up ${codegenInputFiles}
to date (causing a complete rebuild of the project). Anyway, you'd have a similar problem if someone created a new source file and didn't add it in ${codegenInputFiles}
, right? So I believe the additional dependency on the include file should be handled the same way.
source to share