Re: Tips to compile very very giant code

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, 2025-02-12 at 09:25 +0100, Florian Weimer via Gcc-help wrote:
> * Bento Borges Schirmer:
> 
> > [3]
> > https://github.com/bottle2/swf2c/blob/88f9ccb7912d55002e87f1efb11f21720d97e4ec/tests/thousands-of-functions.c
> 
> You should turn L and B into proper functions instead of macros, then
> compilation time will decrease significantly.  If compilation time is
> still too high, consider adopting a table-based approach.


And your compiled C or C++ code should preferably be made of translation units
(C or C++ files) not bigger than about ten thousands lines each.

Observe that the C++ code of GCC don't have any source file bigger than 60KLOC
(the biggest one being ./gcc/cp/parser.cc and ./libstdc++-
v3/testsuite/20_util/to_chars/double.cc ...) and that generated C++ code (e.g.
_GccTrunk/gcc/insn-dfatab.c ...) has at most 210KLOC.


See also https://arxiv.org/abs/1109.0779 and (if you want to generate C code
which is huge to benchmark compile time) 
https://github.com/bstarynk/misc-basile/blob/master/manydl.c

My advice is to refactor your human written C or C++ files to have no more than
ten thousands lines each. (with C++ templates the compilation time can still
take dozen of minutes in pathological cases).

In some cases (see GCC or Firefox source code for examples) you want to generate
C or C++ code from higher-level files (e.g. using your generator or GNU bison or
ANTR). In those cases ensure that the generated C or C++ code is not too big or
is split in several files.

Then for development and debugging you compile with gcc -O2 -g2

Once you have tested enough the executable and need to benchmark it use link-
time optimizations, so compile and link with gcc -flto -O3 ; the build time is
then more than doubled (and the performance might increase by perhaps 20% if you
are lucky in coding something that GCC optimizes well).

You could also focus your development (and human refactoring/optimization)
efforts on those routines which are profiled to consume most of the CPU runtime.
You will need to learn about profiling and profiling guided optimizations.

At last, be sure to compile on a powerful computer. With as much as RAM and
cores you can afford (and SSD disks). Use make -j for parallel building.

Don't forget to backup your human written source code on external devices. These
are the most costly (to develop) things, and you need to backup (probably
remotely).

Regards.

-- 
Basile STARYNKEVITCH           <basile@xxxxxxxxxxxxxxxxx>
8 rue de la Faïencerie
92340 Bourg-la-Reine,          France
http://starynkevitch.net/Basile & https://github.com/bstarynk 




[Index of Archives]     [Linux C Programming]     [Linux Kernel]     [eCos]     [Fedora Development]     [Fedora Announce]     [Autoconf]     [The DWARVES Debugging Tools]     [Yosemite Campsites]     [Yosemite News]     [Linux GCC]

  Powered by Linux