I believe the end-result will convert a substantial fraction of GCC’s functions into being methods, so this is going to touch a lot of code.
I’m proposing numerous changes here, but I believe each one can be broken down into self-contained patches.
A few of the changes touch numerous source files. I’m not sure if this work can sanely be done on a branch: merging would be too painful.
I intend to never break the build: an evolution of the code towards statelessness, without big “flag days” or other stuff that makes the tree unconsumable.
Hence I would prefer that this work go into trunk (targeting 4.9).
However I appreciate:
- I’m a relative newcomer to GCC development (I’m the author of the Python plugin for gcc, and have been working on it for about 2 years), but I’ve only been working on GCC itself for a couple of months.
- There are performance concerns about these changes.
So what’s probably more palatable to the GCC core maintainers is a hybrid:
- low-risk patches go to trunk
- higher-risk patches to to a branch
- split up the work to try to keep the “merge delta” reasonably low: some of the changes to trunk will not make sense except as enabling work for changes to the branch.
One way of this is:
- do the conversion of the code to classes on trunk, keeping everything as “static”: functions become static methods, global variables become static data members. This should have no performance impact.
- maintain a branch in which the various “static” things above become non-static.
Another example, the conversion of passes from C structs to C++ classes contains a mixture of autogenerated and handwritten code. Part of this is a move of the pass tree to a passes.def file - and this needs constant rebasing as passes get moved around by other gcc developers. So if I am destined for a branch, I’d at least want the option of tactical merges here and there.
Question: if it’s a branch, is it a branch in git, or a branch in svn? (or both, somehow?)
As of r201230 in trunk, there is a new class gcc::context, which is intended to encapsulate the state of the compiler.
The singleton instance is g:
extern gcc::context *g;
which was chosen to minimize typing (both in code and in the debugger).
Many things will gain a context *. Although this is good from a state-removal perspective, there’s a danger here that this could become a big blob, or rats nest of interdependencies, where everything in the compiler can access anything else in the compiler.
I think that having a context * where you need it is sufficiently useful that a “good intentions”/”consenting adults” approach will be acceptable for the initial iteration of this work for mitigating the above risk.
Ultimately we may want to pass in something more restrictive e.g. just a gc_heap * so that objects don’t get tightly coupled.
Although there’s been some talk of removing GTY, I plan to work with the existing code, without requiring other features to land, and that means dealing with GC and PCH.
There are two possible ways in which context instances could interact with the GC:
- have the context instances be GC-managed: all parallel contexts share the same heap, and rewriting the GC code to be thread-safe, or
- have the context manage GC, so that the state of GC is per-context: each context has its own GC heap, entirely independent of each other context’s GC heap. You can’t share GC pointers between contexts.
I don’t think (a) is feasible.
The GC is written with the assumption that it only runs at explicitly-controlled times.
For example, the code is full of places where refs to GC-managed data are stored on the stack, but there is no mechanism for tracking on-stack GC roots during a mark-and-sweep. In a multithreaded process using GCC’s code, if one thread wants to garbage-collect, all other threads would need to also be at a location where it’s safe to GC.
Hence (a) would require all threads to synchronize on GC-safe locations.
It would also require a substantial rewrite of PCH-handling, since PCH files are essentially a dump of the state of the GC-heap.
It seems much simpler to me to go with (b): multiple independent GC-heaps, where “context” objects sit below garbage-collection.
Proof-of-concept patch posted as http://gcc.gnu.org/ml/gcc-patches/2013-06/msg00878.html
I have an (unposted) patch which moves many of the cgraph_ functions to be methods of a new class callgraph.
See below in “Middle-end classes” for how this looks.
Status: Not yet ready; remaining work:
- integrate the class with GTY
- integrate the class with “context”
New file gcc/context.h which ultimately would declare something like this:
class context
{
public:
/* Instance of the garbage collector. */
MAYBE_STATIC gc_heap *m_heap;
/* Instance of the callgraph. */
MAYBE_STATIC callgraph *m_cgraph;
/* Pass management. */
MAYBE_STATIC pass_manager *m_passes;
/* Important objects. */
MAYBE_STATIC struct gcc_options m_global_options;
MAYBE_STATIC frontend *m_frontend;
MAYBE_STATIC backend *m_backend;
MAYBE_STATIC FILE * m_dump_file;
MAYBE_STATIC int m_dump_flags;
// etc
MAYBE_STATIC location_t m_input_location;
/* State shared by many passes. */
MAYBE_STATIC struct df_d *m_df;
MAYBE_STATIC redirect_edge_var_state *m_edge_vars;
/* Passes that have special state-handling needs. */
MAYBE STATIC mudflap_state *m_mudflap;
}; // class context
#if GLOBAL_BUILD
/* Global singleton instance of the context. */
extern context the_uni;
#endif
(it would be initially be empty, but would be built up field by field as patches are accepted).
context.h will likely be included by everything, so the context’s fields have some indirection to avoid users of context.h requiring other header files, and thus everything requiring every header file.
The “context” instance can be thought of as the “root” object of global state: if you have a context* you can reach many other useful objects directly. Similarly, many objects have a reference back to their context*
Singletons would make natural GC roots, but gengtype only supports pointers as GC roots, not structs.
I tried registering the singleton context using ggc_register_root_tab, which adds it to extra_root_vec - but this is only used by the garbage collector - it isn’t used by pch.
Hence the various objects referenced through the context never made it into the pch file, and it went “boom” due to pch nuking the heap prior to restoring the heap from PCH (thus leaving the unreached objects with poisoned byte-values).
ggc_mark_roots traverses
- gt_ggc_rtab
- extra_root_vec
whereas gt_pch_save traverses:
- gt_ggc_rtab
- gt_pch_cache_rtab
Currently there doesn’t seem to be a way to add a new callback (or root tab) that’s used by both (gt_ggc_rtab is constant, written out by gengtype).
We could add a way to add callbacks to both, but I think we need to specialcase the singletons inside ggc and pch, explicitly calling their traversal hook there at the appropriate times. Hence it makes sense to have a single context object even in a global-state build: this is the single root struct for GGC; its traversal hooks lead to every other singleton being traversed. As we move global variables into singletons, gt_ggc_rtab will slowly become empty: the only GC root will be the context singleton.
Every pass instance “knows” which context it is in, so every “execute” hook can easily determine which its context, and put this into the per-pass state.
Hence the context* is easily accessed during the top-level function calls within optimization passes, and by anything that can access per-pass state.
How do we get at context from deep within code that doesn’t have easy access to it? (e.g. helper functions and macros).
LLVM solves this by having every type object have a context*: you can always easily find a type object. This is probably too expensive memory-wise to be acceptable to GCC, so we need a different approach.
I propose we use thread-local storage for this:
#if SHARED_BUILD
/* Read a thread-local pointer: */
extern __thread context *g;
#else
/* Access the global singleton: */
extern context *g;
#endif
This approach has the advantage of relative simplicity, and is efficient for the non-shared case (where the pointer will be effectively ignored, as everything will be going through “static”).
(I would have prefered to avoid relying on TLS, since it makes client code need to take this it account when it manages its own threads, but the alternatives are all much clunkier, or introduce unacceptable increases in memory usage).
Currently-existing GCC plugins are expecting to be run from inside a traditional GCC where there is a single instance of state, and I intend to continue that model.
The shared-library approach supports reusing parts of GCC code to build other kinds of tools, and plugins may or may not make sense in such tools (perhaps being initialized once per-context?)
However this is out-of-scope for this iteration.
(perhaps this is analagous to embedding vs extending in the Python world; see http://docs.python.org/2/extending/embedding.html).
A plugin that wants to interact with a shared-library build of GCC could potentially get at the context through the g pointer above.
I’ve been writing scripts to make it easier to automatically refactor the GCC code (e.g. respecting whitespace conventions, whilst not touching whitespace in lines we don’t touch, generating ChangeLogs etc):