From f9852f064f4c40f0260cede6f69fa9d190336e85 Mon Sep 17 00:00:00 2001 From: David Malcolm Date: Tue, 17 Nov 2015 06:52:28 -0500 Subject: [PATCH 01/35] Selftest framework (unittests v4) This is effectively v4 of the unittests proposal; for the earlier versions see: * v1: https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00765.html * v2: https://gcc.gnu.org/ml/gcc-patches/2015-06/msg01224.html * v3: https://gcc.gnu.org/ml/gcc-patches/2015-10/msg02947.html This patch adds a selftest.h/.c to gcc, with an API loosely modelled on gtest (though without the use of CamelCase): it supports enough of the gtest API to enable the tests that I wrote to run with minimal changes. It adds a -fself-test option, which runs the test at the end of cc1,cc1plus, etc, similar to how the plugin worked in v2 and v3 of the kit. The tests themselves are in the followup patches. I moved the tests from gcc/testsuite/unittests into gcc. Where possible, I moved the tests directly into the end of an implementation file e.g. gcc/testsuite/unittests/test-vec.c became an addition to gcc/vec.c; other files don't have a suitable .c file e.g. for parametrized container types where there's just a .h, so I created e.g. gcc/hash-map-tests.c (Irritatingly, touching vec.c means that parts of the selftest code become needed by e.g. collect2 so that vec.o can link; similarly for input.o, which gains the location-handling selftests). Everything apart from the -fself-test option itself is guarded by #if CHECKING_P, so this all compiles away in a production build to a "sorry, not supported". Like gtest, tests are automatically discovered, using some global object ctor magic. Sadly this doesn't work for some reason for source files which are purely tests, so as a nasty workaround I #include those files from toplev.c I didn't document -fself-test; should I? (maybe just in the internal docs?) This iteration of the patchkit doesn't yet do anything to automatically invoke -fself-test; I've been running it by hand when compiling an empty .c file, so it's been running inside cc1. How should this be hooked up? Currently -fself-test requires an input file; should it? (But if not, what should the driver run? All of them?). Requiring an input file, and providing an empty one may be simplest. I anticipate a discussion about output formats; the current output format is deliberately very verbose, to ensure that we're capturing everything of interest. It contains file/line info of each EXPECT/ASSERT so that it's easy in Emacs to click in the compilation buffer to go to specific assertions. NOTE: bitmap_test.bitmap_single_bit_set_p: test starting ../../src/gcc/bitmap.c:2172: PASS: bitmap_test.bitmap_single_bit_set_p: EXPECT_FALSE (bitmap_single_bit_set_p (b)) ../../src/gcc/bitmap.c:2175: PASS: bitmap_test.bitmap_single_bit_set_p: EXPECT_TRUE (bitmap_single_bit_set_p (b)) ../../src/gcc/bitmap.c:2176: PASS: bitmap_test.bitmap_single_bit_set_p: EXPECT_EQ (42, bitmap_first_set_bit (b)) ../../src/gcc/bitmap.c:2179: PASS: bitmap_test.bitmap_single_bit_set_p: EXPECT_FALSE (bitmap_single_bit_set_p (b)) ../../src/gcc/bitmap.c:2180: PASS: bitmap_test.bitmap_single_bit_set_p: EXPECT_EQ (42, bitmap_first_set_bit (b)) ../../src/gcc/bitmap.c:2183: PASS: bitmap_test.bitmap_single_bit_set_p: EXPECT_TRUE (bitmap_single_bit_set_p (b)) ../../src/gcc/bitmap.c:2184: PASS: bitmap_test.bitmap_single_bit_set_p: EXPECT_EQ (1066, bitmap_first_set_bit (b)) NOTE: bitmap_test.bitmap_single_bit_set_p: test ending NOTE: bitmap_test.clear_bit_in_middle: test starting ../../src/gcc/bitmap.c:2135: PASS: bitmap_test.clear_bit_in_middle: EXPECT_EQ (100, bitmap_count_bits (b)) ../../src/gcc/bitmap.c:2139: PASS: bitmap_test.clear_bit_in_middle: EXPECT_TRUE (changed) [...snip...] NOTE: 491 pass(es); 0 failure(s) Potentially we could have some kind of verbosity flag for the selftests, I guess. I did attempt to use "inform" to report results, but I don't think this is appropriate for the most verbose form of output: (A) some tests manipulate cfun and hence lead to strange "In function ..." messages as they run based on the changes to cfun (B) I want to write selftests for the diagnostics subsystem itself I've successfully bootstrapped®rtested the combination of these patches on x86_64-pc-linux-gnu and manually verified -fself-test (with the exception of the final patch "RFC: Add ggc-tests.c" which has some issues, as noted in that patch). How does this look? Notes on porting tests from gtest: * Fixtures inherit from ::selftest::test, rather than from gtest's ::testing::Test * There's no support yet for Setup and Teardown vfuncs (which presumably would be "setup" and "teardown") * I only implemented what I needed * I didn't implement type-parametrized testing, so for now I've dropped test-wide-int.c (which is the only test I had that was parametrized over multiple types). gcc/ChangeLog: * Makefile.in (OBJS-libcommon): Add selftest.o. (OBJS-libcommon-target): Add selftest.o. (COLLECT2_OBJS): Add selftest.o. * common.opt (fself-test): New. * selftest.c: New file. * selftest.h: New file. * toplev.c: Include selftest.h. Add includes of function-tests.c, hash-map-tests.c, hash-set-tests.c, rtl-tests.c as a workaround for a linker issue. (toplev::run_self_tests): New. (toplev::main): Handle -fself-test. * toplev.h (toplev::run_self_tests): New. --- gcc/Makefile.in | 7 +- gcc/common.opt | 4 + gcc/selftest.c | 152 +++++++++++++++++++++++++++++++ gcc/selftest.h | 270 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ gcc/toplev.c | 53 +++++++++++ gcc/toplev.h | 2 + 6 files changed, 485 insertions(+), 3 deletions(-) create mode 100644 gcc/selftest.c create mode 100644 gcc/selftest.h diff --git a/gcc/Makefile.in b/gcc/Makefile.in index 673f87d..33dd75b 100644 --- a/gcc/Makefile.in +++ b/gcc/Makefile.in @@ -1543,13 +1543,14 @@ OBJS = \ # no target dependencies. OBJS-libcommon = diagnostic.o diagnostic-color.o diagnostic-show-locus.o \ pretty-print.o intl.o \ - vec.o input.o version.o hash-table.o ggc-none.o memory-block.o + vec.o input.o version.o hash-table.o ggc-none.o memory-block.o \ + selftest.o # Objects in libcommon-target.a, used by drivers and by the core # compiler and containing target-dependent code. OBJS-libcommon-target = $(common_out_object_file) prefix.o params.o \ opts.o opts-common.o options.o vec.o hooks.o common/common-targhooks.o \ - hash-table.o file-find.o + hash-table.o file-find.o selftest.o # This lists all host objects for the front ends. ALL_HOST_FRONTEND_OBJS = $(foreach v,$(CONFIG_LANGUAGES),$($(v)_OBJS)) @@ -1986,7 +1987,7 @@ gcc-nm.c: gcc-ar.c cp $^ $@ COLLECT2_OBJS = collect2.o collect2-aix.o tlink.o vec.o ggc-none.o \ - collect-utils.o file-find.o hash-table.o + collect-utils.o file-find.o hash-table.o selftest.o COLLECT2_LIBS = @COLLECT2_LIBS@ collect2$(exeext): $(COLLECT2_OBJS) $(LIBDEPS) # Don't try modifying collect2 (aka ld) in place--it might be linking this. diff --git a/gcc/common.opt b/gcc/common.opt index 682cb41..95376bf 100644 --- a/gcc/common.opt +++ b/gcc/common.opt @@ -2057,6 +2057,10 @@ fselective-scheduling2 Common Report Var(flag_selective_scheduling2) Optimization Run selective scheduling after reload. +fself-test +Common Var(flag_self_test) +Run self-tests. + fsel-sched-pipelining Common Report Var(flag_sel_sched_pipelining) Init(0) Optimization Perform software pipelining of inner loops during selective scheduling. diff --git a/gcc/selftest.c b/gcc/selftest.c new file mode 100644 index 0000000..abb6585 --- /dev/null +++ b/gcc/selftest.c @@ -0,0 +1,152 @@ +/* A self-testing framework, for use by -fself-test. + Copyright (C) 2015 Free Software Foundation, Inc. + +This file is part of GCC. + +GCC is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation; either version 3, or (at your option) any later +version. + +GCC is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +#include "config.h" +#include "system.h" +#include "coretypes.h" +#include "selftest.h" + +#if CHECKING_P + +using namespace selftest; + +/* Helper function for ::selftest::run_all_tests. */ + +static int +test_comparator (const void *p1, const void *p2) +{ + const test *t1 = *static_cast (p1); + const test *t2 = *static_cast (p2); + return strcmp (t1->get_name (), t2->get_name ()); +} + +/* Locate and run all tests. + Return the number of failures that occurred. */ + +int +selftest::run_all_tests () +{ + /* Create all the tests, in an arbitrary order (based on + the order of construction of the global "registrator" instances). */ + runner r; + auto_vec tests; + for (registrator *iter = registrator::get_first (); + iter; + iter = iter->get_next ()) + { + test *t = iter->make (&r); + tests.safe_push (t); + } + + /* Sort the tests into a predictable order. */ + tests.qsort (test_comparator); + + /* Run all the tests, in order. */ + unsigned i; + test *t; + FOR_EACH_VEC_ELT (tests, i, t) + { + r.begin_test (t); + t->run (); + r.end_test (t); + } + + /* Cleanup. */ + FOR_EACH_VEC_ELT (tests, i, t) + delete t; + + return r.get_num_failures (); +} + +/* Implementation of class ::selftest::runner. */ + +/* ::selftest::runner's constructor. */ + +runner::runner () +: m_passes (0), + m_failures (0) +{ +} + +/* ::selftest::runner's destructor. */ + +runner::~runner () +{ + fprintf (stderr, "NOTE: %i pass(es); %i failure(s)\n", + m_passes, m_failures); +} + +/* Notify the user that a particular test is about to be run. */ + +void +runner::begin_test (test *t) +{ + fprintf (stderr, "NOTE: %s: test starting\n", t->get_name ()); +} + +/* Record and report the successful outcome of some aspect of a test. */ + +void +runner::pass (const char *file, int line, test *t, const char *msg) +{ + fprintf (stderr, "%s:%i: PASS: %s: %s\n", file, line, t->get_name (), msg); + m_passes++; +} + +/* Record and report the failed outcome of some aspect of a test. */ + +void +runner::fail (const char *file, int line, test *t, const char *msg) +{ + fprintf (stderr, "%s:%i: FAIL: %s: %s\n", file, line, t->get_name (), msg); + m_failures++; +} + +/* Notify the user that a particular test has finished running. */ + +void +runner::end_test (test *t) +{ + fprintf (stderr, "NOTE: %s: test ending\n", t->get_name ()); +} + +/* Implementation of class ::selftest::registrator. */ + +/* This constructor is run before main; to avoid relying on + anything, it simply builds a singly-linked list of + instances. */ + +registrator::registrator (registrator::callback cb) +: m_cb (cb), m_next (NULL) +{ + if (first) + { + /* Add this to the front of the list. */ + m_next = first; + first = this; + } + else + first = this; +} + +/* The singleton head of the registrator linked list. */ + +registrator *registrator::first; + +#endif /* #if CHECKING_P */ diff --git a/gcc/selftest.h b/gcc/selftest.h new file mode 100644 index 0000000..1c55881 --- /dev/null +++ b/gcc/selftest.h @@ -0,0 +1,270 @@ +/* A self-testing framework, for use by -fself-test. + Copyright (C) 2015 Free Software Foundation, Inc. + +This file is part of GCC. + +GCC is free software; you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation; either version 3, or (at your option) any later +version. + +GCC is distributed in the hope that it will be useful, but WITHOUT ANY +WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +#ifndef GCC_SELFTEST_H +#define GCC_SELFTEST_H + +/* The selftest code should entirely disappear in a production + configuration, hence we guard all of it with #if CHECKING_P. */ + +#if CHECKING_P + +namespace selftest { + +class test; +class runner; +class registrator; + +/* The entrypoint for running all tests. */ + +extern int run_all_tests (); + +/* The class ::selftest::runner is responsible for gathering results, + and for output. */ + +class runner +{ +public: + runner (); + ~runner (); + + void begin_test (test *t); + void pass (const char *file, int line, test *t, const char *msg); + void fail (const char *file, int line, test *t, const char *msg); + void end_test (test *t); + + int get_num_failures () const { return m_failures; } + +private: + int m_passes; + int m_failures; +}; + +/* The class ::selftest::test is a base class from which specific + tests inherit, via the TEST and TEST_F macros below. */ + +class test +{ + public: + virtual ~test () {} + virtual void run () = 0; + + const char *get_name () const { return m_name; } + + void set_name (const char *name) { m_name = name; } + void set_runner (runner *r) { m_runner = r; } + + protected: + void pass (const char *file, int line, const char *msg) + { + m_runner->pass (file, line, this, msg); + } + void fail (const char *file, int line, const char *msg) + { + m_runner->fail (file, line, this, msg); + } + + /* We don't yet implement setup & teardown hooks. */ + + private: + const char *m_name; + runner *m_runner; +}; + +/* An implementation detail of automatic test registration. + Global instances are created via the REGISTER_TEST below. + The constructor runs before main, wiring them up into a + singly-linked list, which can be traversed by + ::selftest::run_all_tests. */ + +class registrator +{ + public: + typedef test *(*callback) (runner *r); + registrator (callback cb); + + static registrator *get_first () { return first; } + registrator *get_next () const { return m_next; } + + test *make (runner *r) const { return m_cb (r); } + + private: + callback m_cb; + registrator *m_next; + static registrator *first; +}; + +} /* end of namespace selftest. */ + +/* Macros for creating test functions. */ + +/* Define a new test, expecting a braced function body to follow. + The function body becomes the implementation of a "run" method of + a new ::selftest::test subclass, which will be instantiated and run + by RUN_ALL_TESTS. */ + +#define TEST(TEST_CASE_NAME, TEST_NAME) \ + IMPL_TEST_SUBCLASS (TEST_CASE_NAME ## _ ## TEST_NAME, \ + ::selftest::test, \ + (#TEST_CASE_NAME "." #TEST_NAME) ) + + +/* As per TEST above, but inheriting from a fixture subclass, rather + than directly from ::selftest::test. */ + +#define TEST_F(FIXTURE_CLASS_NAME, TEST_NAME) \ + IMPL_TEST_SUBCLASS (FIXTURE_CLASS_NAME ## _ ## TEST_NAME, \ + FIXTURE_CLASS_NAME, \ + (#FIXTURE_CLASS_NAME "." #TEST_NAME) ) + +/* Macros for writing tests. */ + +/* Evaluate EXPR and coerce to bool, issuing PASS if it is true, + FAIL if it false. */ + +#define EXPECT_TRUE(EXPR) \ + SELFTEST_BEGIN_STMT \ + const char *desc = "EXPECT_TRUE (" #EXPR ")"; \ + bool actual = (EXPR); \ + if (actual) \ + pass (__FILE__, __LINE__, desc); \ + else \ + fail (__FILE__, __LINE__, desc); \ + SELFTEST_END_STMT + +/* Evaluate EXPR and coerce to bool, issuing PASS if it is false, + FAIL if it true. */ + +#define EXPECT_FALSE(EXPR) \ + SELFTEST_BEGIN_STMT \ + const char *desc = "EXPECT_FALSE (" #EXPR ")"; \ + bool actual = (EXPR); \ + if (actual) \ + fail (__FILE__, __LINE__, desc); \ + else \ + pass (__FILE__, __LINE__, desc); \ + SELFTEST_END_STMT + +/* Evaluate EXPECTED and ACTUAL and compare them with ==, issuing PASS + if they are equal, FAIL if they are non-equal. */ + +#define EXPECT_EQ(EXPECTED, ACTUAL) \ + SELFTEST_BEGIN_STMT \ + const char *desc = "EXPECT_EQ (" #EXPECTED ", " #ACTUAL ")"; \ + if ((EXPECTED) == (ACTUAL)) \ + pass (__FILE__, __LINE__, desc); \ + else \ + fail (__FILE__, __LINE__, desc); \ + SELFTEST_END_STMT + +/* Evaluate EXPECTED and ACTUAL and compare them with !=, issuing PASS + if they are non-equal, FAIL if they are equal. */ + +#define EXPECT_NE(EXPECTED, ACTUAL) \ + SELFTEST_BEGIN_STMT \ + const char *desc = "EXPECT_NE (" #EXPECTED ", " #ACTUAL ")"; \ + if ((EXPECTED) != (ACTUAL)) \ + pass (__FILE__, __LINE__, desc); \ + else \ + fail (__FILE__, __LINE__, desc); \ + SELFTEST_END_STMT + +/* Evaluate EXPECTED and ACTUAL and compare them with strcmp, issuing + PASS if they are equal, FAIL if they are non-equal. */ + +#define EXPECT_STREQ(EXPECTED, ACTUAL) \ + SELFTEST_BEGIN_STMT \ + const char *desc = "EXPECT_STREQ (" #EXPECTED ", " #ACTUAL ")"; \ + const char *expected_ = (EXPECTED); \ + const char *actual_ = (ACTUAL); \ + if (0 == strcmp (expected_, actual_)) \ + pass (__FILE__, __LINE__, desc); \ + else \ + fail (__FILE__, __LINE__, desc); \ + SELFTEST_END_STMT + +/* Evaluate PRED1 (VAL1), issuing PASS if it is true, FAIL if + it false. */ + +#define EXPECT_PRED1(PRED1, VAL1) \ + SELFTEST_BEGIN_STMT \ + const char *desc = "EXPECT_PRED1 (" #PRED1 ", " #VAL1 ")"; \ + bool actual = (PRED1) (VAL1); \ + if (actual) \ + pass (__FILE__, __LINE__, desc); \ + else \ + fail (__FILE__, __LINE__, desc); \ + SELFTEST_END_STMT + +/* As per EXPECT_TRUE, but immediately end the test (via return) if + if fails. */ + +#define ASSERT_TRUE(EXPR) \ + SELFTEST_BEGIN_STMT \ + const char *desc = "ASSERT_TRUE (" #EXPR ")"; \ + bool actual = (EXPR); \ + if (actual) \ + pass (__FILE__, __LINE__, desc); \ + else \ + { \ + fail (__FILE__, __LINE__, desc); \ + return; \ + } \ + SELFTEST_END_STMT + + /* A macro for running all tests, returning the number of failures. */ + +#define RUN_ALL_TESTS() \ + ::selftest::run_all_tests () + + +/* The remaining macros are implementation details, for internal use. */ + +/* A macro for registering a test subclass, by creating a global object + with a non-trivial ctor. */ + +#define REGISTER_TEST(SUBCLASSNAME) \ + static selftest::registrator registrator_for_ ##SUBCLASSNAME(&SUBCLASSNAME::make) + +/* This macro is used to implement TEST and TEST_F. It creates a new + subclass of ::selftest::test, and begins the definition of a "run" + method for the subclass, expecting a braced method body to follow. */ + +#define IMPL_TEST_SUBCLASS(SUBCLASS_NAME, BASE_CLASS, NAME) \ + class SUBCLASS_NAME : public BASE_CLASS \ + { \ + public: \ + void run (); \ + static ::selftest::test *make (::selftest::runner *r) \ + { \ + ::selftest::test *t = new SUBCLASS_NAME (); \ + t->set_name (NAME); \ + t->set_runner (r); \ + return t; \ + } \ + }; \ + REGISTER_TEST(SUBCLASS_NAME); \ + void SUBCLASS_NAME::run () + +#define SELFTEST_BEGIN_STMT do { +#define SELFTEST_END_STMT } while (0) + +#endif /* #if CHECKING_P */ + +#endif /* GCC_SELFTEST_H */ diff --git a/gcc/toplev.c b/gcc/toplev.c index 580c03a..1203dc2 100644 --- a/gcc/toplev.c +++ b/gcc/toplev.c @@ -87,6 +87,10 @@ along with GCC; see the file COPYING3. If not see #include "xcoffout.h" /* Needed for external data declarations. */ #endif +#include "selftest.h" + +#include + static void general_init (const char *, bool); static void do_compile (); static void process_options (void); @@ -2031,6 +2035,52 @@ toplev::start_timevars () timevar_start (TV_TOTAL); } +/* For some tests, there's a natural source file to place them in. + For others, they can live in their own "foo-tests.c" file. + Ideally, these "foo-tests.c" files would be added to OBJS in + Makefile.in. However, for some reason that approach doesn't + work: the tests don't get run.. The linker appears to be discarding + the global "registrator" instances in files which are purely + test cases (apart from ggc-tests.c, which works for some + reason; perhaps the GC roots is poking the linker in such a way + as to prevent the issue). + + Hence as a workaround, we instead directly include the files here. */ + +#if CHECKING_P + +#include "function-tests.c" +#include "hash-map-tests.c" +#include "hash-set-tests.c" +#include "rtl-tests.c" + +#endif /* #if CHECKING_P */ + +/* Handle -fself-test. */ + +void +toplev::run_self_tests () +{ +#if CHECKING_P + /* Reset some state. */ + input_location = UNKNOWN_LOCATION; + bitmap_obstack_initialize (NULL); + + /* Run the tests. */ + int result = RUN_ALL_TESTS(); + + /* Ensure that a test failure leads to the process exiting with + a non-zero exit code. */ + if (result) + error ("at least one test failure occurred"); + + /* Cleanup. */ + bitmap_obstack_release (NULL); +#else + sorry ("self-tests are not enabled in this build"); +#endif /* #if CHECKING_P */ +} + /* Entry point of cc1, cc1plus, jc1, f771, etc. Exit code is FATAL_EXIT_CODE if can't open files or if there were any errors, or SUCCESS_EXIT_CODE if compilation succeeded. @@ -2098,6 +2148,9 @@ toplev::main (int argc, char **argv) if (warningcount || errorcount || werrorcount) print_ignored_options (); + if (flag_self_test) + run_self_tests (); + /* Invoke registered plugin callbacks if any. Some plugins could emit some diagnostics here. */ invoke_plugin_callbacks (PLUGIN_FINISH, NULL); diff --git a/gcc/toplev.h b/gcc/toplev.h index 0beb06e..06923cf 100644 --- a/gcc/toplev.h +++ b/gcc/toplev.h @@ -42,6 +42,8 @@ private: void start_timevars (); + void run_self_tests (); + bool m_use_TV_TOTAL; bool m_init_signals; }; -- 1.8.5.3