From 9fdd45d48d265140040d9df6f33460d14552a6a4 Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Wed, 16 May 2018 10:33:00 -0400
Subject: [PATCH 53/67] Update the README

---
 000-README.rst | 209 ++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 201 insertions(+), 8 deletions(-)

diff --git a/000-README.rst b/000-README.rst
index def56e6..05e17a1 100644
--- a/000-README.rst
+++ b/000-README.rst
@@ -5,13 +5,84 @@ Compiler-assisted Performance Analysis for GCC
 Summary
 =======
 
+I want to provide "better" information on how GCC optimizes code:
+
+(a) both for us (GCC developers), and
+
+(b) for advanced end-users
+
+where "better information" includes:
+
+* "actionable": (a) can get a sense of how an optimization can be
+  improved (bug-fixing), (b) can get an idea of which command-line
+  flags to try, or how to rework the code
+
+
 Statement of the problem
 ========================
 
+* Limitations of `-fopt-info`
+
+* LLVM's experience with remarks
+
+  * LLVM Remarks talk by Adam Nemet of Apple from November 2016:
+    2016 LLVM Developers’ Meeting: A. Nemet “Compiler-assisted Performance Analysis"
+      https://www.youtube.com/watch?v=qq0q1hfzidg
+      http://llvm.org/devmtg/2016-11/Slides/Nemet-Compiler-assistedPerformanceAnalysis.pdf
+    (part of http://www.llvm.org/devmtg/2016-11/ )
+
+In both cases, the problem is:
+
+* "wall of text": too much data, can't see what is important
+
+  * specifically: what did the optimizers do to the hot code?
+    * what *didn't* they do, and why not?
+
+Constraints
+===========
+
+* Preserve existing `-fopt-info` overall (whilst allowing for minor changes
+  to formatting *within* individual messages).
+
+* No extra time/memory cost for the default "disabled" case.  I assume it's
+  acceptable to spend a little more time/memory when the user opts-in by
+  enabling this instrumentation.
+
+
 Solution
 ========
 
-Consider::
+Solution is:
+
+* to capture "optimization records", emitting them in a format
+  that retains all pertinent information that can be easily parsed
+
+* to associate the optimization records with code hotness,
+  and to provide a report that prioritizes the records, starting with
+  the hottest code.
+
+I've implemented an HTML report generator, inspired by Adam Nemet's LLVM
+work, using the existing places where we're emitting `-fopt-info`
+information.
+
+Differences from Adam Nemet's work:
+
+* I've added hierarchical records, so that there can be a nesting
+  structure of optimization notes (otherwise there's still too much
+  to decipher).
+
+* as well as source location within the user's code, I'm capturing
+  source location within *GCC* for where the optimization happens.
+
+* LLVM is using YAML for some reason; I used JSON.  Given that I'm
+  capturing some different things, I didn't attempt to use the same
+  file format as LLVM.
+
+
+Example 1
+*********
+
+Consider this `-fopt-info` code (from `FIXME`)::
 
   if (dump_enabled_p ())
     {
@@ -25,13 +96,15 @@ This purely emits text to the destination of `-fopt-info`, e.g.::
 
   foo.c:15:3: note: Symbolic number of iterations is (unsigned int) n_9(D)
 
+e.g. to stderr, and/or the dumpfile for the current pass.
+
 This can be rewritten as::
 
   if (optinfo_enabled_p ())
     {
-      emit_optinfo_at_vect_location (OPT_RECORD_REMARK)
+      OPTINFO_VECT_NOTE
         << "symbolic number of iterations is "
-        << number_of_iterations;
+        << details (number_of_iterations);
     }
 
 It can then be emitted in up to three ways:
@@ -40,7 +113,7 @@ It can then be emitted in up to three ways:
 
     foo.c:15:3: note: symbolic number of iterations is (unsigned int) n_9(D)
 
-  (albeit with capitalization changes)
+  (albeit with capitalization changes), to the `-fopt-info` destination(s).
 
 (b) It can be emitted as a "remark" through the diagnostics subsystem::
 
@@ -49,7 +122,7 @@ It can then be emitted in up to three ways:
 (c) It is emitted in JSON form through `-fsave-optimization-record`:
 
       {
-          "kind": "remark",
+          "kind": "note",
           "count": {
               "quality": "guessed_local",
               "value": 9.5563e+08
@@ -63,16 +136,132 @@ It can then be emitted in up to three ways:
           "message": [
               "symbolic number of iterations is ",
               {
-                  "expr": "(unsigned int) n_9(D)"
+                  "expr": "(unsigned int) n_8(D)"
               }
           ],
+          "impl_location": {
+              "line": 1674,
+              "file": "../../src/gcc/tree-vect-loop.c",
+              "function": "vect_analyze_loop_form"
+          },
           "function": "compute_sum"
       },
 
     Note how this captures metadata such as profile count, function name,
     pass, etc.
 
-    FIXME: should capture the location of the expressions (so we can do hyperlinks).
+
+Example 2
+*********
+
+Consider this `-fopt-info` code (from `vect_create_data_ref_ptr`)::
+
+  if (dump_enabled_p ())
+     {
+       tree dr_base_type = TREE_TYPE (DR_BASE_OBJECT (dr));
+       dump_printf_loc (MSG_NOTE, vect_location,
+                        "create %s-pointer variable to type: ",
+                        get_tree_code_name (TREE_CODE (aggr_type)));
+       dump_generic_expr (MSG_NOTE, TDF_SLIM, aggr_type);
+       if (TREE_CODE (dr_base_type) == ARRAY_TYPE)
+         dump_printf (MSG_NOTE, "  vectorizing an array ref: ");
+       else if (TREE_CODE (dr_base_type) == VECTOR_TYPE)
+         dump_printf (MSG_NOTE, "  vectorizing a vector ref: ");
+       else if (TREE_CODE (dr_base_type) == RECORD_TYPE)
+         dump_printf (MSG_NOTE, "  vectorizing a record based array ref: ");
+       else
+         dump_printf (MSG_NOTE, "  vectorizing a pointer ref: ");
+       dump_generic_expr (MSG_NOTE, TDF_SLIM, DR_BASE_OBJECT (dr));
+       dump_printf (MSG_NOTE, "\n");
+     }
+
+where the information is built up piecewise, with conditional logic.
+
+This purely emits text to the destination of `-fopt-info`, e.g.::
+
+    foo.c:12:5: note: create vector_type-pointer variable to type: vector(4) int  vectorizing a pointer ref: *arr_9(D)
+
+e.g. to stderr, and/or the dumpfile for the current pass.
+
+This can be rewritten by exposing the `pending_optinfo`::
+
+  if (optinfo_enabled_p ())
+    {
+      tree dr_base_type = TREE_TYPE (DR_BASE_OBJECT (dr));
+      pending_optinfo info = OPTINFO_VECT_NOTE
+       << optinfo_printf ("create %s-pointer variable to type: ",
+                          get_tree_code_name (TREE_CODE (aggr_type)))
+       << slim (aggr_type);
+      if (TREE_CODE (dr_base_type) == ARRAY_TYPE)
+       info << "  vectorizing an array ref: ";
+      else if (TREE_CODE (dr_base_type) == VECTOR_TYPE)
+       info << "  vectorizing a vector ref: ";
+      else if (TREE_CODE (dr_base_type) == RECORD_TYPE)
+       info << "  vectorizing a record based array ref: ";
+      else
+       info << "  vectorizing a pointer ref: ";
+      info << slim (DR_BASE_OBJECT (dr));
+    }
+
+The `pending_optinfo` is emitted when it goes out of scope.  As before, it
+can be emitted in up to three ways:
+
+(a) as before via `-fopt-info`::
+
+    foo.c:12:5: note: create vector_type-pointer variable to type: vector(4) int  vectorizing a pointer ref: *arr_9(D)
+
+  (albeit with capitalization changes), to the `-fopt-info` destination(s).
+
+(b) It can be emitted as a "remark" through the diagnostics subsystem::
+
+    foo.c:15:3: remark:   create vector_type-pointer variable to type: ‘vector(4) int’  vectorizing a pointer ref: ‘*arr_9(D)’ [pass=vect]
+
+(c) It is emitted in JSON form through `-fsave-optimization-record`:
+
+      {
+          "kind": "note",
+          "count": {
+              "quality": "guessed_local",
+              "value": 8.60067e+08
+          },
+          "location": {
+              "line": 15,
+              "file": "foo.c",
+              "column": 3
+          },
+          "pass": "vect",
+          "message": [
+              "create vector_type-pointer variable to type: ",
+              {
+                  "expr": "vector(4) int"
+              },
+              "  vectorizing a pointer ref: ",
+              {
+                  "location": {
+                      "line": 16,
+                      "file": "foo.c",
+                      "column": 5
+                  },
+                  "expr": "*arr_9(D)"
+              }
+          ],
+          "impl_location": {
+              "line": 4702,
+              "file": "../../src/gcc/tree-vect-data-refs.c",
+              "function": "vect_create_data_ref_ptr"
+          },
+          "function": "compute_sum"
+      },
+
+    Note how this has captured the source location of the pointer ref.
+
+HTML generator:
+
+* https://github.com/davidmalcolm/gcc-opt-viewer
+
+Examples of HTML output:
+
+* https://dmalcolm.fedorapeople.org/gcc/2018-05-15/
 
 
 Known unknowns
@@ -91,10 +280,14 @@ Known unknowns
   collection of related messages from one subroutine.
 
 * Should optimization info be internationalized/localized? Currently
-  `-fopt-info` isn't.  I say let's not i18n it.
+  `-fopt-info` isn't.  I say let's *not* i18n it.
+
 
 Notes to self
 =============
 
 Working copy:
   /home/david/coding-3/gcc-git-static-analysis/src
+
+Backups:
+* https://dmalcolm.fedorapeople.org/gcc/2018-05-14/gcc-git-static-analysis-opt-records-relative-to-r259650/
-- 
1.8.5.3

