Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12391

Missing EventSequence names in the JSON profile

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Backend
    • ghx-label-3

    Description

      When retrieving profiles in JSON format, I realized theĀ event_sequences are represented as an array of sequences instead of a map:

      "event_sequences": [{
              "offset": 0,
              "events": [{
                              "label": "Metadata of all 0 tables cached",
                              "timestamp": 13596041
                      }, {
                              "label": "Analysis finished",
                              "timestamp": 84345355
                      }, {
                              "label": "Authorization finished (noop)",
                              "timestamp": 84499136
                      }, {
                              "label": "Value transfer graph computed",
                              "timestamp": 105314431
                      }, {
                              "label": "Single node plan created",
                              "timestamp": 110919848
                      }, {
                              "label": "Distributed plan created",
                              "timestamp": 115423306
                      }, {
                              "label": "Planning finished",
                              "timestamp": 187260424
                      }]
      }, {
              "offset": 0,
              "events": [{
                              "label": "Query submitted",
                              "timestamp": 125578
                      }, {
                              "label": "Planning finished",
                              "timestamp": 197022887
                      }, {
                              "label": "Submit for admission",
                              "timestamp": 197377109
                      }, {
                              "label": "Completed admission",
                              "timestamp": 197591734
                      }, {
                              "label": "Ready to start on 1 backends",
                              "timestamp": 197690271
                      }, {
                              "label": "All 1 execution backends (1 fragment instances) started",
                              "timestamp": 207852394
                      }, {
                              "label": "Rows available",
                              "timestamp": 209102814
                      }, {
                              "label": "First row fetched",
                              "timestamp": 300799735
                      }, {
                              "label": "Last row fetched",
                              "timestamp": 300879566
                      }, {
                              "label": "Released admission control resources",
                              "timestamp": 302725368
                      }, {
                              "label": "Unregister query",
                              "timestamp": 306153314
                      }]
      }],

      The EventSequence names (e.g. "Query Compilation", "Query Timeline") are missing.

      Related codes are in be/src/util/runtime-profile.cc
      https://github.com/apache/impala/blob/57964c83d37bdd61ec08b738c56725940138ab7d/be/src/util/runtime-profile.cc#L1398

      void RuntimeProfile::ToJsonSubclass(
          Verbosity verbosity, Value* parent, Document* d) const {
        Document::AllocatorType& allocator = d->GetAllocator();
        // 1. Events
        {
          lock_guard<SpinLock> l(event_sequence_lock_);
          if (!event_sequence_map_.empty()) {
            Value event_sequences_json(kArrayType); // <-- We explicitly use array here
            for (EventSequenceMap::const_iterator it = event_sequence_map_.begin();
                 it != event_sequence_map_.end(); ++it) {
              Value event_sequence_json(kObjectType);
              it->second->ToJson(verbosity, *d, &event_sequence_json);
              event_sequences_json.PushBack(event_sequence_json, allocator); // <-- 'it->first' is not used. It's the name of the EventSequence.
            }
            parent->AddMember("event_sequences", event_sequences_json, allocator);
          }
        }
      

      We can change 'event_sequences_json' to use kObjectType and add the EventSequence names.

      Attachments

        Activity

          People

            Unassigned Unassigned
            stigahuang Quanlong Huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: