Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9148

AuthorizationStmtTest.testColumnMaskEnabled seems flaky

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • Frontend
    • None

    Description

      Recently we have seen failed AuthorizationStmtTest.testColumnMaskEnabled occasionally with the following error messages (e.g., https://jenkins.impala.io/job/gerrit-verify-dryrun/5194/consoleFull).

      Impala does not support row filtering yet. Row filtering is enabled on table: functional.alltypes_view
      expected:
      Impala does not support column masking yet. Column masking is enabled on column: functional.alltypes_view.string_col
      

      Taking a look at the testColumnMaskEnabled(), we can see the related SQL statement is

      select string_col from functional.alltypes_view;
      

      I found that for this SQL statement, authorizeRowFilterAndColumnMask() in RangerAuthorizationCheker.java will be called first (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L183-L200). There will be two privilege requests, one request for column, and the other for table. The function authorizeRowFilter() is the only function that could produce the error message above (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L295-L308). Specifically, this error would be generated if plugin_.evalRowFilterPolicies(req, null).isRowFilterEnabled() returns true (https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java#L303).

      I have taken a brief look at isRowFilterEnabled(), and found that it will return true only if there is some policy on the Ranger server that specifies the policy of row filtering (according to my current understanding). However, in testColumnMaskEnabled() (https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java#L2836), we only add a policy for column masking. Therefore, I suspect it may be possible that some other tests added to the Ranger server some policy for row filtering but did not properly do the cleanup of this row filtering policy afterwards.

      To address this issue, we should add some logic to clean up the policies stored on the Ranger server before running this JUnit test. This JUnit test assumes that the Ranger server does not store any policies related to column masking and row filtering before the testing.

       

       

      Attachments

        Issue Links

          Activity

            People

              fangyurao Fang-Yu Rao
              fangyurao Fang-Yu Rao
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: