Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3957

Test failure in S3 build: TestLoadData.test_load: wrong # of files in destination

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • Impala 2.7.0
    • Impala 2.7.0
    • Backend

    Description

      http://sandbox.jenkins.cloudera.com/job/impala-cdh5-trunk-core-s3/141/

      19:30:59 =================================== FAILURES ===================================
      19:30:59  TestLoadData.test_load[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
      19:30:59 [gw1] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test-s3/repos/Impala/bin/../infra/python/env/bin/python
      19:30:59 metadata/test_load.py:81: in test_load
      19:30:59     self.run_test_case('QueryTest/load', vector)
      19:30:59 common/impala_test_suite.py:332: in run_test_case
      19:30:59     pytest.config.option.update_results)
      19:30:59 common/test_result_verifier.py:377: in verify_raw_results
      19:30:59     VERIFIER_MAP[verifier](expected, actual)
      19:30:59 common/test_result_verifier.py:215: in verify_query_result_is_equal
      19:30:59     assert expected_results == actual_results
      19:30:59 E   assert Comparing QueryTestResults (expected vs actual):
      19:30:59 E     'Loaded 4 file(s). Total files in destination location: 8' != 'Loaded 4 file(s). Total files in destination location: 7'
      19:30:59 ---------------------------- Captured stderr setup -----------------------------
      19:30:59 -- connecting to: localhost:21000
      19:30:59 -- executing against localhost:21000
      19:30:59 drop table if exists functional.test_load_nopart;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 drop table if exists functional.test_load;
      19:30:59 
      19:30:59 MainThread: Starting new HTTPS connection (1): impala-cdh5-trunk.s3.amazonaws.com
      19:30:59 -- executing against localhost:21000
      19:30:59 create table functional.test_load like functional.alltypes location 's3a://impala-cdh5-trunk/test-warehouse/test_load';
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 create table functional.test_load_nopart like functional.alltypesnopart location 's3a://impala-cdh5-trunk/test-warehouse/test_load_nopart';
      19:30:59 
      19:30:59 ----------------------------- Captured stderr call -----------------------------
      19:30:59 -- executing against localhost:21000
      19:30:59 use functional;
      19:30:59 
      19:30:59 SET disable_codegen=False;
      19:30:59 SET abort_on_error=1;
      19:30:59 SET exec_single_node_rows_threshold=0;
      19:30:59 SET batch_size=0;
      19:30:59 SET num_nodes=0;
      19:30:59 -- executing against localhost:21000
      19:30:59 alter table functional.test_load add partition
      19:30:59 (year=2009, month=1);
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 alter table functional.test_load add partition
      19:30:59 (year=2010, month=1);
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 insert overwrite table functional.test_load partition (year=2009, month=1)
      19:30:59 select id, bool_col, tinyint_col, smallint_col, int_col, bigint_col,
      19:30:59 float_col, double_col, date_string_col, string_col, timestamp_col
      19:30:59 from functional.alltypes where year = 2009 and month = 1;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 select count(*) from functional.test_load;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 select count(*) from functional.test_load where year=2010;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 load data inpath 's3a://impala-cdh5-trunk/test-warehouse/test_load_staging/1/100101.txt'
      19:30:59 into table functional.test_load partition(year=2010, month=1);
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 select count(*) from functional.test_load where year=2010 and month=1;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 load data inpath 's3a://impala-cdh5-trunk/test-warehouse/test_load_staging/2/100101.txt'
      19:30:59 into table functional.test_load partition(year=2010, month=1);
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 select count(*) from functional.test_load where year=2010 and month=1;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 load data inpath 's3a://impala-cdh5-trunk/test-warehouse/test_load_staging/3/100101.txt'
      19:30:59 overwrite into table functional.test_load partition(year=2010, month=1);
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 select count(*) from functional.test_load where year=2010 and month=1;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 select count(*) from functional.test_load where year=2009;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 select count(*) from functional.test_load_nopart;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 load data inpath 's3a://impala-cdh5-trunk/test-warehouse/test_load_staging/4/'
      19:30:59 overwrite into table functional.test_load_nopart;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 select count(*) from functional.test_load_nopart;
      19:30:59 
      19:30:59 -- executing against localhost:21000
      19:30:59 load data inpath 's3a://impala-cdh5-trunk/test-warehouse/test_load_staging/5/'
      19:30:59 into table functional.test_load_nopart;
      19:30:59 
      19:30:59 MainThread: Comparing QueryTestResults (expected vs actual):
      19:30:59 'Loaded 4 file(s). Total files in destination location: 8' != 'Loaded 4 file(s). Total files in destination location: 7'
      19:30:59  generated xml file: /data/jenkins/workspace/impala-umbrella-build-and-test-s3/repos/Impala/logs/ee_tests/results/TEST-impala-parallel.xml 
      

      This is the first time I've seen this.

      Attachments

        Activity

          People

            sailesh Sailesh Mukil
            tarmstrong Tim Armstrong
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: