Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9001

Ship with log4j.properties file that has a reliable time based rolling policy

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Labels:
      None

      Description

      The hive log gets locked by the hive process and cannot be rolled in windows OS.
      Install Hive in Windows, start hive, try and rename hive log while Hive is running.
      Wait for log4j tries to rename it and it will throw the same error as it is locked by the process.

      The changes in https://issues.apache.org/bugzilla/show_bug.cgi?id=29726 should be integrated to Hive for a reliable rollover.

      1. HIVE-9001.1.patch
        4 kB
        Hari Sankar Sivarama Subramaniyan

        Issue Links

          Activity

          Hide
          hsubramaniyan Hari Sankar Sivarama Subramaniyan added a comment -

          cc-ing Sushanth Sowmyan for reviewing this change.

          Show
          hsubramaniyan Hari Sankar Sivarama Subramaniyan added a comment - cc-ing Sushanth Sowmyan for reviewing this change.
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12684492/HIVE-9001.1.patch

          ERROR: -1 due to 5 failed/errored test(s), 6695 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_aggregate
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mapjoin_mapjoin
          org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection
          org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Delimited
          org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyCommit
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1941/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1941/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1941/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 5 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12684492 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12684492/HIVE-9001.1.patch ERROR: -1 due to 5 failed/errored test(s), 6695 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_aggregate org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mapjoin_mapjoin org.apache.hive.hcatalog.streaming.TestStreaming.testEndpointConnection org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Delimited org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyCommit Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1941/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1941/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1941/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed This message is automatically generated. ATTACHMENT ID: 12684492 - PreCommit-HIVE-TRUNK-Build
          Hide
          sushanth Sushanth Sowmyan added a comment -

          Looks good to me except for one minor fix - in the change to data/conf/hive-log4j.properties, you seem to have a comment section that is space-aligned to go to the next line rather than having newlines, which makes it one long comment line in the patch. I can fix this myself before committing if that's okay with you.

          Show
          sushanth Sushanth Sowmyan added a comment - Looks good to me except for one minor fix - in the change to data/conf/hive-log4j.properties, you seem to have a comment section that is space-aligned to go to the next line rather than having newlines, which makes it one long comment line in the patch. I can fix this myself before committing if that's okay with you.
          Hide
          hsubramaniyan Hari Sankar Sivarama Subramaniyan added a comment -

          Sushanth Sowmyan Made the space adjustments and overwrote the previous patch since this is a very minor change.

          Thanks
          Hari

          Show
          hsubramaniyan Hari Sankar Sivarama Subramaniyan added a comment - Sushanth Sowmyan Made the space adjustments and overwrote the previous patch since this is a very minor change. Thanks Hari
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12685179/HIVE-9001.1.patch

          ERROR: -1 due to 2 failed/errored test(s), 6695 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_aggregate
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1964/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1964/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1964/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12685179 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12685179/HIVE-9001.1.patch ERROR: -1 due to 2 failed/errored test(s), 6695 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_aggregate org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1964/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1964/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1964/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12685179 - PreCommit-HIVE-TRUNK-Build
          Hide
          sushanth Sushanth Sowmyan added a comment -

          +1, committing to trunk.

          Show
          sushanth Sushanth Sowmyan added a comment - +1, committing to trunk.
          Hide
          sushanth Sushanth Sowmyan added a comment -

          Committed to trunk. Thanks, Hari!

          Show
          sushanth Sushanth Sowmyan added a comment - Committed to trunk. Thanks, Hari!
          Hide
          leftylev Lefty Leverenz added a comment -

          Does this need any documentation, or is it just a bug fix?

          Show
          leftylev Lefty Leverenz added a comment - Does this need any documentation, or is it just a bug fix?
          Hide
          sushanth Sushanth Sowmyan added a comment -

          I think this does need documentation in https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ErrorLogs - it adds an optional mode of logging. Hari Sankar Sivarama Subramaniyan, could you please add info to that wiki?

          Show
          sushanth Sushanth Sowmyan added a comment - I think this does need documentation in https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ErrorLogs - it adds an optional mode of logging. Hari Sankar Sivarama Subramaniyan , could you please add info to that wiki?
          Show
          hsubramaniyan Hari Sankar Sivarama Subramaniyan added a comment - Added the documentation in https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ErrorLogs Thanks Hari
          Hide
          sushanth Sushanth Sowmyan added a comment -

          Lefty Leverenz : Just checking about procedure - is the process that we remove the TODOC15 flag now that Hari has added info for this on the wiki?

          Show
          sushanth Sushanth Sowmyan added a comment - Lefty Leverenz : Just checking about procedure - is the process that we remove the TODOC15 flag now that Hari has added info for this on the wiki?
          Hide
          leftylev Lefty Leverenz added a comment -

          Yes, you can remove TODOC15 as soon as it's documented. We don't use a DOC-DONE label, but comments show that the documentation was done.

          When I do the doc, sometimes I leave TODOC## pending review (unless I'm sure of the information). But there's no need for developers to wait for review once the information is in the doc.

          Show
          leftylev Lefty Leverenz added a comment - Yes, you can remove TODOC15 as soon as it's documented. We don't use a DOC-DONE label, but comments show that the documentation was done. When I do the doc, sometimes I leave TODOC## pending review (unless I'm sure of the information). But there's no need for developers to wait for review once the information is in the doc.
          Hide
          sushanth Sushanth Sowmyan added a comment -

          Thanks for the clarification!

          Show
          sushanth Sushanth Sowmyan added a comment - Thanks for the clarification!
          Hide
          brocknoland Brock Noland added a comment -

          Sushanth Sowmyan can you please review HIVE-9407?

          Show
          brocknoland Brock Noland added a comment - Sushanth Sowmyan can you please review HIVE-9407 ?

            People

            • Assignee:
              hsubramaniyan Hari Sankar Sivarama Subramaniyan
              Reporter:
              hsubramaniyan Hari Sankar Sivarama Subramaniyan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development