[MAPREDUCE-7368] DBOutputFormat.DBRecordWriter#write must throw exception when it fails - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.3.1
Fix Version/s: 3.4.0
Component/s: mrv2
Labels:
- pull-request-available

Target Version/s:

3.4.0, 3.3.1
Hadoop Flags:

Reviewed

Description

When the DBRecordWriter#write fails with an SQLException the problem is not propagated but printed in System.err instead.

public void write(K key, V value) throws IOException {
      try {
        key.write(statement);
        statement.addBatch();
      } catch (SQLException e) {
        e.printStackTrace();
      }
    }

The consumer of this API has no way to tell that the write failed. Moreover, the exception is not present in the logs which makes the problem very hard debug and can easily lead to data corruption since clients can easily assume that everything went well.

Attachments

Issue Links

relates to

HIVE-25717 INSERT INTO on external MariaDB/MySQL table fails silently

Open

supercedes

MAPREDUCE-2548 Log improvements in DBOutputFormat.java and CounterGroup.java

Resolved

links to

GitHub Pull Request #3671

Activity

People

Assignee:: Stamatis Zampetakis

Reporter:: Stamatis Zampetakis

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 17/Nov/21 16:01

Updated:: 11/Feb/24 23:19

Resolved:: 08/Dec/21 11:11

Time Tracking

Estimated:

Not Specified

Remaining:

Logged: