[IMPALA-6943] ORC support with full functionality - ASF JIRA

Agile Board

Attach files

Attach Screenshot

Add vote

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Epic
Status: Open
Priority: Critical
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: Backend
Labels:
None

Epic Name:
Basic ORC Support
Epic Color:
ghx-label-2

Description

Support basic functionality for reading ORC file format tables including stability works for edge cases. This is the first milestone to make ORC support GA. Other advanced features will be tracked in IMPALA-9072.

Attachments

Issue Links

Add Link

contains

IMPALA-8801 Add DATE type support to ORC scanner

Closed

Delete this link

is related to

IMPALA-9040 Read performance improvements in ORC support

Open

Delete this link

IMPALA-9072 Advanced features and write support in ORC support

Open

Delete this link

Issues in epic

quick-create-issue-for-epic-label

IMPALA-5717	Support reading primitive types from ORC format files	Resolved	Quanlong Huang	Actions
IMPALA-6503	Support reading complex types from ORC format files	Resolved	Quanlong Huang	Actions
IMPALA-6772	Enable test_scanners_fuzz for ORC format	Resolved	Quanlong Huang	Actions
IMPALA-7730	Improve ORC File Format Timezone issues	Open	Csaba Ringhofer	Actions
IMPALA-8046	Support CREATE TABLE from an ORC file	Resolved	Norbert Luksa	Actions
IMPALA-8184	Add timestamp validation to Orc scanner	Resolved	Csaba Ringhofer	Actions
IMPALA-9042	Support reading full-ACID ORC tables	Resolved	Zoltán Borók-Nagy	Actions
IMPALA-9074	Add support for zstd in ORC	Resolved	Quanlong Huang	Actions
IMPALA-9174	Revisit the memory pattern of the ORC scanner	Resolved	Zoltán Borók-Nagy	Actions
IMPALA-9175	Revisit the error handling logics in ORC scanner	In Progress	Norbert Luksa	Actions
IMPALA-9226	Improve string allocations of the ORC scanner	Resolved	Norbert Luksa	Actions
IMPALA-9228	ORC scanner could be vectorized	Resolved	Gabor Kaszab	Actions
IMPALA-9249	HdfsOrcScanner crash on resolving file schema	Resolved	Csaba Ringhofer	Actions
IMPALA-9250	HdfsOrcScanner crash in calling orc::createReader	Resolved	Quanlong Huang	Actions
IMPALA-9251	HdfsOrcScanner crash in AssembleRows	Resolved	Quanlong Huang	Actions
IMPALA-9277	Crash due to unhandled exception thrown from orc::ColumnSelector::updateSelectedByTypeId	Resolved	Zoltán Borók-Nagy	Actions
IMPALA-9288	Crash due to memory leak in scan node of orc scanner	Resolved	Zoltán Borók-Nagy	Actions
IMPALA-9290	ORC scanner should support schema evolution between date and timestamp types	In Progress	Norbert Luksa	Actions
IMPALA-9297	HdfsOrcScanner crash in orc::RleDecoderV2::next	Resolved	Zoltán Borók-Nagy	Actions
IMPALA-9306	HdfsOrcScanner crashes in orc::readFully	Resolved	Quanlong Huang	Actions
IMPALA-9324	HdfsOrcScanner crashes in DCHECK failure in OrcSchemaResolver::BuildSchemaPath	Resolved	Quanlong Huang	Actions
IMPALA-9410	Support resolving ORC file columns by names	Resolved	Gergely Fürnstáhl	Actions
IMPALA-13053	Update test_max_nesting_depth test to write Orc with Impala	In Progress	Michael Smith	Actions