[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[jira] [Created] (FLINK-11184) Rework TableSource and TableSink interfaces

Timo Walther created FLINK-11184:

             Summary: Rework TableSource and TableSink interfaces
                 Key: FLINK-11184
             Project: Flink
          Issue Type: New Feature
          Components: Table API & SQL
            Reporter: Timo Walther
            Assignee: Timo Walther

There are a couple of shortcomings with the current {{TableSource}} and {{TableSink}} interface design. Some of the issues are covered in a [basic design document|] that was published a while ago.

The design document has not been updated for some time and partially overlaps with the [current SQL DDL discussion|] for the {{CREATE TABLE}} statement on the ML.

What needs to be solved:
- How to unify sources and sinks in regards of schema and time attributes?
- How to define watermarks, timestamp extractors or timestamp ingestion?
- How to define primary keys and partitioning keys?
- How to differentiate between update modes for tables (i.e. how to read from a append, retraction, or update table)?
- How to express all of the above without pulling in to many dependencies on other Flink modules if source and sink interfaces are located in {{flink-table-spi}} package?

As of the current state of the discussion, it seems that we might extend {{TableSchema}} to allow for returning the information above and remove current interfaces such as {{DefinedRowtimeAttribute}} or {{DefinedFieldMapping}}.

This message was sent by Atlassian JIRA