Skip to content

[Proposal] Remove idle conjuncts in olap scan node which had alreadly push to Storage Engine #4986

Description

@HappenLee

Motivation

Now, Doris's OLAP_SCAN_NODE can push the conjuncts condition to the storage engine.
But after push it down to storage engine, the conditions still need to be filtered in the OLAP_SCANNER.

It's is a very unreasonable way. Therefore, we need to figure out the filtering conditions that have been pushed down to the storage engine. Delete these pushed down conditions on the OLAP SCANNER. We should not waste CPU resources on the conditions that have been filtered.

It can bring us the following benefits:

I Had Do POC in the data of SSB:

select * from customer where C_NATION = "CHINA" order by C_CUSTKEY limit 10, 15; 

There are 3000W rows of data

Cost TimeOrigin DorisDelete Push Down Conjuncts
5.2s4.3s

It's about 20% performance improvement

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions