Skip to content

STORM-1778: scheme extension framework for KafkaSourceProvider and csv s… - #1410

Closed
chuanlei wants to merge 2 commits into
apache:masterfrom
chuanlei:feature-scheme-extension-framework-and-csv-support
Closed

STORM-1778: scheme extension framework for KafkaSourceProvider and csv s…#1410
chuanlei wants to merge 2 commits into
apache:masterfrom
chuanlei:feature-scheme-extension-framework-and-csv-support

Conversation

@chuanlei

@chuanleichuanlei commented May 11, 2016

Copy link
Copy Markdown

feature: scheme extension framework for KafkaSourceProvider and csv scheme support

HeartSaVioR
HeartSaVioR reviewed May 17, 2016
View reviewed changes
Comment threadexternal/sql/storm-sql-kafka/src/jvm/org/apache/storm/sql/kafka/CsvScheme.java
}

@Override
public List<Object> deserialize(ByteBuffer ser) {

@HeartSaVioRHeartSaVioRMay 17, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I think we can have some test cases on this.
  2. StringScheme#deserializeString doesn't use ByteBuffer#toString, and javadoc says it provides information about instance state, not content: https://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html#toString()

@HeartSaVioR

HeartSaVioR commented May 17, 2016

Copy link
Copy Markdown
Contributor

Btw, how about adding TSV? It's similar approach to CSV so easy to address.

vesense
vesense reviewed May 26, 2016
View reviewed changes
Comment threadexternal/sql/storm-sql-kafka/src/jvm/org/apache/storm/sql/kafka/CsvScheme.java
@Override
public List<Object> deserialize(ByteBuffer ser) {
String str = ser.toString();
String[] parts = str.split(",");

@vesensevesenseMay 26, 2016

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not standard CSV format, something like a,"b,c",d is also valid. We can use TSV format to address.

@chuanlei

chuanlei commented May 27, 2016

Copy link
Copy Markdown
Author

@vesense@HeartSaVioR
Thank you for the suggestion, I will fix it asap.

@vesense

vesense commented Nov 16, 2016

Copy link
Copy Markdown
Member

@chuanlei I think this issue has been fixed by STORM-2173. Can you close this?
BTW, theCsvScheme in STORM-2173 uses standard RFC4180 CSV parser supporting strings like a,"b,c",d.

d2r pushed a commit to d2r/storm that referenced this pull request Oct 16, 2018
Derek Dagit
Closes Pull Requests unchanged in 2018
fcda81c
We are closing stale Pull Requests to make the list more manageable.
Please re-open any Pull Request that has been closed in error.
Closesapache#608Closesapache#639Closesapache#640Closesapache#648Closesapache#662Closesapache#668Closesapache#692Closesapache#705Closesapache#724Closesapache#728Closesapache#730Closesapache#753Closesapache#803Closesapache#854Closesapache#922Closesapache#986Closesapache#992Closesapache#1019Closesapache#1040Closesapache#1041Closesapache#1043Closesapache#1046Closesapache#1051Closesapache#1078Closesapache#1146Closesapache#1164Closesapache#1165Closesapache#1178Closesapache#1213Closesapache#1225Closesapache#1258Closesapache#1259Closesapache#1268Closesapache#1272Closesapache#1277Closesapache#1278Closesapache#1288Closesapache#1296Closesapache#1328Closesapache#1342Closesapache#1353Closesapache#1370Closesapache#1376Closesapache#1391Closesapache#1395Closesapache#1399Closesapache#1406Closesapache#1410Closesapache#1422Closesapache#1427Closesapache#1443Closesapache#1462Closesapache#1468Closesapache#1483Closesapache#1506Closesapache#1509Closesapache#1515Closesapache#1520Closesapache#1521Closesapache#1525Closesapache#1527Closesapache#1544Closesapache#1550Closesapache#1566Closesapache#1569Closesapache#1570Closesapache#1575Closesapache#1580Closesapache#1584Closesapache#1591Closesapache#1600Closesapache#1611Closesapache#1613Closesapache#1639Closesapache#1703Closesapache#1711Closesapache#1719Closesapache#1737Closesapache#1760Closesapache#1767Closesapache#1768Closesapache#1785Closesapache#1799Closesapache#1822Closesapache#1824Closesapache#1844Closesapache#1874Closesapache#1918Closesapache#1928Closesapache#1937Closesapache#1942Closesapache#1951Closesapache#1957Closesapache#1963Closesapache#1964Closesapache#1965Closesapache#1967Closesapache#1968Closesapache#1971Closesapache#1985Closesapache#1986Closesapache#1998Closesapache#2031Closesapache#2032Closesapache#2071Closesapache#2076Closesapache#2108Closesapache#2119Closesapache#2128Closesapache#2142Closesapache#2174Closesapache#2206Closesapache#2297Closesapache#2322Closesapache#2332Closesapache#2341Closesapache#2377Closesapache#2414Closesapache#2469
d2r pushed a commit to d2r/storm that referenced this pull request Oct 16, 2018
Derek Dagit
Closes Pull Requests unchanged in 2018
fdb95d0
We are closing stale Pull Requests to make the list more manageable.
Please re-open any Pull Request that has been closed in error.
Closesapache#608Closesapache#639Closesapache#640Closesapache#648Closesapache#662Closesapache#668Closesapache#692Closesapache#705Closesapache#724Closesapache#728Closesapache#730Closesapache#753Closesapache#803Closesapache#854Closesapache#922Closesapache#986Closesapache#992Closesapache#1019Closesapache#1040Closesapache#1041Closesapache#1043Closesapache#1046Closesapache#1051Closesapache#1078Closesapache#1146Closesapache#1164Closesapache#1165Closesapache#1178Closesapache#1213Closesapache#1225Closesapache#1258Closesapache#1259Closesapache#1268Closesapache#1272Closesapache#1277Closesapache#1278Closesapache#1288Closesapache#1296Closesapache#1328Closesapache#1342Closesapache#1353Closesapache#1370Closesapache#1376Closesapache#1391Closesapache#1395Closesapache#1399Closesapache#1406Closesapache#1410Closesapache#1422Closesapache#1427Closesapache#1443Closesapache#1462Closesapache#1468Closesapache#1483Closesapache#1506Closesapache#1509Closesapache#1515Closesapache#1520Closesapache#1521Closesapache#1525Closesapache#1527Closesapache#1544Closesapache#1550Closesapache#1566Closesapache#1569Closesapache#1570Closesapache#1575Closesapache#1580Closesapache#1584Closesapache#1591Closesapache#1600Closesapache#1611Closesapache#1613Closesapache#1639Closesapache#1703Closesapache#1711Closesapache#1719Closesapache#1737Closesapache#1760Closesapache#1767Closesapache#1768Closesapache#1785Closesapache#1799Closesapache#1822Closesapache#1824Closesapache#1844Closesapache#1874Closesapache#1918Closesapache#1928Closesapache#1937Closesapache#1942Closesapache#1951Closesapache#1957Closesapache#1963Closesapache#1964Closesapache#1965Closesapache#1967Closesapache#1968Closesapache#1971Closesapache#1985Closesapache#1986Closesapache#1998Closesapache#2031Closesapache#2032Closesapache#2071Closesapache#2076Closesapache#2108Closesapache#2119Closesapache#2128Closesapache#2142Closesapache#2174Closesapache#2206Closesapache#2297Closesapache#2322Closesapache#2332Closesapache#2341Closesapache#2377Closesapache#2414Closesapache#2469
@d2rd2r mentioned this pull request Oct 16, 2018
Merged
bipinprasad pushed a commit to bipinprasad/storm that referenced this pull request Oct 17, 2019
Aaron GreschGitHub Enterprise
Merge pull requestapache#1410from agresch/agresch_ystorm_6578_2x
4ef7494
YSTORM-6578 add log entries for YCA cert updates
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@chuanlei@HeartSaVioR@vesense