ext-opencsv

ext-opencsv

A powerful and flexible CSV extension library built on top of OpenCSV, providing enhanced features for reading and writing CSV files with support for batch processing, nested objects, and collections.

Features

• 🚀 Fluent Builder API - Easy-to-use builder pattern for both reading and writing
• 📦 Batch Processing - Efficient memory management with configurable batch sizes
• 🔗 Nested Object Support - Export/import nested objects using dot notation
• 📚 Collection Handling - Handle Lists and Sets within your data models
• 🏷️ Annotation-Based - Use @Exportable annotation for field mapping
• ⚙️ Highly Configurable - Custom delimiters, quote characters, and escape characters
• 🔄 Row Transformation - Transform CSV rows into Java objects with custom logic
• 💾 Memory Efficient - Stream processing with batch handling for large files

Installation

Gradle

dependencies {
implementation 'com.javaquery:ext-opencsv:1.0.0'
}

Maven

<dependency>
<groupId>com.javaquery</groupId>
<artifactId>ext-opencsv</artifactId>
<version>1.0.0</version>
</dependency>

Quick Start

Writing CSV Files

Basic Example

importcom.javaquery.opencsv.writer.CsvWriter;
importjava.io.File;
importjava.util.List;
// Create your data objectsList<Customer> customers = getCustomers();
// Write to CSVCsvWriter.<Customer>builder()
.headers(List.of("First Name", "Last Name", "Email"))
.keys(List.of("firstName", "lastName", "email"))
.data(customers)
.toFile(newFile("customers.csv"))
.write();

Writing with Nested Objects

CsvWriter.<Customer>builder()
.headers(List.of("First Name", "Last Name", "Passport Number", "Passport Country"))
.keys(List.of("firstName", "lastName", "passport.passportNumber", "passport.country"))
.data(customers)
.toFile(newFile("customers.csv"))
.write();

Writing with Collections

When your objects contain collections (List or Set), the writer automatically creates multiple rows for each collection item:

CsvWriter.<Customer>builder()
.headers(List.of("First Name", "Last Name", "Address Line", "City", "State"))
.keys(List.of("firstName", "lastName", "addresses.addressLine1", "addresses.city", "addresses.state"))
.data(customers)
.toFile(newFile("customers_addresses.csv"))
.write();

Custom CSV Format

CsvWriter.<Customer>builder()
.headers(List.of("First Name", "Last Name", "Email"))
.keys(List.of("firstName", "lastName", "email"))
.data(customers)
.toFile(newFile("customers.csv"))
.delimiter('|') // Pipe-delimited
.quoteChar('\'') // Single quote
.escapeChar('\\') // Backslash escape
.lineEnd("\r\n") // Windows line ending
.includeHeader(false) // Exclude header row
.write();

Reading CSV Files

Basic Example

importcom.javaquery.opencsv.reader.CsvReader;
importcom.javaquery.helper.BatchProcessor;
importjava.io.File;
importjava.util.ArrayList;
importjava.util.List;
List<Customer> allCustomers = newArrayList<>();
CsvReader.<Customer>builder()
.source(newFile("customers.csv"))
.rowTransformer((headers, rowValues, previousRow) -> Customer.builder()
.firstName(rowValues[0])
.lastName(rowValues[1])
.email(rowValues[2])
.build()
)
.batchProcessor(batch -> {
allCustomers.addAll(batch);
// Or process batch (e.g., save to database)
})
.batchSize(1000)
.read();

With Completion Callback

CsvReader.<Customer>builder()
.source(newFile("customers.csv"))
.rowTransformer((headers, rowValues, previousRow) -> Customer.builder()
.firstName(rowValues[0])
.lastName(rowValues[1])
.email(rowValues[2])
.build()
)
.batchProcessor(newBatchProcessor<Customer>() {
@OverridepublicvoidonBatch(List<Customer> batch) {
// Process each batchSystem.out.println("Processing batch of " + batch.size() + " customers");
customerRepository.saveAll(batch);
}
@OverridepublicvoidonComplete(inttotalProcessed, inttotalBatches) {
System.out.println("Processed " + totalProcessed + " records in " + totalBatches + " batches");
}
})
.batchSize(500)
.read();

Skip Invalid Rows

CsvReader.<Customer>builder()
.source(newFile("customers.csv"))
.rowTransformer((headers, rowValues, previousRow) -> {
try {
returnCustomer.builder()
.firstName(rowValues[0])
.lastName(rowValues[1])
.age(Integer.parseInt(rowValues[2]))
.build();
} catch (NumberFormatExceptione) {
// Return null to skip invalid rowsreturnnull;
}
})
.batchProcessor(batch -> allCustomers.addAll(batch))
.read();

Custom CSV Format

CsvReader.<Customer>builder()
.source(newFile("customers.tsv"))
.delimiter('\t') // Tab-delimited
.quoteChar('\'') // Single quote
.escapeChar('\\') // Backslash escape
.skipLines(1) // Skip first line (e.g., metadata)
.rowTransformer((headers, rowValues, previousRow) -> Customer.builder()
.firstName(rowValues[0])
.lastName(rowValues[1])
.build()
)
.batchProcessor(batch -> allCustomers.addAll(batch))
.batchSize(2000)
.read();

Access Previous Row During Transformation

CsvReader.<Customer>builder()
.source(newFile("customers.csv"))
.rowTransformer((headers, rowValues, previousRow) -> {
Customercustomer = Customer.builder()
.firstName(rowValues[0])
.lastName(rowValues[1])
.build();
// Access previous row for contextif (previousRow != null) {
// Use previous row data for processingcustomer.setSameAddressAsPrevious(true);
}
returncustomer;
})
.batchProcessor(batch -> allCustomers.addAll(batch))
.read();

Data Model Setup

Use the @Exportable annotation to mark fields for CSV export:

importcom.javaquery.annotations.Exportable;
publicclassCustomer {
@Exportable(key = "firstName")
privateStringfirstName;
@Exportable(key = "lastName")
privateStringlastName;
@Exportable(key = "email")
privateStringemail;
@Exportable(key = "age")
privateIntegerage;
@Exportable(key = "passport")
privatePassportpassport;
@Exportable(key = "addresses")
privateSet<Address> addresses;
// Getters and setters
}

Nested Object Example

publicclassPassport {
@Exportable(key = "passportNumber")
privateStringpassportNumber;
@Exportable(key = "country")
privateStringcountry;
@Exportable(key = "expirationDate")
privateStringexpirationDate;
}

Collection Example

publicclassAddress {
@Exportable(key = "addressLine1")
privateStringaddressLine1;
@Exportable(key = "city")
privateStringcity;
@Exportable(key = "state")
privateStringstate;
@Exportable(key = "zipCode")
privateStringzipCode;
}

Advanced Usage

Custom Batch Processing Strategy

classDatabaseBatchProcessorimplementsBatchProcessor<Customer> {
privatefinalCustomerRepositoryrepository;
privatefinalintcommitThreshold;
privateinttotalSaved = 0;
@OverridepublicvoidonBatch(List<Customer> batch) {
repository.saveAll(batch);
totalSaved += batch.size();
if (totalSaved >= commitThreshold) {
repository.flush();
totalSaved = 0;
}
}
@OverridepublicvoidonComplete(inttotalProcessed, inttotalBatches) {
repository.flush(); // Final flushSystem.out.println("Import complete: " + totalProcessed + " records");
}
}
// UsageCsvReader.<Customer>builder()
.source(newFile("customers.csv"))
.rowTransformer(this::transformRow)
.batchProcessor(newDatabaseBatchProcessor(customerRepository, 10000))
.batchSize(1000)
.read();

Dynamic Header Mapping

CsvReader.<Customer>builder()
.source(newFile("customers.csv"))
.rowTransformer((headers, rowValues, previousRow) -> {
Customercustomer = newCustomer();
// Find column index dynamicallyfor (inti = 0; i < headers.length; i++) {
switch (headers[i].toLowerCase()) {
case"first name":
case"firstname":
customer.setFirstName(rowValues[i]);
break;
case"last name":
case"lastname":
customer.setLastName(rowValues[i]);
break;
case"email":
case"email address":
customer.setEmail(rowValues[i]);
break;
}
}
returncustomer;
})
.batchProcessor(batch -> allCustomers.addAll(batch))
.read();

Error Handling

try {
CsvWriter.<Customer>builder()
.headers(List.of("First Name", "Last Name", "Email"))
.keys(List.of("firstName", "lastName", "email"))
.data(customers)
.toFile(newFile("customers.csv"))
.write();
} catch (IOExceptione) {
System.err.println("Failed to write CSV: " + e.getMessage());
}
try {
CsvReader.<Customer>builder()
.source(newFile("customers.csv"))
.rowTransformer(this::transformRow)
.batchProcessor(this::processBatch)
.read();
} catch (IOExceptione) {
System.err.println("Failed to read CSV: " + e.getMessage());
} catch (IllegalArgumentExceptione) {
System.err.println("Configuration error: " + e.getMessage());
}

Configuration Options

CsvWriter Options

Option	Type	Default	Description
headers	List<String>	Required (if includeHeader=true)	Column headers
keys	List<String>	Required	Field keys (supports dot notation)
data	Iterable<T>	Required	Data to write
toFile	File	Required	Destination file
delimiter	char	,	Field delimiter
quoteChar	char	"	Quote character
escapeChar	char	"	Escape character
lineEnd	String	\n	Line ending
includeHeader	boolean	true	Include header row

CsvReader Options

Option	Type	Default	Description
source	File	Required	Source CSV file
rowTransformer	CsvRowTransformer<T>	Required	Row transformation function
batchProcessor	BatchProcessor<T>	Required	Batch processing handler
delimiter	char	,	Field delimiter
quoteChar	char	"	Quote character
escapeChar	char	"	Escape character
skipLines	int	0	Number of lines to skip
batchSize	int	1000	Records per batch

Performance Tips

1. Batch Size: Choose appropriate batch size based on your memory constraints

   • For large files: 500-1000 records
   • For small files: 5000-10000 records

2. Memory Management: The reader processes files in batches to avoid loading entire file into memory

3. Collection Handling: Be aware that writing collections creates multiple rows per parent object

4. Null Transformers: Return null from rowTransformer to skip invalid rows without throwing exceptions

Requirements

• Java 11 or higher
• OpenCSV 5.12.0 or higher

License

This library is part of the JLite project. See LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Support

For issues, questions, or contributions, please visit the GitHub repository.

Author

Vicky Thakor
JavaQuery

---

Version: 1.0.0
Last Updated: December 2025