Mathielen Import Engine Bundle

Introduction

This is a bundle for the mathielen/import-engine library. It provides an easy way to configure a full-blown data importer for your symfony2 project.

Installation

This library is available on Packagist:

To install it, run:

$ composer require mathielen/import-engine-bundle

Then add the bundle to app/AppKernel.php:

publicfunctionregisterBundles()
{
returnarray(
...
newMathielen\ImportEngineBundle\MathielenImportEngineBundle(),
...
);
}

If you want to make use of excel files, please also make sure to include phpoffice/phpexcel in your project:

$ composer require phpoffice/phpexcel

Configuration

Add your importer configurations in your app/config/config.yml.

Full example:

mathielen_import_engine:
#configure storageproviders, that are used in all importersstorageprovider:
default:
type: directoryuri: /tmp/somedirupload:
type: uploaduri: "%kernel.root_dir%/Resources/import"doctrine:
type: doctrinequeries: #a list of DQL-Statements, Entity-Classnames, filenames or directories
- SELECT id FROM Acme\DemoBundle\Entity\Person P WHERE P.age > 10 #dql statement
- Acme\DemoBundle\Entity\ImportData #entity classname- %kernel.root_dir%/dql/mysql.dql #file with dql statement in it- %kernel.root_dir%/other-dql #directorydbal:
type: dbalqueries: %kernel.root_dir%/sql/ #same like doctrineservices:
type: serviceservices:
#the services export_serviceA and export_serviceB must be configured in DICexport_serviceA: [exportMethod1, exportMethod2] #restrict to specific methods of serviceexport_serviceB: ~ #every method of service can be used#configure your Importersimporters:
your_importer_name:
#some context information that is passed through the whole processcontext:
key: value#automaticly recognize this importer by meeting of the conditions belowpreconditions:
format: excel #format of data must be [csv, excel, xml]fieldcount: 2#must have this number of fieldsfields: #these fields must exist (order is irrelevant)
- 'header2'
- 'header1'fieldset: #all fields must exist exactly this order
- 'header1'
- 'header2'filename: 'somefile.xls'#filename must match one of these regular expression(s) (can be a list)#use an object-factory to convert raw row-arrays to target objectsobject_factory:
type: jms_serializer #[jms_serializer, default]class: Acme\DemoBundle\ValueObject\MyImportedRow#add mappingmappings:
#simple a-to-b mappingsource-field1: target-field1#convert the field (but dont map)source-field2: #converts excel's date-field to a Y-m-d string (you can use your own service-id here)converter: mathielen_importengine.converter.excel.genericdate#map and convertsource-field3:
to: target-field3converter: upperCase #use a converter that was registered with the converter-provider#validate imported datavalidation:
source: #add constraints to source fieldsheader1: emailheader2: notemptytarget: ~ #activate validation against generated object from object-factory (via annotations, xml)#or supply list of constraints like in source#target of importtarget:
type: service #[service, doctrine, file]service: '@import_service'#service name in DICmethod: processImportRow #method to invoke on service

Minimum example:

mathielen_import_engine:
importers:
minimum_importer:
target:
type: fileuri: /tmp/myfile.csvformat: csvanother_minimum_importer:
target:
type: fileuri: "@='%kernel.root_dir%/../output_'~date('Y-m-d')~'.csv'"#this uses symfony expression language#to create the filename. Just prefix your#expression with @=format: { type: csv, arguments: [','] } #delimiter is now ',' 

Check out the Testsuite for more information.

Usage

On the command line

Show your configured Import profiles

$ app/console importengine:list

Let the framework discover which importer suites best (auto discovery)

Uses the storageprovider "default" if not also given as argument.

$ app/console importengine:import /tmp/somedir/myfile.csv

Import myfile.csv with "your_importer_name" importer

Uses the storageprovider "default" if not also given as argument.

$ app/console importengine:import -i your_importer_name /tmp/somedir/myfile.csv

Generate a JMS Serializer-annotated ValueObject class for an arbitrary import source (ie. a file)

$ app/console importengine:generate:valueobject data/myfile.csv Acme\\ValueObject\\MyFileRow src

Use the importer within a controller / service

namespaceAppBundle\Controller;
useSensio\Bundle\FrameworkExtraBundle\Configuration\Route;
useSensio\Bundle\FrameworkExtraBundle\Configuration\Method;
useSymfony\Bundle\FrameworkBundle\Controller\Controller;
class DefaultController extends Controller
{
/** * Import a given file, that was POST'ed to the HTTP-Endpoint /app/import * * Using the default sorage provider * * The importer is auto-discovered with the format of the file * * @Route("/app/import", name="homepage") * @Method("POST") */publicfunctionimportAction(\Symfony\Component\HttpFoundation\Request$request)
{
//create the request for the import-engine$importRequest = new \Mathielen\ImportEngine\ValueObject\ImportRequest($request->files->getIterator()->current());
/** @var \Mathielen\ImportEngine\Import\ImportBuilder $importBuilder */$importBuilder = $this->container->get('mathielen_importengine.import.builder');
$import = $importBuilder->build($importRequest);
/** @var \Mathielen\ImportEngine\Import\Run\ImportRunner $importRunner */$importRunner = $this->container->get('mathielen_importengine.import.runner');
$importRun = $importRunner->run($import);
return$this->render('default/import.html.twig', $importRun->getStatistics());
}
}