Building a modular data reader system

Posted on

Problem

Context

I have a project which has an importation module to import content of Excel files.

Originally, importation was possible only for .xls file (< 2003) through Interop assemblies through a pretty ugly non-modular architecture. But there was a need to support importation of .xlsx files.

I have refactored the module to build a system of data readers. Depending on the extension of the files that is imported, a data reader class is instantiated accordingly to be able to read the data from either an Excel <2003 file or an Excell >2003 file transparently.

So I have two classes Excel2003DataReader and OpenXmlDataReader both based on the following interface:

public interface IImporterDataReader: IDisposable
{
    object DataFile { get; }

    bool HasValue(string workSheetName, int rowIdx, int columnIdx);

    string GetValue(string workSheetName, int rowIdx, int columnIdx);

    List < RangeItem > GetRangeValues(string workSheetName, int rowIdx, int columnIdx, ExtractDirection direction = ExtractDirection.Horizontal, int amountExtraValues = 0);
}​

The IImporterDataReader instance is then passed a parameter to the Import method:

void Import(XmlDocument calculator, IImporterDataReader dataSource, Dictionary<string, string> parameters = null)

So dataSource.GetValue() can be used inside the method without worrying what kind of Excel file is actually imported.

The problem

The problem is that this system is too tightly adressed to Excel files. What I need to be able to reader xml files or json files as well ?

I was thinking of abstracting the way the parameters are passed to the methods HasValue()and GetValue() so they could be different per data reader implementations: have workSheetName, row and column properties for excel files, have an xpath property for xml files, etc.

Could be something like this:

public interface IImporterDataReader: IDisposable
{
    object DataFile { get; }

    bool HasValue(IDataAccessor accessor);

    string GetValue(IDataAccessor accessor);

    // Let's forget about this one right now
    //List < RangeItem > GetRangeValues(string workSheetName, int rowIdx, int columnIdx, ExtractDirection direction = ExtractDirection.Horizontal, int amountExtraValues = 0);
}​

Question

I haven’t honestly tried implementing such a mechanism, this is still in my mind.

  • Is this an interesting path to go for to implement such a mechanism ?
  • Also what could the accessor look like ?

I have the feeling this is a bit too complicated for the purpose, but maybe not.

Solution

Since there should be a relation between your importer data reader and data accessor ( importer should know how to use data accessor to access data.) I suggest that you use generics to achieve such coherence.(Since each importer needs its unique data then IDataAccessors can’t be replaced by each other thus I don’t think that’s a good idea to introduce an abstraction for them)

public abstract ImporterDataReader<TAccessor>:IImporterDataReader
{
public abstract bool HasValue(TAccessor accessor);

public abstract string GetValue(TAccessor accessor);
}

Then you can have concrete classes for each type of Accessor:

public class JsonDataReader:ImporterDataReader<JsonDataAccessor>
{
....
}

Leave a Reply

Your email address will not be published. Required fields are marked *