CSV to Metastock Data Conversion

This project will take a CSV file which contains either tick by tick data or 'bar' data which is collected by a separate system and turn this data into Metastock compatible data files and directory structures so the data can be read by Metastock. The data will come in as intraday data, but must be converted into daily data simply by assigning new, sequential dates to the intraday ticks or bars.

## Deliverables

## Detailed Description

### Inputs

There will be two inputs into the program. The first is a directory path for the output files, the second is the input csv file name. This input file name may include a directory path.

The csv formatted data files contain financial market data (primarily futures contracts traded on the Chicago Mercantile Exchange) that has been collected through a separate collection process. The program will need to read and parse these files

#### Data file Name

The name of the data file will include the "collection date", (in i.e. the file will be named similar to [url removed, login to view]). This date must be preserved from the filename in order to modify the symbol name (this is discussed later).

#### Description of data file

The input file is an ascii, csv file. The first line contains the titles:

Symbol,date,time,open,high,low,close,volume

Each subsequent line will contain a line as follows.

ESU0,2010-08-18,16:50:00, 543.75, 543.75, 543.75, 543.75,-1

There will be multiple symbols in each input file. Although they are intermixed on the input, the output must separate them into different data files.

### Outputs

#### Creating the Metastock output

The Metastock data structure consists of "index" files - EMASTER and MASTER, and a set of data files which contain the actual security data. This document IS NOT a reference for the file structure. There are references on the internet which discuss the actual file formats. I can supply the Metastock developers kit documentation, but it is not complete in that it does not provide functionality for writing these files, only reading them. I can also supply the C/C++ header files from the MDK, but this is not a guarantee that this is precisely the structure of the file.

Contained within the directory path there may (probably will) already exist ***EMASTER*** and ***MASTER*** files. Therefore, if these files exist, the program is required to read and modify the existing files, NOT create new ones. ONLY if the ***EMASTER*** and ***MASTER*** files do not exist should the program create new ones.

#### Transformation Summary

| Input Field Name | Output Field Name | Transformation |

| Symbol | Ticker | The symbol that is read from the input file must be modified to include the collection date. For example, if the symbol from the csv file is 6EZ0 and the collection date is 20101011 then the new ticker becomes 6EZ0_20101011 |

| Date | Date | The date in the output field WILL NOT be the same date as the input, with the exception of the first date. Beginning with the second input data row for any symbol, the date will increase one day at a time for each new record in the input file. Weekends will NOT be included in the output file.

For example, if the collection date is 20101011, and you are writing the 6th record for a given ticker symbol, the date for this row will be 2010-10-18. |

| Time | N/A | The time field is ignored |

| Open | Open | None |

| High | High | None |

| Low | Low | None |

| Close | Close | None |

| Volume | Volume | None |

### Other Output

The program should report to the command line the number of lines processed, the number of unique symbols processed, and the number of data rows per symbol. For example, the output might look like:

XXXXX_20101011 processed 15062 lines

8 symbols

6EZ0 3704 rows

6JZ0 1405 rows ....

## Error Handling

The program should gracefully handle errors. Particularly it should not damage the index and data files for the Metastock data if it encounters an error. Similarly, it should not leave the input file open.

## Language/Tool Requirements

This program must be built in either Perl or C/C++. If the program is built in C/C++, the stand-alone executable must be delivered along with all source files and a GNU compatible Makefile. The source code must be able to be compiled using gcc and the standard libraries delivered with gcc. If Perl is used, then just the script file is required, plus any non-standard libraries.

This program must be able to be built and run on Windows XP with cygwin . The ability to run on Mac OS X 10.5.8 is a plus.

## Testing Methodology

The program will be tested as follows:

1. In the case of C/C++, I will build the application using a gcc compiler and the provided Makefile.

2. I will run the application and verify the total number of lines processed from a new data file.

3. I will check each of the resulting data files from inside Metastock to ensure they have the expected number of records each.

4. I will create a excel chart using the CSV input data and I will visually compare this chart to the data using Metastock's graphing functions.

C Programming Engineering Mac OS Microsoft PHP Project Management Script Install Shell Script Software Architecture Software Testing Windows Desktop

Project ID: #3783396

About the project