Data formats
Why are data formats important?
The choice of formats for data use, analysis, back-up and storage strongly affects the lifespan of data and any eventual re-use.
Computer programs are needed to render all digital data meaningful and accessible. Accordingly, all digital data is software-dependent. Therefore data is always at risk of becoming unusable through software obsolescence.
The best way to guarantee long-term data usability and access is to use standard formats that most software can easily interpret.
This generally means using non-proprietary software, and
software formats based on open standards, such as:
- Open Document Format (ODF)
- ASCII
- tab-delimited format
- comma-separated values
- XML
instead of using proprietary formats.
Some proprietary formats, such as SPSS, and Microsoft's Office programs, are likely to be accessible for some time. However, to ensure longevity, these formats should probably be converted to open formats prior to any long term data deposit.
Data issues to consider include:
- could the hardware, software or digital storage media fail or become obsolete within the project's lifetime?
- what impact would such a failure have on the project?
- what level of technical support is available long term?
- can existing data be migrated easily to new hardware and software platforms?
Be sure to list in your data management plan:
- what data formats will be used
- what special software will be used, if any
- what the expected data volume will be
- what the data access requirements are
Researchers should also consider the longevity of any hardware used to create, analyse or visualise research data as well.
The Australian Code for the Responsible Conduct of Research is clear about your responsibilities around the management of data. The Code expects researchers to:
2.6.4 Retain research data, including electronic data, in a durable, indexed and retrievable form.
2.6.5 Maintain a catalogue of research data in an accessible form.
Best Practice
- Choose non-proprietary, well-documented file formats that will last as long as the Code specifies.
- Save your files in their original state - uncompressed and unencrypted.
- Choose formats that will not be complicated to migrate to new formats.
- If your work needs specific software to be understood, archive the software and its documentation as well as the data.
Tip
Format checklists
The Australian National Data Service provides advice on formats.
The UK Data Archive has a list of recommended file formats. They also provide advice on the suitability of formats for file conversion.
Loading