nifti2_data_format > NIfTI-2: a 64-bit update of NIfTI1 (approved)
Showing 1-6 of 6 posts10 Subscribers
replyDisplay: Results per page:
Mar 15, 2011  10:03 AM
NIfTI-2: a 64-bit update of NIfTI1 (approved)
A 64-bit update to the NIfTI format (approved by the NIFTI Data Format Working Group on March 16, 2011)

The NIfTI committee, in conjunction with public discussion on the NITRC forum, have created a final version for the NIfTI-2 format that is a very simple extension of the current NIfTI-1 format, but updated to allow 64-bit storage and addressing for large images and matrices.  

It is intended that this format:
 - will enable the storage of large images and matrices, with all dimensions being coded by 64-bit integers rather than the current limitation of 16-bit signed integers (which currently gives a restrictive 32767 size limit in each dimension)
 - will be very simple to implement and update software (a couple of hours coding or less generally)
 - contains the same information and logic as a NIfTI-1 file (no new fields and all current fields still retained)
 - will support the existing file formats and naming: a .nii single-file, a .hdr/.img file-pair and their gzipped versions
 - has a very simple test (see below) to determine if the file is NIfTI-1 or NIfTI-2
 - does not replace NIfTI-1 in the short term but is supported alongside it

Comments
This revised version is the result of initial committee discussions and public discussion on the NITRC forum.In addition to this format, we are seeking comments relating to larger-scale changes and requests for other types of format change be posted to a separate NITRC discussion list for more advanced neuroimaging formats (www.nitrc.org/forum/forum.php?forum_id=1942)

Supporting software
 - AFNI, BrainVisa, BrainVoyager, Caret, Connectome Workbench, Fiswidgets, FreeSurfer, FSL, ITK, LONI-DIRAC, Mango, MRIcron, NiBabel, R, SPM and TractoR have all agreed to support the NIfTI-2 version in their upcoming releases, but that NIfTI-1 will remain a default output, or a configurable default, in the short-term
 - the sourceforge supporting libraries in niftilib will be updated so that they seemlessly support both NIfTI-1 and NIfTI-2
 - conversion utilities to and from NIfTI-1 will be made available via sourceforge as well as in some of the software packages mentioned above

Changes to the header
The changes to the NIfTI header structure are the following (also see below for the full header struct):
   - short dim[8] becomes int64_t dim[8]
   - float intent_p1 becomes double intent_p1
   - float intent_p2 becomes double intent_p2
   - float intent_p3 becomes double intent_p3
   - float pixdim[8] becomes double pixdim[8]
   - float vox_offset becomes int64_t vox_offset
   - float scl_slope becomes double scl_slope
   - float scl_inter becomes double scl_inter
   - float cal_max becomes double cal_max
   - float cal_min becomes double cal_min
   - float slice_duration becomes double slice_duration
   - float toffset becomes double toffset
   - short slice_start becomes int64_t slice_start
   - short slice_end becomes int64_t slice_end
   - char slice_code becomes int slice_code
   - char xyzt_units becomes int xyzt_units
   - short intent_code becomes int intent_code
   - short qform_code becomes int qform_code
   - short sform_code becomes int sform_code
   - float quatern_b becomes double quatern_b
        and similarly for quatern_c, quatern_d, qoffset_x, qoffset_y, qoffset_z
   - float srow_x[4] becomes double srow_x[4]
        and similarly for srow_y[4], srow_z[4]
   - char magic[4] becomes char magic[8]
   - char unused_str[15] is added at the end of the header   
   - removing previously unused fields: data_type, db_name, extents, session_error, regular, glmax, glmin  
   - reordering of some of the fields to ensure 8-byte alignments for all doubles 
The size and type of all other fields remains unchanged.
The sizeof_hdr must store 540 for a NIfTI-2 file instead of 348 for NIfTI-1.
Extension information is still held in the first 4 bytes after this header (bytes 541-544) in the same way as in NIfTI-1.
Note that the total block size of 544 (header + 4 bytes) retains the 16-byte alignment in NIfTI-1 (348+4=352), as needed for convenient memory-mapping.
The changes of float to double are generally made to allow for more accurate mapping of intensities or indices when dealing with very large arrays, and unused_str is added to ensure the 16-byte alignment holds. The 8-byte alignment of doubles is important of some platforms and necessitated shifting of several smaller fields.
In addition, a range of custom codes for NIFTI_INTENT_*, NIFTI_XFORM_*, NIFTI_UNITS_* and NIFTI_SLICE_* will be created, although these are intended for temporary use only and users are encouraged to register any codes that are useful in the long-term with the NIfTI committee in order for them to become usable by all and keep NIfTI an open standard without private or hidden information.

Compatibility
This format is not, and cannot be, bit-wise compatible with the previous NIfTI-1 (or ANALYZE) formats and so will not work with existing NIfTI-1 software directly. In the short term this can be solved via conversion utilities. A NIfTI-2 image will not be recognised as a valid NIfTI-1 image by existing software and so no incorrect processing or analyses should occur. In the longer term most code should be very easily converted to work with both NIfTI-2 and NIfTI-1 by using the updated sourceforge libraries or making equivalent changes internally.  The change in format is intentionally very minimal and should be very easy to code.

Determining the NIfTI version
The following pseudo-code shows one way that a file can be tested to see: (a) if it is a NIfTI image file, (b) what version of NIfTI it is, and (c) whether byte-swapping is required.

read in the first 4 bytes from the file
let d = the content of these bytes, formatted as a 32-bit int
if (d==348) then it is a NIfTI-1 file, no byte-swapping required
else if (swap_4bytes(d)==348) then it is a NIfTI-1 file, but with byte-swapping required
else if (d==540) then it is a NIfTI-2 file, no byte-swapping required
else if (swap_4bytes(d)==540) then it is a NIfTI-2 file, but with byte-swapping required
else it is not a valid NIfTI file
read in magic[] string from appropriate place (depending on if it is NIfTI-1 or NIfTI-2)
check validity of the NIfTI file by testing the magic[] string (should contain "ni1" or "n+1" followed by \0 for NIfTI-1, and "ni2" or "n+2" followed by \0 and 4 extra signature bytes for NIfTI-2)

if it passed the above tests then read in the full header into the appropriate NIfTI-1 or NIfTI-2 struct

 
Magic Signature
The magic string is expanded to 8 bytes with the first 4 bytes containing ones of the strings "ni2" or "n+2", terminated with \0.  The next four bytes form a magic signature much the same as used by the PNG format (http://www.libpng.org/pub/png/pngintro.html), to detect for file transfer errors involving newline characters.  The extra four bytes are the same as the last four in the PNG format - that is: \r \n \032 \n (0D 0A 1A 0A)
The magic string has a different offset from the start of the file in NIfTI-1 and NIfTI-2 (344 and 4 bytes respectively).

Timeline
Barring any unknown unknowns or unforseeable unforseen problems, it is intended to begin rolling out this format in practice in March-April 2011.  Example datasets and updated code will also be provided soon.
 
C Header Struct
/*! \struct nifti_2_header
    \brief Data structure defining the fields in the nifti2 header.
           This binary header should be found at the beginning of a valid
           NIFTI-2 header file.
 */
                        /*************************/  /************************/ /************/
struct nifti_2_header { /* NIFTI-2 usage         */  /* NIFTI-1 usage        */ /*  offset  */
                        /*************************/  /************************/ /************/
int   sizeof_hdr;     /*!< MUST be 540           */  /* int sizeof_hdr; (348) */  /*   0 */
char  magic[8] ;      /*!< MUST be valid signature. */  /* char magic[4];     */  /*   4 */
short datatype;       /*!< Defines data type!    */  /* short datatype;       */  /*  12 */
short bitpix;         /*!< Number bits/voxel.    */  /* short bitpix;         */  /*  14 */
int64_t dim[8];       /*!< Data array dimensions.*/  /* short dim[8];         */  /*  16 */
double intent_p1 ;    /*!< 1st intent parameter. */  /* float intent_p1;      */  /*  80 */
double intent_p2 ;    /*!< 2nd intent parameter. */  /* float intent_p2;      */  /*  88 */
double intent_p3 ;    /*!< 3rd intent parameter. */  /* float intent_p3;      */  /*  96 */
double pixdim[8];     /*!< Grid spacings.        */  /* float pixdim[8];      */  /* 104 */
int64_t vox_offset;   /*!< Offset into .nii file */  /* float vox_offset;     */  /* 168 */
double scl_slope ;    /*!< Data scaling: slope.  */  /* float scl_slope;      */  /* 176 */
double scl_inter ;    /*!< Data scaling: offset. */  /* float scl_inter;      */  /* 184 */
double cal_max;       /*!< Max display intensity */  /* float cal_max;        */  /* 192 */
double cal_min;       /*!< Min display intensity */  /* float cal_min;        */  /* 200 */
double slice_duration;/*!< Time for 1 slice.     */  /* float slice_duration; */  /* 208 */
double toffset;       /*!< Time axis shift.      */  /* float toffset;        */  /* 216 */
int64_t slice_start;  /*!< First slice index.    */  /* short slice_start;    */  /* 224 */
int64_t slice_end;    /*!< Last slice index.     */  /* short slice_end;      */  /* 232 */
char  descrip[80];    /*!< any text you like.    */  /* char descrip[80];     */  /* 240 */
char  aux_file[24];   /*!< auxiliary filename.   */  /* char aux_file[24];    */  /* 320 */
int qform_code ;      /*!< NIFTI_XFORM_* code.   */ /* short qform_code;      */  /* 344 */
int sform_code ;      /*!< NIFTI_XFORM_* code.   */ /* short sform_code;      */  /* 348 */
double quatern_b ;    /*!< Quaternion b param.   */ /* float quatern_b;       */  /* 352 */
double quatern_c ;    /*!< Quaternion c param.   */ /* float quatern_c;       */  /* 360 */
double quatern_d ;    /*!< Quaternion d param.   */ /* float quatern_d;       */  /* 368 */
double qoffset_x ;    /*!< Quaternion x shift.   */ /* float qoffset_x;       */  /* 376 */
double qoffset_y ;    /*!< Quaternion y shift.   */ /* float qoffset_y;       */  /* 384 */
double qoffset_z ;    /*!< Quaternion z shift.   */ /* float qoffset_z;       */  /* 392 */
double srow_x[4] ;    /*!< 1st row affine transform. */  /* float srow_x[4];  */  /* 400 */
double srow_y[4] ;    /*!< 2nd row affine transform. */  /* float srow_y[4];  */  /* 432 */
double srow_z[4] ;    /*!< 3rd row affine transform. */  /* float srow_z[4];  */  /* 464 */
int slice_code ;      /*!< Slice timing order.   */  /* char slice_code;      */  /* 496 */
int xyzt_units ;      /*!< Units of pixdim[1..4] */  /* char xyzt_units;      */  /* 500 */
int intent_code ;     /*!< NIFTI_INTENT_* code.  */  /* short intent_code;    */  /* 504 */
char intent_name[16]; /*!< 'name' or meaning of data. */ /* char intent_name[16]; */  /* 508 */
char dim_info;        /*!< MRI slice ordering.   */      /* char dim_info;        */  /* 524 */
char unused_str[15];  /*!< unused, filled with \0 */                                  /* 525 */
} ;                   /**** 540 bytes total ****/
typedef struct nifti_2_header nifti_2_header ;
 
[b][b][b][b][b][b] [/b][/b][/b][/b][/b][/b]
reply reply with quote
Mar 15, 2011  11:03 AM
RE: 64-bit update to the NIfTI format
[First attempt was a disaster coz editor_plugin malfunctioned on my browser. Attempt 2 has formating error. This is attempt 3]

Originally posted by Mark Jenkinson:
Determining the NIfTI version
The following pseudo-code shows how a file can be tested to see: (a) if it is a NIfTI image file, (b) what version of NIfTI it is, and (c) whether byte-swapping is required.
read in the first 4 bytes from the file
let d = the content of these bytes, formatted as a 32-bit int
if (d==348) then it is a NIfTI-1 file, no byte-swapping required
else if (swap_4bytes(d)==348) then it is a NIfTI-1 file, but with byte-swapping required
else if (d==508) then it is a NIfTI-2 file, no byte-swapping required
else if (swap_4bytes(d)==508) then it is a NIfTI-2 file, but with byte-swapping required
else it is not a valid NIfTI file



Can I suggest putting the magic string test earlier:
read in the first 4 bytes,
let d = the content of these bytes, formatted as a 32 bit int
read in the next 8 bytes
let m = the content of these bytes, formatted as char[8]
jump to the magic string location for NIfTI1. Read in 4 bytes
let m1 - content of these bytes, formatted as char[4];

let byteSwap = UNKNOWN
let actualHdrSize = UNKNOWN
let magicStringType = UNKNOWN

if(d==348 or d==508) then
   let actualHdrSize=d
   let byteSwap=FALSE
else if (swap_4 bytes(d)==348 or swap_4bytes(d)==508) then
   let actualHdrSize = swap_4bytes(d)
   let byteSwap=TRUE

if(testNifTI1MagicString(m1)==TRUE) then
   magicStringType = NIFTI-1
if(testNifTI2MagicString(m) == TRUE) then
   magicStringType = NIFTI-2

if(actualHdrSize==348 and magicStringType == NifTI-1) then we have NifTI 1. STOP
if(actualHdrSize==348 and magicStringType == UNKNOWN) then we have Analyze File. STOP
if(actualHdrSize==508 and magicStringType == NIFTI-2) then we have NifTI 2. STOP
FAIL as we don't have Nifti1 or 2 or Analyze.

C Header Struct[/b]



(1) datatype, intent_code:
can we bump it up to integer to increase the range of available value.

I am glad that the committee agree to make a range of values available for customization. For intent_code at least, I think the existing code fill up almost the full spectrum, meaning available range for customization is small and have to be squeeze in. This means it is more likely that anyone using customization might find their value used by other person. Granted, this is an issue for the developer to sorted out between themselves. However, it will be good to provide sufficient range that collision rarely happens.
For others, it is easier if I simply have to check a bit to tell whether someone had used a custom value, i.e. I will prefer

if ( intent_code & 1<<64 ) then custom intent code used

Instead of
if (25530<25556) then custom intent code used.


(2)slice_start, slice_end :

Is it big enough to accommodate all possible slice value, especially since dim[3] is now int64. Recommend bump them up to int64 as well.


(3) qform_code, sform_code:

Bump up to int for the same reason as datatype??


(4) quantern_a:

Is it still pixdim[0]?
reply reply with quote
Mar 15, 2011  11:03 AM
RE: 64-bit update to the NIfTI format
Sorry, editor_plugin.js failed on my browser (corrected)
reply reply with quote
Mar 16, 2011  08:03 AM
RE: 64-bit update to the NIfTI format
One more thing to ask, which I only realized once I start to think through how I am going to use NifTI-2: Any chance of requiring all unused field and empty byte after the string termination character for the field to be set to \0?

Right now with NifTI-1 I have to use nifti_tool -diff to  compare two headers, because it ignore rubbish that might be present in unused field or empty space after string termination character. it would be useful to be able to use standard UNIX diff as well. Furthermore, the proposal works better with revision control system: the difference in unused field or space after termination character will cause them to treat two identical nifti files (in information) as different.
reply reply with quote
Mar 16, 2011  03:03 PM
RE: 64-bit update to the NIfTI format
Thank you for your thoughtful response to the format revision.

We have taken aboard you comments and made changes to the revised format (the original message in this thread has been edited appropriately).  In particular, we agreed that it seemed necessary for consistency to have slice_start and slice_end set to int64_t.  In addition, and to ensure 8-byte alignment, we have upgraded the other fields to int, except for datatype which we feel still has sufficient capacity (with at least 240 codes still left, and there being a limit to how many different types of data might be stored without requiring extensions to decode the datatypes).

The code range for the custom codes is still to be defined, but it is likely to be a limited range (and not set bitwise) since the use of custom codes is only encouraged as a very temporary measure, with the expectation that useful elements will be registered and obtain official codes.  Otherwise there is a risk of the format becoming less open and accessible for all users. 

As a consequence the header is now 540 bytes + 4 bytes for the initial extension info (total of 544 bytes).

The unused_str will be forced to be filled with \0 characters.

Finally, in answer to your last question, the quatern_a is still stored in the same way (via pixdim[0]) in order to retain the same logic as NIfTI-1 and make the implementation change easier for developers.  I fully expect that future formats will not follow this scheme, and this should be raised in that forum, but for now it is the most consistent and simplest way of modifying the existing format.
reply reply with quote
Mar 17, 2011  06:03 AM
RE: 64-bit update to the NIfTI format
Originally posted by Mark Jenkinson:
Thank you for your thoughtful response to the format revision.

We have taken aboard you comments and made changes to the revised format (the original message in this thread has been edited appropriately).  In particular, we agreed that it seemed necessary for consistency to have slice_start and slice_end set to int64_t.  In addition, and to ensure 8-byte alignment, we have upgraded the other fields to int, except for datatype which we feel still has sufficient capacity (with at least 240 codes still left, and there being a limit to how many different types of data might be stored without requiring extensions to decode the datatypes).

The code range for the custom codes is still to be defined, but it is likely to be a limited range (and not set bitwise) since the use of custom codes is only encouraged as a very temporary measure, with the expectation that useful elements will be registered and obtain official codes.  Otherwise there is a risk of the format becoming less open and accessible for all users. 

As a consequence the header is now 540 bytes + 4 bytes for the initial extension info (total of 544 bytes).

Thank you for considering and adopting some of my proposal. I'm sorry I caused you more work. It was much more work than I was anticipating
The unused_str will be forced to be filled with \0 characters.

Finally, in answer to your last question, the quatern_a is still stored in the same way (via pixdim[0]) in order to retain the same logic as NIfTI-1 and make the implementation change easier for developers.  I fully expect that future formats will not follow this scheme, and this should be raised in that forum, but for now it is the most consistent and simplest way of modifying the existing format.

I agree with your decision. My original intention is just to conform quatern_a is still pixdim[0] as I had inferred it to be.

Thank you.
reply reply with quote
reply