NAME

GRsetchunk, GRgetchunkinfo, GRetchunkcache - GR Chunking Routines

SYNOPSIS

#include <hdf.h>

extern intn GRsetchunk
(
int32 riid,
HDF_CHUNK_DEF chunk_def,
int32 flags
);

extern intn GRgetchunkinfo
(
int32 riid,
HDF_CHUNK_DEF *chunk_def,
int32 *flags
);

extern intn GRsetchunkcache
(
int32 riid,
int32 maxcache,
int32 flags
);

PARAMETERS

int32 riid: IN: raseter access id.
HDF_CHUNK_DEF chunk_def: IN: chunk definition.
int32 flags (GRsetchunk): IN: flags.
HDF_CHUNK_DEF *chunk_def: IN/OUT: chunk definition.
int32 *flags: IN/OUT: flags.
int32 maxcache: IN: max number of chunks to cache.
int32 flags (GRsetchunkcache): IN: flags = 0, HDF_CACHEALL.

DESCRIPTION

GRsetchunk

This routine makes the GR a chunked GR according to the chunk definition passed in.

The dataset currently cannot be special already i.e. NBIT, COMPRESSED, or EXTERNAL. This is an Error.

The defintion of the "HDF_CHUNK_DEF" union with relvant fields is:

typedef union hdf_chunk_def_u
{
int32  chunk_lengths[2];  Chunk lengths along each dimension.
       {
         int32     chunk_lengths[2]; Chunk lengths along each dimension.
         int32     comp_type;        Compression type
         comp_info cinfo;            Compression info struct
       }comp;
} HDF_CHUNK_DEF

The variable agruement 'flags' is a bit-or'd value which can currently be 'HDF_CHUNK' or 'HDF_CHUNK | HDF_COMP'.

The simplist is the 'chunk_lengths' array specifiying chunk lengths for each dimension where the 'flags' argument set to

COMPRESSION is set by using the 'HDF_CHUNK_DEF' union to set the appropriate compression information along with the required chunk lengths for each dimension. The compression information is the same as that set in 'GRsetcompress()'. The bit-or'd 'flags' argument' is set to

See the example in pseudo-C below for further usage.

The maximum number of Chunks in an HDF file is 65,535.

The performance of the GRxxx interface with chunking is greatly affected by the users access pattern over the image and by the maximum number of chunks set in the chunk cache. The cache contains the Least Recently Used(LRU cache replacment policy) chunks. See the routine GRsetchunkcache() for further info on the chunk cache and how to set the maximum number of chunks in the chunk cache. A default chunk cache is always created.

The following example shows the organization of chunks for a 2D array.

e.g. 4x4 array with 2x2 chunks. The array shows the layout of chunks in the chunk array.

    4 ---------------------
      |         |         |
Y     |  (0,1)  |  (1,1)  |
^     |         |         |
|   2 ---------------------
|     |         |         |
|     |  (0,0)  |  (1,0)  |
|     |         |         |
|     ---------------------
|     0         2         4
---------------> X
 --Without compression--:
{
 HDF_CHUNK_DEF chunk_def;
 .......
 -- Set chunk lengths --
 chunk_def.chunk_lengths[0]= 2;
 chunk_def.chunk_lengths[1]= 2;
 -- Set Chunking --
 GRsetchunk(riid, chunk_def, HDF_CHUNK);
 ......
}
 --With compression--:
{
 HDF_CHUNK_DEF chunk_def;
 .......
 -- Set chunk lengths first --
 chunk_def.chunk_lengths[0]= 2;
 chunk_def.chunk_lengths[1]= 2;
 -- Set compression --
 chunk_def.comp.cinfo.deflate.level = 9;
 chunk_def.comp.comp_type = COMP_CODE_DEFLATE;
 -- Set Chunking with Compression --
 GRsetchunk(riid, chunk_def, HDF_CHUNK | HDF_COMP);
 ......
}.

GRgetchunkinfo

This routine gets any special information on the GR. If its chunked, chunked and compressed or just a regular GR. Currently it will only fill the array of chunk lengths for each dimension as specified in the "HDF_CHUNK_DEF" union. It does not tell you the type of compression or the compression parameters used. You can pass in a NULL for "chunk_def" if don't want the chunk lengths for each dimension. Additionaly if successfull it will return a bit-or'd value in "flags" indicating if the GR is:

       Chunked                  -> flags = HDF_CHUNK
       Chunked and compressed   -> flags = HDF_CHUNK | HDF_COMP
       Non-chunked              -> flags = HDF_NONE
e.g. 4x4 array - Pseudo-C
{
 int32   rcdims[3];
 HDF_CHUNK_DEF rchunk_def;
 int32   cflags;
 ...
 rchunk_def.chunk_lengths = rcdims;
 GRgetchunkinfo(sdsid, &rchunk_def, &cflags);
 ...
}.

GRsetchunkcache

Set the maximum number of chunks to cache.

The cache contains the Least Recently Used(LRU cache replacment policy) chunks. This routine allows the setting of maximum number of chunks that can be cached, "maxcache".

The performance of the GRxxx interface with chunking is greatly affected by the users access pattern over the dataset and by the maximum number of chunks set in the chunk cache. The number chunks that can be set in the cache is process memory limited. It is a good idea to always set the maximum number of chunks in the cache as the default heuristic does not take into account the memory available for the application.

By default when the GR is created as a chunked element the maximum number of chunks in the cache "maxcache" is set to the number of chunks along the last dimension.

The values set here affects the current object's caching behaviour.

If the chunk cache is full and "maxcache" is greater then the current "maxcache" value, then the chunk cache is reset to the new "maxcache" value, else the chunk cache remains at the current "maxcache" value.

If the chunk cache is not full, then the chunk cache is set to the new "maxcache" value only if the new "maxcache" value is greater than the current number of chunks in the cache.

Use flags argument of "HDF_CACHEALL" if the whole object is to be cached in memory, otherwise pass in zero(0). Currently you can only pass in zero.

See GRsetchunk() for a description of the organization of chunks in a GR .