WALZ - WAvelet Lidar Zipper

0.1

WALZ is a framework for storage, access, visualization and analysis of 3D point clouds in general and large LIDAR point clouds in particular.

Purpose

The goals are to:

• Compress data lossless and/or lossy, to reduce the amount of data to store and to transfer, e.g. from a server
• Enable flexible access to low resolution overviews without traversing all points
• Enable full resolution in specific regions-of-interest without traversing all points
• Provide progressive improvements to loaded data
• Provide an easy to use framework for automatic object reconstruction

Important Classes

The most important classes are:

• laserlib::io. Base class for different input/output (file) formats. The class provides virtual methods for reading, writing, compressing and decompressing point clouds. Data are typically processed in blocks of some size (see laserlib::ParametersWalz::cChunkSize), rather than reading writing all the data at once.
• laserlib::walz. A sub class to laserlib::io. Implements the wavelet based zipper, which is the main point with the entire library...
• laserlib::Container. Aggregates vectors of coordinatesand auxilary data for a block. Also aggregates a bounding box for the block.
• laserlib::ParametersWalz. Defines parameters, such as block size, overview level, wavelet type etc. The parameters are used both when reading and writing files.
• laserlib::FileConverter. A high level class to convert from one file format to another. Provides stubs which are called at certain events, e.g. laserlib::FileConverter::OnBlockIsRead, which can be implemented in sub classes to interact with the data and behaviours, see sgis::image::WalzFileConverterWX for an example.
• laserlib::ErrorHandler. A base class for handling errors, warnings, and messages. Can be inherited to not just printf the errors, see sgis::image::WalzErrorHandler for an example.

Use Case

Use Case 1: Convert ASCII file to WALZ file

Convert from an ASCII XYZ format ( laserlib::EFileType::EFT_XYZ ) to the WALZ format (laserlib::EFileType::EFT_WALZ):

// Set an inherited error handler. This one pops up
// error dialogs using wxWidgets
laserlib::ErrorHandler::Set( new sgis::image::WalzErrorHandler() );
        
// Create a file converter and provide the file names in the constructor
// This file converter overloads FileConverter::OnBlockIsRead and FileConverter::OnBlockIsCompressed
sgis::image::WalzFileConverterWX fileConverter( "G:\\data\\test.xyz", "G:\\data\\test.walz" );

// Create a parameter struct and fill with data
laserlib::ParametersWalz param;
param.cTruncateThreshold = 0.0;
param.cOverviewFactor = 1;
param.cnEntries = 6;
param.cLog2ChunkSize = 15;
param.cChunkSize = laserlib::io::Log2ToCount( param.cLog2ChunkSize );
param.cWaveletType = laserlib::EWT_PACKBITS;    // or EWT_DAUB4 or EWT_HAAR_UNSYMMETRIC

// Convert from input to output file with the given parameters
if ( !fileConverter.Convert( param ) )
{
        // Error if we get here
        wxLogError(_("Could not convert files"));
}

// Delete the error handler when you are done
delete laserlib::ErrorHandler::Get();

Use Case 2: Get overview over large area

Get a low resolution overview over the entire area. Implement laserlib::io::GetOverview( int const & )

Todo:

TBD

Use Case 3: Get all points within a region

Get all points within a certian region of interest. Implement laserlib::io::GetPoints( BoundingBox const&, Container& )

Todo:

TBD

Use Case 4: Evaluate the effect of the accuracy

Setting the Parameters::cAccuracy value to a larger value will make a coarser histogram. How does that effect quality and file size?

Todo:

TBD

Use Case 5: Truncating values close to histogram peaks

Truncating values close to histogram peaks will effect quality, but not very much.

Todo:

TBD

Use Case 6: Analyzing detail coefficients for object reconstruction

Values far from histogram peaks contain information on discontinuities. How can that be used for object reconstruction? E.g. set break points at the peaks and vectorize between them.

Todo:

TBD

File Format

The file format for laserlib::walz is binary (i.e. not ASCII) and is composed as follows:

Byte	Description
1-4	4 chars with fixed values: 'W', 'A', 'L', 'Z'
5-8	2 unsigned short defining major and minor file versions. Currently major=0, minor=1.
9-10	1 unsigned short defining laserlib::EWaveletType
11-12	1 unsigned short defining laserlib::ECompressionType
13--	Block by block of data

Each block of data has the following structure:

Byte	Description
1-4	1 unsigned int. Bytes to follow: The size of the block after this unsigned int has been read.
5-52	6 double defining bounding box as top left origin (x0,y0,z0) and size (dx,dy,dz)
53-60	1 double defining point accuracy in [m], e.g. 0.01.
61-64	1 unsigned int defining the number of points in the block
65-68	1 unsigned int defining the size of the compressed buffer
69-	The compressed buffer (size read above)
After buffer	Iff EWT_PACKBITS is used: 3 unsigned int defining the bit length of each coordinate

Todo:

The file format is not settled and will change from time to time.

Todo:

All bounding boxes should be at the beginning of the file together with pointers to where the blocks start

Todo:

Bit length per coordinate should only be included if laserlib::EWaveletType == EWT_PACKBITS

Todo:

The file format does not consider least(most)-significant-bit (LSB/MSB) or byte order